Intelligent legal file retrieval method and system based on multi-modal heterogeneous large model

Abstract

The application discloses a kind of legal dossier intelligent retrieval method and system based on multimodal heterogeneous big model, comprising: obtaining original legal dossier data containing text, image and table, carrying out multimodal analysis and structured preprocessing;Through multimodal heterogeneous big model, each modal depth feature is extracted and is fused using multimodal cross attention mechanism;Case element sparse graph is constructed and is optimized by optimization algorithm;Robust learning real value multi-index model algorithm is applied to weight adjustment and vector optimization;Through random recovery sampling technology, candidate case subset is screened and multidimensional matching calculation is carried out;Similar historical case list is visualized and interactive analysis and processing are carried out.The application improves the accuracy and efficiency of legal dossier intelligent retrieval by multimodal fusion understanding, sparse graph optimization and robust learning technology, provides accurate case reference and decision support for legal workers.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for intelligent retrieval of legal files based on a multimodal heterogeneous large model, belonging to the fields of legal technology and intelligent information retrieval technology. Background Technology

[0002] The field of LegalTech has developed rapidly in recent years, especially driven by artificial intelligence, with intelligent information processing and retrieval becoming an important research direction. This field is dedicated to using advanced technologies to improve the efficiency, accuracy, and intelligence of legal work, such as contract review, legal consultation, evidence analysis, and, as this solution focuses, intelligent retrieval of legal files.

[0003] In the area of ​​intelligent retrieval of legal files, common technologies include traditional keyword-matching search engines. These technologies are simple and fast, but they struggle to understand semantic relevance and are prone to missing documents whose keywords do not perfectly match but whose content is relevant. Another approach is text retrieval systems based on Natural Language Processing (NLP) technologies. These systems utilize pre-trained language models such as word vectors, TF-IDF, BM25 algorithms, or the increasingly popular BERT to calculate semantic similarity. These technologies have made significant progress in pure text retrieval, enabling a better understanding of query intent and text content.

[0004] The most relevant prior art to this invention utilizes artificial intelligence models, particularly large language models, for semantic understanding and retrieval of legal documents. These techniques typically begin by preprocessing the legal documents through word segmentation, entity recognition, and relation extraction, then convert the text content into vector representations and store them in a vector database. When a user inputs a query, the system similarly converts the query into a vector and identifies the most relevant document by calculating the similarity (e.g., cosine similarity) between the query vector and the document vectors in the database. Some systems have also begun to attempt to process multimodal information, such as extracting text and images from scanned PDF files, but this usually involves processing different modalities separately or simply concatenating them, failing to achieve deep multimodal information fusion and collaborative understanding.

[0005] Despite advancements in existing technologies, several technical challenges and limitations remain when processing heterogeneous legal files containing various formats such as text, images, and tables. First, current technologies lack the capacity for fusion and understanding of multimodal heterogeneous data, making it difficult to fully uncover the inherent connections and complementarities between different modalities, resulting in an inaccurate grasp of the overall case. Second, when faced with massive amounts of legal documents, there is still significant room for improvement in the accuracy and recall of efficiently extracting core case information and conducting precise and rapid similar case searches, particularly in scenarios requiring deep semantic matching and complex case comparisons, where the performance of existing technologies often falls short of the needs of legal professionals. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for intelligent retrieval of legal case files based on a multimodal heterogeneous large model, aiming to solve the technical problems in the prior art, such as insufficient ability to integrate and understand multimodal heterogeneous data, inaccurate extraction of core case information, and low efficiency and accuracy of large-scale legal document retrieval.

[0007] To achieve the above objectives, the technical solution provided by the present invention is as follows:

[0008] A method for intelligent retrieval of legal files based on a multimodal heterogeneous large model includes the following steps:

[0009] Obtain raw legal case file data containing text, images, and tables; perform multimodal parsing and structured preprocessing on the raw legal case file data to obtain a structured and standardized multimodal data set.

[0010] For the structured and standardized multimodal dataset, deep features of each modality are extracted using a pre-defined multimodal heterogeneous large model and fused using a multimodal cross-attention mechanism to obtain a unified multimodal fusion feature vector;

[0011] Based on the unified multimodal fusion feature vector and the structured and standardized multimodal data set, a sparse graph of case elements is constructed and optimized by a sparse graph optimization algorithm to obtain a structured set of key case information elements and a case element graph.

[0012] The structured set of key case information elements, the case element graph, and the unified multimodal fusion feature vector are subjected to weight adjustment and vector optimization using a robust learning real-valued multi-exponential model algorithm to obtain a robust semantic vector index library.

[0013] Based on user query requests, and combined with the robust semantic vector index library, a subset of candidate cases is selected through random recycling sampling technology and multi-dimensional matching calculation is performed to obtain a list of similar historical cases sorted by comprehensive relevance.

[0014] The list of similar historical cases sorted by comprehensive relevance is visualized and interactively analyzed to obtain a user interface that supports decision support.

[0015] Furthermore, the process of performing multimodal parsing and structured preprocessing on the original legal case file data to obtain a structured and standardized multimodal data set includes:

[0016] The original legal case file data is subjected to format recognition and content type determination to obtain a classified list of original documents and preliminary metadata;

[0017] Based on the file type identifier in the preliminary metadata, OCR recognition and text extraction are performed on text files in the original file list of the classification, and cleaning processes such as removing illegal characters, unified encoding, and removing irrelevant information are performed to obtain the preliminarily cleaned plain text content.

[0018] Based on the image file identifiers in the preliminary metadata, the image files in the original file list of the classification are subjected to format unification conversion, size normalization, image enhancement and noise reduction processing to obtain a standardized set of image files;

[0019] Based on the table file identifier in the preliminary metadata, the table files in the original file list of the classification are parsed and structured extracted, and data type unification and cell content cleaning are performed to obtain structured table data.

[0020] The structured and standardized multimodal data set is obtained by associating and integrating the pre-cleaned plain text content, the standardized image file set, and the structured tabular data.

[0021] Furthermore, the step of extracting deep features of each modality through a preset multimodal heterogeneous large model and fusing them using a multimodal cross-attention mechanism to obtain a unified multimodal fusion feature vector includes:

[0022] The plain text sequences in the structured and standardized multimodal dataset are encoded using a pre-trained language model fine-tuned in the legal domain to obtain high-dimensional semantic feature vectors of the text modality.

[0023] For the image files in the structured and standardized multimodal dataset, feature extraction is performed using a pre-trained visual model to obtain the visual feature vectors of the image modalities;

[0024] The structured tabular data in the structured and standardized multimodal dataset is encoded using a pre-trained tabular processing model to obtain the structured semantic feature vector of the tabular modality.

[0025] The high-dimensional semantic feature vector of the text modality, the visual feature vector of the image modality, and the structured semantic feature vector of the table modality are aligned in feature space through a linear projection layer to obtain the dimensionally aligned features of each modality.

[0026] Based on the modal features after dimensional alignment, multimodal cross-attention is used for deep fusion, and the unified multimodal fusion feature vector is obtained through aggregation operation.

[0027] Furthermore, the construction of the sparse graph of case elements and its optimization using a sparse graph optimization algorithm to obtain a structured set of key case information elements and a case element graph includes:

[0028] Based on the unified multimodal fusion feature vector and the structured and standardized multimodal data set, a preliminary list of key information elements for potential cases is obtained by applying the named entity recognition model, the relation extraction model, and the keyword extraction algorithm.

[0029] The preliminary list of key information elements for potential cases is standardized and entity disambiguated to obtain a standardized set of key information elements.

[0030] Based on the standardized set of key information elements and the original legal case file data, the semantic relationships between elements are analyzed and elements are regarded as nodes and relationships as edges, resulting in an initial case element graph containing nodes and edges.

[0031] The initial case element graph is optimized by drawing on the idea of ​​sparse graph optimization and using a graph cutting algorithm to obtain an optimized sparse graph of case elements.

[0032] Based on the optimized sparse graph of case elements, core case key information elements are extracted and structured to obtain the structured set of case key information elements and the case element graph.

[0033] Furthermore, the robust learning real-valued multi-exponential model algorithm is applied to adjust weights and optimize vectors to obtain a robust semantic vector index library, including:

[0034] For each case in the historical case database, prepare the corresponding structured set of key case information elements, the case element graph, and the unified multimodal fusion feature vector;

[0035] For each historical case, the unified multimodal fusion feature vector is used to learn weights and adjust parameters based on the importance of its key information elements, the structural characteristics of the case element graph, and the contribution of different modal data, so as to obtain the adjusted and optimized robust case semantic vector.

[0036] The robust case semantic vectors of all historical cases are collected and constructed using an approximate nearest neighbor search index structure to obtain the robust semantic vector index library.

[0037] Furthermore, the step of filtering candidate case subsets through random sampling and performing multidimensional matching calculations to obtain a list of similar historical cases sorted by comprehensive relevance includes:

[0038] The user query request is subjected to multimodal parsing and structured processing to obtain the structured multimodal data of the query;

[0039] For the structured multimodal data of the query, feature extraction and fusion are performed through a preset multimodal heterogeneous large model to obtain the multimodal fused feature vector of the query;

[0040] Based on the multimodal fusion feature vector of the query and the structured multimodal data of the query, key information elements are extracted and graphs are constructed to obtain the key information elements and query element graphs of the query.

[0041] Based on the multimodal fusion feature vector of the query and the robust semantic vector index library, a fast preliminary retrieval is performed using online random sampling technology with random recycling to obtain a subset of highly relevant candidate cases;

[0042] For each candidate case in the highly relevant candidate case subset, the vector semantic similarity, cross-modal fine-grained interaction matching score, and key information element graph structured comparison score are comprehensively calculated, and combined with the weight information determined by the robust learning real-valued multi-index model algorithm, to obtain the comprehensive relevance score between each candidate case in the highly relevant candidate case subset and the query.

[0043] Based on the comprehensive relevance score, the candidate cases in the subset of highly relevant candidate cases are sorted in descending order, and the Top-K results are selected to obtain the list of similar historical cases sorted by comprehensive relevance.

[0044] Furthermore, the comprehensive calculation of vector semantic similarity, cross-modal fine-grained interaction matching score, and key information element graph structured comparison score includes:

[0045] Based on the multimodal fusion feature vector of the query and the robust case semantic vector of the candidate case, cosine similarity or other semantic distance is calculated to obtain the vector semantic similarity score;

[0046] For the structured multimodal data of the query and the original multimodal data corresponding to the candidate cases in the highly relevant candidate case subset, a lightweight multimodal cross-attention method is applied for local alignment and matching scoring to obtain a cross-modal fine-grained interactive matching score.

[0047] Based on the query key information elements, the query element graph, and the case element graph of the candidate cases in the highly relevant candidate case subset, the structural similarity, node matching degree, and relationship consistency are calculated using a graph matching algorithm to obtain the key information element graph structured comparison score.

[0048] The vector semantic similarity score, the cross-modal fine-grained interaction matching score, and the key information element graph structured comparison score are weighted and fused to obtain the comprehensive relevance score.

[0049] Based on the weight information about the importance of different elements or modes learned by the robust learning real-valued multi-index model algorithm, the comprehensive relevance score is adjusted and optimized to obtain the final comprehensive relevance score.

[0050] Furthermore, the visualization and interactive analysis of the list of similar historical cases sorted by comprehensive relevance to obtain a user interface supporting decision assistance includes:

[0051] Based on the list of similar historical cases sorted by comprehensive relevance, core information is extracted from the metadata, key information element set and case element map of each case to obtain structured result data;

[0052] The structured result data is used to generate list views, comparison views, graph association views, and statistical analysis views, resulting in a preliminary user interface with multiple presentation methods.

[0053] Based on the initial user interface, interactive filtering controls such as drop-down menus, sliders, and checkboxes are integrated, and drill-down functionality is provided to obtain an enhanced user interface that supports dynamic filtering and drill-down.

[0054] Based on the results of the user's interactive operations through the enhanced user interface, decision support information including highlighted legal provisions, summary of judge's opinions, and analysis of the reasons for the judgment results is provided to obtain the final user interface;

[0055] The final user interface is optimized and adjusted to ensure that it provides users with case references and supports case analysis and decision-making assistance, thus obtaining the user interface that supports decision-making assistance.

[0056] Furthermore, the provision of decision support information based on the results of user interaction through the enhanced user interface, including highlighted legal provisions, summaries of judicial opinions, and analysis of the reasons for the judgment, includes:

[0057] Based on the user's selected cases and focus of attention, relevant legal provisions are extracted from the original case file content and matched with the disputed points of the queried cases to obtain the most relevant original legal provisions highlighted.

[0058] For similar cases that users are interested in, the key judicial opinions and reasoning logic of the judges are extracted from their judgments, and then the text is summarized and structured to obtain a summary of judicial opinions.

[0059] Based on the differences in the judgments of multiple similar cases, this paper analyzes the differences in fact-finding, application of law, and procedural handling that result from the different judgments, and obtains an analysis of the reasons for the judgments.

[0060] The most relevant legal provisions highlighted in the original text, the summary of the judge's opinion, and the analysis of the reasons for the judgment result are integrated and displayed to obtain comprehensive decision-making support information;

[0061] Based on the user's business scenarios and professional needs, the comprehensive decision support information is adjusted and optimized to obtain personalized decision support information.

[0062] This invention also provides an intelligent legal file retrieval system based on a multimodal heterogeneous large model, comprising:

[0063] The multimodal data preprocessing module is used to acquire raw legal case file data containing text, images and tables, and to perform multimodal parsing and structured preprocessing on the raw legal case file data to obtain a structured and standardized multimodal data set.

[0064] The multimodal feature fusion module is used to extract deep features of each modality from the structured and standardized multimodal data set through a preset multimodal heterogeneous large model and fuse them using a multimodal cross-attention mechanism to obtain a unified multimodal fusion feature vector.

[0065] The key information extraction module is used to construct a sparse graph of case elements based on the unified multimodal fusion feature vector and the structured and standardized multimodal data set, and optimize it through a sparse graph optimization algorithm to obtain a structured set of key case information elements and a case element graph.

[0066] The semantic vector indexing module is used to apply a robust learning real-valued multi-exponential model algorithm to adjust the weights and optimize the vectors of the structured case key information element set, the case element map, and the unified multimodal fusion feature vector, so as to obtain a robust semantic vector index library.

[0067] The intelligent retrieval module is used to filter a subset of candidate cases and perform multi-dimensional matching calculations based on user query requests and the robust semantic vector index library, through random recycling sampling technology, to obtain a list of similar historical cases sorted by comprehensive relevance.

[0068] The visualization module is used to visualize and interactively analyze the list of similar historical cases sorted by comprehensive relevance, resulting in a user interface that supports decision support.

[0069] The beneficial effects of this invention are:

[0070] 1. Through deep feature extraction and fusion representation of multimodal heterogeneous large models, a deep understanding of legal case files containing various heterogeneous data such as text, images, and tables was achieved, effectively uncovering the intrinsic connections and complementarities between different modal information, and improving the accuracy of grasping the overall picture of the case;

[0071] 2. Case key information element extraction and association construction based on sparse graph model and requirement cut idea: By constructing case information elements into a graph, and pruning and optimizing the graph with the optimization idea of ​​requirement cut, it can more accurately identify and extract the most core key information combination and its internal relationship in the case. It overcomes the limitations of traditional methods that only rely on surface text features or simple entity links, and improves the ability to understand the deep semantic structure of complex cases.

[0072] 3. Robust multimodal semantic vector construction based on robust learning real-valued multi-exponential model: By introducing robust learning algorithm, the multimodal fusion feature vector of the case is adaptively adjusted and optimized according to the importance of key information elements of the case and the contribution of multimodal features. This makes it more robust to problems such as noise information, missing data and uneven quality of information of different modalities, thereby generating more reliable and discriminative case semantic representations and improving the accuracy of subsequent retrieval.

[0073] 4. An efficient online retrieval and multi-dimensional matching and ranking mechanism integrating random sampling is used. When searching through massive historical cases, the efficient online random sampling technology of random sampling is first used to quickly narrow down the candidate set. Then, on the candidate set, vector semantic similarity, fine-grained multimodal cross-attention matching, and structured comparison of key information element graphs are combined to perform multi-dimensional fine matching and comprehensive ranking. This achieves a significant improvement in retrieval efficiency while ensuring retrieval accuracy and recall, and can quickly respond to user queries.

[0074] 5. Through the visualization and interactive analysis support of multi-dimensional search results, it provides users with a variety of visualization methods, including list view, comparison view, graph association view and statistical analysis view, and supports interactive filtering and drill-down analysis, which greatly improves users' understanding and utilization efficiency of search results, and provides legal professionals with more accurate and comprehensive case references and decision-making assistance. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 This is a flowchart of the intelligent legal file retrieval method based on a multimodal heterogeneous large model according to the present invention;

[0077] Figure 2 This is a structural block diagram of the intelligent legal file retrieval system based on a multimodal heterogeneous large model, according to the present invention. Detailed Implementation

[0078] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0079] In some of the processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as S11, S12, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0080] It will be understood by those skilled in the art that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application’s specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein may include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0081] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have a meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0082] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Throughout the description, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0083] It is important to note that the similarity calculation of legal texts differs fundamentally from that of general texts. Legal similarity requires not only superficial similarity of the texts, but also similarity in legal characterization, legal reasoning logic, and legal application. For example, two cases may use different wording, but if the facts are similar and the applicable law is consistent, they should be considered highly similar; conversely, if the wording is similar but the legal characterization differs (such as "legitimate self-defense" versus "excessive self-defense"), the similarity should be low. Existing technologies primarily rely on general language models for semantic similarity calculations, which struggle to fully capture the essential legal characteristics such as the legal reasoning process, the relationships between legal elements, and differences in legal characterization, resulting in insufficient legal relevance in search results.

[0084] Specifically, legal semantic similarity calculation faces the following unique characteristics and challenges: First, it's not just about literal similarity, but also about similar legal meaning. Two cases may use different words, but if the facts are similar and the applicable law is consistent, they should be judged as having high similarity. Conversely, if the literal similarity is but the legal characterization is different, the similarity should be low. This requires the similarity calculation model to understand the semantic relationships between legal concepts. Second, it needs to consider the logic of legal reasoning. Ideal legal semantic similarity should not only match factual elements but also capture the similarity of the legal reasoning process. For example, if two cases both use the same method of analyzing the elements of the crime and follow the reasoning chain of "fact finding → legal application → legal characterization → judgment," even if the specific factual details differ, their legal reasoning logic is similar. Third, there is the issue of semantic matching between cases and legal provisions. In legal retrieval scenarios, it is often necessary to match case facts with legal provision texts, or to retrieve cases applicable to a legal provision based on that provision. However, case texts and legal provision texts differ greatly in style and structure, making it difficult for traditional text similarity calculation methods to effectively handle such cross-style matching problems.

[0085] Existing technologies have significant shortcomings in addressing the aforementioned issues. While methods based on general pre-trained language models (such as BERT and RoBERTa) can capture some semantic information, they lack legal domain knowledge and struggle to understand legal terminology, relationships between legal concepts, and legal reasoning patterns. This can lead to misclassifying cases with different legal characterizations but similar wording as highly similar, or cases with the same legal meaning but different wording as lowly similar. Methods based on keyword matching or traditional information retrieval techniques rely more on surface textual features and fail to capture deep legal semantics and reasoning logic. Some methods that attempt to introduce legal knowledge graphs or ontology can utilize relationships between legal concepts to some extent, but they often rely on manually constructed knowledge bases with limited coverage and difficulty in capturing case-specific reasoning logic. Furthermore, most existing technologies focus only on the textual modality, lacking effective mechanisms for fusing and understanding multimodal information such as image evidence and tabular data contained in legal files, resulting in an inaccurate grasp of the overall case.

[0086] Therefore, how to construct an intelligent legal file retrieval system that can deeply understand legal knowledge, effectively integrate multimodal information, and accurately capture legal reasoning logic and legal qualitative characteristics, so as to achieve true legal similarity calculation, is a technical problem that urgently needs to be solved in this field.

[0087] Example 1

[0088] like Figure 1 As shown, the intelligent legal file retrieval method based on a multimodal heterogeneous large model provided by this invention includes the following steps:

[0089] Step S1: Obtain the original legal case file data containing text, images and tables, and perform multimodal parsing and structured preprocessing on the original legal case file data to obtain a structured and standardized multimodal data set;

[0090] Step S1.1: Perform format recognition and content type determination on the original legal case file data to obtain a classified list of original documents and preliminary metadata;

[0091] In this embodiment, raw legal case file data containing text, images, and tables is first acquired. This data may come from court case databases, law firm case files, or other legal data sources. Raw legal case files typically contain files in various formats, such as PDF judgments, Word complaints, JPG evidence photos, and Excel lists of evidence. This heterogeneous data requires systematic processing to be effectively utilized.

[0092] These raw legal case file data undergo multimodal parsing and structured preprocessing. For text data, OCR recognition (if it is a scanned document), text extraction, format cleaning, and standardization are performed. For image data, the system performs format unification, size adjustment, image enhancement, and noise reduction. For tabular data, the system performs structure recognition, content extraction, and data type unification. These processing steps ensure that legal case file data from different sources and in different formats can be converted into a standard format that the system can process.

[0093] Through this series of preprocessing operations, a structured and standardized multimodal dataset is ultimately obtained. This dataset includes cleaned and standardized plain text sequences (such as the main text of judgments and descriptions of case facts), processed image files (such as evidence photographs and crime scene investigation diagrams), and structured tabular data (such as evidence lists and case statistics tables). This data is organized into a unified format and linked to the original case files using unique identifiers, laying the foundation for subsequent feature extraction and analysis.

[0094] In step S1.1, preliminary format identification and content type determination are performed on original legal files from different channels (such as the court system, law firm archives, scanners, etc.). These files include various file formats such as PDF, Word documents, JPG / PNG images, and Excel spreadsheets. The format type (e.g., PDF, Word, image, etc.) and approximate content type (e.g., judgment, indictment, evidence materials, etc.) of each file are identified through methods such as file extension, file header information, and content feature analysis. The output list of categorized original files includes information such as file path and filename, while the preliminary metadata includes identifying information such as file type and approximate content type.

[0095] Step S1.2: Based on the file type identifier in the preliminary metadata, perform OCR recognition and text extraction on the text files in the original file list of the classification, and perform cleaning processing to remove illegal characters, uniform encoding and remove irrelevant information to obtain the preliminarily cleaned plain text content;

[0096] In step S1.2, based on the file type identifier in the preliminary metadata output in step S1.1, files identified as text files (such as Word and TXT) or mixed-format files containing a large amount of text (such as PDF) are processed. For formats such as Word and TXT that directly support text extraction, the system directly extracts the text content. For PDF files, it first attempts to extract embedded text. If extraction fails or the quality is poor, it calls an OCR engine (such as Tesseract OCR, optimized for legal terminology) to convert the image to text. After the text is extracted, a series of cleaning processes are performed, including removing illegal characters (such as control characters, special symbols, etc.), unifying the encoding format (such as converting to UTF-8), removing extra spaces and line breaks, and removing irrelevant information such as headers and footers, ultimately obtaining the preliminarily cleaned plain text content.

[0097] Step S1.3: Based on the image file identifiers in the preliminary metadata, perform format unification conversion, size normalization, image enhancement, and noise reduction on the image files in the original file list of the classification to obtain a standardized set of image files;

[0098] In step S1.3, based on the image file identifiers in the preliminary metadata output in step S1.1, image objects identified as image files (such as JPG, PNG, TIFF) or extracted from PDFs are processed. The processing includes format unification (e.g., converting all images to PNG format for subsequent processing), size normalization (adjusting images to a uniform size or proportion according to downstream model requirements), image enhancement (e.g., contrast adjustment and sharpening to improve text or detail clarity), and noise reduction (e.g., using median filtering or Gaussian filtering to remove scanning noise or speckles). These processes aim to improve image quality, making it more suitable for subsequent feature extraction and analysis, ultimately resulting in a standardized set of image files.

[0099] Step S1.4: Based on the table file identifier in the preliminary metadata, the table files in the original file list of the classification are parsed and structured, and data type unification and cell content cleaning are performed to obtain structured table data;

[0100] In step S1.4, based on the table file identifier in the preliminary metadata output in step S1.1, the table structures identified as table files (such as Excel, CSV) or detected in PDF and Word documents are processed. For native table files such as Excel and CSV, their row and column data and cell content are directly parsed; for tables in documents, table detection and recognition algorithms (such as deep learning-based Table Detection models or a combination of rules and heuristics) are used to locate table areas and extract their cell content and row and column relationships. After extraction, the table data is cleaned, such as removing redundancy caused by merged cells and unifying data types (such as unifying dates in different formats into a standard format and unifying amounts into the same unit), finally obtaining structured table data (e.g., represented in JSON or DataFrame format).

[0101] Step S1.5: Based on the pre-cleaned plain text content, the standardized image file set, and the structured tabular data, associate and integrate them to obtain the structured and standardized multimodal data set.

[0102] In step S1.5, the preliminarily cleaned plain text content output from step S1.2, the standardized image file set output from step S1.3, and the structured tabular data output from step S1.4 are correlated and integrated. This ensures that these different modal data from the same original file can be uniquely identified and mapped to the original file, for example, by establishing a unified document ID and modality type identifier. The integrated dataset contains the plain text sequence, image file set, and structured tabular data corresponding to each file, forming a structured and standardized multimodal data set, providing a foundation for subsequent feature extraction and analysis.

[0103] Step S2: For the structured and standardized multimodal data set, extract the deep features of each modality through a preset multimodal heterogeneous large model and fuse them using a multimodal cross-attention mechanism to obtain a unified multimodal fusion feature vector;

[0104] In this embodiment, deep feature extraction and fusion are performed on the structured and standardized multimodal data set output in step S1. The core of this step is to use a pre-trained large model optimized for different modal data to extract deep semantic features of each modality, and then organically fuse these features through a multimodal cross-attention mechanism.

[0105] For textual data, pre-trained language models (such as Legal-BERT or RoBERTa-Legal) fine-tuned on legal domain corpora are used for processing. These models have been pre-trained on a large number of legal texts and are able to understand the language expressions and concepts specific to law. Through these models, the system can extract high-dimensional semantic feature vectors from legal texts, which capture deep information such as legal concepts, factual descriptions, and legal reasoning in the text.

[0106] For image data, pre-trained visual models (such as Vision Transformer or ResNet) are used for processing. These models can extract rich visual features from images, including information such as shape, texture, and spatial relationships. In legal cases, image evidence (such as accident scene photos, injury photos, etc.) often contains important case information. Through these visual models, this visual information can be transformed into feature vectors that can be processed by computers.

[0107] For tabular data, pre-trained models specifically optimized for table structures (such as TableBERT or Tapas) are used for processing. These models can understand the relationships between the row and column structure and cell content of a table, thereby extracting the structured information contained in the table. In legal cases, tables are often used to present important information such as lists of evidence and detailed financial statements. Through these table processing models, the system can effectively understand and utilize this structured data.

[0108] After extracting the feature vectors from each modality, these feature vectors, which come from different modalities and have different dimensions and distribution characteristics, need to be aligned and fused. First, a linear projection layer or other alignment techniques are used to map the feature vectors from different modalities to a unified feature space, ensuring their comparability and fusionability. Then, a multimodal cross-attention mechanism is employed for deep fusion. This mechanism allows features from different modalities to query and enhance each other, capturing deep cross-modal correlations. For example, evidence described in text can be associated with visual content in an image, and data in a table can be cross-verified with textual descriptions.

[0109] Finally, a unified multimodal fusion feature vector is obtained through aggregation operations (such as weighted averaging, concatenation followed by a fully connected layer, etc.). This vector integrates information from text, images, and tables, and can comprehensively represent the core semantic content of legal case files, providing a solid foundation for subsequent extraction and retrieval matching of key case information.

[0110] Step S2.1: Encode the plain text sequence in the structured and standardized multimodal data set using a pre-trained language model fine-tuned in the legal domain to obtain a high-dimensional semantic feature vector of the text modality;

[0111] In step S2.1, the plain text sequences from the multimodal dataset output in step S1 are input into a pre-trained language model (such as Legal-BERT, RoBERTa-Legal, etc.) that has been fine-tuned on legal domain corpora. These models have been pre-trained or fine-tuned on large-scale legal text corpora, enabling them to better understand the specific language and semantics of legal texts. Through the model's encoding layer, high-dimensional semantic feature vectors for each text sequence are obtained. These vectors capture the deep semantic information of the text, including features related to legal concepts, factual descriptions, and legal reasoning.

[0112] In this embodiment, the pre-trained language model fine-tuned for the legal domain employs a multi-stage training strategy. First, BERT-base-Chinese is selected as the base model, and then a large-scale corpus of legal documents is used for domain-adaptive fine-tuning. This corpus includes various judgments, rulings, indictments, and legal texts, covering multiple legal domains such as civil, criminal, and administrative law, ensuring the model can understand a wide range of legal contexts. The fine-tuning process combines a masked language model task with a document-level next-sentence prediction task, enabling the model to simultaneously learn lexical-level semantics and sentence-level logical relationships. To adapt to the specificities of the legal domain, the original vocabulary was expanded, adding approximately 5000 legal terms, such as specific legal provisions and legal procedural terms. Model training uses a mini-batch training method with a batch size of 32, a learning rate of 2e-5 with linear decay, and is trained for three full epochs on eight V100 GPUs. After this domain-adaptive fine-tuning, the model's accuracy on legal text classification and similarity calculation tasks is about 8.5% higher than the general BERT model, and its F1 score on legal entity recognition tasks is about 6.7% higher, demonstrating good legal semantic understanding capabilities.

[0113] Step S2.2: For the image files in the structured and standardized multimodal dataset, perform feature extraction using a pre-trained visual model to obtain the visual feature vector of the image modality;

[0114] In step S2.2, the image files from the multimodal dataset output in step S1 are input into a pre-trained visual model (such as Vision Transformer (ViT), ResNet, or a model optimized for document images). These models can extract rich visual features from images, such as shape, texture, and spatial relationships. Through the model's feature extraction network, the system obtains a visual feature vector for each image. These vectors represent the visual information in the image and are crucial for understanding the content of evidence photos, charts, signatures, and other image content.

[0115] Step S2.3: Encode the structured tabular data in the structured and standardized multimodal dataset using a pre-trained tabular processing model to obtain the structured semantic feature vector of the tabular modality;

[0116] In step S2.3, the structured tabular data (first converted into a serialized representation or graph structure representation suitable for model input) from the multimodal dataset output in step S1 is input into a pre-trained model (such as TableBERT, TaPas, TUTA, etc.) specifically designed for processing tabular data. These models can understand the structural information (such as row and column relationships) and content information (such as cell values ​​and their semantics) of the table. Through the model's encoding mechanism, a structured semantic feature vector is obtained for each table. These vectors capture the structured information in the table, such as case statistics, evidence lists, and monetary details.

[0117] Step S2.4: Align the high-dimensional semantic feature vector of the text modality, the visual feature vector of the image modality, and the structured semantic feature vector of the table modality in the feature space through a linear projection layer to obtain the dimensionally aligned features of each modality.

[0118] In step S2.4, the text semantic feature vector output in step S2.1, the image visual feature vector output in step S2.2, and the table structured semantic feature vector output in step S2.3 are aligned in feature space. Since feature vectors from different modalities may have different dimensions and distribution characteristics, they are mapped to a unified feature space dimension through a linear projection layer or other alignment techniques (such as OptimalTransport). This step ensures the comparability and fusionability of features from different modalities, laying the foundation for subsequent multimodal fusion and obtaining the dimension-aligned features of each modality.

[0119] Step S2.5: Based on the modal features after dimensional alignment, perform deep fusion using multimodal cross-attention and obtain the unified multimodal fusion feature vector through aggregation operation.

[0120] In step S2.5, a multimodal cross-attention mechanism is used to deeply fuse the dimension-aligned features output from step S2.4. This can be implemented using multiple Transformer Encoder layers, where the query comes from one modality and the key and value come from one or more other modalities, or using a more complex cross-attention mechanism. This mechanism allows features from different modalities to query and enhance each other, capturing deep cross-modal correlations. For example, evidence mentioned in text can be associated with visual content in an image, and data in a table can be cross-verified with textual descriptions. Finally, through pooling or other aggregation operations (such as concatenating the weighted fused features from each modality and then passing them through a fully connected layer), a unified multimodal fusion feature vector is obtained for each legal case file. This vector integrates information from different modalities and can comprehensively represent the core semantics of the case.

[0121] The multimodal cross-attention mechanism in this embodiment is a neural network structure that efficiently fuses information from different modalities. The core design principle of this mechanism is to allow features from different modalities to query and enhance each other, thereby capturing deep semantic relationships between modalities. In its implementation, features from text, image, and table modalities are first mapped to the same-dimensional feature space through a linear projection layer, ensuring direct interaction. Then, a multi-layer cross-attention network is constructed, with each layer containing three parallel cross-attention modules, using text, image, and table modalities as query sources respectively. In each cross-attention module, the query source modality can focus on information from other modalities; for example, the text query module can focus on content in images and tables that is semantically relevant to the current text. To capture richer cross-modal relationships, eight parallel attention heads are designed, each capable of learning different types of cross-modal association patterns. After attention computation, residual connections and layer normalization combine the original features with the attention-enhanced features to form an updated feature representation. This design effectively avoids the vanishing gradient problem and preserves the feature information of the original modality. The entire cross-attention network is stacked with four layers, allowing for full interaction and fusion of information from different modalities. Finally, through weighted averaging or special pooling operations, the updated features from the three modalities are integrated into a unified multimodal fusion feature vector. Experiments show that this cross-attention mechanism can improve cross-modal understanding accuracy by approximately 12% compared to simple feature concatenation or averaging, especially when dealing with cases where there is complementary information between modalities.

[0122] One of the core innovations of this application lies in achieving a leap from superficial semantic similarity to legal meaning similarity through a multimodal heterogeneous large-scale model deeply optimized in the legal domain. Specifically, the legal domain fine-tuned pre-trained language model used in this application is not simply trained on legal texts, but is specially optimized for the task of judging legal meaning similarity.

[0123] In constructing the training data, this application places particular emphasis on legal similarity labeling, rather than simple textual surface similarity. Specifically, senior legal experts were invited to label a large number of case pairs. The labeling criteria explicitly require that similarity be judged based on legal characterization, legal application, and legal reasoning logic, without considering the degree of textual surface similarity. For example, cases with the same legal characterization but different wording are labeled as high similarity. Typical examples include one case described as "the defendant breached the contract by failing to pay for goods," and another described as "the defendant failed to fulfill the payment obligation under the contract." Although the wording is completely different, the legal characterization of both is "breach of contract," and they should be labeled as high similarity. Conversely, cases with similar wording but different legal characterizations are labeled as low similarity. Typical examples include one case involving "legitimate self-defense," and another involving "excessive self-defense." Although both contain the keyword "defense," the legal characterizations are completely different, and they should be labeled as low similarity. These case pairs may have some similarity at the textual level, but their legal characterizations are completely different, and they must be clearly distinguished in the training data.

[0124] Through this carefully designed annotation strategy, the model learns the deep semantic relationships between legal concepts and the logical patterns of legal reasoning during training. The model can understand that although "intent" and "negligence" differ by only one word, they represent completely different subjective elements in a particular law; while both "contract termination" and "contract rescission" lead to contract invalidity, their legal causes and consequences differ. The model can also learn typical patterns of legal reasoning, such as the logical chain of "meeting the elements of a crime → applying relevant legal provisions → making a legal characterization → arriving at a judgment," and the reasoning structure of "existing statutory grounds → satisfying procedural requirements → producing legal consequences." This deep knowledge of the legal field means that the semantic feature vectors extracted by the model are no longer simple textual surface features, but contain rich legal meaning information.

[0125] Furthermore, this application incorporates a contrastive learning mechanism and a difficult negative sample mining strategy during model training. The contrastive learning mechanism requires the model to bring cases with similar legal characterizations closer together in the vector space and to push cases with different legal characterizations further apart, even if they may exhibit opposite characteristics in terms of textual surface similarity. The difficult negative sample mining strategy specifically selects case pairs that are highly similar in textual surface but have different legal characterizations as negative samples, thereby strengthening the model's ability to distinguish differences in legal meaning.

[0126] After the aforementioned optimization training specifically targeting legal similarity, the legal domain fine-tuning model of this application improved the accuracy of legal text classification and similarity calculation tasks by approximately 8.5% compared to the general BERT model, and improved the F1 score of the legal entity recognition task by approximately 6.7%. More importantly, in the task of judging legal similarity (i.e., determining whether two cases have the same legal characterization and consistent legal application), the accuracy reached 91.5%, an improvement of 24.2 percentage points compared to 67.3% using only the general BERT model, and an improvement of 11.7 percentage points compared to 79.8% using the legal domain fine-tuning model without specific similarity optimization. In particular, on difficult samples of "different legal characterizations but similar texts," the accuracy of this application's model was 88.2%, while the general method was only 54.6%, an improvement of 33.6 percentage points; on difficult samples of "same legal characterizations but different wording," the recall of this application's model was 89.7%, while the general method was only 61.3%, an improvement of 28.4 percentage points. These data fully demonstrate that the model in this application has successfully achieved a deep understanding of legal meaning and is able to make similarity judgments beyond the surface features of the text.

[0127] Step S3: Based on the unified multimodal fusion feature vector and the structured and standardized multimodal data set, construct a sparse graph of case elements and optimize it using a sparse graph optimization algorithm to obtain a structured set of key case information elements and a case element graph;

[0128] In this embodiment, based on the unified multimodal fusion feature vector output in step S2 and the structured multimodal data output in step S1, a sparse graph of case elements is constructed and optimized to extract key information elements of the case and their relationships. The innovation of this step lies in organizing the case information into a graph structure and refining it by drawing on the optimization ideas of sparse graphs, thereby more accurately capturing the core elements and internal logic of the case.

[0129] First, guided by multimodal fusion feature vectors, potential key case information elements are extracted from structured multimodal data. The system applies a named entity recognition model optimized for the legal field to identify entities such as parties, courts, legal provisions, time, location, and amount; a relation extraction model is used to discover relationships between entities; and a keyword extraction algorithm is used to identify important concepts and terms in the case. These initially extracted elements constitute the basic elements of case information.

[0130] Next, these initially extracted elements undergo standardization and entity disambiguation. Standardization ensures that the same information in different forms (such as dates in different formats, amounts in different units, or names of people or organizations expressed in different ways) can be represented uniformly. Entity disambiguation ensures that the same entity mentioned in different contexts can be correctly identified and associated, avoiding information duplication and confusion. This step outputs a standardized, unique set of key information elements.

[0131] Next, the semantic relationships between these standardized elements are analyzed, and an initial case element graph is constructed. In this graph, each key information element is represented as a node, and the relationships between elements are represented as edges connecting the nodes. For example, there may be a "litigation" relationship between the plaintiff and the defendant, an "application" relationship between a fact and a legal provision, and a "correspondence" relationship between the points of contention and the judgment. These nodes and edges together constitute the initial graph representing the structure of case information.

[0132] Next, the initial case element graph is optimized. Here, drawing inspiration from the optimization concept of the "Requirement Cut problem on sparse graphs," an objective function is designed to retain the element nodes and edges with the most information and the strongest connections, while removing redundant or secondary information. By executing appropriate graph cutting algorithms or subgraph search algorithms, the system obtains a sparser but more reflective element subgraph of the case. This optimization can be understood as minimizing the complexity of the graph while preserving its key structure and information, making it more focused on the core elements and relationships of the case.

[0133] Finally, the most crucial case information elements are extracted from the optimized case element sparse graph and organized into a structured form. Simultaneously, the system retains the sparse graph itself as a representation of the inherent logical relationships between these elements. This structured organization can involve classifying and sorting elements according to specific categories (such as basic case information, party information, points of contention, legal basis, and judgment results). Ultimately, the system outputs a structured set of key case information elements and a case element graph representing the inherent logical relationships between these elements. Together, these constitute a refined representation of the core case information, providing high-quality structured information for subsequent case retrieval and matching.

[0134] Step S3.1: Based on the unified multimodal fusion feature vector and the structured and standardized multimodal data set, apply the named entity recognition model, relation extraction model and keyword extraction algorithm to obtain a preliminary list of key information elements of potential cases;

[0135] In step S3.1, guided by the unified multimodal fusion feature vector output in step S2, and combined with the structured multimodal data (plain text sequences, image descriptions, and structured tabular data) output in step S1, the Named Entity Recognition (NER) model, Relation Extraction (RE) model, and keyword extraction algorithm are applied to initially extract potential key case information elements and their attributes from the data of each modality. The NER model is optimized for the legal field and can identify entities such as parties, law firms, courts, legal provisions, time, place, and amount; the RE model can discover relationships between entities; and the keyword extraction algorithm helps identify important concepts and terms in the case. The output of this step is a list of initially extracted potential key case information elements, containing various information elements that may be important in the case.

[0136] Step S3.2: Standardize and disambiguate the preliminary extracted list of key information elements for potential cases to obtain a standardized set of key information elements;

[0137] In step S3.2, the preliminary list of key information elements for potential cases extracted in step S3.1 is standardized and entity disambiguated. Standardization includes unifying date formats (e.g., unifying different formats such as "January 1, 2023", "2023-01-01", and "01 / 01 / 2023" into a standard format), converting monetary units (e.g., unifying "10,000 yuan" and "10,000 yuan" into the same unit), and normalizing personal / organization names (e.g., unifying abbreviations, full names, and aliases to a standard representation). Entity disambiguation distinguishes entities with the same name mentioned in different contexts, ensuring each entity has a unique identifier. The output of this step is a standardized and unique set of key information elements, providing clear and consistent foundational data for subsequent relationship mining and graph construction.

[0138] Step S3.3: Based on the standardized set of key information elements and the original legal case file data, analyze the semantic relationships between elements and treat elements as nodes and relationships as edges to obtain an initial case element graph containing nodes and edges;

[0139] In step S3.3, based on the standardized set of key information elements output from step S3.2 and the original case file content, the semantic relationships between the elements are further analyzed. These relationships can be a "litigation" relationship between the plaintiff and the defendant, an "application" relationship between fact A and legal provision B, a "correspondence" relationship between the judgment and the focus of the dispute, etc. Each key information element is considered as a node in a graph, and the relationships between elements are considered as edges in the graph (which can have weights or type labels). For example, one node can be "plaintiff Zhang San" and another node can be "defendant Li Si," and the edge between them can be a "litigation" relationship. The weight represents the importance or certainty of the relationship. The output of this step is an initial case element graph containing nodes and edges, representing the elements in the case and their relationships in graph form.

[0140] Step S3.4: The initial case element graph is optimized by referring to the sparse graph optimization idea and using a graph cutting algorithm to obtain an optimized case element sparse graph;

[0141] In step S3.4, the initial case element graph output in step S3.3 is optimized. Drawing inspiration from the optimization concept of the "Requirement Cut problem on sparse graphs," an objective function is designed. This function aims to retain the element nodes and edges with the most information and the strongest connections, while removing redundant or secondary information. By executing appropriate graph cutting algorithms or subgraph search algorithms (e.g., based on node importance scores, edge weights, graph connectivity, etc.), the case element graph is optimized to obtain a sparser but more reflective subgraph of the case's core elements. This optimization can be understood as minimizing the graph's complexity while preserving its key structure and information, making it more focused on the core elements and relationships of the case.

[0142] This embodiment employs a graph optimization algorithm based on the Requirement Cut concept, a graph structure simplification method specifically designed for legal case element relationship networks. In legal case file analysis, the relationships between case elements are often highly complex, forming dense relationship networks. However, not all relationships are equally important for understanding the core of the case. This algorithm aims to retain the element nodes and relationship edges that best represent the core of the case, while removing redundant or secondary information. Specifically, an importance score is first calculated for each node in the case element graph. This score comprehensively considers the element's frequency of occurrence in the document, the element's centrality in the graph network (calculated using an improved PageRank algorithm), and the semantic relevance of the element to the case conclusion (estimated through multimodal fusion features). Simultaneously, a weight is assigned to each edge between nodes, representing the strength and importance of the relationship. This weight is calculated based on the co-occurrence statistics and semantic association strength of the two elements. Based on this, an optimization objective is defined: to minimize the total number of edges in the graph while ensuring sufficient retention of core case information. The algorithm employs a greedy strategy, first calculating the information loss ratio of each edge (i.e., the ratio of edge weight to the importance of its connected nodes), and then gradually removing edges from low to high according to this ratio until the total importance of the remaining nodes reaches a preset threshold (usually set to 70% of the total importance of the original graph). To maintain the connectivity and structural integrity of the graph, key bridging edges are identified and retained through graph theory analysis; these are edges whose removal would cause the graph to split into multiple unconnected components. Furthermore, domain-knowledge-based constraint rules are introduced to ensure that certain types of legal relationships (such as 'application of legal provisions' relationships) are not over-pruned. After this optimization, the final sparse graph of case elements typically retains about 30% of the edges and 80% of the nodes of the original graph, but can express more than 90% of the core case information, greatly improving the efficiency and accuracy of subsequent processing and analysis.

[0143] Step S3.5: Based on the optimized case element sparse graph, extract the core case key information elements and organize them in a structured manner to obtain the structured case key information element set and the case element graph.

[0144] In step S3.5, the most core key case information elements (such as high-centrality nodes and substructures satisfying specific patterns, as shown in the figure) are extracted from the optimized case element sparse graph output in step S3.4 and organized in a structured manner. Simultaneously, the sparse graph itself is retained as a representation of the inherent logical relationships between these elements. The structured organization can involve classifying and sorting the elements according to specific categories (such as basic case information, party information, points of contention, legal basis, and judgment results). The output of this step is a structured set of key case information elements and a case element graph representing the inherent logical relationships between these elements; together, they constitute a refined representation of the core case information.

[0145] Another core innovation of this application lies in capturing and representing legal reasoning logic by constructing a case element graph. It should be noted that the case element graph constructed in this application is not a simple entity co-occurrence network or knowledge graph, but a structured representation specifically designed to represent legal reasoning logic and legal qualitative information.

[0146] From a legal perspective, the nodes in a case element diagram not only include general entities (such as names of people, places, and organizations), but more importantly, they include special entities and concepts with legal significance. These legal nodes include: specific legal provisions, which directly represent the applicable legal basis in the case; legal concept nodes, such as "legitimate defense," "breach of contract," "force majeure," and "material misunderstanding," which represent the legal characterization or legal grounds involved in the case; disputed issues, such as "whether the statute of limitations has expired" and "whether the contract is valid," which represent the core issues in the case that require legal judgment; legal fact nodes, such as "failure to perform contractual obligations on time," "signing a contract despite knowing the truth," and "causing property damage to the plaintiff," which represent the legally recognized facts of the case; and judgment outcome nodes, such as "supporting the plaintiff's claims," ​​"dismissing the plaintiff's claims," ​​and "ordering the defendant to compensate for losses," which represent the court's final judgment. The existence of these legal nodes makes the element diagram not merely a simple list of case information, but a structured expression of the case's legal attributes. Their presence and attributes in the diagram directly reflect the legal characterization and legal characteristics of the case.

[0147] From the perspective of legal relationships, the edges in the case element diagram represent relationships in a legal sense, rather than simple textual co-occurrence, spatial distance, or temporal sequence. These legal relationships include: "application" relationships, indicating that a specific legal provision applies to a certain legal fact or point of contention, such as "the defendant failed to perform its contractual obligations on time (legal fact node) → application → a certain legal provision (legal provision node)". This relationship reflects the logic of legal application and is the core link of legal reasoning; "constitute" relationships, indicating that a certain behavior or fact constitutes a certain legal concept or legal characterization. This relationship reflects the reasoning process from fact to legal characterization; "lead to" relationships, indicating that a certain legal characterization or legal cause leads to a certain legal consequence; "correspondence" relationships, indicating the correspondence between the point of contention and the judgment result; and "basis" relationships, indicating that the judgment result is based on a certain legal basis.

[0148] The establishment of these legal relationships is not based on simple text distance or co-occurrence statistics, but on a deep semantic understanding of the reasoning sections in legal documents through a relation extraction model. The relation extraction model used in this application is specially trained to identify typical reasoning patterns and expressions in legal documents, such as reasoning markers and logical connectors unique to legal documents, like "because...therefore...", "based on...it is determined to be...", "applicable to...judgment...", "whereas...it is believed...", "in conclusion...", "in accordance with the provisions of...". By identifying these patterns, the model can accurately extract causal relationships, applicable relationships, and constitutive relationships from the legal reasoning chain, rather than simply establishing undifferentiated associations between entities appearing in the same paragraph or sentence.

[0149] From the overall legal perspective of the graph structure, the topological structure of a case element graph reflects the legal reasoning chain and logical hierarchy of the case. A typical civil case element graph may present the following structure: Plaintiff's claim node → Point of contention node → Legal fact determination node → Applicable legal provision node → Legal characterization node → Judgment result node. These nodes are connected by equilateral links such as "proposed", "corresponding", "determined", "applied", "constituted", "leads to", and "basis", forming a complete reasoning chain from the raising of the dispute to the determination of facts, application of law, legal characterization, and final judgment.

[0150] If the element diagrams of two cases are structurally similar—for example, both containing similar node types (both have nodes representing the legal concept of "breach of contract" and applicable legal provisions), similar edge relationships (both exhibit a reasoning path of "facts → application → legal provision → constitution → legal characterization → result → judgment"), and similar topological structures (similar hierarchical and branching structures of the reasoning chain)—then it means that the two cases are similar in legal reasoning logic. Even if they differ in specific factual details, party information, and disputed amounts, from a legal perspective, their approaches to problem-solving, applicable laws, and reasoning logic are consistent, and therefore they should be judged as legally similar. This is precisely the core value of legal case retrieval: finding historical cases that, although differing in factual details, share the same legal issues, similar legal reasoning, and consistent application of law, providing a reference for the handling of current cases.

[0151] Conversely, if the element diagrams of two cases differ significantly in structure—for example, one case's element diagram contains a judgment node for the legal concept of "legitimate defense," while the other case's element diagram contains a node for the legal concept of "excessive defense"—even if both cases involve the same keywords such as "defense" in their surface texts, the core nodes and reasoning paths of the element diagrams are completely different due to their entirely different legal characterizations. Therefore, they should be judged as dissimilar in a legal sense and should not be returned as similar cases.

[0152] Step S4: Apply the robust learning real-valued multi-exponential model algorithm to the structured case key information element set, the case element map, and the unified multimodal fusion feature vector to perform weight adjustment and vector optimization, and obtain a robust semantic vector index library;

[0153] In this embodiment, the structured case key information element set and case element map output in step S3, as well as the unified multimodal fusion feature vector output in step S2, are weighted and optimized using a robust learning real-valued multi-exponential model algorithm to construct a robust semantic vector index library. The innovation of this step lies in introducing a robust learning algorithm, which makes the generated case semantic vectors more robust to problems such as noise, missing data, and uneven quality of information from different modalities.

[0154] First, retrieve each case from the historical case database and prepare a corresponding structured set of key case information elements, a case element map, and a unified multimodal fusion feature vector for each case. These data serve as the foundational input for subsequent feature vector optimization.

[0155] Next, a robust learning real-valued multi-exponential model algorithm is applied to the multimodal fusion feature vectors of each historical case. The core idea of ​​this algorithm is to adaptively adjust the weights and representations of the feature vectors according to the specific circumstances of the case. Specifically, the algorithm considers the following aspects: the importance of key information elements (e.g., some elements have a greater impact on the case outcome), the structural characteristics of the case element graph (e.g., some relationships have higher confidence or are more critical), and the actual contribution of different modalities of data to the case (e.g., image evidence is particularly critical in one case, while textual descriptions may be more important in another).

[0156] Based on these considerations, the algorithm learns a set of weights or adjustment parameters to adjust the original multimodal fusion feature vector. This adjustment makes the vector more resistant to noise interference (e.g., the influence of unimportant elements or low-quality modal data is weakened) and enhances the expression of core semantic information. This can be understood as the model learning how to robustly and adaptively "reshape" the feature vector according to the specific circumstances of the case, making it more discriminative and robust.

[0157] This process generates adjusted and optimized robust case semantic vectors for each historical case. These vectors represent the core semantics of the case more accurately and robustly than the original multimodal fusion feature vectors, exhibiting stronger stability and reliability, especially when facing real-world challenges such as uneven data quality, missing information, or noise interference.

[0158] Finally, robust case semantic vectors from all historical cases are collected, and an efficient Approximate Nearest Neighbor (ANN) search index structure (such as IndexIVFPQ and HNSWlib in FAISS) is used to construct a vector index library. During the construction process, the system selects appropriate index types and parameters (such as the number of cluster centers and quantization bits) based on the distribution and quantity of vectors to balance retrieval speed and accuracy. This index structure can quickly find the vector most similar to the query vector in a large-scale vector set, providing efficient support for subsequent case retrieval.

[0159] Finally, an optimized robust semantic vector index library containing a massive amount of historical cases was output, serving as the foundation for efficient and accurate case retrieval and supporting rapid similar case search and multi-dimensional matching calculation in subsequent steps.

[0160] Step S4.1: Obtain each case from the historical case database and prepare the corresponding structured case key information element set, case element map, and unified multimodal fusion feature vector;

[0161] In step S4.1, each case in the historical case database is retrieved, and for each case, a structured set of key case information elements, a case element map (output from step S3), and a unified multimodal fusion feature vector (output from step S2) that have been processed in previous steps are prepared. The purpose of this step is to prepare the necessary input data for subsequent feature vector weighting and adjustment, ensuring that each historical case has a complete set of key information elements, a case element map, and a multimodal fusion feature vector.

[0162] Step S4.2: For the unified multimodal fusion feature vector of each historical case, based on the importance of its key information elements, the structural characteristics of the case element graph, and the contribution of different modal data, apply the robust learning real-valued multi-exponential model algorithm to learn the weights and adjust the parameters to obtain the adjusted and optimized robust case semantic vector;

[0163] In step S4.2, a robust learning real-valued multi-exponential model algorithm is applied to the multimodal fusion feature vector of each historical case prepared in step S4.1. The core idea of ​​this algorithm is to learn a set of weights or adjustment parameters based on the key information element set of the case (e.g., some elements are more important), the structural characteristics of the case element graph (e.g., some relationships have higher confidence), and the actual contribution of different modalities in the case (e.g., image evidence is particularly crucial in a certain case). These parameters are then used to adjust the original multimodal fusion feature vector, making it more resistant to noise interference (e.g., the influence of unimportant elements or low-quality modal data is weakened) and enhancing the expression of core semantic information. This can be understood as the model learning how to robustly and adaptively "reshape" the feature vector according to the specific circumstances of the case, making it more discriminative and robust. The output of this step is the adjusted and optimized robust case semantic vector, which more accurately and robustly represents the core semantics of the case than the original multimodal fusion feature vector.

[0164] The robust learning real-valued multi-exponential model algorithm in this embodiment is a weight learning method specifically designed for the characteristics of multimodal legal data. The core idea of ​​this algorithm is to adaptively learn the importance distribution of case features, making the final feature representation highly robust to noise and missing data. In its implementation, the importance of key information elements in each case is first assessed. This assessment is based on a combination of factors, including the element's position in the case (e.g., whether it appears in the reasoning section of the judgment), the frequency of its citation, and its relevance to the case outcome. Simultaneously, the contribution of different modalities of data (text, images, tables) to the case is evaluated; for example, in cases where photographic evidence is crucial, the contribution of the image modality is given higher weight. After obtaining this importance and contribution information, the robust learning model is trained iteratively. This model employs a random deactivation strategy similar to Dropout, randomly removing some features or reducing their weights during training, forcing the model to learn more robust representations. To address the long-tail distribution problem common in legal documents (i.e., certain rare case types or legal application situations), the model introduces a focus loss function and a class-balanced sampling strategy. The training process was conducted on a dataset containing approximately 500,000 case samples, using the Adam optimizer with an initial learning rate of 0.001, dynamically adjusted using a cosine annealing strategy. Ultimately, the algorithm generates a set of adaptive weights based on the specific circumstances of each case. These weights are used to adjust the original feature vectors, making them more focused on the core semantics of the case and reducing the impact of noise and non-critical information, thereby significantly improving the discriminative power and stability of the vector representation.

[0165] Step S4.3: Collect the robust case semantic vectors of all historical cases, construct them using an approximate nearest neighbor search index structure, and obtain the robust semantic vector index library.

[0166] In step S4.3, robust semantic vectors generated in step S4.2 for all historical cases are collected, and an efficient Approximate Nearest Neighbor (ANN) search index structure (such as IndexIVFPQ or HNSWlib in FAISS) is used to construct a vector index library. During the construction process, the system selects appropriate index types and parameters (such as the number of cluster centers and quantization bits) based on the distribution and quantity of vectors to balance retrieval speed and accuracy. This index structure can quickly find the vector most similar to the query vector in a large-scale vector set, providing efficient support for subsequent case retrieval. The output of this step is an optimized robust semantic vector index library containing a massive number of historical cases, which is the foundation for the system's efficient case retrieval.

[0167] Step S5: Based on the user query request, and in conjunction with the robust semantic vector index library, a subset of candidate cases is selected through random recycling sampling technology and multi-dimensional matching calculation is performed to obtain a list of similar historical cases sorted by comprehensive relevance.

[0168] In this embodiment, based on the user's query request and the robust semantic vector index library constructed in step S4, a subset of candidate cases is selected through random retrieval sampling technology and multi-dimensional matching calculations are performed to finally obtain a list of similar historical cases sorted by comprehensive relevance. The innovation of this step lies in integrating the efficient online retrieval of random retrieval sampling with the multi-dimensional matching and ranking mechanism, which significantly improves retrieval efficiency while ensuring retrieval accuracy.

[0169] First, the system processes the user's query request. This request may be a case description in natural language, uploaded case file materials, or a specific combination of key information. The system performs multimodal parsing and structured processing on these queries, similar to the processing of the original legal case files in step S1, converting the queries into structured multimodal data that the system can process.

[0170] Next, a similar processing flow as in step S2 is applied to the structured multimodal data of the query. Feature extraction and fusion are performed using a pre-defined multimodal heterogeneous large model to obtain the multimodal fused feature vector of the query. Simultaneously, the system also applies a similar processing flow as in step S3 to extract key information elements from the query and construct a query element graph, preparing for subsequent fine-grained matching.

[0171] Then, based on the query's multimodal fusion feature vector and robust semantic vector index, a fast preliminary retrieval is performed using an efficient online random sampling technique achieved through random recycling. This technique does not perform a brute-force search of the entire index; instead, it quickly locates a sub-region containing potentially highly relevant cases using a low-computational-cost method (such as using a simplified distance metric or hierarchical sampling strategy) and samples from it. Here, "random recycling" refers to a mechanism that ensures the efficiency and coverage of the sampling process while avoiding repeated and inefficient random exploration. This step significantly reduces the number of cases that need to be processed in subsequent fine-grained matching, improving retrieval efficiency.

[0172] Next, a multi-dimensional fine-grained matching process is performed on the initially selected subset of highly relevant candidate cases. This includes three main aspects: First, the cosine similarity or other semantic distance between the multimodal fusion feature vector of the query and the robust semantic vector of the candidate case is calculated to obtain a vector semantic similarity score. Second, if the query contains multiple modal information, the system applies a lightweight multimodal cross-attention mechanism between the specific modal information of the query and the original multimodal information corresponding to the candidate case to perform more detailed local alignment and matching scoring, obtaining a cross-modal fine-grained interactive matching score. Third, the structural similarity, node matching degree, and relationship consistency between the key information elements / graph of the query and the case element graph of the candidate case are compared to obtain a key information element graph structured comparison score.

[0173] Then, the matching scores from these three dimensions are weighted and fused to obtain a comprehensive relevance score. During the fusion process, the system considers the weight information about the importance of different elements or modalities learned in step S4 by the robustly learned real-valued multi-index model, adjusting and optimizing the comprehensive relevance score to obtain the final comprehensive relevance score. This multi-dimensional matching and weighted fusion ensures the comprehensiveness and accuracy of the retrieval results, and can capture the deep semantic connections between queries and cases.

[0174] Finally, the candidate cases are sorted in descending order based on the final comprehensive relevance score, and the Top-K results are selected (K is a preset value or a user-specified number). This sorting ensures that the most relevant cases are listed first, allowing users to view them preferentially. The system outputs a list of similar historical cases sorted by comprehensive relevance, with each case accompanied by a summary of its key information, relevance score, and matching highlights, providing users with ordered, high-quality search results and laying the foundation for subsequent visualization and interactive analysis.

[0175] Step S5.1: Perform multimodal parsing and structured processing on the user query request to obtain the structured multimodal data of the query;

[0176] In step S5.1, the user-input query request (which may be a case description in natural language, uploaded partial case file materials, or a specific combination of key information) undergoes multimodal parsing and structured processing. This process is similar to the processing of the original legal case file in step S1, including text extraction, image processing, and table parsing, with the aim of converting the user query into structured multimodal data that the system can process. The output of this step is the structured multimodal data of the query, which forms the basis for subsequent feature extraction and matching.

[0177] Step S5.2: For the queried structured multimodal data, perform feature extraction and fusion through a preset multimodal heterogeneous large model to obtain the queried multimodal fusion feature vector;

[0178] In step S5.2, the structured multimodal data output from step S5.1 is processed using a similar workflow to step S2, employing a pre-defined multimodal heterogeneous large model for feature extraction and fusion. Specifically, features are extracted from different modalities of the query data, such as text, images, and tables, and then fused using a multimodal cross-attention mechanism to obtain the final multimodal fused feature vector. This vector represents the semantic content of the user's query and is crucial for subsequent retrieval.

[0179] Step S5.3: Based on the multimodal fusion feature vector of the query and the structured multimodal data of the query, extract key information elements and construct a graph to obtain the key information elements and query element graph of the query;

[0180] In step S5.3, based on the multimodal fusion feature vector of the query output in step S5.2 and the structured multimodal data of the query output in step S5.1, a similar processing flow as in step S3 is applied to extract key information elements and construct a graph. The system identifies key information elements (such as parties involved, case facts, points of contention, etc.) from the query and constructs a query element graph representing the relationships between these elements. The output of this step is the key information elements of the query and the query element graph, which provide a structured query representation for subsequent fine-grained matching.

[0181] Step S5.4: Based on the multimodal fusion feature vector of the query and the robust semantic vector index library, perform a fast preliminary retrieval using online random sampling technology with random recycling to obtain a subset of highly relevant candidate cases;

[0182] In step S5.4, based on the query multimodal fusion feature vector (or its reduced-dimensional representation) output in step S5.2 and the robust semantic vector index library output in step S4, a fast preliminary retrieval is performed using an efficient online random sampling technique achieved through random recycling. This technique performs a fast preliminary retrieval in a large-scale index (e.g., using a computationally inexpensive distance metric or hierarchical sampling strategy), or uses methods such as random projection to quickly locate a sub-region containing potentially highly relevant cases and sample from it, rather than performing a brute-force search on the entire index library. Here, "random recycling" refers to ensuring the efficiency and coverage of the sampling process through some mechanism, while avoiding repeated and inefficient random exploration. The goal is to quickly recall a manageable but high-quality subset of candidate cases with low computational cost. The output of this step is a subset of highly relevant candidate cases, which greatly reduces the number of cases that need to be processed in subsequent fine-grained matching, thus improving retrieval efficiency.

[0183] The random retrieval sampling technique in this embodiment is an innovative optimization method for large-scale vector index retrieval, aiming to balance retrieval efficiency and accuracy. Traditional vector retrieval methods either perform a full search (computationally expensive) or use approximate nearest neighbor algorithms (potentially missing important results). The random retrieval sampling technique employs an iterative optimization strategy to maximize retrieval quality with limited computational resources. Specifically, the system first randomly selects a small subset of vectors (typically 1% of the total) from the vector index as an initial candidate set. Then, it calculates the similarity between the query vector and these candidate vectors and assigns an importance weight to each vector based on the similarity score. The weight calculation uses an exponential function with a temperature parameter, which controls the smoothness of the distribution; lower temperatures cause the distribution to concentrate more on high-similarity vectors. Next, probabilistic sampling is performed based on these weights to generate a new candidate set. The key retrieval strategy is that the previous candidate set is not completely discarded; instead, the best portion (typically 30%) is retained and merged with the newly sampled candidate set. This strategy ensures that high-quality candidates are not lost during the randomization process while maintaining sufficient exploratory power. The entire process iterates five times. With each iteration, the quality of the candidate set gradually improves, ultimately resulting in a high-quality subset of candidate cases. Another innovation of this technique is the adoption of an adaptive early stopping strategy. When the quality improvement of the candidate set falls below a threshold after two consecutive iterations, the iteration process is terminated early, further improving efficiency. Experimental results show that, compared with traditional methods, this technique can achieve a recall rate close to over 90% of the full search while retrieving only 5% of the data, significantly improving the system's ability to process massive legal case databases.

[0184] Step S5.5: For each candidate case in the highly relevant candidate case subset, comprehensively calculate the vector semantic similarity, cross-modal fine-grained interaction matching score, and key information element graph structured comparison score, and combine the weight information determined by the robust learning real-valued multi-index model algorithm to obtain the comprehensive relevance score between each candidate case in the highly relevant candidate case subset and the query.

[0185] Step S5.5.1: Based on the multimodal fusion feature vector of the query and the robust case semantic vector of the candidate case, calculate the cosine similarity or other semantic distance to obtain the vector semantic similarity score;

[0186] In step S5.5.1, the cosine similarity or other semantic distance (such as Euclidean distance, dot product, etc.) between the query multimodal fusion feature vector output in step S5.2 and the robust semantic vector of the candidate cases (obtained from the index or recalculated) is calculated. This calculation measures the overall similarity between the query and the candidate cases in the semantic space and is the most basic similarity metric. The output of this step is a vector semantic similarity score, which reflects the degree of matching between the query and the candidate cases in overall semantics.

[0187] Step S5.5.2: Apply lightweight multimodal cross-attention to the original multimodal data corresponding to the candidate cases in the highly relevant candidate case subset of the query structured multimodal data for local alignment and matching scoring to obtain cross-modal fine-grained interactive matching scores;

[0188] In step S5.5.2, if the query contains multiple modalities (e.g., a user uploaded a document fragment containing images), a lightweight multimodal cross-attention mechanism is applied between the specific modal information of the query and the original multimodal information corresponding to the candidate cases to perform more granular local alignment and matching scoring. This mechanism can capture fine-grained interactions between different modalities, such as the correspondence between the text description in the query and the image content in the candidate cases, or the consistency between the chart in the query and the data description in the candidate cases. The output of this step is a cross-modal fine-grained interaction matching score, which provides more granular matching information than vector similarity.

[0189] Step S5.5.3: Based on the query key information elements, the query element graph, and the case element graph of the candidate cases in the highly relevant candidate case subset, calculate the structural similarity, node matching degree, and relationship consistency using a graph matching algorithm to obtain the key information element graph structured comparison score;

[0190] In step S5.5.3, the structural similarity, node matching degree, and relationship consistency between the query key information elements / graph output in step S5.3 and the case element graph of the candidate cases are compared. Graph matching algorithms (such as subgraph isomorphism detection and graph edit distance calculation) or graph embedding-based similarity calculation methods can be used for this comparison. This comparison can assess the degree of matching between the query and candidate cases in terms of structured information, such as whether similar party relationships, applicable legal logic, or case fact organization are involved. The output of this step is the key information element graph structured comparison score, reflecting the degree of matching between the query and candidate cases in terms of structured information.

[0191] Step S5.5.4: The vector semantic similarity score, the cross-modal fine-grained interaction matching score, and the key information element graph structured comparison score are weighted and fused to obtain the comprehensive relevance score;

[0192] In step S5.5.4, the vector semantic similarity score output in step S5.5.1, the cross-modal fine-grained interaction matching score output in step S5.5.2, and the key information element graph structured comparison score output in step S5.5.3 are weighted and fused. The fusion can be a simple linear weighted combination or a more complex non-linear combination (such as through a small neural network). This fusion considers matching information from different dimensions and can provide a more comprehensive similarity assessment. The output of this step is a comprehensive relevance score, which integrates matching information from multiple dimensions.

[0193] The multi-dimensional matching score fusion algorithm in this embodiment employs an adaptive weighting strategy, dynamically adjusting the importance of matching scores across different dimensions based on query characteristics and case attributes. The system first calculates matching scores for three key dimensions: vector semantic similarity score (measuring the overall semantic match between the query and candidate cases), cross-modal fine-grained interaction matching score (measuring the local correspondence between different modal information), and key information element graph structured comparison score (measuring the similarity of case structure and element relationships). These three dimensions capture matching information at different levels and require comprehensive consideration. When fusing these scores, the system first determines the initial weight configuration based on the query type. For example, for queries primarily using natural language descriptions, the vector semantic similarity score receives higher weight; for queries containing a large number of images, the cross-modal interaction matching score is more important; and for queries explicitly requiring structured comparison, the weight of the element graph comparison score is increased. After the initial weights are determined, the system dynamically fine-tunes them based on case-specific weight information provided by a robust learning model. This fine-tuning considers case characteristics such as case complexity, the quality of modal information, and the completeness of the element graph. For example, for cases with incomplete feature maps but high-quality text, the weight of graph structure comparison is reduced while the weight of text semantic matching is increased. Furthermore, a weight optimization mechanism based on historical feedback is introduced. By recording user feedback on search results, the weight configuration is continuously adjusted and optimized, making the system increasingly intelligent and personalized with use. The final comprehensive relevance score is a weighted sum of the scores from these three dimensions, and is normalized to ensure comparability between scores from different cases. Experiments show that this adaptive fusion method improves average accuracy by approximately 15% compared to a fixed-weight fusion strategy, while also increasing user satisfaction.

[0194] Step S5.5.5: Based on the weight information about the importance of different elements or modes learned by the robust learning real-valued multi-index model algorithm, adjust and optimize the comprehensive relevance score to obtain the final comprehensive relevance score.

[0195] In step S5.5.5, the comprehensive relevance score output in step S5.5.4 is adjusted and optimized by incorporating the weight information regarding the importance of different elements or modalities learned from the robust real-valued multi-index model in step S4.2. This adjustment considers the relative importance of different elements and modalities in specific case types or query scenarios, making the relevance assessment more accurate and targeted. The output of this step is the final comprehensive relevance score, which is the case relevance assessment result after multiple considerations and optimizations.

[0196] Step S5.6: Based on the comprehensive relevance score, sort the candidate cases in the subset of highly relevant candidate cases in descending order and select the Top-K results to obtain the list of similar historical cases sorted by comprehensive relevance.

[0197] In step S5.6, based on the comprehensive relevance score of each candidate case output in step S5.5, these candidate cases are sorted in descending order, and the Top-K results are selected (K is a preset value or a user-specified number). This sorting ensures that the most relevant cases are listed first, making it easier for users to view them first. The output of this step is a list of similar historical cases sorted by comprehensive relevance. Each case can be accompanied by its key information summary, relevance score, and matching highlights, providing users with orderly and high-quality search results.

[0198] In the similarity calculation stage, this application achieves accurate judgment of legal significance similarity through multi-dimensional matching calculation, especially the structured comparison of key information element graphs. It is particularly important to emphasize that the key information element graph structured comparison score in this application is not a simple graph structure similarity calculation, but a multi-level comparison mechanism specifically designed for legal significance similarity.

[0199] First, at the level of legal characterization consistency comparison, the system determines whether the legal nature of two cases is the same by comparing whether the element graphs of the query case and the candidate cases contain the same or similar legal characterization nodes. Specifically, the system extracts nodes of all legal concept types from the element graphs and calculates the matching degree of these legal characterization nodes. If the element graph of the query case contains a "constitutes a breach of contract" node, while the candidate case contains a "does not constitute a breach of contract" node or a "constitutes a material misunderstanding" node, even if other factual elements (such as the identities of the parties, the subject matter of the contract, the transaction amount, etc.) are highly similar, the legal characterization consistency score of the candidate case will be very low because the core legal characterization is different. "Constitutes a breach of contract" means that one party intentionally conceals the truth or fabricates facts, causing the other party to fall into a mistaken understanding. The legal consequences are that the contract is voidable and the breaching party may be liable for damages. The legal characterization and legal consequences of "does not constitute a breach of contract" or "constitutes a material misunderstanding" are completely different. Therefore, the difference in legal characterization is the primary criterion for judging the similarity of legal meaning.

[0200] Secondly, at the level of legal reasoning path similarity comparison, the system determines whether the legal reasoning logic of the two cases is consistent by comparing the similarity of the reasoning paths in the element graphs of the query case and the candidate cases. Specifically, the system identifies the reasoning paths in the element graphs from fact nodes to legal characterization nodes and then to judgment result nodes, and analyzes whether the edge type sequences and node type sequences in these paths are similar.

[0201] Third, at the level of comparing the consistency of legal application, the system determines whether the legal application of the two cases is consistent by comparing the legal application nodes and their relationships in the element graphs of the query case and the candidate cases. Consistency in legal application is an important indicator of the similarity of legal meaning.

[0202] When comparing the application of laws, the system not only compares whether the article numbers of the legal provisions are the same, but also analyzes whether the specific circumstances under which the legal provisions are applied match.

[0203] Fourth, at the level of comparing the correspondence of disputed issues, the system determines whether the core legal issues of two cases are the same by comparing the disputed issue nodes in the element graphs of the query case and candidate cases and their correspondence with the judgment results. The disputed issue is the core legal problem of a case; different disputed issues mean that the cases involve different legal problems. For example, the core dispute in two contract disputes is "whether the contract is voidable due to a material misunderstanding." Even if the contract types and subject matter are different, because the core legal issues are the same (both determining whether a material misunderstanding occurred and whether the contract is voidable), it should be determined that the disputed issues are the same and have legal reference value.

[0204] The system also analyzes the correspondence between the points of contention and the judgment results. For example, if the point of contention in a query case is "whether the statute of limitations has expired," and the judgment result is "the statute of limitations has expired, and the claim is dismissed," and the point of contention in a candidate case is also "whether the statute of limitations has expired," and the judgment result is also "the statute of limitations has expired, and the claim is dismissed," then the two are completely consistent in their points of contention and judgment conclusions, and should be judged as highly similar. Conversely, if the point of contention in a candidate case is also "whether the statute of limitations has expired," but the judgment result is "the statute of limitations has not expired, and the claim is supported," then although the points of contention are the same, the judgment conclusions are opposite. The system will reflect this difference in the overall score, allowing users to refer to cases with the same conclusion as well as to understand cases with opposite conclusions and their reasons.

[0205] Fifth, at the level of comparing the consistency of legal consequences, the system determines whether the legal consequences of two cases are similar by comparing the judgment result nodes and their legal nature in the element graphs of the query case and candidate cases. The similarity of legal consequences is an important manifestation of the similarity of legal meaning. For example, if two cases both rule that "the defendant bears liability for compensation," although the specific compensation amounts and types of liability differ (one is an accident, the other is product liability), the nature of the legal consequences is the same (both involve bearing civil liability for compensation), and therefore they should be judged as having similar legal consequences.

[0206] When comparing legal consequences, the system distinguishes between the nature and specific amount of the consequences. For example, in civil cases, "a judgment ordering the defendant to compensate for losses," "a judgment ordering the defendant to return property," "a judgment ordering the defendant to continue performing the contract," and "a judgment ordering the termination of the contract" are legal consequences of different natures. Even if they all support the plaintiff's claims, their legal natures are different.

[0207] By comparing the legal significance at the five levels mentioned above (consistency in legal characterization, similarity in legal reasoning path, consistency in legal application, correspondence of points of contention, and consistency in legal consequences), the structured comparison score of the key information element diagram can accurately reflect the similarity of cases in core legal dimensions such as legal characterization, legal reasoning, legal application, points of contention, and legal consequences. This ensures that the retrieved similar cases are not only similar on the surface of the text, but also similar in legal significance, providing truly valuable legal references for legal professionals.

[0208] In the process of fusing multidimensional matching scores, this application particularly emphasizes the importance of the key information element graph structured comparison score. Regarding the fusion weight configuration, for legal case retrieval scenarios, the initial weight of the key information element graph structured comparison score is typically set to 0.5, higher than the vector semantic similarity score (0.3) and the cross-modal interaction matching score (0.2), to ensure that legal similarity dominates the final relevance judgment. This weight configuration strategy is based on extensive experimental verification: when the element graph comparison score weight is below 0.4, the system tends to return cases with similar text but different legal characterizations; when the weight is above 0.6, it may be too strict, causing some cases with different wording but similar legal meanings to be missed; a weight of 0.5 achieves the best balance between precision and recall.

[0209] In addition, the system dynamically adjusts weights based on query type and case characteristics. For example, if the element graph of a query case is very complete and contains a clear legal reasoning chain, the system will appropriately increase the weight of the element graph comparison score (up to 0.6-0.7) to fully utilize structured information for accurate matching; if the element graph of a query case is incomplete (e.g., the user only provides a simple case description, and the system extracts limited elements), the system will correspondingly decrease the weight of the element graph comparison score and increase the weight of semantic vector similarity to avoid misjudgments due to insufficient element graph information.

[0210] Through the above technical solution, this application achieves true legal similarity calculation, accurately identifying cases with the same legal characterization, similar legal reasoning, and consistent legal application, while effectively excluding cases with similar text but different legal meanings, providing legal professionals with high-quality case retrieval services. Experimental data shows that this application achieves an accuracy rate of 91.5% in judging legal meaning similarity, and an accuracy rate of 88.2% on difficult samples with "different legal characterizations but similar texts," which is 33.6 percentage points higher than the 54.6% of the general method; the recall rate on difficult samples with "the same legal characterization but different wording" is 89.7%, which is 28.4 percentage points higher than the 61.3% of the general method, fully demonstrating the effectiveness and advancement of the technical solution of this application.

[0211] Step S6: Visualize and interactively analyze the list of similar historical cases sorted by comprehensive relevance to obtain a user interface that supports decision support.

[0212] Step S6.1: Based on the list of similar historical cases sorted by comprehensive relevance, extract core information from the metadata, key information element set and case element map of each case to obtain structured result data;

[0213] In step S6.1, core information is extracted from the list of similar historical cases sorted by relevance output in step S5 for each case. This information comes from the case's metadata (such as case number, court of trial, judgment date, etc.), key information element set (such as parties, case facts, points of contention, legal basis, judgment result, etc.), and case element diagram (reflecting the relationships between elements). This information is organized according to a predefined template or dynamically generated view, ready for visualization. The output of this step is structured result data, providing a foundation for subsequent visualization interface generation.

[0214] Step S6.2: Generate a list view, comparison view, graph association view, and statistical analysis view from the structured result data to obtain a preliminary user interface with multiple presentation methods;

[0215] In step S6.2, based on the structured result data output in step S6.1, various visualization views are generated. The list view displays a sorted list of cases, including basic information such as case number, cause of action, court of trial, judgment date, and relevance score. The comparison view allows users to select several cases and display their similarities and differences with the queried case in key information elements (such as fact-finding, applicable law, and points of contention). The graph association view visualizes the case element graph of the queried case and a similar case, highlighting matching nodes and paths. The statistical analysis view statistically analyzes certain features in the search results (such as judgment type and frequency of relevant legal provisions) and presents them in chart form (such as bar charts and pie charts). The output of this step is a preliminary user interface with multiple presentation methods, providing users with the ability to view and understand the search results from multiple perspectives.

[0216] Step S6.3: Integrate interactive filtering controls such as drop-down menus, sliders, and checkboxes on the basis of the initial user interface, and provide drill-down functionality to obtain an enhanced user interface that supports dynamic filtering and drill-down;

[0217] In step S6.3, based on the interface generated in step S6.2, interactive filtering controls (such as drop-down menus, sliders, checkboxes, etc.) are integrated, allowing users to dynamically filter and select search results based on multiple dimensions, including case type, region, trial level, court, year, applicable legal provisions, and the scope of the judgment. Simultaneously, the system provides drill-down functionality, allowing users to click on specific cases to view more detailed case file information or jump to the original documents. These interactive functions enable users to explore and analyze search results more flexibly, conducting refined filtering and in-depth research according to their needs. The output of this step is an enhanced user interface supporting dynamic filtering and drill-down, providing more powerful interactive capabilities.

[0218] Step S6.4: Based on the results of the user's interactive operation through the enhanced user interface, provide decision support information including highlighted legal provisions, summary of judge's opinions, and analysis of the reasons for the judgment, to obtain the final user interface;

[0219] Step S6.4.1: Based on the case and focus of the user's selection, extract relevant legal provisions from the original case file content and match them with the disputed focus of the queried case to obtain the most relevant original legal provisions highlighted.

[0220] In step S6.4.1, relevant legal provisions are extracted from the original case file content based on the user's selected case and focus of attention. The matching relationship between these provisions and the points of contention in the queried case is analyzed to determine the most relevant provisions. Then, the original text of these provisions is highlighted, emphasizing the parts particularly relevant to the queried case (such as specific clauses, keywords, etc.). This highlighting helps users quickly locate and understand the legal basis, providing direct support for legal analysis. The output of this step is the highlighted original text of the most relevant legal provisions, providing users with accurate legal references.

[0221] Step S6.4.2: For similar cases that users are interested in, extract the judges' key judgment opinions and reasoning logic from their judgments, and perform text summarization and structured organization to obtain a summary of the judgment opinions;

[0222] In step S6.4.2, for similar cases of interest to the user, key judicial arguments and reasoning logic are extracted from their judgments. Text analysis techniques are used to identify the core arguments, legal reasoning, and conclusions in the judgments, followed by text summarization (possibly using extractive or generative summarization techniques) and structured organization (e.g., by category, such as fact-finding, applicable law, and reasoning). This process makes complex legal reasoning clearer and easier to understand. The output of this step is a summary of judicial arguments, providing the user with a concise understanding of the judicial reasoning in similar cases.

[0223] Step S6.4.3: Based on the differences in the judgment results of multiple similar cases, analyze the differences in fact-finding, legal application, and procedural handling caused by the different judgment results, and obtain the analysis of the reasons for the judgment results;

[0224] In step S6.4.3, an in-depth analysis is conducted based on the differences in judgments from multiple similar cases. The system compares these cases in terms of fact-finding (e.g., whether the determination of key facts differs), application of law (e.g., whether the applicable legal provisions differ), and procedural handling (e.g., whether different procedural paths were used), and attempts to explain how these differences led to different judgments. This analysis helps users understand the influencing factors and patterns of change in case outcomes. The output of this step is a causal analysis of the judgment results, providing users with a deeper understanding of the differences between cases.

[0225] Step S6.4.4: Integrate and display the highlighted original text of the most relevant legal provisions, the summary of the judge's opinion, and the analysis of the reasons for the judgment result to obtain comprehensive decision support information;

[0226] In step S6.4.4, the highlighted original text of the most relevant legal provisions output in step S6.4.1, the summary of the judge's opinion output in step S6.4.2, and the causal analysis of the judgment output in step S6.4.3 are integrated and displayed. Integration can be done by presenting these information side-by-side or by organizing them according to a logical relationship (such as from legal provisions to application to result). This integrated display allows users to obtain comprehensive decision-making support information in a unified view. The output of this step is comprehensive decision-making support information, providing users with multifaceted legal analysis support.

[0227] Step S6.4.5: Based on the user's business scenario and professional needs, adjust and optimize the presentation of the comprehensive decision support information to obtain personalized decision support information.

[0228] In step S6.4.5, the comprehensive decision support information output in step S6.4.4 is adjusted and optimized based on the user's business scenario (such as litigation preparation, case research, legal consultation, etc.) and professional needs (such as the needs of different roles such as lawyers, judges, and scholars). Adjustments may include highlighting key information, changing the presentation method, and adjusting the use of professional terminology. This personalization makes the decision support information more tailored to the user's specific needs and usage habits. The output of this step is personalized decision support information, providing customized legal support for different users.

[0229] Step S6.5: Optimize and adjust the final user interface to ensure that it provides users with case references and supports case analysis and decision assistance, thereby obtaining the user interface that supports decision assistance.

[0230] In step S6.5, the final user interface obtained in step S6.4 is optimized and adjusted to ensure usability, readability, and functionality. Optimization may include adjustments to the interface layout, improvements to the interaction flow, and enhancements to the visual design. These optimizations aim to enable the user interface to better support users in case analysis and decision-making. The output of this step is a decision-supporting user interface, which is the final output of the system, providing users (lawyers, judges, etc.) with accurate and comprehensive case references and supporting them in conducting efficient case analysis and decision-making assistance.

[0231] Example 2

[0232] like Figure 2 As shown, the present invention also provides a legal file intelligent retrieval system based on a multimodal heterogeneous large model, comprising:

[0233] The multimodal data preprocessing module 201 is used to acquire raw legal case file data containing text, images and tables, and to perform multimodal parsing and structured preprocessing on the raw legal case file data to obtain a structured and standardized multimodal data set.

[0234] The multimodal feature fusion module 202 is used to extract deep features of each modality from the structured and standardized multimodal data set through a preset multimodal heterogeneous large model and fuse them using a multimodal cross-attention mechanism to obtain a unified multimodal fusion feature vector.

[0235] The key information extraction module 203 is used to construct a sparse graph of case elements based on the unified multimodal fusion feature vector and the structured and standardized multimodal data set, and optimize it through a sparse graph optimization algorithm to obtain a structured set of key case information elements and a case element graph.

[0236] The semantic vector indexing module 204 is used to apply a robust learning real-valued multi-exponential model algorithm to adjust the weights and optimize the structured case key information element set, the case element graph, and the unified multimodal fusion feature vector to obtain a robust semantic vector index library.

[0237] The intelligent retrieval module 205 is used to filter a subset of candidate cases and perform multi-dimensional matching calculations based on the user query request and the robust semantic vector index library, through random recycling sampling technology, to obtain a list of similar historical cases sorted by comprehensive relevance.

[0238] The visualization module 206 is used to visualize and interactively analyze the list of similar historical cases sorted by comprehensive relevance, and to obtain a user interface that supports decision assistance.

[0239] The multimodal data preprocessing module 201 implements the function of step S1, the multimodal feature fusion module 202 implements the function of step S2, the key information extraction module 203 implements the function of step S3, the semantic vector indexing module 204 implements the function of step S4, the intelligent retrieval module 205 implements the function of step S5, and the visualization module 206 implements the function of step S6. For the specific implementation of each module, please refer to the detailed description of the corresponding step in Example 1.

[0240] The intelligent legal case file retrieval method and system based on a multimodal heterogeneous large model provided by this invention effectively improves the accuracy and efficiency of intelligent legal case file retrieval through multimodal fusion understanding, sparse graph optimization and robust learning technologies, providing legal professionals with accurate case references and decision support.

[0241] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-modal heterogeneous large model-based legal file intelligent retrieval method, characterized in that, The method comprises the following steps: obtaining original legal file data containing text, images and tables, performing multi-modal analysis and structured preprocessing on the original legal file data to obtain a structured and standardized multi-modal data set; extracting deep features of each modality from the structured and standardized multi-modal data set by using a pre-set multi-modal heterogeneous large model and fusing the features by using a multi-modal cross attention mechanism to obtain a unified multi-modal fusion feature vector; based on the unified multi-modal fusion feature vector and the structured and standardized multi-modal data set, constructing a case element sparse graph and optimizing the graph by using a sparse graph optimization algorithm to obtain a structured set of case key information elements and a case element graph; applying a robust learning real-valued multi-index model algorithm to weight adjustment and vector optimization on the structured set of case key information elements, the case element graph and the unified multi-modal fusion feature vector to obtain a robust semantic vector index library, including: obtaining each case in a historical case database and preparing the corresponding structured set of case key information elements, case element graph and unified multi-modal fusion feature vector; applying a robust learning real-valued multi-index model algorithm to learn weights and adjust parameters according to the importance of key information elements, the structural characteristics of case element graphs and the contribution of different modal data of the unified multi-modal fusion feature vector of each historical case to obtain robust case semantic vectors after adjustment and optimization; collecting the robust case semantic vectors of all historical cases and constructing an approximate nearest neighbor search index structure to obtain the robust semantic vector index library; based on a user query request, combining the robust semantic vector index library, screening a candidate case subset by using a randomness recycling sampling technique and performing multi-dimensional matching calculation to obtain a similar historical case list sorted by comprehensive correlation, including: extracting key information elements and constructing graphs based on the multi-modal fusion feature vector of the query and the structured multi-modal data of the query to obtain query key information elements and query element graphs; using an online random sampling technique with randomness recycling to perform fast preliminary retrieval based on the multi-modal fusion feature vector of the query and the robust semantic vector index library to obtain a high-correlation candidate case subset; for each candidate case in the high-correlation candidate case subset, comprehensively calculating vector semantic similarity, cross-modal fine-grained interaction matching score and key information element graph structural comparison score, and combining weight information determined by a robust learning real-valued multi-index model algorithm to obtain the comprehensive correlation score of each candidate case in the high-correlation candidate case subset with the query; sorting the candidate cases in the high-correlation candidate case subset in descending order based on the comprehensive correlation score and selecting top-K results to obtain the similar historical case list sorted by comprehensive correlation; performing visual presentation and interactive analysis and processing on the similar historical case list sorted by comprehensive correlation to obtain a user interface supporting decision assistance.

2. The method of claim 1, wherein, The multi-modal analysis and structured preprocessing of the original legal document data obtains a structured and standardized multi-modal data set, including: Format recognition and content type judgment are performed on the original legal document data to obtain a classified original file list and preliminary metadata; Based on the file type identification in the preliminary metadata, OCR recognition and text extraction are performed on the text files in the classified original file list, and cleaning processing such as removing illegal characters, unifying encoding, and removing irrelevant information is performed to obtain preliminary cleaned pure text content; Based on the image file identification in the preliminary metadata, format unified conversion, size normalization, image enhancement, and denoising processing are performed on the image files in the classified original file list to obtain a set of standardized processed image files; Based on the table file identification in the preliminary metadata, the table files in the classified original file list are parsed and structured extracted, and data type unification and cell content cleaning are performed to obtain structured table data; The preliminary cleaned pure text content, the set of standardized processed image files, and the structured table data are associated and integrated to obtain the structured and standardized multi-modal data set.

3. The method of claim 1, wherein, The multi-modal heterogeneous large model extracts deep features of each modality and fuses them using a multi-modal cross-attention mechanism to obtain a unified multi-modal fusion feature vector, including: The structured and standardized multi-modal data set includes pure text sequences, which are encoded by a legal domain fine-tuned pre-trained language model to obtain high-dimensional semantic feature vectors of the text modality; The structured and standardized multi-modal data set includes image files, which are feature-extracted by a pre-trained visual model to obtain visual feature vectors of the image modality; The structured and standardized multi-modal data set includes structured table data, which is encoded by a pre-trained table processing model to obtain structured semantic feature vectors of the table modality; The high-dimensional semantic feature vectors of the text modality, the visual feature vectors of the image modality, and the structured semantic feature vectors of the table modality are aligned in feature space by a linear projection layer to obtain dimensionally aligned modality features; Based on the dimensionally aligned modality features, a multi-modal cross-attention is used for deep fusion and an aggregation operation is performed to obtain the unified multi-modal fusion feature vector.

4. The method of claim 1, wherein, The construction of a case element sparse graph and the optimization of a sparse graph optimization algorithm obtain a structured set of case key information elements and a case element graph, including: Based on the unified multi-modal fusion feature vector and the structured and standardized multi-modal data set, a named entity recognition model, a relationship extraction model, and a keyword extraction algorithm are applied to obtain a list of preliminary extracted potential case key information elements; The list of preliminary extracted potential case key information elements is standardized and entity disambiguated to obtain a set of standardized key information elements; Based on the standardized key information element set and the original legal file data, semantic relationships between elements are analyzed and elements are regarded as nodes and relationships are regarded as edges, to obtain an initial case element graph containing nodes and edges; Based on the optimized case element sparse graph, core case key information elements are extracted and structured, to obtain the structured case key information element set and the case element graph. The steps of obtaining the multi-modal fusion feature vector of the query and the multi-modal fusion feature vector of the query include:

5. The method of claim 1, wherein, The multi-modal analysis and structured processing of the user query request are performed to obtain structured multi-modal data of the query; The feature extraction and fusion of the structured multi-modal data of the query are performed through a pre-set multi-modal heterogeneous model to obtain a multi-modal fusion feature vector of the query. The comprehensive calculation of the vector semantic similarity score, the cross-modal fine-grained interaction matching score, and the key information element graph structured comparison score includes:

6. The method of claim 5, wherein, Based on the multi-modal fusion feature vector of the query and the robust case semantic vector of the candidate case, the cosine similarity or other semantic distance is calculated to obtain a vector semantic similarity score; The lightweight multi-modal cross-attention is applied to the local alignment and matching scoring of the structured multi-modal data of the query and the original multi-modal data corresponding to the candidate case in the high-correlation candidate case set, to obtain a cross-modal fine-grained interaction matching score; Based on the query key information elements, the query element graph, and the case element graph of the candidate case in the high-correlation candidate case set, the graph matching algorithm is used to calculate the structural similarity, node matching degree, and relationship consistency to obtain a key information element graph structured comparison score; The vector semantic similarity score, the cross-modal fine-grained interaction matching score, and the key information element graph structured comparison score are weighted and fused to obtain the comprehensive correlation score; Based on the weight information about the importance of different elements or modalities learned by the robust learning real-valued multi-index model algorithm, the comprehensive correlation score is adjusted and optimized to obtain the final comprehensive correlation score. The visual presentation and interactive analysis processing of the similar historical case list sorted by the comprehensive correlation degree are performed to obtain a user interface supporting decision assistance, including:

7. The method of claim 1, wherein, Based on the similar historical case list sorted by the comprehensive correlation degree, core information is extracted from the metadata, key information element set, and case element graph of each case to obtain structured result data; The structured result data is used to generate a list view, a comparison view, a graph association view, and a statistical analysis view to obtain a preliminary user interface containing multiple presentation modes; On the basis of the preliminary user interface, interactive filtering controls including a drop-down menu, a slider, and a check box are integrated, and a drilling function is provided to obtain an enhanced user interface supporting dynamic filtering and drilling; ​ Based on the result of the user's interaction with the enhanced user interface, decision support information including highlighted original text of statutes, summary of judicial views, and analysis of reasons for the judgment result is provided, and a final user interface is obtained; The final user interface is optimized and adjusted to provide case references and support for case analysis and decision support for the user, and the user interface supporting decision support is obtained.

8. The method of claim 7, wherein, The decision support information based on the result of the user's interaction with the enhanced user interface includes: Based on the user-selected case and focus, relevant statutes are extracted from the original case file content and matched with the dispute focus of the query case for analysis, and the most relevant highlighted original text of statutes is obtained; For similar cases of user interest, key judicial views and reasoning logic are extracted from the judgment documents, and text summaries and structured organizations are performed to obtain summaries of judicial views; Based on the differences in the judgment results of multiple similar cases, differences in fact determination, legal application, and procedural handling that lead to different judgment results are analyzed to obtain an analysis of reasons for the judgment results; The highlighted most relevant original text of statutes, the summary of judicial views, and the analysis of reasons for the judgment results are integrated and displayed to obtain comprehensive decision support information; Based on the user's business scenario and professional needs, the comprehensive decision support information is adjusted and optimized to obtain personalized decision support information.

9. A legal document intelligent retrieval system based on a multi-modal heterogeneous large model, characterized in that, It includes: A multi-modal data preprocessing module is used to obtain original legal file data containing text, images, and tables, and to perform multi-modal analysis and structured preprocessing on the original legal file data to obtain a structured and standardized multi-modal data set; A multi-modal feature fusion module is used to extract deep features of each modality and fuse them using a multi-modal cross-attention mechanism based on the structured and standardized multi-modal data set and a pre-set multi-modal heterogeneous large model to obtain a unified multi-modal fusion feature vector; A key information extraction module is used to construct a case element sparse graph based on the unified multi-modal fusion feature vector and the structured and standardized multi-modal data set, and to optimize it using a sparse graph optimization algorithm to obtain a structured set of case key information elements and a case element graph. A semantic vector index module is configured to apply a robust learning real-valued multi-index model algorithm to weight adjustment and vector optimization on the structured case key information element set, the case element graph, and the unified multi-modal fusion feature vector, to obtain a robust semantic vector index library, including: obtaining each case in a historical case database, preparing the corresponding structured case key information element set, case element graph, and unified multi-modal fusion feature vector; applying a robust learning real-valued multi-index model algorithm to learn weights and adjust parameters according to the importance of key information elements, the structural characteristics of case element graphs, and the contribution of different modal data of the unified multi-modal fusion feature vector of each historical case, to obtain a robust case semantic vector after adjustment and optimization; collecting the robust case semantic vectors of all historical cases, and constructing an approximate nearest neighbor search index structure to obtain the robust semantic vector index library; An intelligent retrieval module is configured to, based on a user query request, combine the robust semantic vector index library, filter a candidate case subset through a random recycling sampling technique, and perform multi-dimensional matching calculation, to obtain a similar historical case list sorted by comprehensive relevance, including: performing key information element extraction and graph construction based on a query multi-modal fusion feature vector and structured multi-modal data to obtain query key information elements and a query element graph; performing fast preliminary retrieval using an online random sampling technique with randomness recycling based on the query multi-modal fusion feature vector and the robust semantic vector index library, to obtain a high-relevance candidate case subset; for each candidate case in the high-relevance candidate case subset, comprehensively calculating vector semantic similarity, cross-modal fine-grained interaction matching score, and key information element graph structural comparison score, and combining weight information determined by a robust learning real-valued multi-index model algorithm, to obtain a comprehensive relevance score of each candidate case in the high-relevance candidate case subset and the query; sorting the candidate cases in the high-relevance candidate case subset in descending order based on the comprehensive relevance score and selecting top-K results, to obtain the similar historical case list sorted by comprehensive relevance; A visual presentation module is configured to perform visual presentation and interactive analysis processing on the similar historical case list sorted by comprehensive relevance, to obtain a user interface supporting decision assistance.

Images