Analysis method and system for model training and extraction of case cause

Abstract

The application discloses a kind of model training and extraction of case cause analysis method and system, by fusing Transformer and graph neural network (GNN) to jointly encode case documents, realize the dual modeling of case text sequence semantics and structure relationship, introduce knowledge graph embedding enhanced model representation, significantly improve the understanding ability of legal professional terms, logic relationship, multi-task learning framework makes model complete cause classification, can also identify key elements, predict law article citation, etc., improve overall understanding depth, overall process degree of automation is high, can greatly improve the accuracy and recall rate of cause extraction, and reduce artificial annotation dependence.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and more particularly to an analytical method and system for training and extracting models of case causes of action. Background Technology

[0002] In early legal information processing systems, rule-based and keyword matching was the mainstream technique for extracting case causes of action. These methods typically rely on a predefined set of rules, dictionaries, and regular expressions defined by legal experts. For example, the system might determine the cause of action by matching the presence of keywords such as "contract dispute" or "tort liability" in the document. However, this method has significant limitations. First, its generalization ability is extremely poor; the system cannot accurately identify new types of cases not covered by the rule base or cases with diverse expressions. Legal language is highly complex and flexible; the same legal concept can have multiple expressions, and simple keyword matching cannot cope with this linguistic diversity. Second, the maintenance and expansion of rules are extremely costly. With updates to laws and regulations and the emergence of new types of cases, legal experts need to constantly update the rule base manually, which is not only time-consuming and labor-intensive but also makes it difficult to ensure the completeness and consistency of the rules. Finally, this method cannot understand the deep semantics and contextual logic of the text, and is prone to erroneous judgments due to keyword ambiguity or misuse; for example, incorrectly classifying "not constituting an infringement" as a "tort dispute."

[0003] In conclusion, an analytical method and system for training and extracting case causes of action are needed to address the shortcomings of existing technologies. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an analysis method and system for model training and extraction of case causes of action, aiming to solve the aforementioned problems.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an analysis method for model training and extraction of case causes of action, comprising the following steps:

[0006] Step S1: Collect data, collect case documents, automatically capture case documents, clean, segment and segment them, and annotate key entities and relationships to obtain structured text;

[0007] Step S2: Create a relationship diagram. Based on the structured text, construct a case relationship diagram containing entities and relationships.

[0008] Step S3: Construct a hybrid neural network model, and use a hybrid model that combines Transformer and graph neural networks to jointly encode the structured text and case relationship graph to obtain a comprehensive representation that combines sequence features and structural features;

[0009] Step S4: Knowledge enhancement, the comprehensive representation is enhanced by pre-trained case knowledge graph embedding vectors;

[0010] Step S5: Output the results. Based on the enhanced comprehensive representation, perform cause-of-fact classification through a multi-task learning framework and output the cause of action for each case.

[0011] Optionally, the data collection in step S1 can be performed in the following manner:

[0012] Step A1: Collect various case documents in batches from public channels, and perform standardized cleaning on the original unstructured documents;

[0013] Step A2: Using a combination of rules and machine learning models, the case documents are segmented according to their inherent structure. Initial segmentation is performed based on keyword matching, and sequence labeling models are used to improve the accuracy of segmentation.

[0014] Step A3: Correctly segment professional terms using a word segmentation model trained on case corpus;

[0015] Step A4: Key entity and relationship annotation. Define entity types and preset relationship types using sequence annotation scheme, construct tuples of entities and relationships between the two parties, and output structured text.

[0016] Optionally, the relationship diagram is established in step S2 in the following way:

[0017] Step B1: Define the node and edge types of the graph based on entities and relations. Generate a graph node for each unique entity and construct directed edges using labeled relation tuples.

[0018] Step B2: Introduce inference rules to supplement implicit relations, use rules to induce or program logic, or implicitly learn such relations through message passing of GNN after graph construction;

[0019] Step B3: Organize and manage graph data using a deep learning framework, initialize feature vectors for each graph node, and construct feature representations for edges in the graph to build a typed directed heterogeneous graph.

[0020] Optionally, step S3 is performed in the following manner:

[0021] Step C1: Use the Transformer encoder to capture sequential features of global semantics and contextual dependencies in case documents;

[0022] Step C2: Encode the constructed case relationship graph using a graph attention network method, learn the low-dimensional vector representation of each node, and assign different weights according to the importance of different neighbors when aggregating neighbor node information to capture the complex relationships between nodes;

[0023] Step C3: Effectively combine the sequence features obtained by the Transformer encoder with the structural features obtained by the GNN encoder, including calculating the attention score between each word feature and all graph features, and then weighting and summing these to obtain a context vector aligned with the sequence features. These context vectors are then concatenated or added together to form a fused feature vector.

[0024] Optionally, step S4 is implemented in the following manner:

[0025] Step D1: Load the knowledge graph embedding, identify all legal entities in the case file, query the pre-trained legal knowledge graph, and retrieve the embedding vector of the corresponding entity;

[0026] Step D2: Integrate static knowledge by using the retrieved knowledge graph embedding vector as an additional feature, combining it with the dynamic structural features learned through GNN and the sequence features extracted by the Transformer encoder, and then inputting it into the attention mechanism for further fusion.

[0027] Step D3: Based on the attention mechanism, calculate the attention weight between each word feature and all graph features, then sum the graph features according to these weights to generate a context vector, and combine this context vector with the original sequence features to form the final fused feature vector.

[0028] Optionally, in step S5, case classification is performed using a multi-task learning framework, including a main task of case classification and a joint training auxiliary task:

[0029] Case cause classification involves modeling the extraction of case causes as a multi-class classification problem. A predefined case cause system is established, and the structure of the case cause analysis model consists of a fully connected layer and a Softmax classifier connected after the enhanced comprehensive representation. The output is the number of predefined case cause categories, and finally, the probability distribution results for each case cause are given.

[0030] Optionally, the loss function for the cause-of-fact analysis is cross-entropy loss:

[0031] In the formula, The label indicates the true cause of action. Predict probabilities for the model.

[0032] Optionally, the joint training auxiliary tasks are key case element identification, case clause citation prediction, and dispute focus identification, with each auxiliary task outputting independently.

[0033] An analysis system for training and extracting case causes of action models, employing the aforementioned analysis method for training and extracting case causes of action models, includes a data acquisition and preprocessing module, a case relationship graph construction module, a hybrid neural network encoding module, a knowledge enhancement module, and a multi-task case cause classification and output module;

[0034] The data acquisition and preprocessing module is used to automatically capture original case documents in batches from the public judicial document platform, perform standardized cleaning of unstructured text, intelligently segment the documents according to logical structure based on rules and sequence labeling models, accurately segment professional terms using a word segmentation model customized for the legal field, identify key legal entities and label the relationships between entities through sequence labeling, and output structured text.

[0035] The case relationship graph construction module is used to construct an initial case relationship graph by taking entities in the structured text as graph nodes and relationships as directed edges. It introduces inference rules or uses implicit learning from GNNs to supplement implicit relationships that are not explicitly labeled but are semantically valid, constructs a typed directed heterogeneous graph, and initializes feature vectors for nodes and edges.

[0036] Optionally, the hybrid neural network encoding module is used to extract the global semantics and contextual dependencies of case documents using a Transformer encoder, aggregate neighbor information on the relationship graph using a graph attention network, learn the structural awareness representation of each entity, and align and fuse sequence features with graph features through a cross-modal attention mechanism to generate a comprehensive semantic representation.

[0037] The knowledge enhancement module identifies legal entities appearing in documents and retrieves their embedding vectors from a pre-trained legal knowledge graph. It combines the knowledge graph embeddings with dynamically learned sequence and graph features, and then integrates multi-source information through an attention mechanism to generate a final context vector rich in external knowledge.

[0038] The multi-task case classification and output module is used to model case identification as a multi-class classification problem. The main task is to output the probability distribution of predefined case categories through a fully connected layer and Softmax, while the auxiliary tasks are to identify key case elements, predict the legal provisions cited in the case, and identify the focus of the dispute.

[0039] The beneficial effects of this invention are:

[0040] 1. In this invention, by integrating Transformer and Graph Neural Network (GNN) to jointly encode case documents, dual modeling of semantic and structural relationships of case text sequences is achieved. Knowledge graph embedding is introduced to enhance model representation, significantly improving the ability to understand legal terminology and logical relationships. The multi-task learning framework enables the model to identify key elements and predict legal citations while completing case classification, thereby improving the overall understanding depth. The overall process is highly automated, which can greatly improve the accuracy and recall of case extraction and reduce the dependence on manual annotation.

[0041] 2. In this invention, the Transformer encoder effectively handles ultra-long case documents, solving the problem of information loss caused by truncation in traditional models. The graph attention network dynamically allocates neighbor weights, enabling the model to focus on relationships that are more critical to the case. Based on the attention mechanism, feature fusion achieves deep integration of sequence and structural information, generating a more comprehensive and discriminative case representation.

[0042] 3. In this invention, a pre-trained legal knowledge graph is embedded to inject explicit case knowledge into the data-driven model. The combination of static knowledge and dynamic context enhances the model's ability to generalize to rare or novel causes of action and avoids misjudgment. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of a method flow of the present invention.

[0044] Figure 2 This is a schematic diagram of step S1 of the present invention.

[0045] Figure 3 This is a schematic diagram of step S2 of the present invention.

[0046] Figure 4 This is a schematic diagram of step S3 of the present invention.

[0047] Figure 5 This is a schematic diagram of step S4 of the present invention.

[0048] Figure 6 This is a schematic diagram of a system structure according to the present invention. Detailed Implementation

[0049] To more clearly illustrate the technical solutions in the embodiments of the invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] like Figures 1 to 5 As shown, an analytical method for training and extracting a model of a case cause of action includes the following:

[0051] Step S1: Collect data, collect case documents, automatically capture case documents, clean, segment and segment them, and annotate key entities and relationships to obtain structured text;

[0052] Data sources: publicly available court judgment documents (such as China Judgments Online), authoritative legal databases (such as Peking University Law Database, Wolverhampton Wanderers, etc.), and other legal and compliant judicial disclosure channels.

[0053] Data collection method: Automated web crawling technology is used to regularly collect civil, criminal, and administrative case documents in batches, ensuring that the collection process complies with laws and regulations and the website's Robots Exclusion Protocol to avoid unauthorized access.

[0054] Data cleaning involves standardizing and cleaning the raw, unstructured documents retrieved from the web scraping process.

[0055] Remove irrelevant information: Use regular expressions to remove noise such as HTML tags, JavaScript code, headers and footers, page numbers, watermarks, and special symbols (such as "□" and "※").

[0056] Standardized formatting: Use UTF-8 encoding for all text; standardize line breaks, spaces, and full-width / half-width characters; merge consecutive blank paragraphs. Example: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] Plaintiff Zhang San alleged that... The text to be cleaned is plain text: "Plaintiff Zhang San alleges...".

[0057] Text segmentation (based on the logical structure of legal documents) utilizes a combination of rules and machine learning models to divide text into paragraphs according to the inherent structure of legal documents.

[0058] Standard paragraph division:

[0059] First section: Court name, case number, and information of the parties (plaintiff, defendant, third party, etc.).

[0060] Factual section: The plaintiff's claims, the defendant's defense, and the facts ascertained by the court.

[0061] Reasoning section: Court opinion, analysis of applicable law, and points of contention.

[0062] Judgment results: main text of the judgment, allocation of litigation costs, notification of the right to appeal, etc.

[0063] Technical Implementation:

[0064] Preliminary segmentation is performed based on keyword matching (such as "after investigation and verification", "this court holds", "the judgment is as follows"); sequence labeling models (such as BiLSTM-CRF) are used to improve the accuracy of segmentation.

[0065] For legal terminology, a segmentation model specifically trained on legal corpora (such as a segmenter fine-tuned with Legal-BERT or jieba segmentation based on dictionary enhancement) is used to ensure that specialized terms are correctly segmented, for example:

[0066] Correctly segment the situation into: "apparent agency", "bona fide acquisition", "liability for breach of contract", and "no-fault liability principle"; avoid incorrectly segmenting it into: "apparent / agency" or "liability for breach of contract".

[0067] Key entity and relationship annotation (building the foundation of structured text)

[0068] Entity annotation specifications

[0069] Labeling method: The BIO (Beginning, Inside, Outside) sequence labeling scheme is adopted;

[0070] Relation annotation specifications define preset relation types and construct triples (entity1, relation, entity2):

[0071] "Plaintiff-Sued-Defendant", "Party-Sign-Contract", "Behavior-Violation-LAW (Legal Provisions)", "Case-Applicable-LAW", and "Company-Legal Representative-PER".

[0072] The annotation process involves manual annotation by trained legal experts; a multi-person cross-annotation + arbitration mechanism is used to ensure consistency and accuracy; semi-automatic annotation is supported: initial annotation is performed using a pre-trained NER model, followed by expert correction.

[0073] Output: Structured text representation, which ultimately transforms the cleaned, segmented, word-segmented, and labeled documents into a structured data format.

[0074] Step S2: Create a relationship diagram. Based on the structured text, construct a case relationship diagram containing entities and relationships.

[0075] Step 1: Define the node and edge types of the graph (heterogeneous graph schema)

[0076] Based on relationships such as entity-event and entity-behavior, construct a heterogeneous information network:

[0077] Node Types:

[0078] Person (natural person, such as plaintiff or defendant), Organization (legal person or institution), Legal Document (legal provisions, judicial interpretation), Contract (contract, agreement), Legal Action (legal act: lawsuit, defense, breach of contract, tort, etc.), DisputeFocus (point of contention), and JudgmentResult (elements of the judgment).

[0079] Edge Types / Relations: sue, sign, violate, apply_to, belong_to, cause, and involve.

[0080] Step 2: Instantiate graph nodes and edges from structured text

[0081] Node creation: Each unique entity generates a graph node; node attributes include: id, text, type, position_in_text (optional); for implicit entities (such as "breach of contract"), virtual nodes can be automatically constructed based on verb phrases.

[0082] Edge creation: Directed edges are constructed directly using the labeled relation triples;

[0083] Example:

[0084] Node e1 (Zhang San) → [sign] → Node e3 (Technical Service Contract)

[0085] Node e2 (Company) → [violate] → Node e4 (Article 509 of the Civil Code)

[0086] Step 3: Introduce inference rules to supplement implicit relationships

[0087] "Deducing implicit indirect relationships through rules" can be achieved through the following logical reasoning:

[0088] Example of a rule:

[0089] If A is the legal representative of Company B, and Company B signed Contract C, then it can be inferred that A signed Contract C on behalf of Company B.

[0090] If behavior X violates legal provision Y, and legal provision Y falls under the Civil Code, then it can be inferred that X is subject to the Civil Code.

[0091] If contract C is subject to legal provision Y, and Y stipulates "liability for breach of contract" → construct a virtual node "breach of contract" and connect C and Y.

[0092] Implementation methods: Use rule engines (such as Drools) or logic programming (such as a subset of Prolog); or implicitly learn such relationships through message passing of GNN after graph construction.

[0093] Step 4: Graph Representation and Storage

[0094] Graph data structures: represented by adjacency lists or sparse matrices; supports heterogeneous graph frameworks (such as HeteroData in PyTorchGeometric, or the Heterograph API in DGL).

[0095] Node feature initialization:

[0096] The initial node embedding can come from: the output of the Transformer encoder on the entity mention context; the pre-trained embedding of the corresponding entity in the Legal Knowledge Graph (LKG) (such as TransE vectors); and concatenation of one-hot type encoding and text embedding.

[0097] Edge features: Edge types can be mapped to learnable embedding vectors; or used as relational attention weight modulators in GAT.

[0098] Output: Case relationship diagram (used for GNN encoding)

[0099] The final constructed graph is a typed directed heterogeneous graph G=(V,E), where:

[0100] V = Ut ∈ TV, where Vt and TV are sets of node types; E = Ur ∈ TE, where Er and TE are sets of edge types; each node v ∈ V has an initial feature vector hv(0); each edge (u, r, v) ∈ E means "u points to v through relation r".

[0101] Step S3: Construct a hybrid neural network model, and use a hybrid model that combines Transformer and graph neural networks to jointly encode the structured text and case relationship graph to obtain a comprehensive representation that combines sequence features and structural features;

[0102] Data preprocessing

[0103] Data cleaning and segmentation: First, the legal documents are cleaned to remove irrelevant information and then divided into logical paragraphs, such as "preface", "facts", "reasons" and "judgment".

[0104] Entity identification and relationship annotation: Deep learning models are used to identify legal entities and their relationships from documents, and these are then annotated according to standards. This step lays the foundation for subsequent graph construction.

[0105] Graph construction

[0106] Entity and Relationship Extraction: A joint extraction model based on deep learning is used to extract legal entities and their relationships from the text, forming a series of entity-relationship triples.

[0107] Graph representation: These triples are constructed into a heterogeneous graph, where nodes represent different types of entities (e.g., parties involved, legal entities), and edges represent relationships between entities (e.g., contractual relationships, tort relationships). Tools that support heterogeneous graph frameworks, such as PyTorch Geometric or DGL, can be considered.

[0108] Model training

[0109] Transformer encoder

[0110] Sequence Feature Extraction: The Transformer encoder is used to capture sequence features in legal documents, particularly global semantics and contextual dependencies. For long texts, an improved Transformer architecture (such as Transformer-XL) can be employed to enhance efficiency and effectiveness.

[0111] GNN encoder

[0112] Structural feature modeling: Encoding is performed on the constructed case relationship graph using methods such as Graph Attention Networks (GAT) to learn the low-dimensional vector representation of each node. Specifically, when aggregating neighbor node information, different weights are assigned based on the importance of different neighbors, thereby more accurately capturing the complex relationships between nodes.

[0113] Feature fusion mechanism

[0114] Synthetic Representation Generation: This paper proposes a feature fusion method based on an attention mechanism to effectively combine the sequence features obtained by the Transformer encoder with the structural features obtained by the GNN encoder. This process may involve calculating the attention score between each word feature and all graph features, weighting and summing these scores to obtain a context vector aligned with the sequence features, and finally concatenating or adding them to form a fused feature vector.

[0115] Step S4: Knowledge enhancement, the comprehensive representation is enhanced by pre-trained case knowledge graph embedding vectors;

[0116] Knowledge Graph Construction

[0117] Collecting and organizing legal data: The first step is to establish a comprehensive legal knowledge graph, which includes, but is not limited to, laws and regulations, judicial interpretations, precedents, and the relationships between them. This information can be obtained from public resources, legal databases, etc., and processed to form structured data.

[0118] Entity recognition and relation extraction: Natural language processing technology is used to identify legal entities (such as parties, judges, and legal provisions) and their relationships (such as "plaintiff-defendant" and "citation-legal provision") from the above data, and convert them into nodes and edges in a graph structure.

[0119] Graph embedding: Graph embedding algorithms (such as TransE and RotatE) are used to convert entities and relations in a knowledge graph into low-dimensional vector representations. These embedding vectors capture the semantic relationships between entities and can be incorporated as additional features into hybrid models.

[0120] Loading knowledge graph embeddings: When the model processes a new case, it first identifies all legal entities mentioned in the text and queries the pre-trained legal knowledge graph to retrieve the embedding vectors of the corresponding entities.

[0121] Integrating static knowledge: The retrieved knowledge graph embedding vectors are used as additional features and combined with the dynamic structural features learned by the GNN and the sequence features extracted by the Transformer encoder. For example, the knowledge graph embeddings can be directly concatenated to the node features of the GNN and then input into the attention mechanism for further fusion.

[0122] Attention-Based Fusion Strategy: An attention-based method is designed to allow the model to dynamically select the most relevant knowledge graph information based on the current task requirements. Specifically, the attention weights between each word feature and all graph features are calculated. Then, the graph features are weighted and summed according to these weights to generate a context vector. Finally, this context vector is combined with the original sequence features to form the final fused feature vector.

[0123] Step S5: Output the results. Based on the enhanced comprehensive representation, perform case classification through a multi-task learning framework and output the case cause of action.

[0124] Case classification is performed using a multi-task learning framework, including the main task of case classification and the auxiliary task of joint training:

[0125] Case cause classification involves modeling the extraction of case causes as a multi-class classification problem. A predefined case cause system is established, and the structure of the case cause analysis model consists of a fully connected layer and a Softmax classifier connected after the enhanced comprehensive representation. The output is the number of predefined case cause categories, and finally, the probability distribution results for each case cause are given.

[0126] The loss function used for cause-of-fact analysis is cross-entropy loss:

[0127] In the formula, The label indicates the true cause of action. Predict probabilities for the model.

[0128] The joint training auxiliary tasks are key case element identification, case clause citation prediction, and dispute focus identification, with each auxiliary task outputting independently.

[0129] This invention achieves dual modeling of the semantic and structural relationships of legal text sequences by integrating Transformer and Graph Neural Network (GNN) for joint encoding of legal documents; it introduces knowledge graph embedding to enhance model representation, significantly improving the ability to understand legal terminology and logical relationships; the multi-task learning framework enables the model to identify key elements and predict legal citations while completing case classification, improving the overall depth of understanding; the overall process is highly automated, which can greatly improve the accuracy and recall of case extraction and reduce reliance on manual annotation.

[0130] Paragraph segmentation is performed by combining rules and machine learning, taking into account both the standardization of legal document format and the flexibility of language, thereby improving the accuracy of segmentation. A customized word segmentation model in the legal field is used to ensure that professional terms such as "bona fide acquisition" and "apparent agency" are correctly segmented, avoiding semantic distortion caused by general word segmentation errors. Standardized cleaning and structured annotation provide high-quality input for subsequent model training, laying the data foundation for high-precision case extraction.

[0131] Constructing typed directed heterogeneous graphs can realistically reflect legal entities (parties, contracts, legal provisions, etc.) and their complex interactions; supplementing implicit relationships (such as "A represents the company in signing the contract") through reasoning rules enhances the semantic integrity of the graph, enabling the model to capture legal facts that are not explicitly stated but are logically valid; initializing features for nodes and edges provides structural support for GNN to effectively aggregate information and improves the quality of graph encoding.

[0132] Transformer encoders (such as Transformer-XL) effectively handle ultra-long legal documents, solving the problem of information loss caused by truncation in traditional models; graph attention networks (GAT) dynamically allocate neighbor weights, enabling the model to focus on relationships that are more critical to the cause of action (such as "plaintiff-breach of contract" rather than "witness"); feature fusion based on attention mechanisms achieves deep integration of sequence and structural information, generating more comprehensive and discriminative case representations.

[0133] By introducing pre-trained legal knowledge graph (LKG) embedding, explicit legal knowledge is injected into the data-driven model, enabling the model to "know the why". The combination of static knowledge (LKG embedding) and dynamic context (GNN+Transformer) enhances the model's ability to generalize to rare or novel causes of action. For example, in cases involving "apparent agency", the model can use LKG to understand its connection with "unauthorized agency" and "bona fide counterparty", avoiding misjudgment.

[0134] The model extracts case causes as a multi-category classification problem, adapting to the standardized case cause system in judicial practice (such as the "Provisions on Case Causes in Civil Cases"); Softmax outputs a probability distribution, supporting Top-K case cause recommendations, facilitating manual review or similar case recommendations; the main task drives the optimization of the entire model, ensuring the high accuracy of the core function—case cause classification.

[0135] Cross-entropy loss is the standard optimization objective for multi-class classification tasks, which can effectively maximize the predicted probability of the true cause of action category. When used in conjunction with Softmax, it provides stable training, fast convergence, and ensures the upper limit of the main task's performance.

[0136] Auxiliary tasks such as key element identification, legal provision citation prediction, and dispute focus identification provide additional supervisory signals, guiding the model to learn more discriminative intermediate representations; multi-task joint training enables the model to fully understand the case, not only knowing "what the cause of action is," but also "why this cause of action is." Experiments show that auxiliary tasks significantly improve the performance of the main task on complex and marginal cases.

[0137] like Figure 6 As shown, an analysis system for model training and extraction of case causes of action is presented. The system employs a model training and extraction analysis method for case causes of action, including a data acquisition and preprocessing module, a case relationship graph construction module, a hybrid neural network encoding module, a knowledge enhancement module, and a multi-task case cause classification and output module.

[0138] The data acquisition and preprocessing module is used to automatically capture original case documents in batches from the public judicial document platform, perform standardized cleaning of unstructured text, intelligently segment the documents according to logical structure based on rules and sequence labeling models, accurately segment professional terms using a word segmentation model customized for the legal field, identify key legal entities and label the relationships between entities through sequence labeling, and output structured text.

[0139] The case relationship graph construction module is used to construct an initial case relationship graph by taking entities in the structured text as graph nodes and relationships as directed edges. It introduces inference rules or uses implicit learning from GNNs to supplement implicit relationships that are not explicitly labeled but are semantically valid, constructs a typed directed heterogeneous graph, and initializes feature vectors for nodes and edges.

[0140] The hybrid neural network encoding module is used to extract global semantics and contextual dependencies of case documents using a Transformer encoder, aggregate neighbor information on the relationship graph using a graph attention network, learn the structure-aware representation of each entity, and align and fuse sequence features with graph features through a cross-modal attention mechanism to generate a comprehensive semantic representation.

[0141] The knowledge enhancement module identifies legal entities appearing in documents and retrieves their embedding vectors from a pre-trained legal knowledge graph. It combines the knowledge graph embeddings with dynamically learned sequence and graph features, and then integrates multi-source information through an attention mechanism to generate a final context vector rich in external knowledge.

[0142] The multi-task case classification and output module is used to model case identification as a multi-class classification problem. The main task is to output the probability distribution of predefined case categories through a fully connected layer and Softmax, while the auxiliary tasks are to identify key case elements, predict the legal provisions cited in the case, and identify the focus of the dispute.

[0143] The case cause-of-fact model training and extraction analysis system proposed in this invention is designed to achieve full automation and intelligence throughout the entire process, from data acquisition, model training, cause-of-fact analysis to security protection. The system mainly consists of four core modules: a data acquisition and preprocessing module, a model training and optimization module, a cause-of-fact extraction and analysis module, and a security and privacy protection module. These four modules collaborate to form a closed-loop, scalable, and highly secure intelligent analysis platform.

[0144] Data acquisition and preprocessing module

[0145] The data acquisition and preprocessing module is the cornerstone of the entire system, responsible for providing high-quality, standardized data input for subsequent model training. This module first automatically crawls massive amounts of legal documents from multiple authoritative data sources (such as publicly available court judgment documents and legal databases), covering various case types including civil, criminal, and administrative cases. The collected raw data is heterogeneous and unstructured, thus requiring a series of meticulous preprocessing operations. The first step is data cleaning, which removes irrelevant information such as HTML tags, special symbols, and redundant spaces, and standardizes the text encoding format. The second step is text segmentation, which, based on the fixed format of legal documents (such as "party information," "basic case information," and "judgment analysis process"), uses a combination of rules and machine learning to segment long texts into paragraphs with clear logical meaning for subsequent processing. Next is legal domain word segmentation, employing a word segmentation model specifically trained on legal corpora to ensure that professional terms such as "breach of contract liability" and "no-fault liability" are correctly segmented. Finally, and most crucially, is data annotation. This module supports both manual and semi-automatic annotation modes. Legal experts annotate documents with case types and can selectively annotate key legal entities (such as parties, claims, and points of contention) and relationships, providing high-quality labeled datasets for subsequent supervised learning and multi-task learning.

[0146] Model Training and Optimization Module

[0147] The model training and optimization module is the core engine of the system, responsible for building and optimizing the deep learning model used for cause-of-fact extraction. The core of this module is the TransGNN hybrid model proposed in this invention. The training process consists of two stages: self-supervised pre-training and multi-task supervised fine-tuning. In the self-supervised pre-training stage, the module utilizes a massive amount of unannotated legal documents and designs specific pre-training tasks (such as masked word prediction and sentence order determination) to allow the model to learn the general language patterns and knowledge of legal texts.

[0148] This phase aims to provide the model with good initial parameters and reduce its reliance on labeled data. In the multi-task supervised fine-tuning phase, the module uses a pre-processed dataset with case labeling and key information annotations to perform end-to-end fine-tuning of the TransGNN model. This phase employs a multi-task learning framework, simultaneously optimizing the loss functions for multiple tasks such as case classification and key element recognition, enabling the model to learn more discriminative features. Furthermore, this module integrates various optimization algorithms, such as learning rate scheduling, gradient pruning, and early stopping, to prevent overfitting and improve convergence speed and final performance. The entire training process is conducted under the strict monitoring of a security and privacy protection module to ensure data and model security.

[0149] Case Extraction and Analysis Module

[0150] The case cause extraction and analysis module is the system's front-end application, directly serving end users (such as judges, lawyers, and legal professionals), providing intelligent case cause extraction and analysis services. When a user uploads a new case document, this module first invokes relevant components from the data acquisition and preprocessing module to quickly standardize the document. Then, the processed text is input into a pre-trained TransGNN model for inference. The model outputs the case cause classification result, along with a series of related analytical information, such as identified key legal elements, predicted cited legal provisions, and a constructed case entity relationship diagram. These analytical results are returned in a structured format (such as JSON) and presented to the user through a web interface or API. This module also provides result visualization functions, such as highlighting key text fragments supporting the case cause determination, helping users understand the model's decision-making process and enhancing the system's interpretability and user trust.

[0151] Security and privacy protection module

[0152] The security and privacy protection module is integrated throughout the system's entire lifecycle, providing comprehensive security for data and models. During data acquisition and storage, this module employs encryption technology to encrypt sensitive data and implements strict access control policies to ensure that only authorized users can access the data. During model training, the module integrates differential privacy technology to ensure that the training process does not leak individual data privacy. During model deployment and inference, it uses technologies such as model encryption and Trusted Execution Environments (TEEs) to protect the model itself from theft or tampering. Furthermore, this module supports federated learning, enabling multiple institutions to jointly train more powerful models without sharing raw data, thereby protecting data sovereignty while achieving knowledge sharing and collaborative model optimization.

[0153] Core model architecture: TransGNN model

[0154] The core of this invention lies in proposing an innovative hybrid model architecture—TransGNN. This architecture cleverly integrates the advantages of Transformer and Graph Neural Networks (GNNs), aiming to comprehensively and deeply understand legal texts from both sequential and structural dimensions. The TransGNN model mainly consists of three parts: a Transformer encoder, a GNN encoder, and a feature fusion mechanism. The Transformer encoder is responsible for capturing the deep semantics and contextual dependencies of the text, while the GNN encoder focuses on modeling the structured information in the text, such as entity relationships. Finally, the feature fusion mechanism effectively combines these two sets of features and introduces an external knowledge graph for knowledge enhancement, thereby generating a comprehensive and rich text representation, providing strong support for subsequent cause-of-law extraction tasks.

[0155] Transformer encoder: handling features of long text sequences

[0156] The Transformer encoder is a fundamental component of the TransGNN model, primarily responsible for processing the sequential features of legal documents and extracting deep semantic information. Considering the typically lengthy nature of legal documents, this invention employs improved Transformer models, such as Transformer-XL or Longformer, to address the excessive computational complexity of standard Transformer models when processing long sequences. These models, by introducing segmented recurrent mechanisms or sparse attention mechanisms, can effectively handle texts of thousands or even tens of thousands of words while maintaining high performance. Through multi-layered self-attention mechanisms, the Transformer encoder can capture the dependencies between any two words in the text, regardless of their distance. This enables the model to better understand the overall semantics of long sentences and paragraphs, providing a solid semantic foundation for subsequent cause-of-fact determination.

[0157] Using TransformerXL to process long legal documents

[0158] To effectively handle extremely long texts such as legal judgments, this invention employs the Transformer-XL model in its Transformer encoder. Unlike the standard Transformer model, Transformer-XL introduces segment-level recurrence and relative positional encoding, enabling it to learn dependencies beyond fixed-length segments. Specifically, when processing the current text segment, the model caches and reuses the hidden state of the previous segment, thus passing information to subsequent segments. This mechanism allows the model to build a long-term "memory," effectively capturing long-distance dependencies across segments. Simultaneously, relative positional encoding allows the model to distinguish words in different positions without relying on absolute positional information, further enhancing the model's flexibility and robustness in handling long texts. By employing Transformer-XL, the model of this invention can better understand the overall structure and logic of legal documents, providing a strong guarantee for accurately extracting the cause of action.

[0159] Extracting global semantics and contextual dependencies

[0160] The core advantage of the Transformer encoder lies in its self-attention mechanism, which calculates the relevance weights of each word in the text to all other words, thereby generating a global contextual representation. In legal texts, the meaning of a word often depends on its surrounding context and may even be related to distant words. For example, the identity of the "plaintiff" and the actions of the "defendant" together determine the nature of the case. The self-attention mechanism can capture this complex dependency, generating an embedding vector for each word that incorporates global information. By stacking multiple layers of Transformer encoders, the model can progressively abstract high-level semantic and syntactic features from low-level lexical features, ultimately forming a vector sequence that comprehensively represents the semantic content of the entire case text, laying a solid foundation for subsequent graph construction and feature fusion.

[0161] Graph Construction and GNN Encoder: Modeling Structural Features of Legal Texts

[0162] Another approach in the TransGNN model utilizes graph neural networks (GNNs) to explicitly model the structured information inherent in legal texts. Legal texts are not simply a collection of words, but rather comprise a series of entities with specific legal significance (such as parties, objects, and legal acts) and the complex relationships between them. To capture this non-linear structural feature, these entities and relationships must first be extracted from the original text and constructed into a case relationship graph. This graph is heterogeneous, where nodes represent different types of entities, and edges represent relationships between entities. Once the case relationship graph is constructed, a powerful GNN encoder, specifically a Graph Attention Network (GAT), is used to process it. The core idea of ​​GAT is to assign different attention weights to different neighbors when aggregating information about neighboring nodes, enabling the model to identify which relationships are more important for understanding the legal status of the current node.

[0163] Extracting entities and relationships from legal texts

[0164] The first and crucial step in constructing a case relationship graph is to accurately extract legal entities and their relationships from unstructured legal text. This invention employs a deep learning-based joint extraction model to accomplish this task.

[0165] This model typically consists of a shared encoder (such as BERT or RoBERTa) and two parallel decoders: one for entity recognition and the other for relation classification. In the entity recognition phase, the model labels each word or sub-word in the text, determining whether it belongs to a predefined legal entity category, such as "PER" (parties), "ORG" (organizations), "LAW" (legal provisions), "CONTRACT" (contract), and "ACTION" (legal act). In the relation extraction phase, the model identifies entity pairs in the text and determines whether a predefined relationship exists between them, such as "belongs to," "signed," "violates," "claims," ​​or "located in." Through this joint extraction process, a lengthy legal document can be transformed into a structured set of entity-relationships containing rich semantic relationships, laying a solid foundation for subsequent case relationship graph construction.

[0166] Construct relationship diagrams such as entity-item and entity-behavior.

[0167] After successfully extracting a series of entity and relation triples (entity 1, relation, entity 2) from the legal text, the next step is to organize these discrete semantic units into a unified relation graph that reflects the overall picture of the case. This invention constructs a Heterogeneous Information Network (HIN), which contains various types of nodes and edges. Node types mainly include: parties (plaintiff, defendant, third party), legal subjects (company, institution), legal objects (contract, property, subject matter), legal acts (signing, performance, breach of contract, tort), legal concepts (rights, obligations, responsibilities), and legal provisions. Edge types are defined according to the extracted relations, such as "party-lawsuit-party," "party-signing-contract," "behavior-cause-result," "case-applicable-legal provision," etc. In the process of constructing the graph, not only are the direct relationships in the original text preserved, but also some implicit indirect relationships are inferred through certain rules. For example, if "A is the legal representative of Company B" and "Company B signed a contract with Company C," then it can be inferred that "A signed the contract on behalf of Company B." This method of constructing graphs allows the core elements of a case and their complex interactions to be represented within a unified framework.

[0168] Applying Graph Attention Network (GAT) for Graph Encoding

[0169] After constructing a heterogeneous graph that reflects the structured information of a case, this invention employs a Graph Attention Network (GAT) as a GNN encoder to learn the low-dimensional vector representation (i.e., embedding) of each node in the graph. The core advantage of GAT lies in its ability to dynamically assign different weights based on the importance of neighboring nodes, thereby more accurately aggregating neighborhood information. This characteristic is particularly important in legal cause-of-fact extraction scenarios. For example, when analyzing a "share transfer dispute" case, the relationship between the "share transfer agreement" node and the "transferor" and "transferee" is clearly more crucial than its relationship with the "witnessing lawyer." By calculating attention coefficients, GAT can automatically assign higher weights to the former, thus referencing more information from the "transferor" and "transferee" when updating the representation of the "share transfer agreement" node. Through iterations of multiple layers of GAT, each node can aggregate information from its higher-order neighbors, thereby generating a node embedding representation that integrates local and global structural information.

[0170] Feature fusion mechanism

[0171] Feature fusion is a core component of the TransGNN model, aiming to effectively combine the sequence features extracted by the Transformer encoder and the structural features extracted by the GNN encoder to form a comprehensive and rich case representation. This paper proposes a feature fusion method based on an attention mechanism. Specifically, firstly, the sequence features output by the Transformer and the graph features output by the GNN are linearly transformed and mapped to the same semantic space. Then, the attention weight between each sequence feature and all graph features is calculated; this weight reflects the relevance of a word in the sequence to an entity or relation in the graph. Based on the calculated attention weights, the graph features are weighted and summed to obtain a context vector aligned with the sequence features. Finally, this context vector is concatenated or added to the original sequence features to obtain the final fused feature. This fusion mechanism enables the model to dynamically select the most relevant structural information for each word, thereby achieving deep fusion of sequence and structural features.

[0172] Integrating the sequence features of Transformer with the structural features of GNN

[0173] This invention designs a feature fusion method based on an attention mechanism, aiming to dynamically and selectively integrate two heterogeneous features from Transformer and GNN. The specific steps are as follows: First, the text sequence feature matrix output by the Transformer encoder (denoted as H_seq) and the graph node feature matrix output by the GNN encoder (denoted as H_graph) are mapped to the same dimension through a linear transformation. Then, the attention score between each word feature in H_seq and all node features in H_graph is calculated. This score is obtained through a learnable function, reflecting the semantic relevance between words and graph nodes. Next, these attention scores are normalized using the softmax function to obtain attention weights. Finally, these weights are used to perform a weighted summation of the node features in H_graph to generate a context structure vector corresponding to the current word. This vector is then concatenated with the original sequence features of that word to form the final fused feature vector. This mechanism enables the model to dynamically generate a representation that integrates global semantics and local structural information for each word in the text, greatly enriching the expressive power of the features.

[0174] Introducing knowledge graph embedding for knowledge enhancement

[0175] Building upon feature fusion, this invention further introduces embedding representations from a Legal Knowledge Graph (LKG) for knowledge enhancement. Before model training, graph embedding algorithms (such as TransE and RotatE) are pre-trained on a large-scale legal knowledge graph to obtain low-dimensional vector representations of all legal entities and relationships. During the feature fusion stage, when the model processes a case, it identifies the legal entities mentioned in the text and retrieves their vector representations from the pre-trained LKG embeddings. These knowledge graph embeddings are then used as additional, static features, fused with dynamic structural features learned through a Generative Neural Network (GNN) and sequential features learned through a Transformer. For example, the knowledge graph embeddings can be concatenated with node features from a GNN before being input into the attention fusion mechanism. In this way, the model not only learns specific information from the current case text but also incorporates general background knowledge from the entire legal knowledge system, thereby improving its reasoning accuracy and generalization ability.

[0176] Model training and optimization algorithms

[0177] Multi-task learning framework

[0178] To improve the model's generalization ability and the accuracy of case cause extraction, this invention employs a multi-task learning (MTL) training framework. The core idea of ​​MTL is to enable the model to learn multiple related tasks simultaneously, sharing underlying representations to facilitate the learning of more general and discriminative features. In this invention, case cause extraction and classification are taken as the main task, and several auxiliary tasks closely related to the main task are designed. These auxiliary tasks include:

[0179] 1) Identification of key legal elements: Identify key legal elements in the case text, such as parties, time, place, actions, and results.

[0180] 2) Legal relationship extraction: Identify the legal relationships between entities, such as contractual relationships, tort relationships, inheritance relationships, etc.

[0181] 3) Legal citation prediction: Predict the legal citations that may be involved in the case.

[0182] 4) Identification of points of contention: Identifying the core points of contention in a case. These auxiliary tasks share the encoder part of the TransGNN model with the main task, and each task has its own independent output layer and loss function.

[0183] During training, model parameters are updated by jointly optimizing the loss functions for all tasks. This training approach forces the model to learn how to identify key elements, understand legal relationships, and predict relevant legal provisions while learning case classification, thus achieving a more comprehensive and in-depth understanding of cases. Experiments show that the multi-task learning framework can significantly improve the performance of the main task, especially when dealing with complex and rare cases.

[0184] Main task: Case classification and extraction

[0185] The core objective of this invention is the accurate classification and extraction of case causes of action. Within the multi-task learning framework of the TransGNN model, this is the most important and central task. One of the model's final output layers is a softmax classifier, whose input is a fused feature vector, processed by the feature fusion module, that comprehensively represents the entire case. The output of this classifier is a probability distribution representing the likelihood that an input case belongs to each predefined cause of action category. During training, a cross-entropy loss function is used to measure the difference between the model's predicted cause of action probability distribution and the true label (one-hot encoding). By minimizing this loss, the model is optimized to maximize its accuracy in the cause of action classification task. This primary task drives the entire model's learning process, ensuring that the model ultimately and accurately performs the core function of cause of action extraction.

[0186] Auxiliary tasks: Identification of key legal elements and prediction of legal citations.

[0187] To enhance the model's understanding of cases, this invention designs several auxiliary tasks that are trained in conjunction with the main task. Among these, the key legal element identification task aims to train the model to identify specific information in the case text that is crucial to determining the cause of action, such as the amount involved, the tools used in the crime, the contract signing date, and the method of infringement. This task is typically modeled as a sequence labeling problem, where the model needs to predict whether each lexical element in the text belongs to a predefined element category. The legal citation prediction task requires the model to predict, based on the facts of the case, the relevant legal citations that the judge might cite in the judgment. This task helps the model establish a mapping relationship between the facts of the case and legal norms, thereby gaining a deeper understanding of legal logic. These auxiliary tasks share the underlying TransGNN encoder with the main task, and the additional supervision signals they provide guide the model to learn richer and more discriminative feature representations, thereby indirectly improving the performance of the main task (cause of action extraction).

[0188] Joint optimization of multi-task loss function

[0189] In a multi-task learning framework, the ultimate optimization objective is the weighted sum of the loss functions of all tasks. Let the loss of the main task (case classification) be L_main, and the losses of each auxiliary task be L_aux1, L_aux2, ..., then the total loss function L_total can be...

[0190] It is represented as:

[0191] L_total = L_main + Σ (λ_i * L_aux_i)

[0192] Here, λ_i represents the weight coefficient for each auxiliary task, used to balance the importance of different tasks in the total loss. These weight coefficients are hyperparameters that can be tuned based on performance on the validation set. During training, the model calculates the gradient of L_total with respect to all model parameters using backpropagation and updates the parameters using an optimizer (such as AdamW). Through joint optimization, the model is forced to learn a general representation that can simultaneously satisfy the needs of multiple tasks. This training paradigm not only improves the performance of the main task but also makes the model more robust and generalizable because it no longer relies solely on specific patterns of a single task but learns more universal legal textual patterns.

[0193] Self-supervised learning strategy

[0194] Considering the high cost and limited quantity of labeled data in the legal field, this invention introduces a self-supervised learning (SSL) strategy to fully utilize the massive amount of unlabeled legal documents. The core idea of ​​SSL is to design clever pre-training tasks that allow the model to learn useful knowledge from unlabeled data. Based on the characteristics of legal texts, the following pre-training tasks are designed:

[0195] 1) Masked Language Model (MLM): Randomly masking some words in text, the model predicts the masked words based on the context. This task can help the model learn the semantics and usage of legal vocabulary.

[0196] 2) Sentence Order Prediction (SOP): Sentences in a legal document are randomly shuffled, and the model is asked to determine whether two sentences are adjacent. This task helps the model understand the logical structure and textual relationships of legal documents.

[0197] 3) Legal Entity Recognition (LER): In the pre-training phase, existing, noisy entity recognition tools are used to automatically label unlabeled data, and then the model learns to predict these entities. This task helps the model initially establish an understanding of legal entities. Through these self-supervised learning tasks, the model can be pre-trained on large-scale unlabeled data, learning general representations of legal language. Then, the pre-trained model is fine-tuned using a small amount of high-quality labeled data to adapt to specific case extraction tasks. This "pre-training-fine-tuning" paradigm not only greatly reduces the dependence on labeled data but also significantly improves the final performance of the model.

[0198] Design a pre-training task for legal texts.

[0199] To better adapt the model to the specificities of the legal field, this invention designs a series of self-supervised pre-training tasks for legal texts. In addition to the general Masked Language Model (MLM), tasks more specific to the legal domain are designed. For example, the legal terminology masking and recovery task prioritizes masking legal technical terms (such as "force majeure" and "bona fide acquisition"), forcing the model to learn the precise meanings and usages of these terms. Another task is legal logical relationship judgment, which presents two sentences extracted from a case description and asks the model to determine the logical relationship between them, such as "causal relationship," "adversative relationship," or "progressive relationship." This helps the model understand the internal logic of the case's development. Furthermore, a pre-training task for predicting legal citations can be designed, i.e., predicting the number of legal citations that might be cited based on the case description. This helps the model establish an initial connection between facts and legal norms. These carefully designed pre-training tasks enable the model to learn rich legal domain knowledge during the pre-training phase.

[0200] Pre-training using a large number of unannotated legal documents

[0201] This invention fully utilizes a massive amount of unlabeled legal documents for self-supervised pre-training. These documents come from a wide range of sources, including publicly available judgments, laws and regulations, judicial interpretations, and contract templates, forming a vast and rich legal corpus. During the pre-training phase, the model performs the aforementioned self-supervised tasks on this data. For example, through the MLM task, the model can learn the contextual semantics of legal vocabulary; through the SOP task, the model can understand the textual structure of legal documents. This process enables the model to learn the general grammar, semantics, and logic of legal language, thereby obtaining good initial parameters. The advantage of this large-scale pre-training is that it gives the model powerful language understanding capabilities and domain adaptability. In the subsequent supervised fine-tuning phase, even with only a small amount of labeled data, the model can quickly converge and achieve excellent performance, effectively alleviating the problem of scarce labeled data in the legal domain.

[0202] Model fine-tuning and optimization

[0203] After completing self-supervised pre-training, the model needs to be fine-tuned using a labeled dataset to adapt it to the specific case cause extraction task. During the fine-tuning phase, a multi-task learning framework is employed to jointly optimize the loss functions of the main task (cause extraction) and multiple auxiliary tasks. To further improve model performance, the idea of ​​contrastive learning is introduced. Specifically, for a given case text, it is treated as a "positive sample," and then one or more "negative samples" are generated through data augmentation techniques (such as word replacement and sentence shuffling). During training, the model is required not only to correctly predict the cause of action but also to distinguish the representation of positive samples from that of negative samples, i.e., to narrow the gap between positive samples and samples of the same class, and to widen the gap between positive samples and samples of different classes. This contrastive learning strategy enhances the model's ability to distinguish subtle differences in case texts, thereby improving the accuracy of cause of action classification. Furthermore, some advanced optimization algorithms, such as the AdamW optimizer and the cosine annealing learning rate scheduler, are employed to accelerate model convergence and avoid overfitting. In terms of model evaluation, in addition to focusing on traditional indicators such as accuracy, recall, and F1 score, evaluation indicators that are closer to the needs of judicial practice have also been introduced, such as the Top-K accuracy of case prediction, in order to more comprehensively measure the performance of the model.

[0204] Fine-tuning the model using labeled datasets

[0205] Following self-supervised pre-training, the model enters the supervised fine-tuning phase. This phase utilizes a dataset meticulously annotated by legal experts, containing case texts and their corresponding correct cause-of-case labels, as well as possible labels for other auxiliary tasks (such as key elements, legal citations, etc.). The fine-tuning process is end-to-end, feeding the annotated data into the pre-trained TransGNN model and updating all model parameters using backpropagation and a gradient descent optimizer (such as AdamW). Because the model has already learned rich legal language knowledge during pre-training, it only needs a small number of iterations on task-specific data during fine-tuning to quickly adapt to and master the specific patterns of cause-of-case extraction. This "pre-training + fine-tuning" paradigm, compared to training from scratch, not only significantly shortens training time and reduces dependence on large-scale annotated data but also typically achieves higher final performance.

[0206] Employing contrastive learning to enhance the model's discriminative ability

[0207] To further enhance the model's ability to distinguish between similar cases, this invention introduces a contrastive learning mechanism during the fine-tuning stage. The core idea of ​​contrastive learning is to "bring similar cases closer together and push dissimilar ones further apart." In the case extraction task, a case text and its correct case label can be considered as a positive sample pair.

[0208] Simultaneously, for the same case text, an incorrect cause-of-fact label can be randomly sampled to form a negative sample pair. During training, the model aims to minimize the distance between positive sample pairs while maximizing the distance between negative sample pairs. In this way, the model is forced to learn subtle features that better distinguish different causes of action. For example, the model needs to learn to differentiate the key differences between "sales contract dispute" and "contract for work dispute," rather than simply memorizing some superficial keywords. This training strategy significantly enhances the model's discriminative ability, especially when handling cases with similar causes of action and complex details.

[0209] Data and model security protection technologies

[0210] Data privacy protection

[0211] In the judicial field, the security and privacy of case data are of paramount importance. This invention places data privacy protection at the core of its system design and integrates a series of advanced technologies. First, during the data collection and storage phase, all data involving sensitive personal information undergoes rigorous anonymization processing; for example, the parties' real names and ID numbers are replaced with anonymous identifiers. Simultaneously, all data is encrypted using high-strength encryption algorithms (such as AES-256) during storage and transmission. Second, during the model training phase, this invention introduces differential privacy technology.

[0212] Differential privacy mathematically guarantees privacy by adding precisely calculated noise to the training data or model gradients, making it impossible for attackers to deduce any individual's sensitive information from the final model parameters. Furthermore, this invention supports the Federated Learning framework, allowing models to be trained distributed across local data held by multiple data holders (such as courts in different regions) without centralizing the original dataset in one place, fundamentally avoiding the risk of data leakage during transmission and centralized storage.

[0213] Training data encryption storage and access control

[0214] This invention employs stringent protection measures for the security of training data. All legal document data used for model training is end-to-end encrypted during storage. This means that the data is encrypted before being written to the storage medium, and even if the storage device is physically compromised, attackers cannot read its contents. Industry-standard strong encryption algorithms, such as AES-256, are used to ensure data confidentiality. Furthermore, the system implements a strict Role-Based Access Control (RBAC) policy. Only authorized users, based on their roles and permissions, can access specific datasets. All data access operations are meticulously logged, forming an audit log for subsequent tracking and review. These measures collectively constitute a multi-layered data security protection system, effectively preventing data leakage and unauthorized access during static storage.

[0215] Applying differential privacy technology to protect training data

[0216] To protect the privacy of individual data during model training, this invention integrates Differential Privacy (DP) technology. Differential privacy is a robust privacy protection framework that ensures the presence or absence of any single data point has no significant impact on the final output by adding precisely calibrated random noise to the data or computation results. In the system of this invention, differential privacy can be applied to multiple stages of model training. For example, noise (gradient perturbation) can be added when calculating model gradients, or noise (parameter perturbation) can be added when updating model parameters. In this way, even if an attacker obtains the final model parameters, they cannot deduce any sensitive information about a specific training sample. Advanced DP implementations such as Rényi Differential Privacy (RDP) are employed to provide strong privacy protection while minimizing the impact on the final model performance.

[0217] Model safety protection

[0218] In addition to data security, this invention also emphasizes the protection of the model itself. The trained cause-of-death extraction model is core intellectual property. It needs to be prevented from being illegally stolen, tampered with, or misused. To this end, this invention employs a series of model security protection technologies. First, model parameters are encrypted during storage and transmission to ensure confidentiality. Second, to prevent reverse engineering of the model, model obfuscation technology is used, which increases the difficulty for attackers to understand and copy the model by changing its structure or parameter representation. Furthermore, this invention introduces digital watermarking technology, embedding an invisible, verifiable, and unique identifier into the model. This watermark can be used to prove ownership of the model and track its illegal dissemination. Finally, in scenarios involving multi-party collaborative training, this invention employs Secure Multi-Party Computation (SMPC) technology to ensure that all parties collaboratively complete model training without disclosing their private data and model parameters.

[0219] Model parameter encryption and anti-reverse engineering

[0220] This invention employs strict encryption and anti-reverse engineering measures for the trained model parameters. The model files are encrypted using high-strength encryption algorithms during storage and distribution, and can only be decrypted and loaded in an authorized runtime environment with the correct key. This effectively prevents the direct theft and copying of model files. To further enhance security, model obfuscation technology is also employed. This involves performing an equivalent transformation on the model's network structure, ensuring that its logic remains unchanged while preserving its functionality.

[0221] This makes the model more complex and harder to understand; or it encodes and compresses the model's weight parameters, increasing the difficulty of reverse engineering. These measures together constitute a solid protection for the intellectual property rights of the model, preventing the core algorithm from being stolen.

[0222] Application of secure multi-party computation in model training

[0223] In scenarios requiring joint model training using data from multiple institutions (e.g., courts in different regions wishing to jointly train a more robust cause-of-fact extraction model but unwilling to share their respective sensitive case data), this invention employs Secure Multi-Party Computation (SMPC) technology. SMPC is a cryptographic technique that allows multiple participants to collaboratively compute an agreed-upon function without revealing their private input data. In model training, this means that each party can compute model gradients locally and then aggregate these encrypted gradients via the SMPC protocol to update the global model. The entire process ensures that no party can see the original data or gradient information of other parties, thus achieving the privacy goal of "the model moves while the data remains stationary." This makes cross-institutional data collaboration possible while strictly adhering to data security and privacy regulations.

[0224] The system of this invention features a modular design, with data → graph construction → encoding → knowledge enhancement → classification, decoupling functions for easy maintenance and expansion; each module works collaboratively to form an end-to-end intelligent analysis pipeline, supporting high-throughput, low-latency batch processing; the system can be integrated into a judicial assistance platform for scenarios such as automatic triage, similar case recommendation, and judicial statistics.

[0225] Hybrid neural network encoding module: Enables dual-channel understanding of "sequence + structure" to overcome the limitations of a single model; Knowledge enhancement module: Connects external knowledge with internal representation to enhance the model's professionalism and interpretability; Multi-task output module: One model with multiple uses, in addition to the cause of action, it can also output elements, legal provisions, and points of contention, enhancing the system's practical value; The overall system meets the requirements for judicial applications in terms of accuracy, efficiency, and security.

[0226] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An analytical method for training and extracting case cause of action models, characterized in that, Includes the following steps: Step S1: Collect data, collect case documents, automatically capture case documents, clean, segment and segment them, and annotate key entities and relationships to obtain structured text; Step S2: Create a relationship diagram. Based on the structured text, construct a case relationship diagram containing entities and relationships. Step S3: Construct a hybrid neural network model, and use a hybrid model that combines Transformer and graph neural networks to jointly encode the structured text and case relationship graph to obtain a comprehensive representation that combines sequence features and structural features; Through the following methods: Step C1: Use the Transformer encoder to capture the sequence features of global semantics and contextual dependencies in the case documents, process the sequence features of the legal documents, and extract the deep semantic information. Use TransformerXL to process long legal documents. When processing the current text segment, the hidden state of the previous segment will be cached and reused, thereby passing the information to the subsequent segments. The self-attention mechanism of the Transformer encoder calculates the relevance weight between each word in the text and all other words, and generates a global contextual representation. Step C2: Encode the constructed case relationship graph using the graph attention network method, learn the low-dimensional vector representation of each node, and assign different weights according to the importance of different neighbors when aggregating neighbor node information to capture the complex relationships between nodes, thereby modeling the structured information contained in the legal text, extracting entities and relationships from the legal text, and through multi-layer GAT iteration, each node can aggregate the information of its higher-order neighbors to generate a node embedding representation that integrates local and global structural information; Step C3: Effectively combine the sequence features obtained by the Transformer encoder with the structural features obtained by the GNN encoder, including calculating the attention score between each word feature and all graph features, and then weighting and summing these to obtain a context vector aligned with the sequence features. The context vector is then concatenated or added together to form a fused feature vector. Effective combination of structural features involves linearly transforming the sequence features output by the Transformer and the graph features output by the GNN to map them to the same semantic space, calculating the attention weights between each sequence feature and all graph features, weighted summing of the graph features to obtain a context vector aligned with the sequence features, and concatenating or adding this context vector with the original sequence features to obtain the final fused features. Step S4: Knowledge enhancement, the comprehensive representation is enhanced by pre-trained case knowledge graph embedding vectors; Step S5: Output the results. Based on the enhanced comprehensive representation, perform cause-of-fact classification through a multi-task learning framework and output the cause of action for each case.

2. The analytical method for model training and extraction of case causes of action according to claim 1, characterized in that, The data collected in step S1 is performed in the following manner: Step A1: Collect various case documents in batches from public channels, and perform standardized cleaning on the original unstructured documents; Step A2: Using a combination of rules and machine learning models, the case documents are segmented according to their inherent structure. Initial segmentation is performed based on keyword matching, and sequence labeling models are used to improve the accuracy of segmentation. Step A3: Correctly segment professional terms using a word segmentation model trained on case corpus; Step A4: Key entity and relationship annotation. Define entity types and preset relationship types using sequence annotation scheme, construct tuples of entities and relationships between the two parties, and output structured text.

3. The analytical method for model training and extraction of case causes of action according to claim 2, characterized in that, The relationship diagram is established in step S2 using the following method: Step B1: Define the node and edge types of the graph based on entities and relations. Generate a graph node for each unique entity and construct directed edges using labeled relation tuples. Step B2: Introduce inference rules to supplement implicit relations, use rules to induce or program logic, or implicitly learn such relations through message passing of GNN after graph construction; Step B3: Organize and manage graph data using a deep learning framework, initialize feature vectors for each graph node, and construct feature representations for edges in the graph to build a typed directed heterogeneous graph.

4. The analytical method for model training and extraction of case causes of action according to claim 1, characterized in that, Step S4 is implemented in the following manner: Step D1: Load the knowledge graph embedding, identify all legal entities in the case file, query the pre-trained legal knowledge graph, and retrieve the embedding vector of the corresponding entity; Step D2: Integrate static knowledge by using the retrieved knowledge graph embedding vector as an additional feature, combining it with the dynamic structural features learned through GNN and the sequence features extracted by the Transformer encoder, and then inputting it into the attention mechanism for further fusion. Step D3: Based on the attention mechanism, calculate the attention weight between each word feature and all graph features, then sum the graph features according to these weights to generate a context vector, and combine this context vector with the original sequence features to form the final fused feature vector.

5. The analytical method for model training and extraction of case causes of action according to claim 1, characterized in that, In step S5, case classification is performed using a multi-task learning framework, including the main task of case classification and the auxiliary task of joint training: Case cause classification involves modeling the extraction of case causes as a multi-class classification problem. A predefined case cause system is established, and the structure of the case cause analysis model consists of a fully connected layer and a Softmax classifier connected after the enhanced comprehensive representation. The output is the number of predefined case cause categories, and finally, the probability distribution results for each case cause are given.

6. The analytical method for model training and extraction of case causes of action according to claim 5, characterized in that, The loss function used in the case analysis is cross-entropy loss: In the formula, The label indicates the true cause of action. Predict probabilities for the model.

7. The analytical method for model training and extraction of case causes of action according to claim 5, characterized in that, The joint training auxiliary tasks are key case element identification, case clause citation prediction, and dispute focus identification, with each auxiliary task outputting independently.

8. An analysis system for model training and extraction of case causes of action, employing the analysis method for model training and extraction of case causes of action as described in any one of claims 1-7, characterized in that, It includes a data acquisition and preprocessing module, a case relationship graph construction module, a hybrid neural network encoding module, a knowledge enhancement module, and a multi-task case classification and output module; The data acquisition and preprocessing module is used to automatically capture original case documents in batches from the public judicial document platform, perform standardized cleaning of unstructured text, intelligently segment the documents according to logical structure based on rules and sequence labeling models, accurately segment professional terms using a word segmentation model customized for the legal field, identify key legal entities and label the relationships between entities through sequence labeling, and output structured text. The case relationship graph construction module is used to construct an initial case relationship graph by taking entities in the structured text as graph nodes and relationships as directed edges. It introduces inference rules or uses implicit learning from GNNs to supplement implicit relationships that are not explicitly labeled but are semantically valid, constructs a typed directed heterogeneous graph, and initializes feature vectors for nodes and edges.

9. The analysis system for model training and extraction of case causes of action according to claim 8, characterized in that, The hybrid neural network encoding module is used to extract the global semantics and contextual dependencies of case documents using a Transformer encoder, aggregate neighbor information on the relationship graph using a graph attention network, learn the structural awareness representation of each entity, and align and fuse sequence features with graph features through a cross-modal attention mechanism to generate a comprehensive semantic representation. The knowledge enhancement module identifies legal entities appearing in documents and retrieves their embedding vectors from a pre-trained legal knowledge graph. It combines the knowledge graph embeddings with dynamically learned sequence and graph features, and then integrates multi-source information through an attention mechanism to generate a final context vector rich in external knowledge. The multi-task case classification and output module is used to model case identification as a multi-class classification problem. The main task is to output the probability distribution of predefined case categories through a fully connected layer and Softmax, while the auxiliary tasks are to identify key case elements, predict the legal provisions cited in the case, and identify the focus of the dispute.

Images