Self-rewarding training-based zero-shot legal document entity recognition method

Abstract

The present application relates to a kind of zero sample legal document entity identification method based on self-reward training, belong to natural language processing field, for the deficiency of prior art, the present application constructs a kind of " sampling generation-compound reward evaluation-online parameter update" closed loop training framework.First, the exploration trajectory containing intermediate analysis information and structured prediction result is generated using strategy model, and a double calibration reward mechanism is introduced, the output format specification is verified and the group consensus evaluation based on multiple generation results consistency is combined, to build high-quality feedback signal independent of artificial labeling, finally, the above feedback signal is used to update the model parameters iteratively online.The present application effectively suppresses the illusion generation and speculative consistency problem of model in zero sample scene through self-reward training paradigm, realizes the continuous self-evolution of model in legal document entity identification task, significantly improves the accuracy and robustness of entity identification.

Description

Technical Field

[0001] This invention relates to a zero-sample legal document entity recognition method based on self-reward training. It is applicable to the automatic identification and classification of legal entities in legal documents such as judgments, indictments, and interrogation records without any manually labeled data. It can be applied to scenarios such as judicial file structuring, legal element extraction, and intelligent case-handling assistance, and belongs to the field of natural language processing technology. Background Technology

[0002] Legal document entity recognition, a fundamental task in legal text processing, aims to identify entity fragments with specific legal semantics (such as party names, amounts involved, etc.) from unstructured legal documents. In traditional supervised learning paradigms, this task typically relies on large-scale, high-quality manually labeled data for model training. However, the legal field is characterized by a wide variety of document types, rapidly evolving terminology, and high privacy sensitivity. Obtaining sufficient labeled data presents both high costs and lengthy processing times, limiting the rapid deployment and generalization capabilities of existing technologies in complex and ever-changing judicial scenarios.

[0003] To address the challenge of data annotation, zero-shot entity recognition technology based on large language models has gradually become a research hotspot. Existing methods mainly employ an "inference enhancement" strategy, which improves performance by designing complex prompt templates, introducing external tools, or performing majority voting during the inference phase when the model parameters are frozen. Although these methods reduce the dependence on labeled data to some extent, they still have significant technical shortcomings in legal document processing scenarios: (1) Lack of parameter update mechanism. Existing zero-shot methods only search within a fixed parameter space and cannot internalize the rules and boundary features for identifying legal entities into model parameters. This non-training paradigm can only achieve instantaneous performance improvement and cannot accumulate experience through historical interactions, making it difficult for the model to achieve continuous capability evolution. (2) Risk of "speculation consistency". In the absence of external supervision, relying solely on the consistency of multiple outputs of the model (i.e., majority voting) as a judgment criterion has blind spots. When the model produces highly consistent "illusory" entity outputs (i.e., entities that do not exist in the original text), the majority voting mechanism will misjudge them as correct results. In rigorous legal scenarios, this "model-conviction error" can lead to a serious crisis of trust. (3) Insufficient constraints on structured output. Legal entity recognition requires that the output result can be accurately parsed and strictly correspond to the original text substrings. Existing methods often lack strong constraint mechanisms on the legality of the output format and the fidelity to the original text, which makes it easy for the model to produce format errors or generate entities that cannot be traced back to the original text during the generation process.

[0004] In summary, existing technologies struggle to simultaneously achieve continuous growth in model capabilities and precise, controllable entity recognition under unsupervised conditions. Therefore, there is an urgent need for a zero-shot legal document entity recognition method based on self-reward training that can utilize the model's own output to construct high-quality feedback signals and achieve self-evolution through parameter updates. Summary of the Invention

[0005] To overcome the shortcomings of existing zero-shot legal document entity recognition technologies, such as the inability to perform fixed parameter reasoning, lack of stability in recognition results, and difficulty in ensuring that the entity output is faithful to the original text, this invention provides a zero-shot legal document entity recognition method based on self-reward training. It constructs a closed-loop continuous self-evolution paradigm that revolves around three stages: "sampling generation - composite reward evaluation - online parameter update". Without the need for any manually labeled data, it drives the model to autonomously improve the recognition accuracy and robustness of key entities in legal documents.

[0006] Terminology Explanation:

[0007] Zero-sample legal document entity recognition: refers to a technology that identifies and classifies predefined legal entities (such as criminal suspects, amount of money involved, etc.) in any input legal document text without any manually labeled data, relying solely on the generation capabilities of unlabeled legal document corpora and large language models, through self-training and feedback mechanisms.

[0008] Self-reward training refers to a training paradigm that does not rely on external manual labels but instead uses the model's own output to construct feedback signals to drive parameter updates. In this invention, the closed-loop framework includes three stages: structured exploration trajectory generation, composite reward evaluation, and online policy optimization.

[0009] Exploration trajectory: refers to a complete output sequence generated by the model for the input text, including two parts: "intermediate analysis information" and "structured prediction results". The intermediate analysis information corresponds to the model's logical reasoning process for legal documents (such as case analysis and element decomposition), and the structured prediction results correspond to the final extracted entity list.

[0010] The present invention adopts the following technical solution:

[0011] A zero-shot legal document entity recognition method based on self-reward training includes the following steps:

[0012] (1) Trajectory generation stage: For a given unlabeled legal document input Entity recognition strategy model Instead of directly outputting entities, it generates multiple exploration trajectories containing thought chains through sampling;

[0013] (2) Composite reward signal construction stage: A composite reward function was designed to generate the final composite reward value for each exploration trajectory from two dimensions: format reward and self-consistency reward;

[0014] (3) Online strategy optimization stage: Based on the composite reward value in step (2), the strategy model parameters are updated to obtain a new strategy model, so that the strategy model can generate a higher quality trajectory in the next iteration.

[0015] Preferably, the specific implementation process of step (1) is as follows:

[0016] The action space of the entity recognition strategy model is defined as generating "inference-prediction" pairs, for the input legal documents. The strategy model is based on the current parameters. conduct Sub-independent sampling generates a set of exploration trajectories. ; Each trajectory Both consist of two parts:

[0017] (1)

[0018] in, This refers to the intermediate analysis information, namely the process by which the model uses natural language to analyze the syntactic structure, case logic, and entity attributes of legal documents; This represents structured prediction results generated based on analysis, formatted as a JSON object or a list of tuples.

[0019] Preferably, the format reward of step (2) is as follows: Prediction results The following verification process is performed sequentially:

[0020] A. Structured parsing verification:

[0021] A structured output pattern for entity recognition results is predefined. This pattern is a standardized data structure format, including but not limited to JSON objects or key-value pair sequences, and includes a predefined entity category field; for the prediction results... Perform the parsing operation and determine whether it meets the following conditions: it can be successfully parsed into a valid structured object using a regular expression or syntax parser; the parsing result contains a predefined entity category field;

[0022] When all of the above conditions are met, the determination is made. The structured parsing is validated; otherwise, the parsing is deemed to have failed.

[0023] B. Source verification:

[0024] Let's assume that from the prediction results The entity set obtained by parsing is Each entity It is represented as a combination of an entity type and an entity text value, that is: ,in Indicates the entity category, Represents the text content of an entity; for each entity Perform the following matching operation: When entering legal documents... Searching for strings Determine if a position interval exists. , making and The substrings in this interval are completely identical; if and only if all entities All can be entered into text If a matching substring is found, the prediction result is deemed to have passed the original text tracing verification; otherwise, the tracing is deemed to have failed.

[0025] When the prediction result If both structured parsing verification and original text tracing verification are satisfied, a positive format reward is given to the exploration trajectory; otherwise, the format reward for the exploration trajectory is set to zero.

[0026] Preferably, the self-consistent reward in step (2) is... Each sampling result Count the frequency of occurrence of each unique entity set, and select the set with the highest number of votes as the pseudo-true value of the current input, denoted as . ;

[0027] If the prediction result of a certain trajectory and If the trajectory is completely consistent, it is determined that the trajectory has a high degree of self-consistency and is given a positive reward.

[0028] Preferably, each trajectory is calculated based on two parts: format reward and self-consistency reward. Final compound reward value :

[0029] (2).

[0030] Preferably, in step (3), the group relative policy optimization algorithm GRPO is used to efficiently optimize the policy model without training an additional value network.

[0031] Preferably, the implementation process of step (3) is as follows:

[0032] In each training step, the current policy model A set of trajectories is generated by sampling the input legal document text fragments; then, the composite reward value of each trajectory is calculated;

[0033] The Within-Group Relative Policy Optimization (GRPO) algorithm estimates the baseline by assessing the relative performance of a set of trajectories and updates the model parameters by maximizing the following objective function. :

[0034] (3)

[0035] in, Indicates the parameters of the policy model The optimization objective; Indicates the input sample Obey the dataset The distribution and the set of exploration trajectories From the old strategy model Under the conditions of sampling, the expectation of the objective function is calculated; Indicates the same input sample The number of exploration trajectories generated, i.e., the number of samplings; Indicates the first A path of exploration, Indicates the length of the trajectory; Representing the trajectory The first in A number of generation positions are used for position-by-position optimization of the sequence-level output; This represents a truncation function used to restrict input values ​​to a range. Inside, among which This operation, set as a preset truncation threshold, limits the magnitude of policy updates to prevent excessive policy shifts during training, thereby improving training stability. Here, represents the corresponding regularization coefficient, used to control the weight of the KL divergence in the overall objective function, thus constraining the policy update from deviating too far from the original distribution. It is usually taken as 0.1; Indicates the location New strategies Compared to the old strategy The probability ratio is used to measure the degree of change of the current strategy relative to the old strategy; Indicates the first The advantage function corresponding to each trajectory; The KL divergence represents the distribution between the old and new strategies, and is used to measure the degree of difference between two probability distributions.

[0036] Preferred, The definition of is:

[0037] (4)

[0038] in Indicates the first Specific word elements generated at each position; Indicates the first The preceding sequence of preorder lexical terms generated at each position, i.e., the trajectory Historical context information; This represents the current policy model to be optimized, with the following parameters: , used for given input and historical generation sequence At that time, for the next word element Model the generation probability; This represents the old policy model before the parameter update, and its parameters are... Provides a baseline probability distribution under the same input conditions.

[0039] Preferred, The definition of is:

[0040] (5)

[0041] in, To prevent the minimum value of division by zero; This function represents the arithmetic mean of the elements in the input set. This function calculates the standard deviation of each element in the input set, and is used to measure the dispersion of the elements in the set.

[0042] Preferred, The calculation formula is:

[0043] (6)

[0044] in This represents the reference strategy model.

[0045] This invention mainly comprises three stages: In the exploration trajectory generation stage, a strategy model is used to perform multiple sampling reasoning on the input unlabeled legal documents to generate exploration trajectories containing intermediate analysis information and structured prediction results, thereby stimulating the model's deep understanding of legal semantic logic and explicitizing its decision-making process; In the composite reward signal construction stage, a dual-calibration reward mechanism is introduced to construct high-quality reward signals. This mechanism guides the output entities to accurately correspond to the original substrings in the legal documents through format rewards, and utilizes self-consistency rewards to mine highly self-consistent pseudo-true values ​​through group consensus, thereby effectively identifying entities and suppressing the illusion generated by the model; In the online strategy optimization stage, the relative advantage of each exploration trajectory is calculated based on the above composite reward signals, and the model parameters are updated online using an intra-group relative strategy optimization algorithm. The updated model is directly used for the next round of sampling tasks. Through the aforementioned closed-loop mechanism, this invention establishes a spiral upward path in which "model reasoning ability" and "reward signal quality" mutually promote each other, effectively solving the problem that traditional zero-sample methods cannot internalize entity recognition rules. It achieves continuous self-evolution of legal document entity recognition ability under completely unsupervised conditions, and is particularly suitable for various judicial application scenarios such as judicial case file structuring, automated extraction of legal elements, and intelligent case-handling assistance.

[0046] For any details not covered in this invention, please refer to the prior art.

[0047] The beneficial effects of this invention are as follows:

[0048] 1. This invention proposes a zero-shot legal document entity recognition method based on self-reward training, requiring no manually labeled data, significantly reducing the construction cost and data threshold of legal intelligence systems. Traditional legal entity recognition methods typically rely on legal experts spending considerable time constructing large-scale, high-quality labeled datasets, and often require re-labeling and retraining for different case types (e.g., civil, criminal) or document types. This invention completely eliminates the dependence on external manual labels, utilizing only unlabeled legal document corpora, and achieving optimization through the model's own exploration and feedback mechanisms. This allows the invention to quickly adapt to newly promulgated laws and regulations or emerging judicial fields, greatly improving the deployment efficiency and generalization ability of legal artificial intelligence systems in low-resource scenarios.

[0049] 2. This invention overcomes the limitations of existing zero-shot methods that rely solely on "inference enhancement" and have fixed parameters, enabling the model to internalize and continuously evolve its knowledge of legal entity recognition. Existing zero-shot techniques (such as cue engineering and contextual example retrieval) typically operate with model parameters frozen, failing to solidify experience in processing legal documents into model capabilities, resulting in performance improvements that are often transient and unstable. This invention employs an online closed-loop training process of "sampling-evaluation-update," utilizing the within-group relative policy optimization algorithm (GRPO) to update model parameters online. This mechanism allows the model to learn from historical inference trajectories, gradually internalizing the boundary determination rules and semantic classification logic of legal entities into the model's inherent parameters, thereby achieving continuous accumulation and self-evolution of model capabilities.

[0050] 3. This invention effectively solves the technical problem of "illusion" and "speculative consistency" that large language models are prone to generate in legal scenarios by introducing a dual calibration reward mechanism that combines format standardization and majority voting. In rigorous judicial applications, entity recognition results must be strictly faithful to the original text. Existing technologies that rely solely on majority voting are prone to leading to high-confidence misjudgments (speculative consistency) of erroneous "illusion" entities. The composite reward function designed in this invention mandates that the output entity must be able to accurately trace back and match the original substring in the legal document. Combined with consistency-based pseudo-truth value screening, this constitutes a dual defense of "formal legality" and "semantic consensus." This design significantly suppresses the risk of the model generating non-existent legal entities and greatly enhances the credibility and accuracy of entity recognition results in high-risk scenarios such as judicial trials and case file reviews.

[0051] 4. This invention constructs a spiral-like improvement mechanism where "model reasoning ability" and "reward signal quality" mutually reinforce each other, achieving continuous self-enhancement of performance under unsupervised conditions. Within the closed-loop framework of this invention, as training iterates, the model, after parameter updates, possesses stronger legal semantic analysis capabilities, thereby generating higher-quality and more consistent exploration trajectories; conversely, higher-quality majority voting results provide the model with more accurate reward signals. This positive feedback loop ensures that the model can steadily improve its understanding depth and entity recognition accuracy of complex legal documents under completely unsupervised conditions. Attached Figure Description

[0052] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0053] Figure 1 This is a schematic diagram of the zero-shot legal document entity recognition method based on self-reward training according to the present invention. Detailed Implementation

[0054] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. However, this is not the only description; all aspects not described in detail herein are based on conventional techniques in the art.

[0055] Example 1

[0056] A zero-shot legal document entity recognition method based on self-reward training, such as Figure 1 It includes the following steps:

[0057] (1) Trajectory generation stage:

[0058] The goal of this phase is to transform the legal document entity recognition task into a generative task that includes explicit reasoning. Given an unlabeled legal document input... (For example, a factual description in a judgment), entity recognition strategy model Instead of directly outputting entities, it generates multiple exploration trajectories containing thought chains through sampling to stimulate the model's deeper understanding of legal semantics;

[0059] The specific implementation process is as follows:

[0060] The action space of the entity recognition strategy model is defined as generating "inference-prediction" pairs, for the input legal documents. The strategy model is based on the current parameters. conduct Sub-independent sampling generates a set of exploration trajectories. ; Each trajectory Both consist of two parts:

[0061] (1)

[0062] in, This represents intermediate analysis information, that is, the process by which the model uses natural language to analyze the syntactic structure, case logic, and entity attributes of legal documents (e.g., "Analysis of this sentence reveals that 'Zhang San' committed theft, therefore he should be identified as a suspect"). This represents structured prediction results generated based on analysis, formatted as a JSON object or a list of tuples.

[0063] This step forces the model to explicitly generate the discrimination logic before outputting the final legal entity, so that subsequent feedback signals can not only evaluate the correctness of the identification results, but also guide the optimization of the reasoning logic.

[0064] (2) Composite reward signal construction stage:

[0065] This phase aims to construct high-quality reward signals without manual annotation.

[0066] A composite reward function was designed to generate the final composite reward value for each exploration trajectory from two dimensions: format reward and self-consistency reward.

[0067] Formatting Rewards: Legal document processing demands extremely high accuracy of information. Formatting rewards aim to constrain the model to generate entity recognition results that are legal and faithful to the original text.

[0068] For any trajectory Prediction results The following verification process is performed sequentially:

[0069] A. Structured parsing verification:

[0070] A structured output pattern for entity recognition results is predefined. This pattern is a standardized data structure format, including but not limited to JSON objects or key-value pair sequences, and includes a predefined entity category field; for the prediction results... Perform the parsing operation and determine whether it meets the following conditions: it can be successfully parsed into a valid structured object using a regular expression or syntax parser; the parsing result contains a predefined entity category field;

[0071] When all of the above conditions are met, the determination is made. The structured parsing is validated; otherwise, the parsing is deemed to have failed.

[0072] B. Source verification:

[0073] Let's assume that from the prediction results The entity set obtained by parsing is Each entity It is represented as a combination of an entity type and an entity text value, that is: ,in Indicates the entity category, Represents the text content of an entity; for each entity Perform the following matching operation: When entering legal documents... Searching for strings Determine if a position interval exists. , making and The substrings in this interval are completely identical; if and only if all entities All can be entered into text If a matching substring is found, the prediction result is deemed to have passed the original text tracing verification; otherwise, the tracing is deemed to have failed.

[0074] When the prediction result If both structured parsing verification and original text tracing verification are satisfied, a positive format reward is given to the exploration trajectory; otherwise, the format reward for the exploration trajectory is set to zero.

[0075] Self-consistent reward: Based on the assumption that "majority voting reflects group consensus", this invention uses the highly self-consistent results of multiple samplings as pseudo-true values.

[0076] for Each sampling result Count the frequency of occurrence of each unique entity set, and select the set with the highest number of votes as the pseudo-true value of the current input, denoted as . ;

[0077] If the prediction result of a certain trajectory and If the trajectory is completely consistent, it is determined that the trajectory has a high degree of self-consistency and is given a positive reward.

[0078] Each trajectory is calculated based on two parts: format reward and self-consistency reward. Final compound reward value :

[0079] (2).

[0080] Under this setting, high-quality predictions that are consistent with the false true value will receive the highest reward; if the prediction result is in the correct format but deviates from the group consensus, a small positive reward (such as 0.1) will be given to maintain the basic ability of the model to output structured data; while outputs that are in the wrong format or contain illusions will receive zero reward.

[0081] (3) Online strategy optimization stage: Based on the composite reward value in step (2), the strategy model parameters are updated to obtain a new strategy model, so that the strategy model can generate a higher quality trajectory in the next iteration.

[0082] The Within-Group Relative Policy Optimization (GRPO) algorithm is used to efficiently optimize the policy model without training an additional value network.

[0083] The implementation process is as follows:

[0084] In each training step, the current policy model A set of trajectories is generated by sampling the input legal document text fragments; then, the composite reward value of each trajectory is calculated;

[0085] The Within-Group Relative Policy Optimization (GRPO) algorithm estimates the baseline by assessing the relative performance of a set of trajectories and updates the model parameters by maximizing the following objective function. :

[0086] (3)

[0087] in, Indicates the parameters of the policy model The optimization objective; Indicates the input sample Obey the dataset The distribution and the set of exploration trajectories From the old strategy model Under the conditions of sampling, the expectation of the objective function is calculated; Indicates the same input sample The number of exploration trajectories generated, i.e., the number of samplings; Indicates the first A path of exploration, Indicates the length of the trajectory; Representing the trajectory The first in A number of generation positions are used for position-by-position optimization of the sequence-level output; This represents a truncation function used to restrict input values ​​to a range. Inside, among which This operation, set as a preset truncation threshold, limits the magnitude of policy updates to prevent excessive policy shifts during training, thereby improving training stability. Here, represents the corresponding regularization coefficient, used to control the weight of the KL divergence in the overall objective function, thus constraining the policy update from deviating too far from the original distribution. It is usually taken as 0.1; Indicates the location New strategies Compared to the old strategy The probability ratio is used to measure the degree of change of the current strategy relative to the old strategy; Indicates the first The advantage function corresponding to each trajectory; The KL divergence represents the distribution between the old and new strategies, and is used to measure the degree of difference between two probability distributions.

[0088] The definition of is:

[0089] (4)

[0090] in Indicates the first Specific word elements generated at each position; Indicates the first The preceding sequence of preorder lexical terms generated at each position, i.e., the trajectory Historical context information; This represents the current policy model to be optimized, with the following parameters: , used for given input and historical generation sequence At that time, for the next word element Model the generation probability; This represents the old policy model before the parameter update, and its parameters are... Provides a baseline probability distribution under the same input conditions.

[0091] By calculating the ratio of the two probabilities mentioned above, we can measure the adjustment magnitude of the current policy relative to the old policy at this position. This allows for effective control over the direction and magnitude of policy updates when combining the truncation function and the dominance function for constraint optimization, thereby improving the stability and convergence of the training process.

[0092] The definition of is:

[0093] (5)

[0094] in, To prevent the minimum value of division by zero; This function represents the arithmetic mean of the elements in the input set. This function calculates the standard deviation of each element in the input set, and is used to measure the dispersion of the elements in the set.

[0095] Through this comparative advantage in computation, the algorithm can automatically adapt the reward scale to legal document samples of varying difficulty.

[0096] In addition, the objective function includes a pruning term. Update the hyperparameters by controlling the policy update magnitude. Determine the clipping range; simultaneously introduce a KL divergence penalty term. Its weight is determined by The adjustment is set to 0.1 to prevent the updated model from deviating excessively from the reference strategy. (Refer to the strategy model) It is initialized from the first policy model at the start of training and remains unchanged during training, such as Figure 1 (Referencing the model in the model) to ensure the stability of the training process. The formula for calculating the KL divergence is:

[0097] (6)

[0098] in This represents the reference strategy model.

[0099] Steps (1) to (3) above constitute a complete online training closed loop. After completing one parameter update, the updated model... This will be directly used as the sampling strategy for the next training step.

[0100] Through this dynamic mechanism, as training progresses: the model's legal semantic analysis and entity recognition capabilities are enhanced; this enhanced model capability leads to an improvement in the quality of majority voting, thus eliminating false ground truth values. More accurate; higher quality pseudo-true values ​​provide more accurate reward signals, further driving model optimization; thus achieving a spiral increase in "reasoning ability" and "reward signal quality" in legal document entity recognition tasks.

[0101] Example 2

[0102] A zero-shot legal document entity recognition method based on self-reward training is proposed to automatically identify predefined entity categories such as "criminal suspect," "time of crime," "location of crime," "stolen property," and "tools of the crime" from unannotated theft-related court documents. This embodiment constructs a self-reward training closed loop, enabling the model to learn autonomously without human intervention. The operation steps are as follows:

[0103] (1) Trajectory generation: Exploring the unlabeled theft judgment corpus as input. Utilizing the current strategy model (e.g., a model initialized based on Qwen3-8B) Execute independently for each text. Secondary sampling. Based on the input text. "On March 15, 2024, at approximately 10:00 PM, the defendant Zhang Moumou, in the underground parking garage of a building in Zhongguancun, Haidian District, Beijing, used a crowbar to break a car window and stole an Apple iPhone 15 and RMB 2,300 in cash from the car of the victim Li Moumou." This example illustrates the exploration trajectory generated by the model. Includes intermediate analysis information Compared with the prediction results .

[0104] For example, the trajectory of a certain sampling The content is: intermediate analysis information “Analyzing the sentence semantics: ‘Zhang Moumou’ is referred to as the defendant and should be identified as a criminal suspect; ‘around 10 PM on March 15, 2024’ is the time the act occurred; ‘crowbar’ was used to break the car window and is a tool used in the crime; ‘Apple iPhone 15’ and ‘RMB 2,300 in cash’ are the objects of theft and are stolen property.” Structured Prediction : [(“Zhang Moumou”, “criminal suspect”), (“around 22:00 on March 15, 2024”, “time of the crime”), (“crowbar”, “tool of the crime”), (“Apple iPhone 15”, “stolen property”), (“RMB 2,300”, “stolen property”)].

[0105] (2) Construction of composite reward signals: The system... The trajectory is evaluated and the reward is calculated. :

[0106] Format reward: Check Does the entity in the sentence actually exist in the original sentence? Assume a certain trajectory... If a "crowbar" is incorrectly predicted as an "iron bar" (not mentioned in the text), the entity cannot be traced, leading to... The format reward is reset to zero. ).

[0107] Self-consistency reward: Analysis of 16 sampling results revealed that "crowbar" appeared as the "tool of the crime" 4 times, and "iPhone 15" appeared 5 times. These high-frequency results were aggregated into pseudo-true values. .

[0108] Dual calibration reward mechanism: for trajectory Its prediction results and Completely identical, and all entities match the original text precisely, therefore receiving the highest reward. For tracks that, while formatted correctly, failed to identify "cash,"... Because of Inconsistency will result in a lower score reward. .

[0109] (3) Online strategy optimization: Based on the reward value calculated above, the advantage value of each trajectory is calculated using the group relative strategy optimization algorithm GRPO. For example, high-quality trajectories With a positive dominance value, the model will increase the probability of generating this type of "analysis logic + precise entity"; while trajectories containing illusions... A negative dominance value suppresses its generation probability. This is achieved by maximizing the objective function. The model parameters are updated in one step to obtain a new model. .

[0110] (4) Continuous self-evolution and testing: updated model The data was used in the sampling phase of the next training step, thus enabling continuous self-evolution. The last updated model was used to handle more complex test text: "Wang Moumou sneaked into a residential area in Chaoyang District at night, used a screwdriver to pry open the battery cover of an electric vehicle, and stole a set of Tianneng brand 48V20Ah lithium batteries." Thanks to the reinforcement of "reasoning logic" and "original text matching" during training, the model was able to output a clear analytical chain: "'Screwdriver' is used for the 'prying' action and is determined to be the tool used in the crime," and accurately extracted the long entity "a set of Tianneng brand 48V20Ah lithium batteries," alleviating the problems of inaccurate entity boundaries or incorrect entity category recognition.

[0111] Example 3

[0112] A zero-shot legal document entity recognition method based on self-reward training is proposed for identifying entities in judgments concerning the crime of illegal possession of drugs. This method is used to identify core entities such as "possessor," "drug name," "drug quantity," and "location of possession" in unmarked judgments. The specific operation steps of this embodiment are as follows:

[0113] (1) Trajectory Generation: A corpus of judgments on illegal possession of drugs is used for batch sampling, employing a strategy model (such as a model initialized based on Qwen3-8B). Input text "Police found a package of white crystalline substance, weighing 12.36 grams, in a hidden compartment of the black shoulder bag carried by the defendant, Zhao Moumou; testing revealed it to contain methamphetamine." The model generates multiple trajectories. For example, the trajectory... Intermediate analysis indicated that "'white crystalline substance' is a suspected drug, but the legally recognized drug name should be based on the identification results, namely 'methamphetamine'." (Predicted results) Includes ("methamphetamine", the name of the drug); and the trajectory The intermediate analysis produced a bias: "'Methamphetamine' is commonly known as crystal meth, therefore it was identified as crystal meth." Prediction result Includes ("methamphetamine", "drug name").

[0114] (2) Construction of composite reward signals: The system... The trajectory is evaluated and the reward is calculated. :

[0115] Formatted reward: Automatically detected by the system, tracking... The word "methamphetamine" was not in the original sentence. The phrase appears in the text (the original sentence only contains "methamphetamine"). Although semantically similar, it violates the "fidelity to the original text" principle for legal entity identification. Therefore, the trajectory is determined. Formatting errors will result in a punitive reward. .

[0116] Self-consistent reward: In In this second sampling, four tracks accurately identified the "black shoulder bag lining" as the "concealment location" and "12.36 grams" as the "quantity of drugs." These high-frequency, consistent results constitute false true values. .

[0117] Dual calibration reward mechanism: trajectory Those that meet both the constraints of the original text (extracting "methamphetamine") and align with the consensus of the group will receive a reward. .

[0118] (3) Online policy optimization: The GRPO algorithm is used for online policy optimization. During this process, the model parameters are updated to penalize the image trajectory. This behavior of "rewriting entities based on external knowledge" also rewards things like trajectories. This behavior of "reasoning based on expert opinions and accurately extracting them" internalizes the legal constraint of "strictly adhering to the original text" into the model parameters.

[0119] (4) Continuous self-evolution and testing: After multiple rounds of "sampling-evaluation-updating", the model demonstrates a high degree of legal rigor when handling new cases. For example, the input: "Investigators found two bags of powder in a hidden compartment of the bedside table in the bedroom of suspect Chen, which was confirmed to contain heroin." The model can automatically ignore the physical description of "powder" through its internalized reasoning ability, and accurately locate "heroin" as the legal name of a drug, and accurately identify the "hidden compartment of the bedside table" as the hiding place, achieving zero-sample entity recognition under the condition of no labeled data.

[0120] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A zero-shot legal document entity recognition method based on self-reward training, characterized in that, Includes the following steps: (1) Trajectory generation stage: For a given unlabeled legal document input Entity recognition strategy model Multiple exploration trajectories containing thought chains are generated through sampling; (2) Composite reward signal construction stage: A composite reward function was designed to generate the final composite reward value for each exploration trajectory from two dimensions: format reward and self-consistency reward; (3) Online strategy optimization stage: Based on the composite reward value in step (2), the strategy model parameters are updated to obtain a new strategy model, so that the strategy model can generate a higher quality trajectory in the next iteration.

2. The zero-shot legal document entity recognition method based on self-reward training according to claim 1, characterized in that, The specific implementation process of step (1) is as follows: The action space of the entity recognition strategy model is defined as generating "inference-prediction" pairs, for the input legal documents. The strategy model is based on the current parameters. conduct Sub-independent sampling generates a set of exploration trajectories. ; Each trajectory Both consist of two parts composition: (1) in, This refers to the intermediate analysis information, namely the process by which the model uses natural language to analyze the syntactic structure, case logic, and entity attributes of legal documents; This represents structured prediction results generated based on analysis, formatted as a JSON object or a list of tuples.

3. The zero-shot legal document entity recognition method based on self-reward training according to claim 2, characterized in that, The format reward of step (2) is given for any trajectory. Prediction results The following verification process is performed sequentially: A. Structured parsing verification: A structured output pattern for entity recognition results is predefined, including but not limited to JSON objects or key-value pair sequences, and includes a predefined entity category field; for the prediction results... Perform the parsing operation and determine whether it meets the following conditions: it can be successfully parsed into a valid structured object using a regular expression or syntax parser; the parsing result contains a predefined entity category field; When all of the above conditions are met, the determination is made. The structured parsing is validated; otherwise, the parsing is deemed to have failed. B. Source verification: Let's assume that from the prediction results The entity set obtained by parsing is Each entity It is represented as a combination of an entity type and an entity text value, that is: ,in Indicates the entity category, Represents the text content of an entity; for each entity Perform the following matching operation: When entering legal documents... Searching for strings ; Determine if a position interval exists. , making and The substrings in this interval are completely identical; if and only if all entities All can be entered into text When a matching substring is found, the prediction result is deemed to have passed the original text source verification. Otherwise, the tracing process is deemed to have failed. When the prediction result If both structured parsing verification and original text tracing verification are satisfied, a positive format reward is given to the exploration trajectory; otherwise, the format reward for the exploration trajectory is set to zero.

4. The zero-shot legal document entity recognition method based on self-reward training according to claim 3, characterized in that, The self-consistent reward in step (2) is for Each sampling result Count the frequency of occurrence of each unique entity set, and select the set with the highest number of votes as the pseudo-true value of the current input, denoted as . ; If the prediction result of a certain trajectory and If the trajectory is completely consistent, it is determined that the trajectory has a high degree of self-consistency and is given a positive reward.

5. The zero-shot legal document entity recognition method based on self-reward training according to claim 4, characterized in that, Each trajectory is calculated based on two parts: format reward and self-consistency reward. Final compound reward value : （2）。 6. The zero-shot legal document entity recognition method based on self-reward training according to claim 5, characterized in that, In step (3), the group relative policy optimization algorithm GRPO is used to efficiently optimize the policy model without training an additional value network.

7. The zero-shot legal document entity recognition method based on self-reward training according to claim 6, characterized in that, The implementation process of step (3) is as follows: In each training step, the current policy model A set of trajectories is generated by sampling the input legal document text fragments; Then, the composite reward value for each trajectory is calculated; The Within-Group Relative Policy Optimization (GRPO) algorithm estimates the baseline by assessing the relative performance of a set of trajectories and updates the model parameters by maximizing the following objective function. : （3） in, Indicates the parameters of the policy model The optimization objective; Indicates the input sample Obey the dataset The distribution and the set of exploration trajectories From the old strategy model Under the conditions of sampling, the expectation of the objective function is calculated; Indicates the same input sample The number of exploration trajectories generated, i.e., the number of samplings; Indicates the first A path of exploration, Indicates the length of the trajectory; Representing the trajectory The first in One generation location; This represents a truncation function used to restrict input values ​​to a range. Inside, among which This is the preset cutoff threshold; The corresponding regularization coefficient is used to control the weight of the KL divergence in the overall objective function, so as to constrain the policy update from deviating too far from the original distribution; Indicates the location New strategies Compared to the old strategy The probability ratio is used to measure the degree of change of the current strategy relative to the old strategy; Indicates the first The advantage function corresponding to each trajectory; The KL divergence represents the distribution between the old and new strategies, and is used to measure the degree of difference between two probability distributions.

8. The zero-shot legal document entity recognition method based on self-reward training according to claim 7, characterized in that, The definition of is: （4） in Indicates the first Specific word elements generated at each position; Indicates the first The preceding sequence of preorder lexical terms generated at each position, i.e., the trajectory Historical context information; This represents the current policy model to be optimized, with the following parameters: , used for given input and historical generation sequence At that time, for the next word element Model the generation probability; This represents the old policy model before the parameter update, and its parameters are... Provides a baseline probability distribution under the same input conditions.

9. The zero-shot legal document entity recognition method based on self-reward training according to claim 8, characterized in that, The definition of is: （5） in, To prevent the minimum value of division by zero; This function represents the arithmetic mean of the elements in the input set. This function calculates the standard deviation of each element in the input set, and is used to measure the dispersion of the elements in the set.

10. The zero-shot legal document entity recognition method based on self-reward training according to claim 9, characterized in that, The calculation formula is: （6） in This represents the reference strategy model.

Images