Case quality evaluation method, computer device, and storage medium

Abstract

The present disclosure relates to a case quality evaluation method, a computer device and a storage medium. It relates to digital data processing and solves the problem of lacking an accurate and efficient automatic case evaluation mechanism. The method comprises: analyzing case data and legal data sets of a case, generating sub-tasks corresponding to each evaluation dimension according to at least one preset evaluation dimension; executing the sub-tasks through multiple agents, and assigning each agent a sub-task; and generating and outputting a unified structured risk consensus according to the reasoning results output by the multiple agents. The technical solution provided by the present disclosure is applicable to the judicial supervision system and realizes low-cost, high-efficiency and high-quality standardized quality evaluation.

Description

Technical Field

[0001] This disclosure relates to digital data processing, and more particularly to a case quality assessment method, computer device, and storage medium applicable to a judicial oversight system. Background Technology

[0002] Case quality review is a crucial step in ensuring judicial fairness and standardizing judicial conduct. It involves a comprehensive review of the entire process, focusing on compliance with case filing regulations, the validity of evidence, procedural legality, and appropriate sentencing. Currently, the review is primarily conducted manually, requiring relevant personnel to review each case file, extract facts, verify legal provisions, and identify potential risks.

[0003] Manual review is inefficient, prone to errors and omissions. Furthermore, cases involve a wide range of real-life situations, require diverse professional knowledge, and involve complex applicable laws and regulations. Manual judgment is susceptible to subjective influence, and different personnel often have different evaluation standards, leading to inconsistent review results that lack objectivity and may even contradict the facts of the case.

[0004] In an effort to overcome the shortcomings of manual review, attempts were made to introduce deep learning models into the review process to partially replace manual review. However, due to the complexity of the case data and the diversity of review dimensions, the deep learning models failed to effectively address the needs of case review.

[0005] In summary, there is a lack of an accurate and efficient automated case review mechanism. Summary of the Invention

[0006] To overcome the problems existing in related technologies, this disclosure provides a case quality assessment method, computer device and storage medium, which is based on multiple intelligent agents to complete the assessment task collaboratively, and conducts assessment from different assessment dimensions to discover risks. It solves the problem of lacking an accurate and efficient automatic case assessment mechanism, and realizes a standardized quality assessment with low cost, high efficiency and high quality.

[0007] According to a first aspect of the present disclosure, a method for case quality assessment is provided, comprising: Analyze the case data and legal dataset of the case, and generate sub-tasks corresponding to each of the pre-defined evaluation dimensions based on at least one pre-defined evaluation dimension; The subtask is executed by multiple agents, and one subtask is assigned to each agent. Based on the reasoning results output by multiple intelligent agents, a unified structured risk consensus is generated and output.

[0008] Furthermore, the evaluation dimensions include at least one or more of the following: Case filing, evidence, procedures, sentencing, The intelligent agent includes at least one or more of the following: Intelligent agents for case filing review, evidence review, process review, and sentencing review. The step of generating and outputting a unified structured risk consensus based on the inference results output by multiple intelligent agents includes: Obtain the reasoning results of each of the aforementioned agents, wherein the reasoning results include at least one or more of the following: The first inference result output by the intelligent agent for case filing review in identifying compliance risks during case filing is... The evidence review agent performs evidence chain integrity and consistency risk identification, outputting the second reasoning result. The third inference result output by the intelligent agent in evaluating the legality risk identification of litigation procedures during process review. The fourth inference result output by the sentencing review agent in identifying the appropriateness risk of implementing sentencing recommendations. The first inference result, the second inference result, the third inference result, and the fourth inference result have a unified output format, which includes at least one or more of the following information: Risk type, description, elements involved, relevant legal provisions, confidence level, and source of evidence; For the reasoning results based on the same case output by different intelligent agents, retrieve the factual findings to determine contradictions; In the event of a contradiction in factual findings, a weighted vote is performed on the inference results based on the authority weight coefficients of each agent to obtain a unified structured risk consensus. The authority weight coefficients indicate the historical accuracy of the agent's assessments. The structured risk consensus includes at least one or more of the following information: Case unique identifier, risk classification, risk type, description, elements involved, related evidence or legal provisions, and confidence level.

[0009] Furthermore, after the step of obtaining the inference results of each of the intelligent agents, the method further includes: For the inference results output by each of the aforementioned agents, calculate the prediction entropy; If the predicted entropy is higher than a preset risk threshold, risk processing is initiated.

[0010] Furthermore, before the step of analyzing the case data and legal dataset and generating sub-tasks corresponding to each of the preset evaluation dimensions, the method further includes: Obtain the original case file data, which includes at least one or more of the following data: Judgment, court transcript, and evidence materials The original case file data contains at least one or more of the following data formats: Text, images, and videos; The case data is generated in structured text form based on the original case file data; The legal dataset is obtained by searching a pre-set legal knowledge base based on the case data. Based on the basic information of the case, the case data in structured text form is associated with the legal dataset in structured data table form and output. The basic information includes at least the unique identifier of the case.

[0011] Furthermore, the step of generating the case data in structured text form based on the original case file data includes: By using overlapping sliding windows, the original case file data is split into paginated case file documents, which contain multiple split text blocks. Information extraction is performed on the case file documents to obtain valid information. The information extraction includes at least one or more of the following processing methods: Handwriting recognition, signature recognition, and text extraction; The extracted valid information is aggregated to generate the case data in structured text form.

[0012] Furthermore, handwriting recognition is performed in the following way: The paginated case file documents are preprocessed using normalization to obtain standardized case file images; The standardized file images are divided into image blocks of fixed size; The image patch is converted into a feature vector according to the following expression. : , in, These are placeholders for categories used to aggregate global features. Let j be the feature vector of the j-th image patch, where j takes values ​​from 1 to N. For position encoding; The feature vector is extracted by a decoder to obtain a long sequence of handwritten semantics; The long program sequence is mapped to a character-level probability distribution, and the character with the highest probability distribution is selected as the recognition result for each character position to generate a handwritten text sequence. The handwritten text sequence is corrected for errors according to the following expression, and the corrected handwritten text sequence O is output: , in, For regularization items in a legal terminology database, To adjust the parameters, L is the target sequence length, and I is the normalized dossier image. The parameters of the model to be trained, For the actual character at position i, - (·) is a negative logarithmic function.

[0013] Furthermore, signature recognition is performed in the following manner: Based on the feature vector, candidate regions are determined; The candidate region is extracted and mapped to a high-dimensional feature space by a signature feature encoder based on a Siamese network to generate a signature feature vector. The signature feature vector is compared with a preset standard signature sample to obtain the signature feature vector that meets the similarity requirements. The signature recognition result is generated and output based on the signature feature vector and location information that meet the similarity requirements.

[0014] Furthermore, the step of retrieving the legal dataset from a preset legal knowledge base based on the case data includes: Both the case data and the legal provisions in the legal knowledge base are converted into high-dimensional dense vectors. The similarity between the high-dimensional dense vector of the case data and the high-dimensional dense vector of the legal provisions is calculated, and the legal dataset is generated based on the legal provisions whose similarity reaches the preset legal provision matching conditions.

[0015] Furthermore, the method also includes: By constructing a fuzzy evaluation matrix and combining it with a weight vector, the comprehensive risk score vector S for each risk point in the structured risk consensus is calculated according to the following expression: , in, Let R be a weight vector of length n, where n is the number of agents participating in the evaluation, m is the total number of risk points, and R is the fuzzy evaluation matrix of each agent for each risk point. The severity of the risk to the quality of the case is determined based on the comprehensive risk scoring vector.

[0016] Furthermore, the method also includes: Based on the case data, the legal dataset, and the structured risk consensus, extract keywords related to the case; Using the aforementioned keywords as the mandatory guiding condition, multiple report candidate sequences are generated, and the probability of each report candidate sequence is calculated according to the following expression. : , in, For the key operators of legal provisions in the legal dataset, Let L be the semantic similarity function, and L be the length of the target sequence. The generated review report text sequence consists of L tokens. X represents the input data. For the 1st to t-1th output words, Given case data X and the generated Given the probability of generating the basic language model for the t-th word, For the t-th word element, Weights are used to convert semantic similarity into probabilities.

[0017] The candidate report sequence with the highest probability value is selected as the evaluation report output.

[0018] According to a second aspect of the embodiments of this disclosure, a computer apparatus is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute the aforementioned case quality assessment method.

[0019] According to a third aspect of the embodiments of this disclosure, a non-transitory computer-readable storage medium is provided, which, when the instructions in the storage medium are executed by a computer's processor, enables the computer to perform the aforementioned case quality assessment method.

[0020] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: parsing case data and legal datasets, generating sub-tasks corresponding to each of the preset evaluation dimensions according to at least one evaluation dimension; executing the sub-tasks through multiple agents, assigning one sub-task to each agent; generating and outputting a unified structured risk consensus based on the reasoning results output by the multiple agents. Based on multiple agents, data of the same case is analyzed and reasoned from different evaluation dimensions, and multiple reasoning results are integrated to obtain a unified structured risk consensus. Based on multiple agents collaboratively completing evaluation tasks, evaluating from different evaluation dimensions to discover risks, this solves the problem of lacking an accurate and efficient automatic case evaluation mechanism, and achieves standardized quality evaluation with low cost, high efficiency, and high quality.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0023] Figure 1 This is a flowchart illustrating a case quality assessment method according to an exemplary embodiment.

[0024] Figure 2 This is a flowchart illustrating yet another case quality assessment method according to an exemplary embodiment.

[0025] Figure 3 This is a flowchart illustrating yet another case quality assessment method according to an exemplary embodiment.

[0026] Figure 4 This is a flowchart illustrating yet another case quality assessment method according to an exemplary embodiment.

[0027] Figure 5 This is a flowchart illustrating yet another case quality assessment method according to an exemplary embodiment.

[0028] Figure 6 This is a flowchart illustrating yet another case quality assessment method according to an exemplary embodiment.

[0029] Figure 7 This is a flowchart illustrating yet another case quality assessment method according to an exemplary embodiment.

[0030] Figure 8 This is a flowchart illustrating yet another case quality assessment method according to an exemplary embodiment.

[0031] Figure 9 This is a flowchart illustrating yet another case quality assessment method according to an exemplary embodiment.

[0032] Figure 10 This is a flowchart illustrating yet another case quality assessment method according to an exemplary embodiment.

[0033] Figure 11 This is a flowchart illustrating yet another case quality assessment method according to an exemplary embodiment.

[0034] Figure 12 This is a flowchart illustrating yet another case quality assessment method according to an exemplary embodiment. Detailed Implementation

[0035] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0036] Currently, the evaluation of case quality is still mainly done manually, requiring relevant personnel to review each case file, extract facts, verify legal provisions, and identify risks.

[0037] Case files typically include printed text, handwritten notes, signatures, images, and videos. Manual review is inefficient and prone to errors, and long files are prone to missing information.

[0038] Furthermore, the cases involve a wide range of real-life areas, are subject to complex laws and regulations, and are easily influenced by subjective factors in human judgment, with different people often having different judgment standards.

[0039] In an effort to overcome the shortcomings of manual review, attempts were made to introduce deep learning models into the review process to partially replace manual review. However, due to the complexity of the case data and the diversity of review dimensions, the deep learning models failed to effectively address the needs of case review.

[0040] In summary, there is a lack of an accurate and efficient automated case review mechanism.

[0041] To address the aforementioned problems, embodiments of this disclosure provide a case quality assessment method, a computer device, and a storage medium. The embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0042] An exemplary embodiment of this disclosure provides a case quality assessment method, based on multi-agent collaborative assessment and the generation of a unified structured risk consensus. The method is used to detect and identify case quality issues. The process for detecting and identifying case quality issues using this method is as follows: Figure 1 As shown, it includes: Step 101: Analyze the case data and legal dataset of the case, and generate sub-tasks corresponding to each of the preset evaluation dimensions.

[0043] The evaluation dimensions include at least one or more of the following: Case filing, evidence, procedures, sentencing, These four evaluation dimensions are key dimensions for evaluating the quality of criminal cases, among which: The case filing dimension refers to the compliance of the case filing process, such as the legality of the competent authority, the accuracy of the application of the crime, and the completeness of the determination of aggravating circumstances; The evidentiary dimension refers to the evidentiary basis for determining the facts of a case, such as the completeness, consistency, and legality of the evidence chain. The process dimension refers to the procedural compliance of the entire case-handling process, such as the time limit for coercive measures, the issuance / service of legal documents, and the compliance of time limits for case-handling stages; Sentencing dimensions refer to the appropriateness and balance of sentencing recommendations, such as the determination of mitigating / aggravating circumstances, the conditions for applying probation, and the reasonableness of the sentencing range.

[0044] According to an exemplary implementation, for cases of dangerous driving, the case filing dimension refers to the review of the case filing stage, and the key points of the review include blood alcohol content, aggravating circumstances, etc.; the evidence dimension refers to the review of the evidence stage, and the key points of the review include the suspect's situation, the vehicle's situation, etc.; the process dimension refers to the review of the process stage, and the key points of the review include coercive measures, document timeliness, etc.; the sentencing dimension refers to the intelligent agent reviewing the sentencing stage, and the key points of the review include probation, leniency, etc.

[0045] Step 102: Execute the sub-task through multiple agents, and assign one sub-task to each agent.

[0046] The intelligent agent includes at least one or more of the following: Intelligent agents for case filing review, evidence review, process review, and sentencing review.

[0047] Each agent corresponds one-to-one with the evaluation dimensions. Each agent has built-in domain knowledge, judgment rules, and large model (LLM) reasoning capabilities, and operates by combining a rule engine with LLM fine-tuning / hint engineering.

[0048] According to an exemplary implementation, a collaborative evaluation system is used to complete the scheduling and control of each intelligent agent. The architecture of the collaborative evaluation system is as follows: Figure 2 As shown, it includes a master scheduler (Orchestrator) and multiple agents that are subject to its scheduling and control.

[0049] The master scheduler uses a multi-agent parallel distribution mechanism to push subtasks to their respective agents, triggering parallel reasoning in each agent. Each agent extracts data from its assigned evaluation dimension, combines it with the evaluation rule base, independently performs the evaluation work, identifies case quality risks in that dimension, and outputs the reasoning results.

[0050] According to an exemplary implementation, taking a dangerous driving crime case as an example, the reasoning process of each intelligent agent is as follows: 1. Case Filing Review Intelligent Agent: Enforcement of compliance risk identification in case filing. This intelligent agent specifically verifies the compliance of cases during the acceptance and filing stages. Its workflow is as follows: 1) Data intake and element extraction: The case review agent accurately extracts key case-filing elements such as "suspected crime", "blood alcohol content (mg / 100ml)", "value of involved property" and "criminal suspect's prior criminal record" from case data in structured text form.

[0051] 2) Rule matching: The case filing review agent has a built-in rule base of the Criminal Law, the Criminal Procedure Law, and the case filing standards of the procuratorate. For example, for the crime of dangerous driving, the rule base clearly defines the statutory aggravating circumstances corresponding to different blood alcohol content ranges (such as ≥200mg / 100ml).

[0052] 3) LLM Reasoning and Verification: The extracted key elements for case filing, along with relevant legal provisions, are input into a finely tuned dedicated LLM. The LLM's task is to determine "whether the charge in this case is accurate given the given blood alcohol content, prior criminal record, etc., whether there are any circumstances where a charge should have been filed but was not, or should not have been filed, and whether the determination of aggravating circumstances is complete." For example, the LLM might reason: "According to Article 133-1 of the Criminal Law, the driver's blood alcohol content is 215mg / 100ml, and he has previously been administratively punished for drunk driving. This should be considered a 'serious offense.' Has this been adequately considered during the case filing process?" 4) Inference Result Output: The case review agent outputs the inference result, which is a structured risk record. According to an exemplary implementation, the format of the output inference result is: {Risk Type: “Jurisdiction / Application of Charges / Determination of Aggravating Circumstances”, Risk Description: “...”, Elements Involved: [...], Relevant Legal Provisions: “...”, Confidence Level: 0.95}.

[0053] 2. Evidence review agent: The process involves identifying risks related to the integrity and consistency of the chain of evidence. The workflow is as follows: 1) Multi-source evidence alignment: The evidence review agent first constructs an "evidence-entity correspondence table" and associates entities such as "suspect identity information (name, ID number)", "motor vehicle information (license plate number, vehicle frame number)" and "key physical evidence number" extracted from case cards, records, expert documents and physical evidence lists.

[0054] 2) Consistency logic check: The intelligent agent calls the rule engine to automatically compare the records of the same entity in different evidence carriers. For example, it compares whether the license plate number on the "Seizure List" is consistent with the license plate number in the "Vehicle Information Query Results"; it compares whether there is a logical conflict between the suspect's self-reported time of the crime in the "Interrogation Record" and the timestamp of the "On-site Surveillance Screenshot".

[0055] 3) Inference Result Output: The output is as follows: the inference result. According to one exemplary implementation, the inference result is a structured record in the following format: {Risk type: "Contradictory evidence / Missing evidence", Contradictory party: ["Record A", "Physical evidence B"], Contradictory content: "...", Influencing factor: "Determination of time of crime"}.

[0056] 3. Process evaluation agent: Identifying risks related to the legality of litigation proceedings. The process review agent specifically examines whether the litigation proceedings comply with statutory time limits and formal requirements. Its workflow is as follows: 1) Automated Validation of Time Limits Rules: The process review agent has a powerful built-in procedural law rule engine that covers the time limits stipulated in the Criminal Procedure Law for each stage. The engine automatically calculates the time interval between adjacent event nodes and compares it with the statutory time limits. For example, it automatically calculates whether "the time from detention to requesting approval for arrest exceeds 37 days (special circumstances)" and "whether the period for review and prosecution by the procuratorate has exceeded the time limit."

[0057] 2) Document completeness check: By analyzing the document content through LLM, check whether necessary legal documents (such as the "Notice of Rights and Obligations" and the "Notification of Expert Opinion") have been mentioned and served in the transcript, or check whether the corresponding documents exist in the case file through OCR recognition results.

[0058] 3) Output of Reasoning Results: Output the reasoning results. According to one exemplary implementation, the reasoning results are structured records, with the following format: {Risk type: "Prolonged detention / service defects", Violation stage: "Review and prosecution stage", Specified time limit: "30 days", Actual time used: "38 days", Relevant document: "Prosecution opinion (xxx)"}.

[0059] 4. Sentencing Review Agent: Identifying risks related to the appropriateness of sentencing recommendations. The sentencing review agent benchmarks the judgment or sentencing recommendation against the original judgment to examine the balance and accuracy of the sentencing process. Its workflow is as follows: 1) Panoramic extraction of sentencing circumstances: The sentencing review intelligence system systematically extracts all "aggravating circumstances" (such as recidivism, principal offender, serious consequences) and "lenient circumstances" (such as surrender, confession, pleading guilty and accepting punishment, compensation and forgiveness, preparation / attempt) from the case data, and pays special attention to the description of "conditions for probation" (such as relatively minor criminal circumstances, remorseful behavior, no risk of recidivism, etc.).

[0060] 2) Sentencing Model Reference and LLM Equilibrium Argument: The sentencing review agent inputs the "charge + basic criminal facts + set of circumstances" of this case into a sentencing prediction model trained based on historical judgment data to obtain a reference range for the sentence.

[0061] 3) Output of Reasoning Results: Output the reasoning results. According to one exemplary implementation, the reasoning results are structured records, with the following format: {Risk Type: “Omission of Circumstances / Uneven Sentencing / Inappropriate Application of Probation”, Omitted Circumstances: “Compensation and Obtaining Forgiveness”, Impact Range: “Potential Reduction of Base Sentence by Less Than 20%”, Reference Sentence Range: “[2 Years, 3 Years]”, Recommended Sentence: “4 Years”}.

[0062] Step 103: Generate and output a unified structured risk consensus based on the reasoning results output by the multiple intelligent agents.

[0063] This step involves integrating the reasoning results of each agent, retrieving and resolving contradictions in fact-finding, and ultimately generating a unified, standardized, structured risk consensus. The specific process is as follows: Figure 3 As shown, it includes: Step 301: Obtain the reasoning results of each of the aforementioned agents.

[0064] The reasoning result includes at least one or more of the following: The first inference result output by the intelligent agent for case filing review in identifying compliance risks during case filing is... The evidence review agent performs evidence chain integrity and consistency risk identification, outputting the second reasoning result. The third inference result output by the intelligent agent in evaluating the legality risk identification of litigation procedures during process review. The fourth inference result output by the sentencing review agent in identifying the appropriateness risk of implementing sentencing recommendations. The first inference result, the second inference result, the third inference result, and the fourth inference result have a unified output format, which includes at least one or more of the following information: Risk type, description, elements involved, relevant legal provisions, confidence level, and source of evidence.

[0065] Among them, the risk type refers to the specific risk category of each assessment dimension, such as "no jurisdiction by the competent authority" in the case filing dimension, "contradictory evidence" in the evidence dimension, "detention beyond the legal time limit" in the process dimension, and "fair circumstances not recognized" in the sentencing dimension; the description is the specific details of the risk point, such as "blood alcohol content of 220mg / 100ml, which is a legally aggravating circumstance, but was not marked in the case filing stage"; the case-related elements are the key case information involved in the risk point, such as the suspect's identity, the vehicle involved, blood alcohol content, etc.; the associated legal provisions are the legal provisions / judicial interpretations on which the risk point is determined, marked with the legal provision ID and core clauses; the confidence level is the probability value (0~1) of the agent's determination of the risk point, reflecting the reliability of the determination result; the source of evidence is the source of the case file data on which the risk point is determined, marked with the page sub-ID and data type (handwritten / printed / signed).

[0066] Step 302: For the reasoning results based on the same case output by different intelligent agents, retrieve the facts to determine contradictions.

[0067] In this step, the reasoning results output by all agents are summarized, and the reasoning results for the same case are globally retrieved using the unique case identifier to identify factual discrepancies between different agents.

[0068] The contradiction in fact-finding mainly refers to conflicting judgments made by different agents regarding the same fact or risk point in the same case. For example, the case-filing review agent might mark "the suspect has a prior criminal record," while the sentencing review agent might fail to recognize this prior record and conclude that "there are no aggravating circumstances." Another example is the evidence review agent determining that "the chain of evidence is complete and without contradiction," while the case-filing review agent might determine that "the evidence for filing a case is insufficient" due to issues with the details of the evidence. The master control scheduler uses preset contradiction retrieval rules to compare the elements involved in the case, fact-finding, and risk point identification in the reasoning results, locates the contradictions, and marks the agents and risk points involved in the contradictions.

[0069] Step 303: In the event of a contradiction in factual findings, the reasoning results of each intelligent agent are weighted and voted on according to their authority weight coefficients to obtain a unified structured risk consensus.

[0070] The authority weight coefficient indicates the historical evaluation accuracy of the agent, and the structured risk consensus includes at least one or more of the following information: Case unique identifier, risk classification, risk type, description, elements involved, related evidence or legal provisions, and confidence level.

[0071] If a contradiction is found in the factual findings, a weighted voting method is used to resolve the contradiction.

[0072] According to one exemplary implementation, data consistency alignment in this step is achieved using a weighted voting game model. Specifically, a directed acyclic graph (DAG) can be used to maintain the dependencies of the evaluation tasks.

[0073] Data consistency alignment is performed based on the following expression to obtain the final structured risk consensus. : , in, The authority weight coefficient of the i-th agent in a specific legal field (dynamically adjusted based on historical evaluation accuracy). Let m be the reasoning result of the i-th agent, ensuring that the system outputs a unique structured risk consensus, where m is the number of agents participating in the reasoning. This represents the set of all possible risk consensus candidate values ​​(e.g., the value space for possible risk categories or types). This is an indicator function that takes the value 1 when the reasoning result of the i-th agent is equal to the candidate value v, and 0 otherwise.

[0074] For each contradictory factual determination or risk point, the reasoning results of each agent are extracted and grouped and aggregated according to different values. For each candidate result, the agents supporting the result are counted and multiplied by their authority weight coefficients for weighted summation to obtain the weighted score of the result. By comparing the weighted scores of each candidate result, the result with the highest score is selected as the final consensus for the contradictory point. If there are cases with the same score, a tie-breaking mechanism is activated, which may select the result with higher confidence, introduce additional agents, or mark it as requiring manual review.

[0075] The authority weight coefficient indicates the historical review accuracy of the agent and is a quantitative indicator of the reliability of the agent's reasoning results. The authority weight coefficient ranges from 0 to 1, and the sum of the authority weight coefficients of all participating agents is 1. The higher the historical review accuracy of an agent, the larger its authority weight coefficient, and the stronger its influence in the weighted voting. For example, if the evidence review agent has a historical review accuracy of 98% for dangerous driving cases, its authority weight coefficient can be set to 0.28; the sentencing review agent has a historical review accuracy of 97%, and its authority weight coefficient can be set to 0.26; the process review agent has a historical review accuracy of 96%, and its authority weight coefficient can be set to 0.24; and the case filing review agent has a historical review accuracy of 95%, and its authority weight coefficient can be set to 0.22, ensuring that the sum of the weights of all agents is 1, and the higher the accuracy, the greater the weight.

[0076] After resolving conflicts through weighted voting, all risk points are integrated to generate and output a unified structured risk consensus. This structured risk consensus includes at least one or more of the following: unique case identifier, risk classification, risk type, description, involved elements, relevant evidence or legal provisions, and confidence level. The risk classification is divided according to evaluation dimensions: case filing risk, evidentiary risk, procedural risk, and sentencing risk, facilitating subsequent risk classification and statistics.

[0077] An exemplary embodiment of this disclosure also provides a case review method, which includes an uncertainty assessment step in the process of generating and outputting a unified structured risk consensus. Difficult cases are identified by calculating the predictive entropy of the reasoning results, triggering a manual review mechanism, thus achieving a combination of "machine review + manual review" to further improve the accuracy and reliability of risk identification. The specific process includes the following steps: like Figure 4 As shown, it includes: Step 401: Obtain the reasoning results of each of the aforementioned intelligent agents.

[0078] The implementation principle of this step is the same as that of obtaining the reasoning result in step 301, and will not be repeated here.

[0079] Step 402: Calculate the prediction entropy for the inference results output by each of the intelligent agents.

[0080] When outputting inference results, each agent calculates the predictive entropy of the inference result based on the Monte Carlo Dropout sampling algorithm. Predictive entropy is a quantitative indicator that measures the uncertainty of the agent's inference results, reflecting the reliability of the agent's judgment on risk points. A higher predictive entropy value indicates greater uncertainty in the agent's judgment of the risk point, and lower reliability of the inference result; conversely, a lower predictive entropy value indicates greater certainty in the agent's judgment of the risk point, and higher reliability of the inference result.

[0081] In this step, while obtaining the inference results of each agent, the corresponding prediction entropy value is extracted, and the prediction entropy value of each risk point is summarized.

[0082] According to an exemplary implementation, the prediction entropy H is calculated using the following expression: , Where x is the input sample, i.e. the case data to be reviewed, including the basic facts of the case, relevant legal documents, evidence, etc.; D is the training dataset, i.e. the historical case data that the agent has learned during the training phase; C is the category space, i.e. the set of all possible judgment results for the risk point. It is the probability that the input sample x belongs to category c, calculated by the agent through Monte Carlo Dropout sampling.

[0083] Step 403: If the predicted entropy is higher than the preset risk threshold, initiate risk processing.

[0084] If the predicted entropy value of a single risk point is higher than the preset risk threshold, the case is determined to have a quality problem risk, and risk handling is initiated.

[0085] The entropy threshold and mean threshold can be dynamically adjusted according to the actual needs of judicial case handling and historical review data. For example, for cases of dangerous driving, the risk threshold can be set to 0.3. When the agent's predicted entropy value for the risk point exceeds 0.3, it indicates that there is significant uncertainty in its reasoning result (probability distribution is p(yes) = 0.65, p(no) = 0.35). The system automatically triggers the risk handling mechanism, marks the case as "requiring manual review" and submits it to senior review experts for review. The entropy threshold can be dynamically adjusted according to the actual needs of judicial case handling and historical review data. For example, differentiated thresholds can be set for different case types, or the threshold standard can be temporarily tightened according to the requirements of special review activities to ensure the accuracy and adaptability of risk identification.

[0086] Risk management includes measures such as activating the "expert intervention" flag and generating and pushing warning messages to ensure that risk identification in complex and difficult cases is not misreported.

[0087] According to one exemplary implementation, risk handling includes triggering a manual review mechanism, pushing information such as case file data, agent reasoning results, and prediction entropy values ​​of the difficult case to the manual review end, whereby professionals conduct manual review of the difficult risk points, verify, correct or supplement the results of the machine review, and output the manual review results.

[0088] The results of manual review and the results of intelligent agent reasoning have a unified output format, which can be directly applied to the subsequent reasoning process to generate structured risk consensus.

[0089] Step 404: For the reasoning results based on the same case output by different intelligent agents, retrieve the facts to determine contradictions.

[0090] Step 405: In the event of a contradiction in factual findings, the reasoning results of each intelligent agent are weighted and voted on to obtain a unified structured risk consensus.

[0091] The implementation principles of steps 404 to 405 are the same as those of steps 302 and 303, and will not be repeated here.

[0092] An exemplary embodiment of this disclosure also provides a case quality assessment method, which preprocesses the raw case file data to transform unstructured raw case file data into structured text case data and structured data table legal datasets, providing a standard data source for subsequent sub-task generation and multi-agent assessment. The specific process is as follows: Figure 5 As shown, it includes: Step 501: Obtain the original case file data.

[0093] The original case file data includes at least one or more of the following: Judgment, court transcript, and evidence materials The original case file data contains at least one or more of the following data formats: Text, images, and videos; The original case file data is the complete case file data generated during the judicial case handling process. In addition to judgments, court transcripts, and evidence materials, it may also include other relevant information.

[0094] According to one exemplary implementation, for cases of dangerous driving, the original case file data also includes specific materials such as blood alcohol content test reports, information on the vehicle involved, documents on compulsory measures, and plea agreements.

[0095] Each case is assigned a unique case identifier, which is used throughout all subsequent preprocessing and review stages to ensure data traceability.

[0096] Step 502: Generate the case data in structured text form based on the original case file data.

[0097] In this step, the data undergoes structuring. Through long document segmentation, multi-type information extraction, and data aggregation, the original case file data, characterized by its disorganized format and scattered information, is transformed into a standardized, structured text format. The specific process is as follows: Figure 6 As shown, it includes: Step 601: Using an overlapping sliding window, the original case file data is split into paginated case file documents by performing ultra-long document segmentation.

[0098] The paginated case file contains multiple segments of text.

[0099] Original case files are typically quite long, and direct processing can easily lead to memory overflow in the model, low computational efficiency, and information gaps.

[0100] In this step, the excessively long case file is segmented using an overlapping sliding window mechanism to generate paginated case file documents.

[0101] Set a fixed-length window size (chunk size) as the scanning baseline and configure a specific stride ratio to generate overlapping areas (i.e., create overlap). As the window slides forward, the newly generated text block will include redundant information from the end of the previous text block (overlap). The window size and overlap area size can be dynamically adjusted according to the format and length of the dossier data; for example, the window size can be set to 512 characters and the overlap area to 100 characters.

[0102] During the segmentation process, the original case file data is segmented segment by segment using a sliding window as the unit, generating multiple text blocks. Each text block is a page, and each page is assigned a page identifier. After segmentation is completed, a paginated case file document is generated. The document contains all the segmented text blocks and associates them with the case unique identifier and the page identifier, ultimately forming an association relationship between the case unique identifier and the sequence of page identifiers for multiple text blocks.

[0103] Step 602: Extract information from the case file documents to obtain valid information.

[0104] The information extraction includes at least one or more of the following processing methods: Handwriting recognition, signature recognition, and text extraction.

[0105] In this step, corresponding methods are used to extract information from different data formats in paginated case file documents, achieving full and accurate extraction of printed text, handwritten content, and signatures. Valid information is the core information required for case quality assessment; therefore, blank spaces, irrelevant annotations, and other invalid information in the file are first removed.

[0106] 1. Handwriting recognition.

[0107] Handwriting recognition targets non-signature handwritten content such as handwritten annotations, handwritten records, and handwritten documents in case files, addressing the challenges of large-scale model recognition for illegible, incomplete, blurry, technically complex, and lengthy handwritten content in judicial case files.

[0108] According to one exemplary implementation, handwriting recognition is performed in the following manner: Figure 7 As shown: Step 701: Perform normalization preprocessing on the paginated case file documents to obtain standardized case file images.

[0109] It can be implemented based on a Vision Transformer (ViT) model that has been customized and fine-tuned for judicial scenarios.

[0110] In this step, the handwritten images in the paginated case file documents undergo normalization preprocessing (e.g., eliminating differences in image size, resolution, brightness, contrast, etc.) to prepare for subsequent image block segmentation.

[0111] Step 702: Divide the standardized file image into image blocks of fixed size.

[0112] The ViT model processes one-dimensional sequence data. In this step, to transform a two-dimensional image into one-dimensional sequence features that the ViT model can process, the dossier image... The image is uniformly divided into N square patches of fixed size (P, P). The side length P of each patch can be adjusted according to the model requirements. The sequence length is then determined. (W is the image width, H is the image height).

[0113] Step 703: Convert the image patch into a feature vector.

[0114] The image patch is converted into a feature vector according to the following expression. : , in, These are placeholders for categories used to aggregate global features. Let j be the feature vector of the j-th image patch, where j takes values ​​from 1 to N. For position encoding.

[0115] In this step, each two-dimensional image patch (P,P) is flattened into a one-dimensional vector, and then mapped to a D-dimensional feature vector using a learnable linear projection matrix E; finally, a learnable positional encoding is added to the projected D-dimensional feature vector. Preserve the positional information of the handwritten characters in the image; finally, add category placeholders. This is used for subsequent aggregation of global features of the entire image. As a prefix, it is concatenated with the features of N image patches sequentially to form an embedding sequence of degree N+1. This sequence is then encoded with learnable locations. The components are added together, preserving the spatial location information of the handwritten characters in the image, ultimately yielding a feature vector with fused location information. This vector is the input to the Transformer Encoder in the ViT model.

[0116] Step 704: Extract features from the feature vector using a decoder to obtain a long sequence of handwritten semantics.

[0117] In this step, the decoder is the Transformer Encoder of the ViT model, which converts the feature vectors... After entering the Transformer Encoder, the pen tip features and texture frequencies in the image are extracted through the multi-head self-attention mechanism to distinguish handwritten strokes and printed characters in terms of frequency domain and spatial distribution.

[0118] ViT determines whether there is handwriting on the page or in the area by calculating the output vector of the class token: , where Q is the query variable obtained through a learnable linear transformation matrix ; K is the key variable obtained through a learnable linear transformation matrix and is used to match with the query variable to calculate the degree of correlation between features; V is the value variable obtained through a learnable linear transformation matrix and contains the actual semantic content of the input features and is used for weighted aggregation according to the weights; is the feature dimension of the query variable and the key variable, which is used as a scaling factor to smooth the dot product result and prevent the gradient from disappearing in the saturation region of the softmax function; softmax is the normalized exponential function and is used to convert the correlation score into a probability distribution with a sum of 1, that is, the attention weight.

[0119] Under the multi-head self-attention mechanism, when the model recognizes scribbled characters, it will automatically associate with the legal term keywords in the context (such as "defendant", "fixed-term imprisonment", etc.) to improve the inference accuracy under incomplete information. When encountering a handwritten area with extremely scribbled handwriting and blurred pixel features (such as the blurred character "self"), the multi-head self-attention first encodes the features of the entire image and the text sequence, then calculates the attention weights between this area and the clear legal terms in the context (such as the printed "defendant", "defense"), and strengthens the highly semantically associated parts. Subsequently, based on the prior probability of legal semantics, global consistency modeling is completed, and the area with a weight significantly higher than the threshold is accurately determined as the handwritten text area, thus effectively overcoming the recognition difficulties caused by information loss.

[0120] After completing the positioning of the handwritten area, subsequent character recognition can be performed on the located handwritten area.

[0121] For lengthy, paginated case files, the corresponding feature vectors are also quite long, potentially leading to memory overflow. This step introduces a sliding window mechanism to process the feature vectors into regions. The lengthy handwritten feature vectors are divided into multiple smaller feature sub-blocks using a sliding window. Convolutional branches are used to extract local texture features from each feature sub-block. Then, the local features of all feature sub-blocks are input into a Transformer layer to capture the handwritten semantic features contained in all feature sub-blocks. Finally, these features are fused to obtain a long sequence of handwritten semantic features for the entire page's handwritten area. This approach avoids memory overflow while ensuring the integrity of the handwritten features.

[0122] Step 705: Map the long program column to a character-level probability distribution, and select the character with the highest probability distribution for each character position as the recognition result to generate a handwritten text sequence.

[0123] In this step, the probability distribution of candidate characters at each character position is calculated based on the following expression. The decoder output is normalized. , in, Let t be the target character predicted by the model at time t, which is the t-th word in the generated sequence. The first to the (t-1)th output tokens are the historical character sequences generated before time t; The standardized case file image features are used as a global context reference for the cross-modal attention mechanism. This indicates the output character at the current position given the input image and the generated sequence. The conditional probability distribution. Let be the hidden state vector output by the decoder at time t. It is a linear mapping matrix. .) is the normalization exponential function, used to transform the original score after linear mapping into a value that conforms to a probability distribution.

[0124] According to one exemplary implementation, = = , ,…, .

[0125] The algorithm uses a greedy search approach to select the character with the highest confidence from the candidate character set, generating a handwritten text sequence. Starting from the first character position in the handwritten area, the algorithm selects the candidate character with the highest probability value in the probability distribution as the recognition result for that position. Following the order of the handwritten characters, the algorithm performs the selection of candidate characters for each character position sequentially, recognizing each character one by one. Finally, the recognition results for all character positions are concatenated in sequence to generate the handwritten text sequence with the highest confidence.

[0126] Step 706: Correct the handwritten text sequence for errors and output the corrected handwritten text sequence.

[0127] In this step, to enhance the model's robustness to incomplete and ambiguous dossiers, a character-level negative log-likelihood loss is introduced into the loss function, along with label smoothing technology.

[0128] The handwritten text sequence is error-corrected according to the following loss function: , in, For regularization items in a legal terminology database, To adjust the parameters, L is the target sequence length, and I is the normalized dossier image. The parameters of the model to be trained, For the actual character at position i, - (·) is a negative logarithmic function. Given a standardized dossier image I and model parameters, In the case where the model correctly predicts the real character at the i-th position. The conditional probability distribution.

[0129] This step refines the initial handwritten text sequence using a loss function, while also enhancing the model's robustness in complex scenarios such as incomplete or blurred content in the dossier.

[0130] 2. Signature recognition.

[0131] The signature recognition system distinguishes between the signatures of parties involved in the case file, case handlers, and other personnel, and ordinary handwritten content.

[0132] According to one exemplary implementation, signature recognition is performed in the following manner: Figure 8 As shown, it includes: Step 801: Determine the candidate region based on the feature vector.

[0133] In this step, based on the feature vector obtained from handwriting recognition, the signature component on the page is located using the YOLOv8 model within the handwriting recognition marked area, and the coordinates (x, y, w, h) of all candidate signature regions are output. Here, x is the pixel coordinate of the center point of the candidate signature region bounding box in the horizontal direction of the image; y is the pixel coordinate of the center point of the candidate signature region bounding box in the vertical direction of the image; w is the width of the candidate signature region bounding box, representing the horizontal pixel span occupied by the signature component in the image; and h is the height of the candidate signature region bounding box, representing the vertical pixel span occupied by the signature component in the image.

[0134] Step 802: Using a signature feature encoder based on a Siamese network, the candidate region is subjected to feature extraction and mapping to a high-dimensional feature space to generate a signature feature vector.

[0135] Because signatures are randomly distributed in legal files (e.g., at the end of the signature box, across the seam, or at the end of the record), the model is trained using a pre-labeled signature bounding box dataset to output the coordinates (x, y, w, h) of candidate regions, thus extracting image patches of the signature regions. To distinguish between a party's signature and general handwritten annotations, a signature feature encoder based on a Siamese network is constructed. The extracted signature region image patches are then processed. The input is fed into a signature feature encoder with shared parameters, mapped to a high-dimensional feature space, to obtain the signature feature vector f: .

[0136] The preprocessed signature image patch is input into the signature feature encoder of the Siamese network, which maps the two-dimensional signature image to a high-dimensional feature space to generate a signature feature vector. The signature feature vector can characterize the visual features of the signature.

[0137] Step 803: Perform a similarity analysis between the signature feature vector and a preset standard signature sample to obtain the signature feature vector that meets the similarity requirements.

[0138] In this step, cosine similarity or Euclidean distance is calculated to measure the degree of match between the current signature and the standard signature samples of the case subjects (such as defendants, lawyers, and judges) stored in the case file database. , in, This refers to the candidate signature feature vector extracted from the current file image to be examined. The formula is a standard signature sample feature vector of the case subject stored in the case file database. The numerator of the formula is the dot product of the two vectors, and the denominator is the product of the lengths of the two vectors.

[0139] In this embodiment, a judicial case file signature library can be preset, which contains standard signature samples of case-related subjects (defendants, lawyers, judges, police officers handling cases, etc.). Each standard signature sample has been pre-converted into a high-dimensional feature vector.

[0140] If the similarity score between the signature feature vector and a certain standard signature sample is greater than or equal to the preset matching threshold, the candidate signature is determined to be a valid signature of the corresponding subject; if the score is lower than the threshold, it is determined that it cannot be matched with the known subject related to the case.

[0141] Step 804: Generate and output the signature recognition result based on the signature feature vector and location information that meet the similarity requirements.

[0142] In this step, the output signature recognition result is in a standardized format and can be directly integrated into the subsequent data aggregation process.

[0143] 3. Text extraction.

[0144] Text extraction is performed on printed text information in the case files, using PaddleOCR technology.

[0145] Step 603: Aggregate the extracted valid information to generate the case data in structured text form.

[0146] In this step, handwriting recognition, signature recognition, and text extraction results are fused to generate case data in structured text format.

[0147] Step 503: Search the case data in the preset legal knowledge base to obtain the legal dataset.

[0148] This step involves retrieving legal basis. Semantic matching of case facts with legal provisions is performed to generate a legal dataset highly relevant to the case.

[0149] The specific process is as follows: Figure 9 As shown, it includes: Step 901: Convert the case data and the legal provisions in the legal knowledge base into high-dimensional dense vectors.

[0150] In this embodiment, a pre-set legal knowledge base is provided, which includes basic laws such as the Criminal Law and the Criminal Procedure Law, judicial interpretations, sentencing guidelines from various regions, case filing standards, rules of evidence, and procedural law provisions. All legal provisions have been structured, and the structured data table format of legal provisions can be directly obtained through retrieval.

[0151] According to one exemplary implementation, a BGE vector model is used as a text encoder. Utilizing the deep bidirectional attention mechanism of the BGE model, all legal provisions in the case data and the legal knowledge base are transformed into high-dimensional dense vectors. This places the case data vectors and legal provision vectors in the same high-dimensional semantic space, allowing the degree of matching between case facts and legal provisions to be determined by calculating the similarity between the vectors.

[0152] Step 902: Calculate the similarity between the high-dimensional dense vector of the case data and the high-dimensional dense vector of the legal provisions, and generate the legal dataset based on the legal provisions whose similarity reaches the preset legal provision matching conditions.

[0153] To achieve millisecond-level response times with massive amounts of legal data, this embodiment introduces the Faiss library to construct a high-performance vector index. Addressing the density of legal data distribution, an inverted file index is employed, dividing the high-dimensional vector space into several cellular regions to enhance retrieval accuracy. Legal provisions with similarity reaching preset matching conditions (e.g., a matching threshold) are selected, and duplicate or irrelevant provisions are removed, ultimately yielding legal bases highly relevant to the case data and generating a legal dataset.

[0154] Step 504: Based on the basic information of the case, associate and output the case data in structured text form with the legal dataset in structured data table form.

[0155] The basic information includes at least the unique identifier of the case.

[0156] In this step, the generated case data is linked and integrated with the legal dataset, using the unique case identifier as the association key. The two are then encapsulated into a structured combined dataset and output. Subsequently, a multi-agent system can parse this structured combined dataset and detect risks of case quality issues.

[0157] An exemplary embodiment of this disclosure also provides a method for case quality assessment. After obtaining a structured risk consensus, risk rating is performed by constructing a fuzzy assessment matrix and calculating a comprehensive risk score to achieve quantitative rating of risk points and determination of the severity of case quality risks, providing accurate quantitative basis for the rectification of judicial case handling. The specific process is as follows: Figure 10 As shown, it includes: Step 1001: By constructing a fuzzy evaluation matrix and combining it with a weight vector, calculate the comprehensive risk score vector for each risk point in the structured risk consensus.

[0158] After all agents have completed their analysis, the master scheduler will aggregate the structured risk records output by the four agents. The system will automatically classify the risks based on the type of risk (entity / procedure), the severity of the legal consequences, and the confidence level given by the agents, in order to reflect the severity of the risks (risk severity includes "critical", "important", "warning", etc.).

[0159] Based on the severity and probability of occurrence of the risk, a fuzzy evaluation matrix R is constructed, and combined with the weight vector W, a comprehensive risk score is calculated.

[0160] The comprehensive risk score vector S for each risk point in the structured risk consensus is calculated according to the following expression: , in, Let be a weight vector of length n, where n is the number of agents participating in the evaluation, and mR be the fuzzy evaluation matrix of each agent for each risk point. Let be the confidence level of the i-th agent in judging the j-th risk point.

[0161] The weight vector W is multiplied by the fuzzy evaluation matrix R using weighted matrix multiplication, resulting in a 1×m dimension row vector S. The j-th element in S is the comprehensive risk score for the j-th risk point. The comprehensive risk score ranges from 0 to 1; a higher score indicates a higher degree of risk severity for that risk point.

[0162] Step 1002: Determine the severity of the risk to the quality of the case based on the comprehensive risk scoring vector.

[0163] In this step, based on the range of values ​​for S, the risk level corresponding to the comprehensive risk score is determined to reflect the severity of the risk to the case quality.

[0164] According to one exemplary implementation, the risk levels, from high to low, are as follows: Four levels enable precise risk classification and diversion.

[0165] An exemplary embodiment of this disclosure also provides a case quality assessment method. After completing risk rating, an assessment report is automatically generated based on case data, legal datasets, and structured risk consensus using a keyword-guided Beam Search algorithm. The specific process is as follows: Figure 11 As shown, it includes: Step 1101: Extract keywords related to the case based on the case data, the legal dataset, and the structured risk consensus.

[0166] The keywords extracted in this step are all from case data, legal data sets, and structured risk consensus.

[0167] According to an exemplary embodiment, the keywords include the following types: I. Keywords for case fact elements, from case data. Such as "blood alcohol content 200mg / 100ml", "drunk driving", "pleading guilty and accepting punishment", "suspect Wang Moumou" in a case of dangerous driving crime; II. Keywords for legal provision terms, from legal data sets. Such as "Article 133-1 of the Criminal Law", "crime of dangerous driving", "filing standard", "compulsory measures", "sentencing range", etc.; III. Keywords related to risks, from structured risk consensus. Such as "filing risk", "evidence contradiction", "extended custody", "important risk", etc.

[0168] After the keyword extraction is completed, a set of guiding keywords exclusive to this report is constructed to provide a guiding basis for the subsequent automatic report generation.

[0169] Step 1102: Using the keywords as mandatory guiding conditions, generate multiple report candidate sequences and calculate the probabilities of each of the report candidate sequences.

[0170] For the report generation based on Constrained Decoding, to ensure that the generated review report meets the rigor of legal official documents, the Beam Search algorithm guided by keywords is introduced when the LLM generates narrative text. Define the probability function of the generated sequence as: , where is the key operator of the legal provisions in the legal data set, is the semantic similarity function, L is the length of the target sequence, is the generated text sequence of the review report, consisting of L word tokens; X is the input data, that is, input information such as case data, legal data sets, and structured risk consensus; y<t is the word tokens from the 1st to the t - 1st output, that is, the content of the 1st to the t - 1st output units, is the probability of generating the t-th word token by the basic language model given the case data X and the already generated ; is the weight for converting the semantic similarity into probability.

[0171] K represents the key operators of legal provisions in the legal dataset, namely the set of all keywords extracted in step 1101, such as "drunk driving", "dangerous driving crime", "risk of filing a case", etc. Let t be the t-th word element, i.e., the token generated in step t. To convert semantic similarity into probability weights, i.e. keyword reward factors, the probability of matching keyword tokens can be increased, thus enabling restricted decoding.

[0172] Step 1103: Select the candidate report sequence with the highest probability value as the evaluation report output.

[0173] In this step, the candidate sequence with the highest probability is selected as the final evaluation report and output.

[0174] An exemplary embodiment of this disclosure also provides a case quality assessment method, the process of using this method to detect and discover case quality problems as follows: Figure 12 As shown, it includes: Step 1201: Obtain the original case file data.

[0175] The original case file data includes at least one or more of the following: Judgment, court transcript, and evidence materials The original case file data contains at least one or more of the following data formats: Text, images, and videos.

[0176] Step 1202: Generate the case data in structured text form based on the original case file data.

[0177] Step 1203: Search the case data in a preset legal knowledge base to obtain the legal dataset.

[0178] Step 1204: Based on the basic information of the case, associate and output the case data in structured text form with the legal dataset in structured data table form, wherein the basic information includes at least the unique identifier of the case.

[0179] Step 1205: Analyze the case data and legal dataset of the case, and generate sub-tasks corresponding to each of the preset evaluation dimensions according to at least one evaluation dimension.

[0180] Step 1206: Execute the sub-task through multiple agents, and assign one sub-task to each agent.

[0181] Step 1207: Generate and output a unified structured risk consensus based on the reasoning results output by the multiple intelligent agents.

[0182] Step 1208: By constructing a fuzzy evaluation matrix and combining it with a weight vector, calculate the comprehensive risk score vector S for each risk point in the structured risk consensus according to the following expression: , in, Let R be a weight vector of length n, where n is the number of agents participating in the evaluation, and R is the fuzzy evaluation matrix of each agent for each risk point. Let be the confidence level of the i-th agent in judging the j-th risk point.

[0183] Step 1209: Determine the severity of the risk to the quality of the case based on the comprehensive risk scoring vector.

[0184] Step 1210: Extract keywords related to the case based on the case data, the legal dataset, and the structured risk consensus.

[0185] Step 1211: Using the keyword as a forced guiding condition, generate multiple report candidate sequences, and calculate the probability of each report candidate sequence according to the following expression. : , in, For the key operators of legal provisions in the legal dataset, Let L be the semantic similarity function, and L be the length of the target sequence. The generated review report text sequence consists of L tokens. X represents the input data. This refers to the words from the 1st to the (t-1th)th output, that is, the contents of the 1st to the (t-1th)th output units. Given case data X and the generated Given the probability of generating the basic language model for the t-th word, The lexical units generated in step t.

[0186] Step 1212: Select the candidate report sequence with the highest probability value as the evaluation report output.

[0187] The implementation principles of steps 1201 to 1212 have been fully explained in the foregoing embodiments and will not be repeated here.

[0188] An exemplary embodiment of this disclosure also provides a computer apparatus, including: processor; Memory used to store processor-executable instructions; The processor is configured to execute the case quality assessment method provided in the embodiments of this disclosure.

[0189] An exemplary embodiment of this disclosure also provides a non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a computer's processor, enable the computer to perform the case quality assessment method provided by embodiments of this disclosure.

[0190] This disclosure provides a method, computer device, and storage medium for case quality assessment. It parses case data and legal datasets, generates sub-tasks corresponding to each preset assessment dimension, and executes these sub-tasks through multiple agents, assigning one sub-task to each agent. Based on the reasoning results output by the multiple agents, a unified structured risk consensus is generated and output. By analyzing and reasoning about the same case data from different assessment dimensions using multiple agents, and integrating the multiple reasoning results, a unified structured risk consensus is obtained. This method, based on the collaborative completion of assessment tasks by multiple agents, assesses risks from different dimensions to identify them, solving the problem of lacking an accurate and efficient automatic case assessment mechanism, and achieving standardized quality assessment with low cost, high efficiency, and high quality.

[0191] Based on multi-agent technology, criminal cases such as dangerous driving cases are reviewed in a full-process, multi-dimensional manner, enabling efficient processing of extremely long case files and precise coverage of key review points.

[0192] The entire process is automated with no human intervention, except for complex cases where expert intervention is triggered. While maintaining high efficiency through automation, the system ensures reliability by introducing human intervention as needed. This improves both the efficiency and standardization of case quality assessment while guaranteeing the impartiality and professionalism of the results. It is suitable for quality assessment of various types of criminal cases, and is particularly well-suited for the large-scale, standardized assessment needs of common criminal cases such as dangerous driving.

[0193] Those skilled in the art will also understand that the various illustrative logical blocks and steps listed in the embodiments of this disclosure can be implemented by electronic hardware, computer software, or a combination of both. Whether such functionality is implemented in hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functionality using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of this disclosure.

[0194] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”

[0195] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding this specification and the accompanying drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”

[0196] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0197] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for evaluating case quality, characterized in that, include: Analyze the case data and legal dataset of the case, and generate sub-tasks corresponding to each of the pre-defined evaluation dimensions based on at least one pre-defined evaluation dimension; The subtask is executed by multiple agents, and one subtask is assigned to each agent. Based on the reasoning results output by multiple intelligent agents, a unified structured risk consensus is generated and output.

2. The case quality assessment method according to claim 1, characterized in that, The evaluation dimensions include at least one or more of the following: Case filing, evidence, procedures, sentencing, The intelligent agent includes at least one or more of the following: Intelligent agents for case filing review, evidence review, process review, and sentencing review. The step of generating and outputting a unified structured risk consensus based on the inference results output by multiple intelligent agents includes: Obtain the reasoning results of each of the aforementioned agents, wherein the reasoning results include at least one or more of the following: The first inference result output by the intelligent agent for case filing review in identifying compliance risks during case filing is... The evidence review agent performs evidence chain integrity and consistency risk identification, outputting the second reasoning result. The third inference result output by the intelligent agent in evaluating the legality risk identification of litigation procedures during process review. The fourth inference result output by the sentencing review agent in identifying the appropriateness risk of implementing sentencing recommendations. The first inference result, the second inference result, the third inference result, and the fourth inference result have a unified output format, which includes at least one or more of the following information: Risk type, description, elements involved, relevant legal provisions, confidence level, and source of evidence; For the reasoning results based on the same case output by different intelligent agents, retrieve the factual findings to determine contradictions; In the event of a contradiction in factual findings, a weighted vote is performed on the inference results based on the authority weight coefficients of each agent to obtain a unified structured risk consensus. The authority weight coefficients indicate the historical accuracy of the agent's assessments. The structured risk consensus includes at least one or more of the following information: Case unique identifier, risk classification, risk type, description, elements involved, related evidence or legal provisions, and confidence level.

3. The case quality assessment method according to claim 2, characterized in that, After the step of obtaining the reasoning results of each of the intelligent agents, the method further includes: For the inference results output by each of the aforementioned agents, calculate the prediction entropy; If the predicted entropy is higher than a preset risk threshold, risk processing is initiated.

4. The case quality assessment method according to claim 1, characterized in that, Before the step of analyzing case data and legal datasets and generating sub-tasks corresponding to each of the preset evaluation dimensions, the method further includes: Obtain the original case file data, which includes at least one or more of the following data: Judgment, court transcript, and evidence materials The original case file data contains at least one or more of the following data formats: Text, images, and videos; The case data is generated in structured text form based on the original case file data; The legal dataset is obtained by searching a pre-set legal knowledge base based on the case data. Based on the basic information of the case, the case data in structured text form is associated with the legal dataset in structured data table form and output. The basic information includes at least the unique identifier of the case.

5. The case quality assessment method according to claim 4, characterized in that, The step of generating the case data in structured text form based on the original case file data includes: By using overlapping sliding windows, the original case file data is split into paginated case file documents, which contain multiple split text blocks. Information extraction is performed on the case file documents to obtain valid information. The information extraction includes at least one or more of the following processing methods: Handwriting recognition, signature recognition, and text extraction; The extracted valid information is aggregated to generate the case data in structured text form.

6. The case quality assessment method according to claim 5, characterized in that, Handwriting recognition is performed using the following method: The paginated case file documents are preprocessed using normalization to obtain standardized case file images; The standardized file images are divided into image blocks of fixed size; The image patch is converted into a feature vector according to the following expression. : ， in, These are placeholders for categories used to aggregate global features. Let j be the feature vector of the j-th image patch, where j takes values ​​from 1 to N. For position encoding; The feature vector is extracted by a decoder to obtain a long sequence of handwritten semantics; The long program sequence is mapped to a character-level probability distribution, and the character with the highest probability distribution is selected as the recognition result for each character position to generate a handwritten text sequence. The handwritten text sequence is corrected for errors according to the following expression, and the corrected handwritten text sequence O is output: ， in, For regularization items in a legal terminology database, To adjust the parameters, L is the target sequence length, and I is the normalized dossier image. The parameters of the model to be trained, For the actual character at position i, - (·) is a negative logarithmic function. Given a standardized dossier image I and model parameters, In the case of correctly predicting the real character at the i-th position The conditional probability distribution.

7. The case quality assessment method according to claim 6, characterized in that, Signature recognition is performed in the following manner: Based on the feature vector, candidate regions are determined; The candidate region is extracted and mapped to a high-dimensional feature space by a signature feature encoder based on a Siamese network to generate a signature feature vector. The signature feature vector is compared with a preset standard signature sample to obtain the signature feature vector that meets the similarity requirements. The signature recognition result is generated and output based on the signature feature vector and location information that meet the similarity requirements.

8. The case quality assessment method according to claim 4, characterized in that, The step of retrieving the legal dataset from a preset legal knowledge base based on the case data includes: Both the case data and the legal provisions in the legal knowledge base are converted into high-dimensional dense vectors. The similarity between the high-dimensional dense vector of the case data and the high-dimensional dense vector of the legal provisions is calculated, and the legal dataset is generated based on the legal provisions whose similarity reaches the preset legal provision matching conditions.

9. The case quality assessment method according to claim 1, characterized in that, The method further includes: By constructing a fuzzy evaluation matrix and combining it with a weight vector, the comprehensive risk score vector S for each risk point in the structured risk consensus is calculated according to the following expression: ， in, Let R be a weight vector of length n, where n is the number of agents participating in the evaluation, m is the total number of risk points, and R is the fuzzy evaluation matrix of each agent for each risk point. The severity of the risk to the quality of the case is determined based on the comprehensive risk scoring vector.

10. The case quality assessment method according to claim 1, characterized in that, The method further includes: Based on the case data, the legal dataset, and the structured risk consensus, extract keywords related to the case; Using the aforementioned keywords as the mandatory guiding condition, multiple report candidate sequences are generated, and the probability of each report candidate sequence is calculated according to the following expression. : ， in, For the key operators of legal provisions in the legal dataset, Let L be the semantic similarity function, and L be the length of the target sequence. The generated review report text sequence consists of L tokens. X represents the input data. For the 1st to t-1th output words, Given case data X and the generated Given the probability of generating the basic language model for the t-th word, For the t-th word element, Weights are used to convert semantic similarity into probabilities; The candidate report sequence with the highest probability value is selected as the evaluation report output.

11. A computer device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the case quality assessment method as described in any one of claims 1 to 10.

12. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of a computer, the computer is able to perform the case quality assessment method as described in any one of claims 1 to 10.

Images