A multi-modal intelligent recording auxiliary decision-making system and method

Abstract

This application provides a multimodal intelligent record-keeping auxiliary decision-making system and method, relating to the field of text data processing technology. The system adopts a layered architecture design, including: an application role layer, providing system access points for different judicial work participants; an intelligent scenario layer, providing intelligent auxiliary functions around key aspects of judicial business; a core function layer, providing evidence management, record management, AI assistant, configuration center, and system management functions; an intelligent system layer, providing technical support for intelligent system applications based on RAG, cognitive graphs, AIAgent, and multimodal large model technologies; a data system layer, providing data modeling, data resources, and data security functions; and an infrastructure layer, providing intelligent computing, storage, and network hardware support. All the above layers are interconnected through standardized interfaces to ensure the system's stability, scalability, and security.

Description

Technical Field

[0001] This application relates to the field of text data processing technology, and in particular to a multimodal intelligent record-keeping auxiliary decision-making system and method. Background Technology

[0002] The traditional manual record-taking method is no longer sufficient to meet the demands of high-quality judicial work, and has gradually exposed significant compliance risks and efficiency bottlenecks, specifically in three core aspects: First, manual recording is prone to omissions of key case information due to human error, differences in recording speed, and other factors, affecting the completeness of subsequent fact-finding; second, the lack of unified standards in record-keeping, with inconsistent formatting and content rigor, may reduce the legal validity of records due to non-compliance with evidentiary requirements, adversely affecting case adjudication; third, facing a large number of records and related evidence, relying on manual sorting and cross-referencing is not only time-consuming and labor-intensive, but also prone to inefficiency due to the limitations of manual analysis, causing delays in case handling and making it difficult to keep pace with efficient law enforcement.

[0003] Driven by practical needs, how to integrate intelligent technologies into smart record-keeping systems is a pressing technical problem that needs to be solved. Such systems, by integrating artificial intelligence technologies such as natural language processing and speech recognition, can achieve intelligent upgrades to the entire record-keeping process: using standardized template generation functions to unify record formats and core elements, improving production efficiency from the source; using real-time intelligent analysis technology to dynamically verify the compliance and completeness of record content, proactively mitigating evidentiary risks; utilizing intelligent comparison algorithms to quickly locate contradictions between different records, reducing manual review costs; and combining with a full-process evidence database management mechanism to establish a one-to-one correspondence between records and evidence, ensuring the integrity and closed loop of the evidence chain. This type of smart record-keeping system not only provides core technical support for the standardization and normalization of judicial work but also effectively improves the overall effectiveness of supervision. Summary of the Invention

[0004] This application provides a multimodal intelligent record-keeping auxiliary decision-making system and method to achieve at least one of the following objectives: 1) Digital Records: By implementing digital record-keeping functions, it is possible to record record materials more conveniently and efficiently, accurately preserve all key information, form different types of record-themed databases, classify them, and improve the digitization of records.

[0005] 2) Digital Standardized Template Integration: Provides a unified template-based standard record-keeping tool. In accordance with national standards, multiple standardized record templates are first created, and then case handlers use the templates to register and record the records, ensuring the validity of each record and improving the standardization of records.

[0006] 3) Introduction of intelligent comparison and analysis model: By determining a unique topic parameter, the intelligent comparison and analysis model uses this topic parameter as a condition to retrieve all records that match the corresponding topic from the record topic database, and intelligently identifies the content of each record. It compares the content according to the preset comparison parameters, and combines the business experience of case handling personnel to quickly and efficiently extract effective record content, providing an effective auxiliary means for actual case handling.

[0007] Firstly, this application provides a multimodal intelligent record-keeping auxiliary decision-making system, the system adopting a layered architecture design, including: The application role layer provides system access points for different roles involved in judicial work. The intelligent scenario layer is used to provide intelligent auxiliary functions around key aspects of judicial business; The core functionality layer provides evidence management, record management, AI assistant, configuration center, and system management functions. The intelligent system layer is used to provide technical support for the intelligent application of the system based on RAG, cognitive graph, AIAgent, and multimodal large model technology; The data architecture layer provides data modeling, data resources, and data security functions. The infrastructure layer provides hardware support for intelligent computing, storage, and networking.

[0008] Preferably, in the above-mentioned multimodal intelligent record-keeping auxiliary decision-making system, the intelligent system layer includes: The retrieval enhancement generation module combines judicial knowledge base retrieval with large model generation to output logical interpretations. The cognitive graph module is used to construct a knowledge network in the judicial field, presenting knowledge relationships in a graph-like manner. The AIAgent module is used to simulate an intelligent case handling agent, automatically triggering tasks and invoking tools. The multimodal large model module is used to integrate text, voice, and image data types to achieve comprehensive analysis.

[0009] Preferably, in the above-mentioned multimodal intelligent record-keeping auxiliary decision-making system, the core functional layer includes: The evidence database management module is used to uniformly manage evidence data, including evidence to be collected, proven evidence, evidence arrangement, and case file generation. The record management module is used to uniformly manage record functions, including record outline, intelligent addition, element analysis, intelligent Q&A, record comparison, and intelligent record inspection; The AI ​​assistant module provides intelligent reminders, intelligent error correction, knowledge Q&A, and case analysis functions. The configuration center module provides template configuration, case configuration, permission configuration, and Q&A configuration. The system management module provides role management, access control, department management, and job management.

[0010] Preferably, in the above-mentioned multimodal intelligent record-keeping auxiliary decision-making system, the record management module further includes: The intelligent record generation unit is used to automatically supplement record content based on case elements; The intelligent transcript comparison unit is used to identify discrepancies and contradictions in statements across multiple transcripts. The intelligent quality inspection unit for transcripts is used to automatically check for grammatical errors, missing elements, and logical conflicts in transcripts.

[0011] Preferably, in the above-mentioned multimodal intelligent record-keeping auxiliary decision-making system, the system further includes a core algorithm model, which is constructed based on a knowledge fine-tuning training method and includes: The input layer is used to receive big data analysis reports and transcripts. The task conversion module is used to convert raw data and business requirements into task instructions that AI can understand; The Prompt engineering module is used to guide model output through standardized prompt templates; The transcript analysis model is used to receive prompt templates and raw text, and to perform complex text analysis. The structured output module is used to convert analysis results into a standardized output format.

[0012] Preferably, in the above-mentioned multimodal intelligent record-keeping auxiliary decision-making system, the training data samples of the record-keeping analysis model are in input-output pair format, wherein: The input is a combination of knowledge fragments and original text, wherein the knowledge fragments are knowledge retrieved from the judicial knowledge base that is related to the content of the current transcript; The output is structured content in JSON or Markdown format.

[0013] Preferably, in the above-mentioned multimodal intelligent record-keeping auxiliary decision-making system, the system adopts a microservice architecture, including: The user access layer provides the web access point and load balancing. The gateway layer is used for request routing, rate limiting, authorization verification, and filtering. The service registration and configuration layer is used for microservice registration and configuration management. The microservice layer includes evidence services, record-keeping services, Office services, and model services; The data storage layer includes Redis, MySQL, OSS, and file storage.

[0014] Secondly, this application provides a multimodal intelligent record-keeping auxiliary decision-making method, based on the multimodal intelligent record-keeping auxiliary decision-making system described above, the method comprising: Receive case information and initiate the case process; Collect evidence data and conduct intelligent review; Transcript analysis model based on multimodal data and knowledge enhancement generates and compares transcripts; Automatically generate case summary materials and reports; Integrate evidence, transcripts, and report data to generate electronic case files.

[0015] Preferably, in the above-mentioned multimodal intelligent record-based decision-making assistance method, the steps of generating and comparing records include: Generate subject identity records based on big data analysis reports; Use a template-based approach to add previous records; Intelligent referencing, evidence comparison, text correction, and element analysis are introduced into the process of record making; Multiple transcripts were compared horizontally and vertically to identify points of conflict.

[0016] Preferably, in the above-mentioned multimodal intelligent record-keeping auxiliary decision-making method, the method further includes: The transcript analysis model is trained by using a knowledge-enhanced fine-tuning method, enabling the model to combine relevant knowledge fragments retrieved from the knowledge base during the analysis process and output structured analysis results.

[0017] The multimodal intelligent record-keeping auxiliary decision-making system and method provided in this application have at least the following beneficial effects: 1) Improved efficiency throughout the entire process: The efficiency of record making is greatly improved through a one-stop closed loop of case-evidence-record-report.

[0018] 2) AI Intelligent Assistance: By reshaping the traditional record-taking mode through artificial intelligence, it achieves full coverage of core functions such as case management, intelligent record generation, evidence association, multi-dimensional intelligent comparison, case fact summary, and automatic compilation of electronic case files.

[0019] 3) Compliance and efficiency assurance: AI is used to automatically compare contradictions in the transcripts, reduce evidence risks, and improve the accuracy of case handling.

[0020] 4) Comprehensive transformation of business model: Break away from the traditional paper-and-pen + manual model, realize online record-keeping of the entire case handling process, and adapt to the requirements of information technology construction. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] Figure 1 A structural diagram of a multimodal intelligent record-keeping auxiliary decision-making system provided in an embodiment of this application; Figure 2 This application provides a structural diagram of the intelligent system layer in a multimodal intelligent record-keeping auxiliary decision-making system. Figure 3 A structural diagram of the core functional layer in a multimodal intelligent record-keeping auxiliary decision-making system provided in this application embodiment; Figure 4 This application provides a structural diagram of a record management module in a multimodal intelligent record-based decision-making system. Figure 5 This application provides an overall architecture diagram of a multimodal intelligent record-keeping auxiliary decision-making system. Figure 6 A technical architecture diagram of a multimodal intelligent record-keeping auxiliary decision-making system provided in this application embodiment; Figure 7 A business process diagram of a multimodal intelligent record-keeping auxiliary decision-making system provided in this application embodiment; Figure 8 A data processing flowchart of a multimodal intelligent record-keeping auxiliary decision-making system provided in this application embodiment; Figure 9 A flowchart illustrating the core algorithm model provided in the embodiments of this application; Figure 10 A flowchart illustrating a multimodal intelligent record-keeping auxiliary decision-making method provided in this application embodiment; Figure 11 This is a flowchart illustrating the generation and comparison of transcripts in a multimodal intelligent transcript-assisted decision-making method provided in an embodiment of this application.

[0023] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0025] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0026] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0027] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings. Example 1

[0028] This application provides a multimodal intelligent record-keeping auxiliary decision-making system. For example... Figure 1 As shown, this multimodal intelligent record-keeping auxiliary decision-making system adopts a layered architecture design, including an application role layer 100, an intelligent scenario layer 200, a core function layer 300, an intelligent system layer 400, a data system layer 500, and an infrastructure layer 600. Specifically, the application role layer 100 provides system access for different judicial work participants; the intelligent scenario layer 200 provides intelligent auxiliary functions around key aspects of judicial business; the core function layer 300 provides evidence management, record management, AI assistant, configuration center, and system management functions; the intelligent system layer 400 provides technical support for the system's intelligent applications based on RAG, cognitive graphs, AI agents, and multimodal large model technologies; the data system layer 500 provides data modeling, data resources, and data security functions; and the infrastructure layer 600 provides intelligent computing, storage, and network hardware support.

[0029] In some embodiments, such as Figure 2 As shown, the intelligent system layer 400 includes a retrieval enhancement generation module 401, a cognitive graph module 402, an AIAgent module 403, and a multimodal large model module 404. The retrieval enhancement generation module 401 combines judicial knowledge base retrieval with large model generation to output a logical interpretation. The cognitive graph module 402 constructs a knowledge network in the judicial field, presenting knowledge relationships in a graph-like manner. The AIAgent module 403 simulates intelligent case handling agency, automatically triggering tasks and calling tools. The multimodal large model module 404 integrates text, voice, and image data types to achieve comprehensive analysis.

[0030] In some embodiments, such as Figure 3 As shown, the core functional layer 300 includes an evidence library management module 301, a record management module 302, an AI assistant module 303, a configuration center module 304, and a system management module 305. The evidence library management module 301 is used to uniformly manage evidence data, including evidence to be collected, proven evidence, evidence arrangement, and case file generation. The record management module 302 is used to uniformly manage record functions, including record outlines, intelligent addition, element analysis, intelligent Q&A, record comparison, and intelligent record inspection. The AI ​​assistant module 303 provides intelligent reminders, intelligent error correction, knowledge Q&A, and case analysis functions. The configuration center module 304 provides template configuration, case configuration, permission configuration, and Q&A configuration. The system management module 305 provides role management, permission management, department management, and position management.

[0031] In some embodiments, such as Figure 4 As shown, the record management module 302 further includes a record intelligent generation unit 3021, a record intelligent comparison unit 3022, and a record intelligent quality inspection unit 3033; wherein, the record intelligent generation unit 3021 is used to automatically supplement the record content based on case elements; the record intelligent comparison unit 3022 is used to identify differences and contradictions in statements among multiple records; and the record intelligent quality inspection unit 3033 is used to automatically check for grammatical errors, missing elements, and logical conflicts in the records.

[0032] In some embodiments, such as Figure 5 The diagram shown is the overall architecture of the multimodal intelligent record-keeping auxiliary decision-making system. The following will combine... Figure 5 This section details the specific functions and implementation process of each layer of the system. The system adopts a layered architecture design, integrating domestically developed technology stacks and intelligent technology capabilities to achieve efficient support for the entire lifecycle management of record keeping. The architecture, from top to bottom, is divided into application roles, intelligent scenarios, core functions, intelligent systems, data systems, and infrastructure. Each layer interacts through standardized interfaces to ensure the system's stability, scalability, and security.

[0033] The application role layer 100 covers judicial work participants such as case handlers, adjudicators, and management personnel, and is the system's service target; among which: Case handling personnel: Frontline enforcers in case handling need to use the system to carry out practical work such as evidence collection and record making, and rely on functions such as intelligent profiling of individuals and intelligent evidence management to assist in the case handling process.

[0034] Adjudicators: Focusing on the case adjudication process, they conduct in-depth analysis of the facts and evidence logic of the case through intelligent element analysis and AI intelligent assistants to ensure the quality of adjudication.

[0035] Management: From a management perspective, leveraging functions such as intelligent fact-finding and intelligent conflict comparison, we grasp the overall situation of the case, identify key decision-making issues, and coordinate the case's progress. Intelligent Scenarios: The Intelligent Scenario Layer 200 focuses on key aspects of judicial operations, providing intelligent auxiliary functions such as intelligent profiling and element analysis for different application roles. It empowers traditional case handling processes with technology, creating intelligent application scenarios and facilitating business operations. Among these: Intelligent Person Profile: Integrates multi-dimensional information about individuals (identity, relationships, historical behavior, etc.) to build a visualized and characteristic person profile, assisting in the rapid identification of key personnel characteristics.

[0036] Intelligent evidence management: covering the entire evidence lifecycle (pending collection → proven evidence → arrangement → case file generation), using intelligent means to improve the efficiency of evidence collection, verification, and organization, and ensure the integrity of the evidence chain.

[0037] Intelligent element analysis: It provides a detailed introduction to the core elements of a case (time, place, behavior, responsibility, etc.), uses AI to sort out the logical relationships between the elements, and uncovers potential connections and loopholes to support the determination of facts.

[0038] AI Intelligent Assistant: As a "digital assistant for case handling", it provides functions such as intelligent Q&A (legal / procedural consultation), material summary, and text correction, reducing the cost of repetitive work.

[0039] Intelligent Fact Summary: Automatically extracts key facts of a case, outlines the context, and transforms scattered evidence and transcripts into clear factual statements, assisting in the drafting of trial reports and other documents.

[0040] Intelligent contradiction comparison: accurately identifies contradictions in evidence and conflicting claims in cases, marks differences and analyzes their causes, helps to focus on the core of the dispute and improve the pertinence of the trial.

[0041] Intelligent case file generation: Based on evidence, transcripts and other materials, it automatically typesets and integrates to generate standardized case files, replacing tedious manual organization and ensuring that case files are standardized and consistent.

[0042] Intelligent record creation: Supports rapid record generation (outline guidance, intelligent addition) and intelligent quality inspection (error correction, citation management), improving the efficiency and quality of record production.

[0043] The core functionality layer 300 includes multi-dimensional functions such as evidence library management and record management, covering the entire process of evidence and record processing. Detailed descriptions are as follows: (1) Evidence database management: Unified management of evidence data.

[0044] Evidence to be collected: Record the list of evidence to be collected and verified, track the collection progress, and ensure that no evidence is missed.

[0045] Verified evidence: Stores evidence whose validity has been verified, connects it to the facts of the case, and forms the basic material for the chain of evidence.

[0046] Evidence arrangement: Sort and group evidence according to the logic of the case (time, cause and effect, etc.), organize the order of evidence presentation, and assist in court hearings and trial statements.

[0047] Case file generation: After summarizing and organizing the evidence, it automatically generates a standardized case file document (including a table of contents and evidence attachments) to meet the needs of archiving and retrieval.

[0048] (2) Record management: Unified management of record functions.

[0049] Transcript outline: Pre-set standardized transcript templates (such as interrogation and inquiry outlines) to guide case handlers to cover key questioning points and avoid omissions.

[0050] Intelligent addition: Based on case elements, automatically supplement the transcript content (such as related evidence information and key points of historical transcripts) to reduce duplicate entry.

[0051] Element analysis: Extract core elements (people, actions, time) from the transcript, verify the completeness and logical rationality of the elements, and assist in fact verification.

[0052] Intelligent Q&A: Built-in legal knowledge Q&A database allows for quick access to answers to procedural and legal questions encountered during case handling, replacing manual searches.

[0053] Disputed Facts and Behaviors Record: Focusing on the record of facts in a case, accurately capturing descriptions of disputed facts and behaviors and evidence connections to support qualitative analysis.

[0054] Excellent Records: Compilation of benchmark record cases for case handling personnel to learn from and refer to, standardizing record production standards.

[0055] Transcript comparison: Cross-compare the contents of multiple transcripts (transcripts from different stages of the same case and from different people) to identify discrepancies and contradictions in the statements.

[0056] Comprehensive transcript: Integrates scattered transcript fragments to generate a complete record of the case process, clearly presenting the whole picture of the event.

[0057] Intelligent Record Inspection: AI automatically checks records (grammatical errors, missing elements, logical conflicts) and outputs quality inspection reports to improve record quality.

[0058] Answering citations: When citing evidence or regulations in the transcript, the source of the material is automatically linked to ensure that the citation is traceable and accurate.

[0059] Material summary: Extract the core information from the transcript (demands, facts, disputes) and generate a concise version of the material, suitable for management to quickly review and approve.

[0060] Text correction: Covers typos, grammatical errors, ambiguous expressions, etc., automatically marks them and suggests corrections to optimize the quality of transcript text.

[0061] (3) AI Assistant: Use AI capabilities to provide convenient assistant functions.

[0062] Intelligent reminders: Based on case progress and process nodes (such as evidence submission deadlines and trial time limits), push reminders for tasks to be completed to avoid missing key actions.

[0063] Intelligent error correction: Covers logical errors in evidence, records, and documents (such as time conflicts and contradictory evidence), marks risk points, and prompts for correction.

[0064] Knowledge Q&A: Construct a judicial knowledge graph (laws, procedures, cases) and use natural language Q&A format to quickly answer business questions.

[0065] Case Analysis: Input case information and automatically match similar cases (cases of the same type and with the same elements), referencing past handling approaches and judgment results to assist in decision-making.

[0066] (4) Configuration Center: Provides basic standardized configuration management.

[0067] Template configuration: Customize templates for transcripts, documents, evidence lists, etc., to adapt to different case types and regional regulations, ensuring standardized output.

[0068] Case configuration: Configure the activation of functional modules and process nodes according to the nature of the case (case facts, disputed facts, crimes, etc.) and complexity to flexibly adapt to business needs.

[0069] Permission configuration: refine role permissions (e.g., case handlers can only edit records, while management can view the entire process) to ensure data security and clear responsibilities.

[0070] Question and Answer Configuration: Maintain the AI ​​assistant's knowledge question and answer database, supporting the addition and modification of question and answer entries, and dynamically updating business knowledge.

[0071] (5) System Management: Provides comprehensive system functionality assurance.

[0072] Role management: Define operational roles within the system (case handling, adjudication, management), associate functional permissions, and adapt to the organizational structure.

[0073] Access control: Detailed access control down to function buttons and data fields (e.g., some users can only view evidence but cannot modify it) to ensure data security.

[0074] Departmental Management: The scope of system use is divided according to relevant institutions and departments, and cross-departmental collaboration and data isolation are managed.

[0075] Job Management: Link jobs with system functions to achieve precise matching of "jobs and functions".

[0076] The intelligent system layer 400 relies on technologies such as RAG and cognitive graphs to construct multimodal large models, providing technical support for intelligent system applications. Among these: RAG (Retrieval Enhanced Generation): Combining judicial knowledge base retrieval with large-scale model generation, when answering case questions, it not only accurately cites laws and cases, but also outputs logical interpretations.

[0077] Cognitive Graph: Constructing a knowledge network in the judicial field (legal connections, case relationships, and evidentiary logic), presenting knowledge connections in a graph-like manner to assist in complex factual reasoning.

[0078] AIAgent: Simulates an "intelligent case handling agent" to automatically trigger tasks (such as evidence expiration reminders and transcript quality inspections) and call tools (legal searches and evidence verification) to process business processes in a closed loop.

[0079] MCP Square: It brings together judicial scenario models and tools (such as evidence analysis models and transcript generation tools) to form a "model application marketplace" that can be called and combined as needed.

[0080] Computing power scheduling: Intelligent allocation of hardware computing power (GPU / CPU) to ensure efficient operation of multiple tasks (simultaneous evidence analysis and transcript generation) and balance system load.

[0081] Model Management: Manage judicial AI models throughout their entire lifecycle (training, deployment, iteration), monitor model performance, and ensure accurate output.

[0082] Task awareness: Identify hidden tasks in the case handling process (such as the need to complete the chain of evidence or verify contradictions) and automatically trigger system function responses.

[0083] Large Model: Based on general / specialized large models, it handles complex tasks such as natural language understanding (semantic analysis of transcripts) and text generation (document writing).

[0084] Comprehensive data collection: Collects judicial data (case processing system, court recordings, and document archives) across systems and terminals, breaking down data silos and enriching training materials.

[0085] Data augmentation: For scarce, low-quality data (such as transcripts of niche cases), use generation and annotation augmentation techniques to improve data quality and support model training.

[0086] Multimodal: Integrating data types such as text (transcripts, documents), voice (conversations, search recordings), and images (evidence photos), and using multimodal models to achieve comprehensive analysis (such as audio transcription + semantic analysis).

[0087] Data governance: Cleaning, labeling, and de-identifying judicial data; standardizing data formats and quality; and ensuring the accuracy and security of AI model training and business applications.

[0088] The data system layer 500 focuses on data value mining, from modeling to security, and constructs a closed-loop management system for judicial data, detailed as follows: (1) Data modeling: Standardized data modeling capabilities.

[0089] Modal processing: Adapting multimodal data such as text, images, and audio to convert them into formats that the model can process (such as text segmentation and image feature extraction).

[0090] Feature modeling: Extracting core features of judicial data (case type features, evidence correlation features) to provide "data tags" for AI analysis.

[0091] Semantic extraction: Mining semantic information from text data (such as behavioral intent and logical relationships of evidence in transcripts) and transforming it into structured knowledge.

[0092] Tag generation: Automatically label data (e.g., "sufficient evidence") to facilitate data classification and retrieval, and model training.

[0093] Vector encoding: Converting judicial elements (people, regulations) into vector form and using vector similarity calculation to achieve intelligent matching (such as similar case retrieval).

[0094] Data cleaning: Remove data noise (duplicate records, erroneous evidence), correct format and content errors, and ensure data quality.

[0095] (2) Data resources: Unified and collaborative management of data resources.

[0096] Behavior log: Records user system operation trajectory (such as evidence upload time, transcript modification record), used to analyze case handling process and optimize function design.

[0097] Evidence Resources: Store all case evidence (electronic documents and images), link it to case information, and form an evidence asset library.

[0098] Q&A Knowledge: Accumulate legal Q&A data (frequently asked questions + standard answers) to continuously enrich the knowledge base of the AI ​​assistant.

[0099] Transcript corpus: Accumulate a massive amount of transcript texts as training material for the model to optimize the performance of transcript generation and analysis functions.

[0100] Case template: Extract the framework of benchmark cases (facts, evidence, and judgment logic) for reference and reuse in similar cases, and standardize the case handling approach.

[0101] (3) Data security: Provide secure data management.

[0102] Access control: Data access permissions are refined based on roles and positions (e.g., management can view all cases, while case handlers can only view their own cases) to prevent data leakage.

[0103] Metadata: Manage data "descriptive information" (such as evidence creation time and the case to which the transcript belongs) to support data retrieval, tracing and management.

[0104] Audit trail: Records the entire process of data operation (who, when, and what was done) for security auditing and accountability.

[0105] De-identification and compliance: Sensitive data (personal privacy, classified information) is automatically de-identified (e.g., the last four digits of the ID card are hidden) to ensure that the use of data is legal and compliant.

[0106] Quality monitoring: Real-time monitoring of data quality (completeness, accuracy), early warning of abnormal data (such as missing evidence, incorrect transcript format), and promotion of data governance.

[0107] Standards Catalog: Formulate standardized specifications for judicial data (formats, field definitions), unify data standards, and improve cross-system compatibility.

[0108] The infrastructure layer 600, based on Ascend and other technologies, provides hardware and network support for intelligent computing, storage, and other aspects to ensure system operation. Among these: Ascend: Huawei's Ascend chip / computing architecture provides the foundation for AI computing power, supporting large-scale model training and inference tasks.

[0109] Intelligent computing: By using intelligent scheduling algorithms, CPU / GPU resource allocation is optimized to improve the efficiency of judicial data processing and model operation.

[0110] Lossless network: Ensures low latency and high reliability of data transmission, avoiding loss or errors in the transmission of critical data such as evidence and transcripts.

[0111] AI Accelerator Card: Dedicated hardware to accelerate AI tasks (such as large model inference and image recognition), shorten computation time, and improve system response speed.

[0112] AI Module: Pre-installed general AI function modules (such as text classification and image recognition) for rapid integration into judicial business scenarios.

[0113] Intelligent edge computing: Deploy edge computing power at case processing terminals (such as mobile law enforcement equipment) to enable rapid local data processing (such as intelligent analysis of on-site records) and reduce reliance on the cloud.

[0114] High-performance storage: Provides large-capacity, high-speed read and write storage resources to securely store core data such as evidence, transcripts, and models, ensuring efficient data retrieval.

[0115] In some embodiments, the system adopts a microservice architecture, including a user access layer, a gateway layer, a service registration and configuration layer, a microservice layer, and a data storage layer; wherein, the user access layer provides a web access entry point and load balancing; the gateway layer is used for request routing, rate limiting, permission verification, and filtering; the service registration and configuration layer is used for microservice registration and configuration management; the microservice layer includes evidence service, record service, Office service, and model service; the data storage layer includes Redis, MySQL, OSS, and file storage.

[0116] For example, such as Figure 6 The diagram shown is a technical architecture diagram of a multimodal intelligent record-keeping auxiliary decision-making system provided in an embodiment of this application. The overall technical architecture of the system is mainly developed using a microservice architecture on the back end, adopting a front-end and back-end separation, while the front end uses web technology. The system framework is mainly based on B / S, i.e., accessed via a browser. It mainly consists of the following: 1) User access layer: Provides the user system access point.

[0117] Web: The user operates the terminal (browser / client) to initiate system access requests (such as case inquiry, record entry), which is the human-computer interaction entry point.

[0118] Load balancing layer (NGINX): As a reverse proxy and load balancer, it receives web requests and distributes the traffic to multiple gateways to avoid single point of pressure and improve system throughput and availability (e.g., 100 user requests are evenly distributed to 5 gateways for processing).

[0119] 2) Gateway Layer: The system's "traffic gatekeeper," preprocessing requests to ensure security and compliance. Routing: Based on the request path (such as " / Evidence Management"), accurately forward the request to the corresponding backend microservice (Evidence Management Service).

[0120] Rate limiting: Limit the number of requests per unit of time (e.g., a maximum of 1000 requests per second) to prevent sudden traffic surges from overwhelming the system.

[0121] Permission verification: Check user tokens and role permissions (e.g., ordinary case handlers cannot access the management decision-making module), and block unauthorized requests.

[0122] Request filtering: Filters illegal requests (such as SQL injection and malicious parameters) to ensure the security of backend services.

[0123] 3) Service Registration and Configuration Layer (NACOS): Provides registration center and configuration center management capabilities. Registration Center: When a backend microservice (server) starts up, it actively "registers" with the registration center so that the system knows "which services are available"; the gateway and other services discover and call services through the registration center (similar to a "service address book").

[0124] Configuration Center: Centrally manages microservice configurations (such as database connections and log levels), automatically pulls the configuration when the service starts, and realizes "configuration modification without restarting the service" (for example, adjusting the evidence storage path without restarting the server one by one).

[0125] 4) Microservice layer (intermediate server cluster): provides microservice capabilities for the cluster.

[0126] Architecture pattern: The system adopts microservice decomposition, splitting the system into multiple independent servers (such as evidence service and transcript service). Each service focuses on a single business ("small and specialized"). FeginRibbon is used to realize remote calls between services and load balancing (for example, when the evidence service calls the transcript service, Ribbon selects the least idle transcript service instance).

[0127] Business microservices (server): These carry core judicial business (evidence management, transcript generation, etc.) and are developed, deployed, and scaled independently.

[0128] Office services: Provide online document editing, text optimization, and other functions.

[0129] Model services: Provide model management, scheduling, and execution capabilities.

[0130] 5) Service Governance and Communication (FeginRibbon): Provides unified communication, monitoring, and scheduling capabilities between services. Fegin simplifies HTTP calls between microservices, allowing services to call other services as if they were calling local methods (e.g., the evidence service uses Fegin to call the "transcription service to get the transcript list"), reducing development complexity.

[0131] Ribbon: Built-in load balancing strategies (round-robin, weighted). When a service is called, it selects the "most suitable" instance from the service instance list in the registry to send the request (e.g., prioritize servers with fast response times) to avoid overloading a single service instance.

[0132] 6) Data storage layer: Unified data storage, supporting both structured and unstructured data. Redis: An in-memory database used as a cache to store frequently accessed data (such as the latest case status and user permissions), accelerating reads (several times faster than querying MySQL) and reducing database pressure.

[0133] MySQL: A relational database that persistently stores structured business data (such as evidence information, transcripts, and user accounts), ensuring reliable data storage.

[0134] OSS (Object Storage Service): Stores large, unstructured files (such as evidence photos and court trial videos), supports massive file fragment storage and fast access, and is more efficient than storing directly in a database.

[0135] File storage (log management): Archive system operation logs (who did what and when), used for troubleshooting, auditing and tracing, and ensuring system maintainability.

[0136] In some embodiments, the business process of this multimodal intelligent record-keeping auxiliary decision-making system is provided. This system constructs a closed-loop management system around the entire lifecycle of records, deeply integrates relevant case handling processes, and reshapes the traditional record-keeping model through digital means. It creates a one-stop online platform for case management, evidence association, record creation, and case file archiving, covering core functions such as standardized record generation, multi-dimensional intelligent comparison, case summary and extraction, and automatic electronic case file compilation. This facilitates the digital transformation of judicial business and significantly improves case handling efficiency. Figure 7 As shown, the business process of this multimodal intelligent record-keeping auxiliary decision-making system includes the following steps S01-S04.

[0137] S01: New case.

[0138] Case management is used for comprehensive management of case information. Case information includes key details such as case number, relevant personnel, affiliated unit, handling department and personnel, case name, case type, preliminary review and filing time, transfer for trial time, case processing time, and case stage. The system also provides a window for uploading attachments such as preliminary review plans and summaries of problem clues to achieve centralized storage and management of case-related materials.

[0139] Starting with new cases, two parallel tracks were simultaneously launched: evidence chain formation and record making.

[0140] S02: Digital management of the chain of evidence.

[0141] The evidence repository management system is used for full lifecycle management of evidence in a case, mainly including three core modules: evidence to be proven, evidence already proven, legal documents, and case files. The system displays evidence according to a list of disputed facts and supports functions such as evidence review, viewing, resubmission, and export. Simultaneously, evidence is automatically associated with index records, enabling the presentation and comparison of evidence. Step S02 includes the following steps S021 and S022.

[0142] S021: Upload evidence.

[0143] According to the investigation and review measures, the case handling personnel upload the corresponding legal documents and evidence, and link them to the disputed facts.

[0144] S022: Intelligent Evidence Review.

[0145] Evidence reviewers examine and process evidence based on its standardization and compliance.

[0146] S03: Record Management.

[0147] Transcript management is the core function of the intelligent transcript system, used for the creation, comparison, and summarization of transcripts. It covers various business scenarios such as transcripts of subject identity, disputed facts, comprehensive transcripts, previous transcripts, transcript creation, transcript comparison, summary materials, and report generation. Step S03 includes the following steps S031-S038.

[0148] S031: Import big data analysis report.

[0149] The system combines a large model to extract key personnel information from the analysis report, generates a profile of the person, and intelligently establishes the subject's identity through written records.

[0150] S032: New disputed facts.

[0151] It supports the creation of disputed facts involving relevant personnel, including title, nature, case type, and overview. Case handlers can easily view the basic information of the disputed facts, related evidence, and case file information, providing comprehensive factual basis and background support for subsequent record-keeping.

[0152] S033: Add previous records.

[0153] Based on the disputed facts, the record-keeping process can begin with a combination of "templates, reuse of best practices, and perspective shifts," thus meeting the practical needs of case handlers with varying levels of experience.

[0154] S034: Record taking.

[0155] The system supports the use of artificial intelligence technology to improve the efficiency of record making, serving the entire record making process. It combines various intelligent tools such as text correction, transcription, element analysis, and evidence comparison to complete intelligent record making, which can effectively reduce the difficulty of record making and reduce problems in the records.

[0156] S035: Intelligent comparison of transcripts.

[0157] The comparison function covers horizontal comparison between different record objects and vertical comparison of different record objects across different records. By intelligently analyzing the contradictions in case elements in different records, it provides detailed comparison results for case handlers and supports tracing the original text of contradictory content, making it easier for case handlers to fully grasp the details of the case and providing support for in-depth investigation and trial of the case.

[0158] S036: Upload the signed transcript.

[0159] Once the transcript is completed and signed, it supports uploading the signed transcript and related documents. The signed transcript will be synchronized to the evidence database management for compliance review.

[0160] S037: Summary of intelligent generation of materials.

[0161] By using AI to intelligently analyze case elements, evidence, and related records, the system automatically extracts core facts and the closed-loop logic of the evidence chain, forming a summary of disputed facts that conforms to the elements constituting a crime.

[0162] S038: Intelligent generation of case reports.

[0163] Integrate data from the entire process, including evidence, records, and case elements, and combine this with laws, regulations, and rules to output standardized report frameworks such as investigation reports and indictment opinions.

[0164] S04: Intelligent generation of electronic case files.

[0165] The system utilizes a multi-layered data-driven approach, combining evidence, transcripts, and reports with automated compilation tools to generate case files by categorizing, organizing, and rearranging information according to preset rules. This allows case handlers to access complete case files at any time, ensuring the orderly storage and transfer of case evidence.

[0166] In some embodiments, the data processing flow of this multimodal intelligent record-keeping auxiliary decision-making system is provided. For example... Figure 8 As shown, the system focuses on the entire process of a case from "adding" to "archiving the case file," using intelligent methods to connect case management, evidence processing, record making, and case file generation. The following is a detailed introduction according to the process nodes: 1) Case Initiation: Case Management, a unified case management module Add a new case: Users enter basic case information (parties, cause of action, etc.), and the case data is stored in the case database as the starting point for subsequent processes.

[0167] Case Inquiry: Through the case database, you can search for historical / current case files, access basic case information, and connect related data such as evidence and transcripts.

[0168] 2) Evidence Management: From Collection to Storage New evidence is divided into "Evidence pending review" (newly collected, unverified) and "Evidence reviewed" (verified and entered the process). Finally, the reviewed evidence is stored in the evidence database.

[0169] Evidence retrieval: Retrieve evidence from the evidence database, supporting evidence retrieval and verification in cases (such as quickly accessing tried evidence during court hearings).

[0170] 3) Record Management: Creation → Intelligent Assistance → Storage Main process flow: Intelligent generation of subject identity records: Based on big data analysis, automatically generate a framework of records related to the identity of the parties involved (such as basic information and related connections).

[0171] Add previous records: Record records of different stages of a case (interrogation, questioning), support adding templates (quickly create records using standardized templates), referencing excellent records (reusing high-quality content from the past), and intelligently generating records from different perspectives (switching and supplementing from the perspective of relevant personnel / witnesses).

[0172] Record keeping: Combining model capabilities (intelligent referencing of previous records, intelligent comparison of presented evidence, etc.) to assist case handlers in improving records, which are then stored in the record database.

[0173] Intelligent transcript comparison: Cross-check multiple transcripts (different stages of the same case, different people) to identify contradictions and logical loopholes in statements.

[0174] Upload signed transcript: After the transcript is signed and confirmed by the party concerned, it is updated to the transcript database and marked as "confirmed".

[0175] 4) Model Capability: An Intelligent Engine Throughout the Process The model incorporates four core intelligent assistance mechanisms to improve efficiency in the evidence and record-keeping processes: Intelligent referencing of previous transcripts: When creating a new transcript, it automatically links to the content of previous transcripts (such as previous statements by relevant personnel) to help quickly supplement information.

[0176] Presenting Evidence / Intelligent Evidence Comparison: When presenting evidence, automatically compare the evidence chain logic (such as whether the timeline conflicts) and point out contradictions.

[0177] Intelligent text correction: Checks for textual errors (typos, grammar) and omissions in expression (such as missing key information) in the transcript, and outputs correction suggestions.

[0178] Intelligent element analysis: Extracts core elements (time, place, behavior) from records / evidence, sorts out the relationships between elements, and assists in the determination of facts.

[0179] 5) Case file generation: Case closing and archiving Intelligent case file generation: Integrates case data (evidence, transcripts, reports), automatically arranges and generates standardized case files, and stores them in the case file database.

[0180] Search case files: Retrieve generated case files from the case file database for archiving, access, and review (such as for inspections by higher authorities or case reviews).

[0181] 6) Additional intelligent assistance: Materials and reports Intelligent summary generation: Automatically extracts key case information (facts, evidence, conclusions) and generates concise summaries (such as case abstracts).

[0182] Intelligent case report generation: Based on evidence and records, automatically generate case reports (such as trial reports and case closure reports), reducing the cost of manual drafting.

[0183] 7) Data closed loop: Inter-database linkage and updates The system comprises three major databases: a case database (basic information), an evidence database (evidence throughout the entire process), a transcript database (transcripts from each stage), and a case file database (final archiving). Data flow is achieved through "query-update" operations, supporting full lifecycle management of cases (adding → processing → archiving → querying).

[0184] In some embodiments, the system further includes a core algorithm model, which is constructed based on a knowledge-fine-tuning training method and includes: The input layer is used to receive big data analysis reports and transcripts. The task conversion module is used to convert raw data and business requirements into task instructions that AI can understand; The Prompt engineering module is used to guide model output through standardized prompt templates; The transcript analysis model is used to receive prompt templates and raw text, and to perform complex text analysis. The structured output module is used to convert analysis results into a standardized output format.

[0185] In some embodiments, the training data samples of the transcript analysis model are in input-output pair format, wherein: The input is a combination of knowledge fragments and original text, wherein the knowledge fragments are knowledge retrieved from the judicial knowledge base that is related to the content of the current transcript; The output is structured content in JSON or Markdown format.

[0186] In some embodiments, such as Figure 9The diagram shown is a flowchart of the core algorithm model. This system is based on a large language model (LLM) intelligent model for judicial records. Its core function is to use AI to extract key information from records and reports, achieving automated and structured processing. Each step is described in detail below: 1) Input layer: Data source.

[0187] Big data analysis reports and transcripts: As raw materials, these include unstructured texts such as case facts, statements from parties involved, and descriptions of evidence (e.g., an interrogation transcript that records the time and behavior of relevant personnel's statements).

[0188] The dashed box indicates expanded tasks, such as "generating subject identity records and comparing records," which means that more judicial scenario tasks can be accessed (automatic extraction of identity information and verification of contradictions among multiple records), enriching the analysis dimensions.

[0189] 2) Task transformation: Intelligent analysis of tasks.

[0190] Transforming "raw data + business needs" into task instructions that AI can understand (such as "extracting the time, place, and tools of the relevant person's crime from the transcript") and clarifying the analysis objectives is a bridge connecting business and technology.

[0191] 3) Prompt Project: Prompt Template.

[0192] Transform the "intelligent analysis task" into a prompt that the large language model can understand, and guide the model to output accurately through standardized templates (such as "Please extract key information from the following transcript in the format of [time, place, person]: xxx transcript content").

[0193] Purpose: To enable AI to understand "what needs to be done and what output format to use", solving the problem of large models "understanding business needs" (for example, judicial scenarios require strict extraction according to evidence standards, and templates will embed such requirements).

[0194] 4) Transcript Analysis Model: Core AI Engine.

[0195] It receives "prompt template + original text" and, relying on the model's language understanding and generation capabilities, processes complex text analysis (such as identifying implicit logical relationships and contradictions in transcripts).

[0196] Key value: By using pre-trained knowledge and reasoning ability, it breaks through the limitations of traditional rule engines (for example, without hardcoding the "crime time extraction rule", the model can automatically identify from the ever-changing records).

[0197] 5) Structured output: Output structured results.

[0198] Structured output: Convert the model analysis results into standardized formats (tables, JSON).

[0199] Key information extraction: Screen out core data (evidence elements, points of conflict, and relationships between people) that are valuable for handling judicial cases, and use them directly in case handling (such as court hearings and report writing).

[0200] 6) Process value: AI application in judicial scenarios.

[0201] This process is essentially a closed loop of "business needs → AI understanding → structured results," which solves the pain points in judicial work such as "large volume of transcripts / reports, low efficiency of manual extraction, and easy omission of key information." It allows AI to truly assist in case handling (such as quickly extracting the timeline of the crime and contradictory evidence from hundreds of pages of transcripts), thereby improving the efficiency and accuracy of case handling.

[0202] 7) Core algorithm model.

[0203] While directly applying the Large Language Model (LLM) to the field of intelligent record analysis has a certain text understanding capability, it usually has some shortcomings, such as a lack of domain expertise, low credibility of analysis conclusions, and uncontrollable output format.

[0204] To address the above issues, this embodiment proposes a knowledge-based fine-tuning training method that allows the model to "see" domain knowledge during the learning process, making its reasoning and judgments more evidence-based and enabling it to act more like a relevant expert rather than just a language expert.

[0205] In this embodiment, the training data samples are formatted as input-output pairs. The model input is a combination of knowledge fragments and raw text. The knowledge fragments are several pieces of knowledge (such as article content) most relevant to the current transcript content retrieved from an existing judicial knowledge base. The raw text is the transcript text to be analyzed or a big data analysis report. The model output format is a structured output in JSON or Markdown format. In the element extraction task, the model output format is {"Time":"","Location":"",...}. The model trained based on the above training set is the transcript analysis model.

[0206] In the intelligent analysis process, different task types correspond to different input content and prompt templates. For example, the prompt template for the transcript comparison task requires the large language model to find contradictions in relevant elements from multiple input transcripts. Upon receiving the transcript text, relevant knowledge points are first retrieved from the knowledge base, and then the knowledge points and text are concatenated into instructions consistent with the training format. After being encapsulated by the prompt template, these instructions can be input into the transcript analysis model for automatic processing and analysis. The results are structured content in JSON or Markdown format, from which key information can be extracted as the final feedback from the system.

[0207] The knowledge enhancement fine-tuning method proposed in this embodiment is not a simple model optimization, but a paradigm innovation for vertical domain applications. It effectively solves the core pain points of general-purpose models, such as insufficient professionalism, lack of basis for reasoning, and non-standard output, providing a new, efficient, and reliable technical path for the practical application of large language models in highly specialized and sensitive fields. Example 2

[0208] This application provides a multimodal intelligent record-keeping auxiliary decision-making method, based on the multimodal intelligent record-keeping auxiliary decision-making system described in any one of Embodiments 1, such as... Figure 10 As shown, the method includes the following steps S10-S50: S10: Receive case information and initiate the case process; S20: Collect evidence data and conduct intelligent review; S30: Transcript analysis model based on multimodal data and knowledge enhancement generates and compares transcripts; S40: Automatically generate case summary materials and reports; S50: Integrate evidence, transcripts, and report data to generate electronic case files.

[0209] In some embodiments, such as Figure 11 As shown, step S30, which generates and compares the transcripts, includes the following steps S301-S304: S301: Generate subject identity records based on big data analysis reports; S302: Add previous records using a template-based approach; S303: Introduce intelligent referencing, evidence comparison, text correction, and element analysis during the record-keeping process; S304: Compare multiple transcripts horizontally and vertically to identify points of conflict.

[0210] In some embodiments, the method further includes: training a transcript analysis model using a knowledge-enhanced fine-tuning method, so that the model combines relevant knowledge fragments retrieved from the knowledge base during the analysis process to output structured analysis results.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A multi-modal smart recording aided decision system, characterized in that, The system adopts a layered architecture design, including: The application role layer provides system access points for different roles involved in judicial work. The intelligent scenario layer is used to provide intelligent auxiliary functions around key aspects of judicial business; The core functionality layer provides evidence management, record management, AI assistant, configuration center, and system management functions. The intelligent system layer is used to provide technical support for the intelligent application of the system based on RAG, cognitive graph, AIAgent, and multimodal large model technology; The data architecture layer provides data modeling, data resources, and data security functions. The infrastructure layer provides hardware support for intelligent computing, storage, and networking.

2. The multi-modal smart-recording aided decision system of claim 1, wherein, The intelligent system layer includes: The retrieval enhancement generation module combines judicial knowledge base retrieval with large model generation to output logical interpretations. The cognitive graph module is used to construct a knowledge network in the judicial field, presenting knowledge relationships in a graph-like manner. The AIAgent module is used to simulate intelligent case handling agency, automatically triggering tasks and calling tools; The multimodal large model module is used to integrate text, voice, and image data types to achieve comprehensive analysis.

3. The multi-modal smart-recording aided decision system of claim 1, wherein, The core functional layer includes: The evidence database management module is used to uniformly manage evidence data, including evidence to be collected, proven evidence, evidence arrangement, and case file generation. The record management module is used to uniformly manage record functions, including record outline, intelligent addition, element analysis, intelligent Q&A, record comparison, and intelligent record inspection; The AI ​​assistant module provides intelligent reminders, intelligent error correction, knowledge Q&A, and case analysis functions. The configuration center module provides template configuration, case configuration, permission configuration, and Q&A configuration. The system management module provides role management, access control, department management, and job management.

4. The multi-modal smart-recording aided decision system of claim 3, wherein, The record management module further includes: The intelligent record generation unit is used to automatically supplement record content based on case elements; The intelligent transcript comparison unit is used to identify discrepancies and contradictions in statements across multiple transcripts. The intelligent quality inspection unit for transcripts is used to automatically check for grammatical errors, missing elements, and logical conflicts in transcripts.

5. The multi-modal smart-recording aided decision system as claimed in claim 1, wherein, The system also includes a core algorithm model, which is constructed based on a knowledge-fine-tuning training method, including: The input layer is used to receive big data analysis reports and transcripts. The task conversion module is used to convert raw data and business requirements into task instructions that AI can understand; The Prompt engineering module is used to guide model output through standardized prompt templates; The transcript analysis model is used to receive prompt templates and raw text, and to perform complex text analysis. The structured output module is used to convert analysis results into a standardized output format.

6. The multimodal intelligent record-keeping auxiliary decision-making system according to claim 5, characterized in that, The training data samples for the transcript analysis model are in input-output pair format, wherein: The input is a combination of knowledge fragments and original text, wherein the knowledge fragments are knowledge retrieved from the judicial knowledge base that is related to the content of the current transcript; The output is structured content in JSON or Markdown format.

7. The multimodal intelligent record-keeping auxiliary decision-making system according to claim 1, characterized in that, The system adopts a microservice architecture, including: The user access layer provides the web access point and load balancing. The gateway layer is used for request routing, rate limiting, authorization verification, and filtering. The service registration and configuration layer is used for microservice registration and configuration management. The microservice layer includes evidence services, record-keeping services, Office services, and model services; The data storage layer includes Redis, MySQL, OSS, and file storage.

8. A multimodal intelligent record-keeping-assisted decision-making method, characterized in that, Based on the multimodal intelligent record-keeping auxiliary decision-making system as described in any one of claims 1 to 7, the method includes: Receive case information and initiate the case process; Collect evidence data and conduct intelligent review; Transcript analysis model based on multimodal data and knowledge enhancement generates and compares transcripts; Automatically generate case summary materials and reports; Integrate evidence, transcripts, and report data to generate electronic case files.

9. The multimodal intelligent record-keeping auxiliary decision-making method according to claim 8, characterized in that, The steps for generating and comparing transcripts include: Generate subject identity records based on big data analysis reports; Use a template-based approach to add previous records; Intelligent referencing, evidence comparison, text correction, and element analysis are introduced into the process of record making; Multiple transcripts were compared horizontally and vertically to identify points of conflict.

10. The multimodal intelligent record-keeping auxiliary decision-making method according to claim 8, characterized in that, The method further includes: The transcript analysis model is trained by using a knowledge-enhanced fine-tuning method, enabling the model to combine relevant knowledge fragments retrieved from the knowledge base during the analysis process and output structured analysis results.

Images