Intelligent patrol auxiliary method and system based on large language model fine tuning

Abstract

The invention relates to the technical field of intelligent patrol assistance, in particular to an intelligent patrol assistance method and system based on large language model fine tuning, and the method comprises the steps: firstly deploying a large language model LLM after instruction fine tuning in a local government affair server; a user uploads a patrol document, and a back end executes preprocessing; constructing a patrol clue vector library, performing semantic comparison on the preprocessed text and the patrol clue vector library, and identifying potential clues; calling a built-in Prompt template to generate a first draft of a patrol report; in combination with a thematic lexicon and LLM situation judgment, labeling type labels and scores for the suspected high-risk paragraphs; cross-text clustering is carried out on the multiple documents, and repeated problem types are recognized; outputting the structured table and the natural language report; according to the method, hundreds of documents can be processed in batches through fusion application of the local deployment instruction fine-tuning large language model and the semantic retrieval technology, automatic deep processing of patrol documents is achieved, and the working efficiency is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent patrol assistance technology, and in particular to an intelligent patrol assistance method and system based on fine-tuning of a large language model. Background Technology

[0002] Inspection is an important institutional arrangement for team supervision, involving the review and analysis of a large number of documents such as meeting minutes and conversation records. In traditional inspection work, the inspection team needs to manually read through a massive amount of information to uncover clues to problems, which is labor-intensive and has limited efficiency.

[0003] In traditional inspection work, inspection teams need to manually review a large number of documents such as meeting minutes and conversation records, which is labor-intensive and prone to overlooking hidden clues. Existing technologies have proposed using natural language processing (NLP) technology to assist inspections. For example, patent CN109710730B describes an NLP-based inspection information system that uses keyword annotation, TF-IDF, TextRank and other methods to screen data. This can standardize the process of discovering problems and reduce manual workload. However, it relies on rules and explicit features and is difficult to deeply mine the implicit semantics contained in the documents. With the development of Large Language Model (LLM) technology, the new generation of models has stronger text understanding and generation capabilities, which can further improve the depth and accuracy of automated processing of inspection data.

[0004] Therefore, in response to the problems mentioned above, this invention proposes an intelligent patrol assistance method and system based on fine-tuning of a large language model. Summary of the Invention

[0005] To overcome the problems of low efficiency and low depth and accuracy of automated processing of patrol data in existing patrol information systems, this invention proposes an intelligent patrol assistance method and system based on fine-tuning of a large language model. It uses artificial intelligence technology to perform deep semantic analysis and processing of patrol data, providing patrol personnel with auxiliary functions such as clue mining, summary generation, and risk warning.

[0006] The technical solution of this invention is: an intelligent patrol assistance method based on fine-tuning of a large language model, comprising the following steps: S1, Deploy the finely tuned large language model LLM on the local government server. The LLM is trained by supervised fine-tuning SFT and direct preference optimization DPO to adapt to the patrol task. S2: Users upload inspection documents through the B / S architecture front end, and the back end performs preprocessing, performing image OCR recognition, document segmentation, sentence segmentation, word segmentation, language detection, and generating embedding codes in sequence. S3. Construct a patrol clue vector library, which stores the embedding vectors of known problem templates. Perform semantic comparison between the preprocessed text and the patrol clue vector library, and identify potential clues from the document through semantic retrieval. S4 calls the built-in Prompt template to input structured instructions into the LLM, generating a draft inspection report containing problem points, evidence citations, and rectification suggestions; S5 combines a thematic thesaurus with LLM contextual judgment to label and score suspected high-risk paragraphs. At the same time, LLM can further output contextual explanations and attach high-risk suggestion paragraphs. S6 performs cross-text clustering on multiple documents to identify recurring problem types; S7 outputs structured tables and natural language reports, supporting Excel / CSV export or Word / PDF format generation.

[0007] Preferably, the Prompt template in step S4 is stored in the system Prompt framework, which supports version control and optimization.

[0008] Preferably, the semantic comparison in step S3 specifically involves using the DeepSeek vector engine to calculate the similarity between the document fragment embedding vector and the risk template vector, thereby achieving high recall rate clue discovery.

[0009] Preferably, the risk scoring mechanism in step S5 includes: triggering LLM contextual reasoning for statements containing high-risk keywords, and outputting a risk score and explanatory paragraph.

[0010] Preferably, the structured table in step S7 includes fields such as: source document, paragraph content, risk type, and score; the natural language report generates a cover page, identified problems, rectification suggestions, and attachment references according to a preset template.

[0011] As a preferred option, an intelligent patrol assistance system based on fine-tuning of a large language model includes: The document upload and preprocessing module supports Word / PDF / image format input and performs OCR, segmentation, sentence segmentation, word segmentation and embedding encoding operations in sequence. The clue extraction module has a built-in patrol clue vector library. It identifies potential clues by comparing text embedding vectors with known question template vectors through semantic retrieval. The report summary and generation module integrates command-line fine-tuning of LLM and calls the built-in Prompt template to generate structured reports; The risk warning module combines a thematic thesaurus with LLM contextual judgment to assign type tags and scores to paragraphs suspected of being high-risk. The problem summary module for inspected units can perform cross-text clustering of multiple documents and automatically summarize the recurring problems found in the inspected units. The output module generates structured tables and natural language reports, supporting export to Excel / CSV / Word / PDF formats.

[0012] Preferably, the report generation module includes a Prompt template management engine that stores Prompt templates for multiple scenarios and supports version control.

[0013] Preferably, the system deployment framework uses Ollam, which isolates the model and data sandbox, restricts all operations to the local network, and has no external data access.

[0014] Preferably, the system also includes an audit log module, which records user upload, analysis and export operations, and tracks model version and Prompt version.

[0015] Preferably, the front end of the output module supports: drag-and-drop file upload, result scoring feedback, supplementary explanation input, and context-based report regeneration.

[0016] The beneficial effects of this invention are: 1. This invention integrates a large language model with semantic retrieval technology through local deployment and fine-tuning, enabling automated deep processing of inspection documents and significantly improving work efficiency. Traditional inspection methods require manual reading of massive amounts of documents and manual annotation of clues, which can take several days or even weeks. In contrast, this system can process hundreds of documents in batches, completing the entire process of OCR recognition, text segmentation, clue mining, and report generation in a short time, saving at least 50% of manual time. This allows inspection personnel to focus on result verification and decision optimization, greatly shortening the overall time cycle of inspection tasks.

[0017] 2. This invention effectively addresses the pain point of overlooking hidden problems in traditional inspections. This system utilizes the deep semantic understanding capabilities of large language models, combined with vectorized semantic retrieval technology, to transform text paragraphs and problem templates into embedded vectors for similarity comparison. It can not only capture direct keywords but also identify potential clues of semantic association. For example, it can accurately classify non-explicit descriptions such as "undisclosed fund flows" as financial anomalies, significantly improving the comprehensiveness and accuracy of problem discovery. Attached Figure Description

[0018] Figure 1 The diagram shown illustrates the workflow of this invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figure 1 This invention provides an embodiment: an intelligent patrol assistance method based on fine-tuning of a large language model, comprising the following steps: S1, Deploy the finely tuned large language model LLM on the local government server. The LLM is trained by supervised fine-tuning SFT and direct preference optimization DPO to adapt to the patrol task. S2: Users upload inspection documents through the B / S architecture front end, and the back end performs preprocessing, performing image OCR recognition, document segmentation, sentence segmentation, word segmentation, language detection, and generating embedding codes in sequence. S3. Construct a patrol clue vector library, which stores the embedding vectors of known problem templates. Perform semantic comparison between the preprocessed text and the patrol clue vector library, and identify potential clues from the document through semantic retrieval. S4 calls the built-in Prompt template to input structured instructions into the LLM, generating a draft inspection report containing problem points, evidence citations, and rectification suggestions; S5 combines a thematic thesaurus with LLM contextual judgment to label and score suspected high-risk paragraphs. At the same time, LLM can further output contextual explanations and attach high-risk suggestion paragraphs. S6 performs cross-text clustering on multiple documents to identify recurring problem types; S7 outputs structured tables and natural language reports, supporting Excel / CSV export or Word / PDF format generation.

[0021] Furthermore, the system utilizes a large model to perform semantic analysis on the uploaded text. By combining a keyword database in the government sector with contextual understanding, it automatically identifies abnormal clues in the document. Specifically, the system encodes text paragraphs and typical risk descriptions into embedding vectors and calculates the similarity between them, thereby discovering semantically related potential clues. This process not only matches explicit keywords but also captures contextual semantics, improving the comprehensiveness of clue discovery.

[0022] The system leverages the language model's summarization and generation capabilities to automatically draft inspection reports. A specially designed prompt template guides the model in extracting and summarizing key information. For example, the system can prompt the model to "generate a summary report of the problems discovered during the inspection based on the following conversation transcript," and the model will output a structured paragraph containing a problem description, relevant evidence, and rectification suggestions. The model is optimized through instruction fine-tuning techniques (such as training with SFT and DPO algorithms), enabling it to more accurately understand the task requirements in inspection scenarios and output high-quality text.

[0023] The system combines a rule-based lexicon and semantic analysis to monitor the risk of uploaded text. For sentences containing high-risk keywords or whose contextual semantics indicate problems, the model triggers a real-time warning. The language model can also make further inferences and judgments on suspected risky content, calculate risk scores, and prompt inspectors to pay attention to potential risk points.

[0024] For multiple documents from the same inspected unit, the system uses a large model to summarize information and cluster problems, automatically generating a list of key issues for that unit. The model can synthesize descriptions from different documents and extract core issues. For example, the model can merge multiple feedback messages to generate summaries such as "Main problems of this unit: chaotic financial approval process, unsound personnel management system," thus improving the efficiency of information integration.

[0025] The processing results are displayed and saved through the results output module: The system features structured table export: Extracted clues, risk points, and issue lists are organized into tables with fields and displayed on the web for human review. Users can further filter, sort, and annotate this data. Table data can be exported to common formats (such as Excel spreadsheets) for easy archiving and statistical analysis.

[0026] Natural Language Report Generation: The system automatically generates a complete inspection report text based on the analysis results, which can be directly exported as a Word or PDF file. The model organizes the extracted main issues, evidence materials, and rectification suggestions into a coherent report according to a preset template, significantly reducing the workload of manual writing. The output report format is standardized and compatible with traditional manually written report formats.

[0027] Furthermore, this invention deploys an intelligent patrol assistance system in a local government-grade server cluster. The hardware platform uses domestically produced AI chips or Nvidia H800 chip clusters to ensure that the computing power meets the inference requirements of large models. The system is designed based on a B / S architecture. The front end is accessed through a web browser, and the back end includes modules such as document preprocessing, semantic processing, and result export. All services run in a closed intranet environment without external network access. After logging into the system through identity authentication, users can drag and drop to upload patrol documents (supporting Word, PDF, text, and image formats) on the front end. Uploaded files are stored in a local encrypted database. For image documents, the system calls the OCR engine to perform text recognition and output structured text. For Word / PDF documents, the text content is directly extracted. The preprocessing module performs multi-level processing on the text: first, it is segmented according to chapter and paragraph rules, then it is segmented into sentences, then it is segmented into words and language detection, and finally, the DeepSeek vector engine is used to generate the embedding encoding of text fragments. All intermediate data is temporarily stored in local memory.

[0028] Furthermore, the intelligent patrol assistance system based on large language model fine-tuning includes: The document upload and preprocessing module supports Word / PDF / image format input and performs OCR, segmentation, sentence segmentation, word segmentation and embedding encoding operations in sequence. The clue extraction module has a built-in patrol clue vector library. It identifies potential clues by comparing text embedding vectors with known question template vectors through semantic retrieval. The report summary and generation module integrates command-line fine-tuning of LLM and calls the built-in Prompt template to generate structured reports; The risk warning module combines a thematic thesaurus with LLM contextual judgment to assign type tags and scores to paragraphs suspected of being high-risk. The problem summary module for inspected units can perform cross-text clustering of multiple documents and automatically summarize the recurring problems found in the inspected units. The output module generates structured tables and natural language reports, supporting export to Excel / CSV / Word / PDF formats.

[0029] The clue extraction module loads a pre-built inspection clue vector library, which contains embedding vectors of typical problem templates. The system uses a large model to perform semantic analysis on the uploaded text, combining a keyword library in the government domain and contextual understanding to automatically identify abnormal clues in the document. Specifically, the system encodes text paragraphs and typical risk descriptions into embedding vectors, calculates the similarity between them, and thus discovers semantically related potential clues. This process not only matches explicit keywords but also captures contextual semantics, improving the comprehensiveness of clue discovery. For example, when a paragraph describes "undisclosed special fund flows," although the keyword "financial anomalies" does not appear, the system automatically labels it as a financial clue because of its extremely high semantic similarity to the template vector. This process is achieved through semantic retrieval technology, surpassing the limitations of traditional keyword matching.

[0030] The report summarization and generation module calls a locally deployed command-based fine-tuning large language model. The model is trained using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). The system has a built-in Prompt template management engine that stores templates for multiple scenarios: the clue recognition template is "Extract inspection clues from the following text and output them in the format of [problem type, description, evidence paragraph]"; the summary generation template is "Please generate a structured report containing problem points, evidence citations, and rectification suggestions"; and the early warning inference template is "Output type labels and scores for high-risk content". For example, when a user selects the "Report Generation" function, the system automatically constructs the Prompt: "System: The following is the content of the inspection document. Please generate a structured report containing problem points, evidence citations, and rectification suggestions. User: [Text to be analyzed]". The model outputs a draft based on this, and the model optimized by SFT / DPO significantly improves the report's standardization and problem coverage.

[0031] The risk warning module runs synchronously. It integrates a thematic thesaurus and links with a large language model. When a high-risk word appears in the text, it triggers the model's contextual reasoning: the model analyzes the context in which the word is located and outputs a risk type label and score (1-5 points).

[0032] The problem summarization module for inspected units performs cross-text clustering on multiple documents from the same inspected unit, identifying recurring problem types and automatically summarizing problems to improve efficiency. For example, if five documents all contain descriptions related to "approval process not executed," the system automatically groups them into the "lack of system implementation" category and outputs a summary statement: "This unit has three high-frequency problems: 1. Disorganized financial approval process (occurred 5 times)...". The processing results are displayed through the output module: a structured table containing fields such as "source document, paragraph content, risk type, and score," supporting online filtering and sorting, and can be exported to Excel / CSV format; a natural language report organized according to a preset template, including a cover page, identified problems, explanations, rectification suggestions, and attachment references, supporting one-click export to Word or PDF format for easy use in official processes.

[0033] Furthermore, regarding user interaction: Inspectors can rate the generated report or add supplementary text descriptions. The system records the feedback and updates the context. When the user clicks "Optimize Report", the supplementary descriptions are automatically converted into a new prompt and attached to the original instruction, triggering the model to regenerate. For example, if the original instruction is to generate a report summary, and the supplementary description is "add an analysis of insufficient supervision of grassroots cadres", the new prompt will be adjusted to "add a section on grassroots cadre supervision to the original report", and the model will output a revised version.

[0034] Furthermore, the system adopts a three-tier security architecture: The hardware layer uses domestically produced equipment or Nvidia H800 clusters, physically isolating it from the external network; The service layer deploys models through the Ollam framework. The models run in a sandbox environment, isolated from the document storage area. The access layer implements RBAC (Role-Based Access Control), allowing users to operate only authorized functions. The system records user upload / analysis / export operations, supporting auditing and accountability. Furthermore, model and Prompt versions are traceable, ensuring the source of output content is verifiable.

[0035] Furthermore, this invention performs fine-tuning of the general large language model for inspection domain instructions (using techniques such as supervised fine-tuning SFT or direct preference optimization DPO), and designs a special Prompt template to guide the model in generating results, thereby improving the model's adaptability to inspection tasks. By continuously iterating the Prompt and training data, the model can accurately understand the types of inspection issues, document formats, and government work contexts, and generate analyses and reports that meet the requirements.

[0036] Furthermore, this invention vectorizes documents and predefined risk descriptions and uses semantic similarity in the embedding space for retrieval. Compared with simple keyword matching, semantic retrieval can capture the potential meaning of text, identify content that is related to risks but not explicitly marked, and improve the coverage of clue discovery. The system can also annotate sentences and segments with high semantic similarity in combination with the context to achieve the identification and classification of potential problems.

[0037] Furthermore, the entire system and large model are deployed on local government-grade servers, ensuring that no sensitive data is transmitted to the external network, thus effectively protecting data security. The system adopts a domestic computing power platform and a private deployment strategy to ensure that "raw data does not leave the domain and data is usable but not visible." During model training and inference, data is encrypted and stored throughout the process, and network access permissions are strictly controlled to meet government data security requirements.

[0038] Furthermore, the system provides a user-friendly web interface that supports uploading various file formats (Word, PDF, images, etc.) and exporting various results (online tables, Word / PDF reports, etc.). The user interface is simple and intuitive, allowing inspectors to upload files by dragging and dropping or clicking, and to browse analysis results and search for clues. The interactive platform also supports keyword search, result annotation, and feedback functions, facilitating manual supervision and correction.

[0039] Furthermore, compared to traditional inspection methods, this invention achieves a high degree of automation from massive document collection to problem discovery. Existing technologies have proven that NLP-assisted inspection information systems can standardize the discovery process, reduce workload, unify problem characterization standards, and improve efficiency. This system further leverages the deep understanding capabilities of large-scale models, achieving not only keyword-level matching but also insight into implicit problems at the paragraph level and even throughout the entire text context, thus more comprehensively and accurately uncovering problem clues. The automatic report generation function also allows inspection team members to focus more on result verification and strategy formulation, rather than tedious data collection and report writing. See Table 1 for details.

[0040] Table 1 Comparison of the present invention with traditional inspection methods project Traditional manual methods Intelligent patrol AI assistant Document processing Manual reading, annotation, and manual statistics of each article Batch processing, semantic detection, and automatic clue extraction Problem detection accuracy Relying on human experience can easily lead to overlooking clues. By combining vector comparison with model judgment, latent problems can be identified. Report writing Manually written, time-consuming, and prone to missing key information. Automatically generated based on templates and generated Prompts, with standardized format. Risk warning It relies heavily on keywords, resulting in significant false positives or false negatives. Keyword + semantic recognition + scoring mechanism: more accurate early warning Data security and compliance Local processing, but inefficient. Fully localized deployment, government-level security controls, and high efficiency. The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An intelligent patrol assistance method based on large language model fine-tuning, characterized in that, Comprise the following steps: S1, in the local government server deployment instruction fine-tuning large language model LLM, the LLM is supervised fine-tuning SFT and direct preference optimization DPO training, adapt to the task of patrol; S2, the user uploads the patrol document through the B / S architecture front end, and the back end executes preprocessing, and sequentially carries out identification, document segmentation, sentence segmentation, word segmentation, language detection and embedding coding operation on picture OCR; S3, a patrol clue vector library is constructed, the vector library stores embedding vectors of known problem templates, the preprocessed text is compared with the patrol clue vector library, and potential clues are identified from the document through semantic retrieval; S4, call built-in Prompt template to input structured instructions to LLM, generate patrol report draft containing problem points, evidence reference and rectification suggestion; S5, combine special topic library with LLM context judgment, label type tag and score for suspected high-risk paragraph, and LLM can further output context explanation and attach high-risk suggestion paragraph; S6, cross-text clustering is carried out on multiple documents to identify repeatedly appearing problem types; S7, output structured table and natural language report, support Excel / CSV export or Word / PDF format generation.

2. The intelligent patrol auxiliary method based on large language model fine tuning according to claim 1, characterized in that: The Prompt template in the step S4 is stored in the system Prompt framework, and version control and optimization are supported.

3. The intelligent patrol auxiliary method based on large language model fine tuning according to claim 1, characterized in that, The semantic comparison of the step S3 is specifically that the similarity of document segment embedding vectors and risk template vectors is calculated using DeepSeek vector engine, so that high recall rate clue discovery is realized. 4.The intelligent patrol assisting method based on large language model fine-tuning according to claim 1, characterized in that, The risk scoring mechanism of the step S5 comprises that the LLM context reasoning is triggered for the sentence containing high-risk keywords, and risk score and explanation paragraph are output. 5.The intelligent patrol assisting method based on large language model fine-tuning according to claim 1, characterized in that, The structured table of the step S7 comprises fields: source document, paragraph content, risk type and score;The natural language report is generated according to the preset template cover, discovery problem, rectification suggestion and attachment reference structure.

6. The intelligent patrol auxiliary system based on large language model fine-tuning, the intelligent patrol auxiliary method based on large language model fine-tuning according to any one of claims 1-5, deployed on a local government server group, characterized in that, Comprise: Document uploading and preprocessing module, support Word / PDF / picture format input, sequentially execute OCR, segmentation, sentence segmentation, word segmentation and embedding coding operation; Clue extraction module, built-in patrol clue vector library, compare text embedding vectors with known problem template vectors through semantic retrieval, identify potential clues; Report abstract and generation module, integrate instruction fine-tuning LLM, call built-in Prompt template to generate structured report; Risk warning module, combine special topic library with LLM context judgment, assign type label and score to suspected high-risk paragraph; Patrolled unit problem induction module, cross-text clustering is carried out on multiple documents to automatically induce repeated problems of the patrolled unit; Output module, generate structured table and natural language report, support Excel / CSV / Word / PDF export.

7. The intelligent patrol auxiliary system based on large language model fine tuning according to claim 6, characterized in that: The report generation module comprises Prompt template management engine, stores multiple scene Prompt templates and supports version control. 8.The intelligent patrol assisting system based on large language model fine-tuning according to claim 6, wherein: The system deployment framework adopts Ollama, model and data sandbox are isolated, all operations are limited to local network, and there is no external data call. 9.The intelligent patrol assisting system based on large language model fine-tuning according to claim 6, wherein: The system also includes an audit log module that records user uploads, analysis, and export operations, and traces model versions and Prompt versions. 10.The intelligent patrol assisting system based on large language model fine-tuning according to claim 6, wherein, The front end of the output module supports file drag-and-drop upload, result scoring feedback, supplementary note input, and context-based report regeneration.

Images