Vertical domain intelligence analysis method based on large language model

Abstract

The invention discloses a vertical domain intelligence analysis method based on a large language model. The method comprises the following steps: receiving a natural language analysis demand of a user; based on pre-constructed vertical domain specific task planning knowledge, intelligently decomposing the demand into a subtask list corresponding to a preset dimension by using a large language model, and semantically matching a proper information acquisition tool for each subtask to generate a structured task plan; the system schedules and executes tool sets of network search, webpage capture, internal query and the like according to a plan, and performs cross verification and cleaning on collected fragmented information; and inputting the verified data into a large language model for integration analysis, and optionally carrying out multi-round iteration self-evaluation optimization to automatically generate a high-quality vertical domain intelligence analysis report. According to the method, full-process and high-intelligence automatic processing of information analysis in the vertical field from demand understanding to report generation is realized, and the method has excellent planning capability, process interpretability, result reliability and excellent modular expansibility.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a vertical field intelligence analysis method based on a large language model. BACKGROUND

[0002] In the current digital market competition environment, enterprises urgently need to grasp the key information such as product updates, market actions and marketing strategies of competitors in real time to formulate effective countermeasures. The traditional manual research method is inefficient, high in cost and difficult to achieve continuous monitoring and analysis; while the existing automatic crawler or search engine tool can obtain data, but lacks semantic understanding ability and cannot autonomously decide the collection path and analysis logic according to complex requirements.

[0003] In recent years, with the development of large language models (such as Qwen, ChatGPT, etc.), intelligent agents (Agentic) with reasoning and tool calling capabilities have gradually emerged. However, the existing technology has not effectively combined task planning, multi-tool scheduling and industry vertical scenarios (such as competitor monitoring), resulting in problems such as inaccurate response and broken execution chain in actual commercial applications.

[0004] Therefore, an intelligent system capable of understanding user high-level intentions, automatically decomposing task steps, coordinating multiple external tools and outputting structured results is urgently needed, especially for business intelligence collection scenarios.

[0005] Therefore, a more efficient vertical field intelligence analysis method is needed. SUMMARY

[0006] The technical problem to be solved by the present application is to overcome the deficiencies of the prior art, solve the efficiency problem of general information research and monitoring tools, and provide a vertical field intelligence analysis method based on a large language model.

[0007] To solve the above technical problems, the present application provides a vertical field intelligence analysis method based on a large language model, characterized in that it comprises the following steps:

[0008] Step 1: receiving an analysis requirement input by a user in natural language;

[0009] Step 2: based on pre-constructed vertical field-specific task planning knowledge, using a large language model to decompose the analysis requirement into a list of sub-tasks corresponding to a plurality of preset dimensions; through the large language model, based on the matching semantics of prompt words and tool descriptions, appropriate information collection tools are assigned to each sub-task to generate a structured task plan containing execution order; the vertical field-specific task planning knowledge is realized by at least one of the following ways: pre-defined structured prompt word template, domain task ontology library or large language model fine-tuned by domain data;

[0010] Step 3: According to the structured task plan, sequentially schedule and execute the corresponding tool set to collect and process target information; the tool set includes at least one of a network search tool, a web scraping tool, and an internal data query tool;

[0011] Step 4: Cross-verify and clean the fragmented information obtained after tool execution to form a verified data set;

[0012] Step 5: Input the verified data set into a large language model for integration and analysis to generate a structured vertical domain intelligence analysis report.

[0013] The step 2 specifically includes: checking and integrating historical dialogue records and task states related to the current analysis requirements to construct context information for task planning; calling a large language model for reasoning based on a preset structured prompt word template and the context information; persistently storing the structured task plan generated by the large language model in a memory database for state management.

[0014] In the step 2, the large language model is a model fine-tuned by domain task planning data, and the large language model controls the randomness and creativity of generating task plans by adjusting its temperature parameter.

[0015] In the step 2, the information collection tools are registered and managed based on a unified standardized interface, supporting dynamic expansion and flexible scheduling of tools.

[0016] In the step 3, the tool set includes:

[0017] A thinking and planning tool for secondary decomposition of complex subtasks in the subtask set;

[0018] An export tool for exporting the finally generated structured analysis report as a standardized document or storing it in a cache.

[0019] In the step 4, the cross-verification and cleaning includes:

[0020] Comparing the consistency of homologous data obtained by different tools;

[0021] Verifying the authority of data sources;

[0022] Checking the timeliness of information.

[0023] In the step 5, the memory database is also used to store the verified data set and the structured vertical domain intelligence analysis report.

[0024] In step 5, the large language model further performs self-evaluation and optimization on the analysis results based on preset multi-dimensional evaluation indicators when generating the structured vertical field intelligence analysis report, to output the report content; the multi-dimensional evaluation indicators include information integrity, data consistency, and difference degree of historical reports.

[0025] The self-evaluation and optimization process specifically performs the following multi-round iteration optimization steps:

[0026] First draft generation and first round evaluation: input the verified data set into the large language model to generate a structured report draft; prompt the large language model to switch to a preset role, and review the structured report draft according to the multi-dimensional evaluation indicators to output a structured evaluation result containing qualitative comments and quantitative scores of each dimension;

[0027] Targeted revision: analyze the structured evaluation result, identify the lowest scoring evaluation dimension and its corresponding qualitative comments; based on the qualitative comments, the large language model automatically plans and triggers the corresponding information collection tool for supplementary query, or directly revises the text content;

[0028] Iteration termination and output: start the second round of evaluation on the revised report to obtain a new round of quantitative scores; compare the quantitative scores of the two rounds, if the overall score or the score of the lowest dimension reaches the preset condition, including the standard termination and convergence termination, terminate the iteration and output the current report as the final output; otherwise, take the current report as the new input and repeat the previous step to perform a new round of revision and evaluation.

[0029] The self-evaluation and optimization process includes:

[0030] Based on the completeness check of the planning list: call the structured task plan generated in step 2 and the subtask list contained therein; traverse whether an independent analysis conclusion paragraph is provided for each subtask in the subtask list in the verification report; if the content of the corresponding subtask is detected to be missing, the model automatically generates a supplementary query subtask for the missing information, and drives the system to schedule related information collection tools or call existing data for information completion, to realize task closed loop;

[0031] Traceability check and conflict resolution: data traceability is performed on key conclusions in the report to check whether the information is from the verified data set in step 4, and the specific source tool is marked; if it is found that the same fact exists in different tool sources with expression conflict, the conflict resolution logic is started, which includes: giving priority to the source with higher authority, or explaining the difference in the report in the form of notes;

[0032] Structured enhancement combined with domain ontology: using the domain task ontology library described in step 2 as a constraint, the report hierarchy is optimized to force the chapter content in the report to strictly follow the sub-dimension order defined in the ontology library and present in sequence.

[0033] The present application achieves the following beneficial effects:

[0034] End-to-end full-process automation from user demand understanding to high-quality report generation is achieved, with high universality, flexibility, interpretability and good scalability, greatly improving the efficiency, depth and intelligent level of enterprises in vertical field intelligence analysis.

[0035] 1. High universality and flexibility: the framework is not bound to a specific field, and by registering different tool sets and domain knowledge, it can quickly adapt to a variety of complex task scenarios such as business competitor monitoring, medical research and development tracking, policy and regulation analysis, market research, etc., and has strong ecological expansion capability.

[0036] 2. End-to-end automation and continuous optimization: full-process automation is achieved from raw user demand understanding, intelligent task planning, tool scheduling execution, information verification and cleaning to report generation. In particular, through a multi-round iterative self-evaluation optimization mechanism, the system can automatically revise and improve the report based on quantitative indicators, ensuring the professional quality of the output results without manual intervention in key steps.

[0037] 3. Good interpretability and traceability: the planning logic of the entire workflow (task decomposition basis, tool matching reason), execution steps, data sources and verification results are clearly recorded and structured stored, making the final analysis conclusion and decision-making process completely transparent, auditable and traceable, meeting the requirements of enterprise-level applications for process credibility.

[0038] 4. Modularity and scalability: the system uses a loosely coupled modular design, allowing for flexible extension of new tools, replacement of different LLM models, enrichment of domain task knowledge base or use of different state storage solutions, and has strong adaptability and long-term evolution potential. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 Method step flowchart for exemplary embodiments of the present application;

[0040] Figure 2 Task planning flowchart in exemplary embodiments of the present application. DETAILED DESCRIPTION

[0041] The application provides a vertical field intelligence analysis method based on a large language model, aiming to solve the problems of low efficiency, lack of intelligent reasoning and information fragmentation in traditional information monitoring (especially friend and competitor monitoring). The core solution is a vertical field intelligence analysis method based on a large language model, and an intelligent analysis system is built based on this method. The system first receives the user's natural language requirements, uses a large language model as the "decision center", and intelligently decomposes complex requirements into structured and executable task plans. The system calls network search, web scraping, internal data query and other tools through an extensible tool scheduling center to automatically collect information, and cross- verifies and cleans the fragmented information to ensure reliability. Finally, the large language model is used again to integrate and analyze the verified information, and a structured vertical field intelligence analysis report (such as a competitive intelligence report) is automatically generated.

[0042] The method and system for running the method according to the application can realize functional decoupling and efficient collaboration through modular design, and the core modules are composed as follows:

[0043] User query receiving module

[0044] As the interaction entrance of the system and the user, it is responsible for accurately receiving the user's natural language form of competition monitoring request, providing initial input for subsequent task processing.

[0045] Task planning and decomposition module

[0046] The "decision center" of the system, using a configured large language model (LLM) as the reasoning engine. Before task planning, the system first accesses the "context memory and state management module" (implemented based on Redis) to load the historical context information associated with the current session ID. Then, after receiving the user query and historical context information, the core reasoning is performed: by constructing a standardized prompt word template, injecting company name, tool capability list, preset domain task knowledge and other key information. After large model reasoning, a structured task plan (JSON format) is output, which explicitly includes a sequence of subtasks and a tool matched for each subtask.

[0047] Tool registration and scheduling module

[0048] The "executor scheduling center" of the system maintains a set of standardized tool interfaces based on the unified base class BaseTool, supports dynamic extension of tools, and the core tools include:

[0049] Web search tool (WebSearchTool): calling search engines to obtain the latest information of competitors, industry dynamics and other external information;

[0050] Webpage Fetching Tool (FetchUrlTool): Directly fetches specific content from designated webpages, enabling precise information collection.

[0051] Internal Data Query Tool (GetProductTool): Interfaces with internal product databases to extract details such as competitor product functions, parameters, and iteration history.

[0052] Export Tool (ExportContent): Exports the final generated analysis report into a standardized document or stores it in Redis cache for subsequent calls.

[0053] Thinking and Planning Tool (ThinkingAndPlanning): A core tool for task planning, responsible for further transforming ambiguous and complex sub-problems into linear executable task flows.

[0054] Large Language Model Interface Module (LLM Interface)

[0055] System "Intelligent Engine Interface" supports flexible switching and parameter configuration of multiple models: By default, a large language model with task planning capabilities such as Qwen3-Planning is used to handle task planning core work, and other LLMs such as Bing, Baidu, and GLM can be configured. The interface supports controlling the randomness of model output by adjusting the temperature parameter, with a typical range of 0.1-0.5. In task planning, the system can dynamically adjust the temperature value based on task complexity - for structured and deterministic tasks, a lower temperature (e.g., 0.1-0.2) is used to ensure stable results; for open and creative tasks, the temperature is increased (e.g., 0.4-0.5) to enhance diversity and innovation. The interface also provides an asynchronous calling mechanism to improve multi-task concurrent processing performance.

[0056] Context Memory and State Management Module

[0057] System "Memory Hub", based on Redis, implements two core functions: First, it records the "TODO list" summary (refContent) in the conversation history, supporting the continuity of cross-round tasks and avoiding repeated execution of completed work; second, it caches tool invocation results, task plans, and other intermediate data, enabling information sharing among multiple modules and ensuring the coherence and consistency of task processing.

[0058] Result Return and Feedback Loop Module

[0059] The system's "stability assurance hub" has a built-in comprehensive error capture and exception fallback mechanism: when a tool call fails, a model inference fails, or data verification fails, it automatically triggers a retry, degradation, or alarm process to ensure continuous service availability; at the same time, this module receives task planning results and drives subsequent modules to call tools in sequence to form a complete execution chain, realizing a closed loop of "planning-execution-feedback".

[0060] The present invention will be further described below with reference to the accompanying drawings and exemplary embodiments:

[0061] like Figure 1 As shown, the entire process of a specific embodiment of the method described in this invention starts with user needs and ends with the generation of a structured report, and is divided into five core steps, as follows:

[0062] Step 1: Requirement access and text integration.

[0063] After a user submits a request for intelligence analysis in a vertical field using natural language (such as "background investigation of XX Technology Co., Ltd."), the system performs two steps in sequence:

[0064] Request access: The system receives user commands through the front-end interactive interface;

[0065] Context integration: The "User Query Parsing and Context Building Module" combines historical dialogue data, existing task status and other information to build a precise query context that includes "original requirements + historical context", providing complete input for subsequent task planning.

[0066] Step 2: Task planning and requirement breakdown.

[0067] like Figure 2 As shown, this embodiment of the invention provides a task planning and requirement decomposition method for complex business needs. This method is initiated after completing step 1, "requirement access and text integration," and is used to automatically transform the user's original query intent into an executable, structured subtask flow, which is the core decision-making link of the entire system. The output of each stage is strictly used as the input of the next stage, ensuring the controllability, interpretability, and repeatability of the task planning results.

[0068] This process is initiated after completing step 1, "Requirement Access and Text Integration," and is used to transform the user's original query intent into an executable, structured flow of subtasks. The core of this process lies in breaking down ambiguous business requirements into explicit task sequences and tool call chains based on contextual information, domain-specific task planning knowledge, and the reasoning capabilities of large language models.

[0069] After the context preparation is completed, the system performs a "build standardized prompt word template" step, in which the system automatically loads vertical domain-specific task planning knowledge according to the demand type, and injects it into a structured prompt word template.

[0070] The vertical domain-specific task planning knowledge is a set of rules and experience relied on by the system in the task planning stage, which is used to guide how to decompose the fuzzy demand of a specific domain (such as competitive product monitoring) into executable information collection actions. For example, the system preloads a "company background investigation" template, which includes:

[0071] "You are a competitive intelligence analysis expert. Please decompose the user's investigation demand about {company name} into the following dimensions of sub-tasks: 1. Basic information (founding time, location, legal person); 2. Financing history (financing round, amount, investment party); 3. Core team (founder, key management personnel background); 4. Intellectual property (patent application situation). Please specify the most suitable information collection tool for each sub-task."

[0072] The vertical domain-specific task planning knowledge can be embodied by a structured domain task ontology library. The ontology library defines all concepts, attributes, relationships and standard task processes involved in specific domain analysis. For example:

[0073] In the field of competitive product monitoring, the system maintains a "competitive analysis ontology", which defines:

[0074] Root node: competitive analysis

[0075] First-level child node (intelligence dimension): company profile, product information, market dynamics, financial performance...

[0076] The intelligence dimension refers to the multiple structured aspects that need to be investigated when analyzing a specific object (such as a competitor), including but not limited to "company background", "product technology", "market dynamics", "marketing strategy", "financing situation".

[0077] Second-level child node (specific task): company profile includes querying business information, querying financing history, querying recruitment dynamics... Each specific task is preloaded with a default tool call sequence (such as querying business information -> [WebSearchTool, FetchUrlTool])

[0078] Subsequently, the system enters the "model configuration" step, which by default calls a large language model dedicated to planning and dynamically adjusts inference parameters such as Temperature, Max Tokens, etc. according to the task type to adapt to the planning needs of tasks of different complexity. The vertical domain-specific task planning knowledge is achieved by using domain-specific data to fine-tune the instructions for the general large language model. The fine-tuned model has the task decomposition logic of the domain internalized in its parameters. For example:

[0079] A large number of daily work records and task decomposition cases of competitive product analysts are collected to form a training data set. Using this data set, the Qwen basic large language model is fine-tuned for instructions, so that the model can automatically generate a structured task list that meets the professional norms of competitive product analysis after receiving the instruction "analyze competitor A".

[0080] Subsequently, the system enters the "LLM inference generates structured task plan" step, where the model is driven by "context + standardized prompt words" to automatically infer a complete structured task plan.

[0081] The matching relationship between tasks and tools is automatically established by the system's task planning module, which combines a hybrid mechanism of model training and rule constraints: through the task intent recognition and semantic matching capabilities of a large language model dedicated to planning, the most suitable tool combination is automatically inferred, ensuring that the planning result is both intelligent and adaptive, and controllable and interpretable. Taking "background investigation of XX Technology Co., Ltd." as an example, the overall demand is broken down into sub-tasks such as "company basic information query", "business scope analysis", "core product analysis", etc., and the execution order and optimal tool configuration for each sub-task are specified. For example, the "company basic information query" task matches WebSearchTool because such information is mainly sourced from public networks (such as the company's official website, business registration website, and news reports); while the "core product analysis" task matches GetProductTool + FetchUrlTool + ContextExtraction, where GetProductTool accesses structured internal product databases, FetchUrlTool retrieves the latest product information from the company's official website and third-party platforms, and ContextExtraction extracts product features and performance indicators from the text.

[0082] The generated task plan will be stored in the Redis database to support subsequent state tracking, task recovery, and cross-stage collaboration. Redis is used as a high-performance state storage medium to enable fast read and write of structured task plans.

[0083] After storage, it enters the "plan verification" judgment node. If the verification result is "verification passed", the system will enter the "output task plan to drive the next stage" step, and hand over the final structured task plan to the downstream tool routing module or execution module for further processing. If the verification result is "verification failed", the system enters "trigger error handling / return abnormal reason", provides clear error types (such as missing key information, tool conflict, plan format exception, etc.), and can support automatic retry or manual intervention.

[0084] Step 3: Tool call and information completion.

[0085] The system parses the task nodes generated in step 2 by the tool registration and scheduling module, and drives the corresponding tool instance to run according to the node type. This step converts static "planning" into dynamic "data collection and processing" actions through standardized calling flow. The specific execution process is as follows:

[0086] Recursive decomposition of complex subtasks (call "ThinkingAndPlanning" or "ThinkingAndPlanning tool"): The system does not directly call the collection tool, but preferentially triggers the thinking and planning tool. The application of this tool in the method is as follows: the current fuzzy subtask is taken as a new input prompt and returned to the large language model for secondary inference, and it is decomposed into a set of linear atomic tasks. This ensures the feasibility of the execution level.

[0087] Serial collection of external intelligence (call WebSearchTool and FetchUrlTool): For atomic tasks that require external information, the system uses a "search-locate-grab" collaborative mode. First, call the WebSearchTool: this tool receives query parameters such as "company name" and "keywords", accesses search engine APIs, and returns an index list containing titles, abstracts, and URLs. Then, the system filters high-value URLs based on relevance scores and passes them to the FetchUrlTool as input parameters. The tool starts a HeadlessBrowser environment to dynamically render target URLs, bypass simple anti-crawling strategies, and accurately extract web text, table data, and publication times, achieving deep acquisition from "index" to "details".

[0088] Directed extraction of internal data (call GetProductTool): For tasks involving private domain knowledge (such as "compare our product parameters"), the system calls internal data query tools. This tool uses pre-configured security credentials to connect to the enterprise's internal ERP or knowledge base, performs SQL / API queries based on entity ID or product model, extracts structured function lists, iteration history, and parameter indicators, and fills in the blind spots of external public information.

[0089] Proliferation of intermediate states (call ExportContent): After each sub-task is executed, the system calls the cache function of the export tool (ExportContent). The application of this tool in the method is not only used to generate the final report, but also responsible for real-time serialization of the raw fragmented data (Raw Data) collected by each tool into JSON format and writing it into Redis cache. This ensures that in long process analysis, even if the system is interrupted, it can continue based on cache data, and at the same time provides a standardized input source for subsequent data cleaning steps.

[0090] Information completion and preliminary reasoning based on LLM: After the tool execution returns data, the system again calls the large language model as a "data organizer". For unstructured text returned by the tool, the model performs information extraction and completion operations to preliminarily integrate structured information sets.

[0091] Step 4: Cross-validation and cleaning.

[0092] To ensure data reliability, the system performs multi-dimensional verification on the information set output in the third stage: by cross-comparing the same source data obtained by different tools (such as whether the "company establishment time" obtained by WebSearchTool and FetchUrlTool is consistent), verifying the authority of the data source (such as whether it comes from an official channel), checking the timeliness of the information (such as whether it is the latest data), eliminating errors, outdated or questionable content, and forming a reliable verified data set.

[0093] More specifically:

[0094] Consistency comparison: Through text similarity calculation or key information (such as date, numerical value) extraction and comparison, judge whether the same source data obtained by different tools is consistent;

[0095] Authority verification: According to the pre-defined authoritative source list (such as official website domain name, government agency domain name), verify the reliability of the data source;

[0096] Timeliness check: Extract the timestamp or publication date in the information and compare it with the current system time to determine whether it is the latest data.

[0097] Finally, the system automatically eliminates errors, outdated or questionable content, forming a reliable verified dataset.

[0098] Step 5: LLM analysis and report generation.

[0099] The verified data is input into a large language model (LLM), which integrates and analyzes the information to generate a structured company background investigation report. In generating the structured vertical domain intelligence analysis report, the LLM further performs self-evaluation and optimization of the analysis results based on preset multi-dimensional evaluation indicators, including information integrity, data consistency, and difference from historical reports, to output report content that better meets the decision-making needs of domain experts.

[0100] The self-evaluation and optimization process is a multi-round iterative optimization process based on a preset, quantifiable evaluation indicator system.

[0101] First draft generation and first round of quantitative evaluation: The LLM first generates a structured report draft based on the verified dataset. Then, the system prompts the model to switch to the 'evaluation expert' role and critically reviews the draft based on preset multi-dimensional evaluation indicators, including but not limited to information integrity, data consistency, and difference from historical reports. The evaluation output is a structured evaluation result, which not only includes qualitative comments for each dimension, but also generates a quantitative score (e.g., using a 0-10 scale) for each dimension.

[0102] Targeted revision and re-evaluation based on evaluation results: The system analyzes the evaluation results to identify the lowest-scoring evaluation dimension and its corresponding qualitative comments. The model then makes targeted revisions based on this. The revision includes two modes:

[0103] Information completion mode: If the comments indicate that specific information is missing (for example, when evaluating a background investigation report for a start-up technology company, if the model finds that the initial draft only lists the CEO information in the "core team" chapter, and misses the CTO and technical team background, resulting in a low score in the "information integrity" dimension), the model will automatically generate a new information query sub-task, trigger supplementary retrieval and text revision of the technical team data, and drive the tool scheduling module to call the corresponding information collection tool (such as WebSearchTool or FetchUrlTool) to perform supplementary retrieval, and integrate the new data into the report.

[0104] Text optimization mode: If the comments point to problems such as ambiguous expressions, unclear logic, or data conflicts, the model directly rewrites, polishes, or reconstructs the report text.

[0105] After revision, the revised report is generated. For example, in the evaluation of a background investigation report of a start-up technology company, if the model finds that the initial draft only lists the CEO's information in the "core team" chapter, and misses the CTO and technical team background, resulting in a lower score in the "information integrity" dimension, the system will automatically trigger the supplementary search and text revision of the technical team data according to this comment. After revision, the system automatically starts the second round of evaluation, and the model scores the revised draft again.

[0106] Iteration termination and output: The system compares the scores of the two rounds before and after. If the overall score or the score of the lowest dimension reaches the preset condition, including the termination of the standard and the convergence termination, the iteration is terminated, and the current report is taken as the final output. Otherwise, repeat the previous step and perform a new round of revision and evaluation.

[0107] The preset condition is condition one (termination of the standard): the overall score or the score of the lowest dimension has reached the preset satisfaction threshold (for example, the overall score is greater than or equal to 8).

[0108] The preset condition is condition two (convergence termination): the improvement amplitude of the scores of the two consecutive rounds is less than a certain minimum value (for example, the improvement score is less than 0.5), indicating that the optimization has approached the limit.

[0109] If any of the termination conditions is met, the iteration stops, and the current report is taken as the final output of the structured vertical field intelligence analysis report. Otherwise, the current report is taken as input, and the "targeted revision" and subsequent evaluation steps are repeated.

[0110] For example, in the initial draft of the background investigation report of the aforementioned start-up technology company, the "information integrity" dimension score is low (for example, 5). The system triggers the information completion mode according to this, automatically plans the "query [company name] CTO background" subtask, calls the WebSearchTool for supplementary search, revises the report after obtaining the information, and improves the dimension score to 8 in the next round of evaluation.

[0111] This mechanism ensures that the report is constantly approaching the preset quality standard during the generation process, realizes the qualitative change from "one-time generation" to "iterative optimization", and significantly improves the reliability and professionalism of the output results.

[0112] Another manifestation of the self-evaluation and optimization process can be: the vertical field-specific task planning knowledge deeply coupled with the core process of the system. This process not only optimizes the report itself, but also aims to ensure that the final output is completely aligned with the initial planning intention, and all conclusions have reliable sources.

[0113] Plan-based completeness check: The system first retrieves the structured task plan (containing the subtask list) generated in 'Step 2'. After generating the report, the large language model will act as an "auditor" and go through the subtask list in the plan item by item to verify whether the final report provides an independent and conclusive analysis paragraph for each subtask. If it detects missing content, the model does not simply perform text polishing, but automatically triggers an "information supplement" sub-process: that is, according to the missing task content, a new information query instruction is generated, and the tool scheduling module is driven to re-call or supplement the relevant information collection tools (such as WebSearchTool) to obtain the required data, and then integrate the new data into the report. For example, when checking a company background investigation report, the model first checks whether the "financing history" subtask in the task list is presented in the final report with a corresponding independent paragraph. If it is found that the "financing history" subtask in the task list is not in the report (content missing), the system will automatically plan and execute a targeted search for the company's financing news, supplement the relevant tools or data to complete the content of this part, and ensure the task loop.

[0114] Traceability verification and conflict resolution: The model performs data traceability on each key factual conclusion (such as product parameters, financial data, and key event time) in the report. The system checks whether the conclusion is derived from the verified data set produced in 'Step 4', and requires the information source to be marked in the report or appendix (for example, note that "the data is from the internal database V2.1 version queried by GetProductTool"). When the system finds that the same fact (such as "the date of company establishment") exists in different sources (such as official annual report FetchUrlTool and commercial database WebSearchTool), it will start the default conflict resolution protocol. The protocol is usually: priority adoption (for example, preferentially adopt data from official websites, government registration, and other authoritative sources), or "doubtful annotation" (when it cannot be decided, explain in the report that there are different opinions and their respective sources). This step greatly enhances the auditability and academic rigor of the report.

[0115] Combined with the structured enhancement of domain ontology - to ensure format professionalism: the system uses the industry standard knowledge structure defined by the domain task ontology library constructed in'step 2' as a strong constraint condition for formatting the report. The model automatically reorganizes and optimizes the chapter structure of the report according to the intelligence dimension hierarchy and order defined in the ontology library. For example, in the "competitive analysis ontology", "company profile" is clearly defined as containing four sub-dimensions of "founding time, registered location, legal representative, registered capital" and presented in this order. The system will force the "company profile" chapter in the final report to strictly follow this structure and order, eliminate redundant information, and complete missing dimensions, thereby generating a professional report that not only has accurate content, but also has highly standardized format and conforms to the inherent cognitive framework of domain experts.

[0116] Through the above steps, the'self-evaluation and optimization' of the report is no longer an isolated post-processing, but a deep quality assurance link closely linked with the pre-task planning and mid-term tool execution, forming a quality control chain throughout the process.

[0117] This embodiment takes the vertical field of "competitive product monitoring" as an example to fully demonstrate the end-to-end automated process from user natural language requirements to structured intelligence reports, fully demonstrating the effectiveness and practicality of the method.

[0118] The beneficial effects achieved by the present application are:

[0119] The end-to-end full-process automation from user demand understanding to high-quality report generation is achieved, with high universality, flexibility, interpretability and good scalability, greatly improving the efficiency, depth and intelligent level of enterprises in vertical field intelligence analysis.

[0120] 1. High universality and flexibility: the framework is not bound to a specific field, and by registering different tool sets and domain knowledge, it can quickly adapt to a variety of complex task scenarios such as business competitive product monitoring, medical research and development tracking, policy and regulation analysis, market research, etc., showing strong ecological expansion capability.

[0121] 2. End-to-end automation and continuous optimization: full-process automation is achieved from original user demand understanding, intelligent task planning, tool scheduling execution, information verification and cleaning to report generation. In particular, through the multi-round iterative self-evaluation and optimization mechanism, the system can automatically revise and improve the report based on quantitative indicators, ensuring the professional quality of the output results without manual intervention in key steps.

[0122] 3. Excellent interpretability and traceability: The planning logic of the entire workflow (the basis for task decomposition and the reasons for tool matching), execution steps, data sources and verification results are clearly recorded and stored in a structured manner, making the final analysis conclusions and decision-making process completely transparent, auditable and traceable, meeting the requirements of enterprise-level applications for process credibility.

[0123] 4. Modularity and Scalability: The loosely coupled modular design allows for flexible expansion with new tools, replacement with different LLM models, enrichment of domain task knowledge bases, or adoption of different state storage schemes, giving the system strong adaptability and long-term evolution potential.

[0124] This invention may have many other embodiments. The embodiments described above are merely specific implementations of this invention, used to exemplify the technical solutions of this invention, and not to limit it. The scope of protection of this invention should cover any modifications, equivalent substitutions, and functional extensions made by those skilled in the art based on the core technical concept and principles of this invention. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, all of which should be included within the scope of protection defined by the claims of this invention.

Claims

1. A vertical domain intelligence analysis method based on a large language model, characterized in that, Includes the following steps: Step 1: Receive the analysis request from the user in natural language input; Step 2: Based on pre-built vertical domain-specific task planning knowledge, the analysis requirements are decomposed into a list of sub-tasks corresponding to multiple preset dimensions using a large language model; Using the large language model, based on the semantic matching of prompt words and tool descriptions, appropriate information collection tools are assigned to each subtask to generate a structured task plan containing the execution order; the vertical domain-specific task planning knowledge is implemented through at least one of the following methods: predefined structured prompt word templates, domain task ontology libraries, or large language models fine-tuned with domain data; Step 3: According to the structured task plan, schedule and execute the corresponding toolset in sequence to collect and process target information; the toolset includes at least one of web search tools, web scraping tools, and internal data query tools; Step 4: Perform cross-validation and cleaning on the fragmented information obtained after the tool is executed to form a validated dataset; Step 5: Input the verified dataset into a large language model for integration and analysis to generate a structured vertical domain intelligence analysis report.

2. The vertical domain intelligence analysis method based on a large language model as described in claim 1, characterized in that: Step 2 specifically includes: checking and integrating historical dialogue records and task statuses related to the current analysis needs to construct contextual information for task planning; calling a large language model for reasoning based on a preset structured prompt word template and the contextual information; and persistently storing the structured task plan generated by the large language model in an in-memory database for state management.

3. The vertical domain intelligence analysis method based on a large language model as described in claim 2, characterized in that: In step 2, the large language model is a model fine-tuned by domain task planning data. The large language model controls the randomness and creativity of the generated task plan by adjusting its temperature parameter.

4. The vertical domain intelligence analysis method based on a large language model as described in claim 1, characterized in that: In step 2, the information collection tool is registered and managed based on a unified standardized interface, supporting dynamic expansion and flexible scheduling of the tool.

5. The vertical domain intelligence analysis method based on a large language model as described in claim 1, characterized in that: In step 3, the toolset includes: Thinking and planning tools are used to perform secondary decomposition of complex subtasks in the aforementioned set of subtasks; An export tool is used to export the final generated structured analysis report as a standardized document or store it in a cache.

6. The vertical domain intelligence analysis method based on a large language model as described in claim 1, characterized in that: In step 4, the cross-validation and cleaning include: Compare the consistency of data from the same source obtained by different tools; Verify the authority of the data source; The timeliness of information verification.

7. The vertical domain intelligence analysis method based on a large language model as described in claim 2, characterized in that: In step 5, the in-memory database is also used to store the verified dataset and the structured vertical domain intelligence analysis report.

8. The vertical domain intelligence analysis method based on a large language model as described in claim 7, characterized in that: In step 5, when generating the structured vertical domain intelligence analysis report, the large language model further performs self-evaluation and optimization of the analysis results based on preset multi-dimensional evaluation indicators to output report content; the multi-dimensional evaluation indicators include information completeness, data consistency and the degree of difference between historical reports.

9. The vertical domain intelligence analysis method based on a large language model as described in claim 8, characterized in that, The self-evaluation and optimization process specifically executes the following multi-round iterative optimization steps: Draft generation and first-round evaluation: Input the validated dataset into the large language model to generate a draft structured report; prompt the large language model to switch to the preset role, review the draft structured report according to the multi-dimensional evaluation indicators, and output a structured evaluation result containing qualitative comments and quantitative scores for each dimension; Targeted revision: Analyze the structured evaluation results to identify the evaluation dimensions with the lowest scores and their corresponding qualitative comments; based on the qualitative comments, the large language model automatically plans and triggers the invocation of corresponding information collection tools for supplementary queries, or directly makes targeted revisions to the text content; Iteration Termination and Output: Initiate the second round of evaluation of the revised report to obtain a new round of quantitative scores; compare the quantitative scores of the two rounds. If the overall score or the score of the lowest dimension meets the preset conditions, the preset conditions include target termination and convergence termination, then terminate the iteration and use the current report as the final output; otherwise, use the current report as the new input, repeat the previous step, and perform a new round of revision and evaluation.

10. The vertical domain intelligence analysis method based on a large language model as described in claim 8, characterized in that, The self-assessment and optimization process includes: Completeness check based on the planning list: retrieve the structured task plan generated in step 2 and its sub-task list; check whether the verification report provides an independent analysis conclusion paragraph for each sub-task in the sub-task list; if the content of the corresponding sub-task is missing, the model automatically generates a supplementary query sub-task for the missing information, and drives the system to schedule relevant information collection tools or retrieve existing data to complete the information, so as to realize the task closed loop. Source tracing and conflict resolution: Data tracing is performed on the key claims in the report to check whether the information originates from the verified dataset in step 4, and the specific source tools are marked; if the same fact is found to have conflicting statements between different tool sources, the conflict resolution logic is initiated, which includes: prioritizing the adoption of the more authoritative source, or explaining the difference in the form of a note in the report; Structured enhancement based on domain ontology: Using the domain task ontology library described in step 2 as a constraint, the hierarchical structure of the report is optimized, forcing the chapter content in the report to be presented in strict accordance with the order of the sub-dimensions defined in the ontology library.

Images