An agile public opinion early warning method and system based on a local large model

Abstract

This invention discloses an agile public opinion early warning method and system based on a local large-scale model, relating to the field of public opinion monitoring technology. The method includes: acquiring public data sources and non-public enterprise data sources; constructing a dynamic topic library and deploying a localized large-scale language model; the dynamic topic library includes user-defined topics, each topic including a topic ID and a natural language description; consuming raw information from a message queue and then constructing a classification task based on all activated topics, sending it to the localized large-scale model for classification inference to obtain one or more matching topic IDs; matching the classification inference results with user-subscribed topics, and pushing the matching results according to corresponding push rules. The method and system provided by this invention, by constructing a dynamic topic library and utilizing a localized large-scale language model as an efficient, secure, and low-cost classifier, achieve a lightweight, locally deployable, and highly flexible agile public opinion early warning system.

Description

Technical Field

[0001] This invention relates to the field of public opinion monitoring technology, and in particular to an agile public opinion early warning method and system based on a local large model. Background Technology

[0002] Public opinion monitoring refers to the activity of organizations or individuals using technological means to automatically collect, analyze, and report publicly available information on the Internet in order to understand the dynamics and opinions of the public, customers, or competitors. With the explosive growth of social media and self-media, public opinion monitoring has become crucial for corporate brand maintenance, crisis public relations, market strategy formulation, and competitor analysis.

[0003] Currently, public opinion monitoring services in the market are mainly provided by large SaaS (Software as a Service) platforms. These platforms are typically comprehensive and have wide coverage, but they also have a series of drawbacks that severely restrict their use by small and medium-sized enterprises (SMEs). The main disadvantages of existing SaaS public opinion monitoring platforms are as follows: First, high operating costs. Traditional SaaS public opinion monitoring platforms usually adopt a model of charging by annual subscription, by the number of keywords, or by the number of information sources. Their target customers are mainly large enterprises and government agencies, and the pricing strategy constitutes an insurmountable barrier for SMEs with limited budgets. Second, high data privacy and security risks. Most SaaS platforms require customers to submit the keywords they care about, the monitoring topics, and even internal information to third-party cloud servers. This means that the company's core trade secrets (such as product codenames under development, potential litigation risks, and key competitor strategies) are completely exposed to third-party service providers, posing extremely high data leakage and security risks. Third, bloated functions and poor flexibility. In order to meet the demand for "large and comprehensive" services, large platforms integrate complex semantic analysis, sentiment scoring, AI report generation, and online troll identification functions. These functions are not only complex and costly to learn for SMEs that only need "rapid alerts," but also result in cumbersome systems and slow responses. Fourth, the tagging systems are rigid and fail to align with business needs. Most existing technologies use general, open tagging systems (e.g., political, economic, livelihood, positive, negative). However, the actual needs of SMEs are often very specific and personalized (e.g., "draft of new regulations for XX industry," "defects in competitor A's XX product," "key blogger B's evaluation of our company's product"). Traditional systems either cannot provide such detailed tags or rely on expensive manual customization services. Fifth, local deployment is difficult. Most SaaS platforms are based on their private cloud architecture and do not provide or provide very few on-premise deployment options. This makes it unusable for enterprises with strict data security requirements (such as the financial, medical, and legal industries). Sixth, traditional AI models have limitations. Some systems attempt to use AI for automatic tagging, but traditional models or general cloud-based large model APIs have problems. On the one hand, keyword matching is too rigid, unable to recognize context, irony, or metaphor, resulting in high false negative and false positive rates. On the other hand, when using general AI models (open-ended) for open-ended tagging, the costs are high (charged per token), and the tagging results are inconsistent and non-convergent (for example, the model might label the same article as "AI" one time and "artificial intelligence" the next), making it difficult to establish subsequent automated filtering and recommendation rules. Seventh, it fails to integrate non-public data sources. Existing technologies almost entirely focus on "public" data from the internet.However, non-public data within enterprises (such as customer tickets, CRM records, and internal mailing lists) is often a "gold mine" for discovering potential crises and opportunities. However, existing SaaS platforms, due to their cloud architecture, are unable to handle this highly sensitive non-public data in terms of both technology and security compliance, resulting in a fragmented and incomplete view of corporate public opinion. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide an agile public opinion early warning method and system based on a local large model. By constructing a lightweight, locally deployable, highly flexible and extremely low-cost public opinion early warning framework, the public opinion monitoring task is transformed from "open semantic understanding" to "closed topic matching", and a localized large language model is used as an efficient, safe and low-cost classifier.

[0005] Firstly, this invention provides an agile public opinion early warning method based on a local large-scale model, including: Multi-source heterogeneous information collection process: acquiring public data sources and non-public enterprise data sources; connecting to non-public enterprise data sources in a secure and pluggable manner through a private data connector; and sending the acquired data sources, after labeling their sources, to a message queue. Business logic configuration process: Build a dynamic topic library and deploy a localized large language model; the dynamic topic library includes user-defined topics, each topic including a topic ID and a natural language description; configure user subscription topics and corresponding push rules; Data processing: Consume raw information from the message queue, then construct a classification task based on all active topics in the dynamic topic library, send it to the localized large model for classification reasoning, and obtain one or more matching topic IDs; store the article text, metadata, and matching topic IDs in the structured information database; The message push process involves querying newly added information in the structured information database, matching it with the user's subscribed topics, and then pushing the matching results according to the corresponding push rules.

[0006] Furthermore, in the process of collecting multi-source heterogeneous information, publicly available structured information sources are obtained by subscribing to polling machines, while unstructured and dynamic information sources are obtained by distributed crawler clusters.

[0007] Furthermore, the business logic configuration process also includes adding, deleting, and modifying topics in the dynamic topic library, as well as managing topic status.

[0008] Furthermore, during the configuration of the business logic, the push rules include push channels, timeliness levels, and silent periods.

[0009] Furthermore, the timeliness level includes instant push and summary push. When configured as summary push, multiple pieces of information are aggregated into a summary through a localized large language model and then sent.

[0010] Secondly, this invention provides an agile public opinion early warning system based on a local large model, comprising: The multi-source heterogeneous information acquisition module is used to acquire public data sources and non-public enterprise data sources; it connects to the non-public enterprise data sources in a secure and pluggable manner through a private data connector; and it sends the acquired data sources to a message queue after labeling their sources. The business logic configuration module is used to build a dynamic topic library and deploy a localized large language model; the dynamic topic library includes user-defined topics, each topic including a topic ID and a natural language description; and configures user subscription topics and corresponding push rules. The data processing module consumes raw information from the message queue, then constructs a classification task based on all active topics in the dynamic topic library, sends it to the localized large model for classification reasoning, and obtains one or more matching topic IDs; it then stores the article text, metadata, and matching topic IDs into a structured information database. The message push process involves querying newly added information in the structured information database, matching it with the user's subscribed topics, and then pushing the matching results according to the corresponding push rules.

[0011] Furthermore, in the multi-source heterogeneous information acquisition module, publicly available structured information sources are obtained through a subscription poller, and unstructured and dynamic information sources are obtained through a distributed crawler cluster.

[0012] Furthermore, the business logic configuration module is also used to add, delete, and modify topics in the dynamic topic library and manage topic status.

[0013] Furthermore, in the business logic configuration module, the push rules include push channels, timeliness levels, and silent periods.

[0014] Furthermore, the timeliness level includes instant push and summary push. When configured as summary push, multiple pieces of information are aggregated into a summary through a localized large language model and then sent.

[0015] The technical solutions provided in the embodiments of the present invention have at least the following technical effects: 1. Achieve unified and secure analysis of internal and external data: Employing an architecture combining a "private data connector" and a "localized large-scale model," it enables the analysis of highly sensitive, non-public data such as CRM, work orders, emails, and reports. Because the LLM is deployed locally, sensitive data remains 100% within the internal network, completely resolving the fundamental technical and compliance challenges of SaaS platforms in handling internal data. It breaks down data silos between "external public opinion" and "internal feedback." Enterprise managers can obtain a complete, interconnected global view, such as automatically linking "negative complaints on social media" with "internal channel messages," thereby achieving earlier and more accurate crisis warnings and root cause analysis.

[0016] 2. Balancing extremely low cost with extremely high data security: By deploying a localized large-scale model, all computation and data processing are completed on the user's local (or private cloud) server. The benefits are twofold: Firstly, (cost) it avoids pay-per-use fees for expensive third-party LLM APIs (such as OpenAI, Google), with marginal costs approaching zero after the initial hardware investment. Secondly, (security) sensitive data such as monitored topics, keywords, and acquired public opinion information are 100% kept within the internal network, resolving the data privacy and trade secret leakage risks associated with SaaS platforms.

[0017] 3. Achieving extremely high monitoring accuracy and consistency: Through an innovative "topic-matching" model, open-ended tagging is transformed into a closed-set classification task. The benefits are twofold: Firstly, (accuracy): Compared to keyword matching, LLM can deeply understand context, metaphors, and irony, accurately determining whether an article is related to the "topic description," significantly reducing false positives and false negatives. Secondly, (consistency): Compared to open-ended AI tagging, the output of this invention is a fixed topic ID, rather than unpredictable natural language tags. This ensures that downstream automated push rules ("When T102 occurs, immediately notify Zhang San") can be executed with 100% reliability.

[0018] 4. Unparalleled Business Agility and Flexibility: By decoupling core logic from the "dynamic topic library," zero-shot classification is achieved using LLM. The benefits are: when business needs change (e.g., to start monitoring a new competitor or a new regulation), users simply add a new topic description in the management interface. The system requires no code modification or model retraining; in the next minute of polling, the new topic is automatically included in the LLM's classification task, enabling instant monitoring of new events.

[0019] 5. Lightweight and Easy-to-Deploy System: A microservices and containerization (Docker) framework can be adopted (all functional modules except the local main model can run as lightweight containers (such as Docker) on the same server's CPU or distributed across multiple virtual machines), focusing on the core "early warning" function. The benefits are twofold: Firstly, (lightweight) it is designed specifically for small and medium-sized enterprises, eliminating the bloated functions of large SaaS platforms, resulting in low system resource consumption and high operating efficiency. Secondly, (easy deployment) the entire system can be packaged as a docker-compose file, enabling "one-click startup" on the customer's server, greatly reducing the complexity of local deployment and maintenance.

[0020] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0021] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0022] Figure 1 This is a flowchart illustrating the overall process of the method in Embodiment 1 of the present invention. Figure 2 This is a flowchart of the agile public opinion early warning execution based on a local large model in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the system structure in Embodiment 2 of the present invention; Figure 4 This is a schematic diagram of the system architecture hierarchy and data flow in Embodiment 2 of the present invention. Detailed Implementation

[0023] This invention provides an agile public opinion early warning method and system based on a local large model. By constructing a lightweight, locally deployable, highly flexible and extremely low-cost public opinion early warning framework, the public opinion monitoring task is transformed from "open semantic understanding" to "closed topic matching". The localized large language model is used as an efficient, safe and low-cost classifier.

[0024] The overall concept of the technical solutions in the embodiments of the present invention is as follows: First, a multi-source heterogeneous information acquisition layer is constructed. This layer adopts a multi-source heterogeneous information acquisition strategy. For structured information sources (such as government websites and mainstream news organizations), a configurable RSS / Atom subscription poller is established to check for updates to the subscription sources at a high frequency (e.g., every 1-5 minutes). For unstructured and dynamic information sources (such as social media bloggers in specific fields and industry forums), a distributed crawler cluster based on technologies such as Scrapy or Playwright is built to perform deep crawling of target sites. For non-open-source data within the enterprise, a set of "Proprietary Data Connectors" is provided. These connectors can securely access internal enterprise systems (such as Zendesk tickets, Salesforce records, internal knowledge bases, etc.) through APIs, database polling, IMAP email crawling, etc. All collected raw information (whether public or private) is labeled with its source and then sent to a message queue (such as RabbitMQ or Redis Pub / Sub) to achieve asynchronous decoupling from subsequent processing layers.

[0025] Second, an innovative data processing layer (Processing Layer) is established. This layer is the core innovation. The system maintains a "Dynamic Theme Database," where users can define, modify, or delete "topics" of interest at any time. A topic is not only a keyword but also a complete natural language description (e.g., "Topic ID: T101, Description: Any news about our competitor 'Company A' releasing a new product or feature"). A Processing Worker consumes raw information from the message queue, cleans it, extracts the main text, and pulls all active topics from the "Dynamic Theme Database." The worker constructs a specific prompt, sending the cleaned main text and all active topic descriptions together to a localized large language model (e.g., qwen2.5:32b) deployed through Ollam. The task of this Prompt is classification rather than questioning (e.g., "Please determine whether the following [article] is related to any topic in the [topic list]? If related, please only return the corresponding topic ID. If not related, please return 'NONE'."). The local LLM performs inference, returning one or more matching topic IDs. Because this is a classification task with a "closed set," the LLM output is highly consistent and accurate, and its computational cost is far lower than that of generative tasks. Finally, the worker stores the article text, metadata (including data sources), and matched topic IDs in a processed database (such as PostgreSQL or Elasticsearch).

[0026] Thirdly, there is a flexible business logic layer. This layer is responsible for managing users, push rules, and the topic library. The user configuration module allows users to subscribe to topic IDs they are interested in. The push rules module allows users to set different push rules for each subscribed topic, including push channels (email, WeChat Work, DingTalk, Slack, Webhook, etc.), timeliness levels (e.g., "instant push" (5-minute intervals), "summary push" (30-minute or 1-hour intervals)), and quiet periods (e.g., do not disturb at night).

[0027] Fourth, establish an agile application and push layer. A scheduler, based on the definitions in the push rules module, periodically (e.g., every 5 minutes) queries the structured information repository for information added within the past 5 minutes that has been subscribed to by the user. The push aggregator, based on the user's timeliness settings, decides whether to push a single message immediately or aggregate multiple messages into a digest. Finally, a multi-channel gateway is responsible for accurately sending the formatted alert information to the target user through the channel selected by the user (Email, Webhook, etc.).

[0028] The essence of this invention lies in the scalability of its framework: First, horizontal expansion (information sources). Simply adding a new RSS source, configuring a new crawler target, or developing a new private data connector at the information collection layer requires no modification to subsequent logic. Second, vertical expansion (monitoring scope). Simply adding a new topic description (whether for public or internal data) to the "dynamic topic library" allows the system to immediately begin monitoring information related to the new topic, without retraining the model, achieving extremely high business agility. Example 1

[0029] This embodiment provides an agile public opinion early warning method based on a local large model, such as... Figure 1 and Figure 2 As shown, including; S1. Multi-source heterogeneous information collection process: Acquire public data sources and non-public enterprise data sources; connect to non-public enterprise data sources in a secure and pluggable manner through private data connectors; after labeling the source of the acquired data sources, send them to the message queue; specifically, obtain public structured information sources (such as government websites and mainstream news organizations) through subscription polling (RSS / Atom), and obtain unstructured and dynamic information sources (such as social media bloggers and industry forums in specific fields) through distributed crawler clusters.

[0030] S2-1, Business Logic Configuration Process: Build a dynamic topic library and deploy a localized large language model (such as Ollama); the dynamic topic library includes user-defined topics, each topic including a topic ID and a natural language description; configure user subscription topics and corresponding push rules; allow users to subscribe to topic IDs they are interested in, and allow users to set different push rules for each subscribed topic.

[0031] By adopting an architecture that combines "private data connectors" with "localized large-scale models," it becomes possible to analyze highly sensitive, non-public data within an enterprise, such as CRM, work orders, emails, and reports. Because the LLM is deployed locally, sensitive data is 100% kept within the internal network, completely solving the fundamental technical and compliance challenges faced by SaaS platforms in handling internal data.

[0032] S2-2 Data Processing Process: The Processing Worker consumes raw information from the message queue, then constructs a classification task based on all active topics in the dynamic topic library, sends it to the localized large model for classification inference, and obtains one or more matching topic IDs (e.g., "Please determine whether the following [article] is related to any topic in the [topic list]? If related, please return only the corresponding topic ID. If not related, please return 'NONE'."). Because this is a classification task with a "closed set," the LLM output is highly consistent and accurate, and the computational cost is much lower than that of generative tasks. Finally, the Processing Worker stores the article text, metadata, and matching topic IDs in a structured information repository.

[0033] Preferably, based on the classification task constructed from all active topics in the dynamic topic library, a two-stage coarse-fine filtering mechanism is also introduced: Level 1 (Coarse Screening): Use lightweight traditional NLP techniques (common keyword vectors) to quickly filter out the top 10 potentially relevant topics.

[0034] Level 2 (Fine Screening): Only detailed descriptions of the Top-10 topics are included in the Prompt and sent to the LLM for final confirmation.

[0035] A user might define 100 topics. If a detailed description of all 100 topics is crammed into the Prompt for each request, it leads to several problems: 1. Exceeding the local model's context window limit; 2. Extremely slow inference speed; 3. Distracted model attention and decreased accuracy. By introducing a two-level coarse-to-fine screening mechanism, computational overhead is significantly reduced without sacrificing accuracy, and it can run on general hardware configurations.

[0036] S3. Message Push Process: The process involves querying the structured information database for newly added information, matching it with the user's subscribed topics, and then pushing the matching results according to the corresponding push rules. Specifically, a scheduler can periodically (e.g., every 5 minutes) query the structured information database for information added within the past 5 minutes that has been subscribed to by the user, based on the user's timeliness settings. The push aggregator determines whether to push a single message immediately or aggregate multiple messages into a digest based on the user's timeliness level settings. Finally, the multi-channel gateway is responsible for accurately sending the formatted alert information to the target user through the channel selected by the user (Email, Webhook, etc.).

[0037] By employing an innovative "topic-matching" model, open-ended tagging is transformed into a closed-set classification task. On one hand, compared to keyword matching, LLM (Local Level Management) can deeply understand context, metaphors, and irony, accurately determining whether an article is relevant to the "topic description," significantly reducing false positives and false negatives. On the other hand, compared to open-ended AI tagging, the output of this invention is a fixed topic ID, rather than unpredictable natural language tags. This ensures that downstream automated push rules ("notify Zhang San immediately when T102 occurs") can be executed with 100% reliability.

[0038] Preferably, the business logic configuration process also includes managing the addition, deletion, and modification of topics in the dynamic topic library, as well as the topic status (e.g., "active" or "dormant"), enabling agile adjustments to the business. When business requirements change (e.g., needing to start monitoring a new competitor or a new regulation), the user only needs to add a new topic description in the management interface. The system requires no code modification or model retraining; in the next minute of polling, it can automatically incorporate the new topic into the LLM classification task, instantly enabling the monitoring of new events.

[0039] Preferably, during the configuration of the business logic, the push rules include push channels (email, WeChat Work, DingTalk, Slack, Webhook, etc.), timeliness levels (e.g., "instant push" (5-minute level), "summary push" (30-minute or 1-hour level)) and silent periods (e.g., do not disturb at night). When configured as summary push, multiple messages are aggregated into a summary through a localized large language model and then sent to meet the diverse subscription needs of users.

[0040] Based on the same inventive concept, this application also provides a system corresponding to the method in Embodiment 1, as detailed in Embodiment 2. Example 2

[0041] This embodiment provides an agile public opinion early warning system based on a local large model, such as Figure 3 As shown, it includes: The multi-source heterogeneous information acquisition module is used to acquire public data sources and non-public enterprise data sources; it connects to the non-public enterprise data sources in a secure and pluggable manner through a private data connector; and it sends the acquired data sources to a message queue after labeling their sources. The business logic configuration module is used to build a dynamic topic library and deploy a localized large language model; the dynamic topic library includes user-defined topics, each topic including a topic ID and a natural language description; and configures user subscription topics and corresponding push rules. The data processing module consumes raw information from the message queue, then constructs a classification task based on all active topics in the dynamic topic library, sends it to the localized large model for classification reasoning, and obtains one or more matching topic IDs; it then stores the article text, metadata, and matching topic IDs into a structured information database. The message push process involves querying newly added information in the structured information database, matching it with the user's subscribed topics, and then pushing the matching results according to the corresponding push rules.

[0042] In one possible implementation, the multi-source heterogeneous information acquisition module obtains publicly available structured information sources by subscribing to a poller, and obtains unstructured and dynamic information sources by a distributed crawler cluster.

[0043] Preferably, the business logic configuration module is also used to add, delete, and modify topics in the dynamic topic library and manage the topic status.

[0044] Preferably, in the business logic configuration module, the push rules include push channels, timeliness levels, and silent periods.

[0045] Preferably, the timeliness level includes instant push and summary push. When configured as summary push, multiple pieces of information are aggregated into a summary through a localized large language model and then sent.

[0046] In one specific embodiment, such as Figure 4 As shown, the system architecture layers are as follows: The first layer is the data sources. This includes public data sources such as "government / news RSS feeds," "social media (Weibo, X)," and "industry forums," as well as non-public data sources such as "enterprise CRM," "customer ticketing systems," and "internal email services." The data flow arrows point to the information collection layer.

[0047] The second layer is the Acquisition Layer. This includes an RSS poller and web crawler service for handling public data, and a private data connector for handling non-public data. All three modules push data to a message queue (e.g., RabbitMQ).

[0048] Third is the Data Processing Layer. This includes processing workers that pull data from the message queue; the Ollam local model service, deployed as qwen2.5:32b and invoked by the processing workers; and a dynamic topic library, which is read by the processing workers and written to by the configuration management interface. After processing, the processing workers write the data to a structured information repository (e.g., Elasticsearch).

[0049] Fourth is the Business Logic Layer. This includes the push scheduler, the on-time query module, and the user / rule base; data is read by the push scheduler and written to the configuration management interface; the configuration management interface is used to manage the dynamic topic library and the user / rule base.

[0050] Fifth is the Application & Push Layer. This includes a multi-channel push gateway that receives instructions from the push scheduler and sends alerts to end users via email, WeChat / DingTalk, webhooks, and other methods.

[0051] Hardware deployment and service configuration In one specific embodiment, the system can be deployed on a single private server or in a customer's private cloud environment.

[0052] Recommended server configuration: It is recommended to use a server with a mid-to-high-end NVIDIA GPU (such as RTX 4070 or higher) to run Ollam local models.

[0053] Service deployment method: Other modules can be run as lightweight containers (such as Docker) on the CPU of the same server, or distributed in multiple virtual machines.

[0054] Detailed description of core software module functions 1. RSS Poller: Responsible for maintaining a list of RSS feeds, using libraries such as feedparser to periodically check for updates, and pushing the links and titles of new entries into the message queue.

[0055] II. Web Crawler Service: For dynamic pages without RSS feeds. Using the Scrapy framework, manage the proxy IP pool and User-Agents, simulate browser behavior, crawl the dynamic content of specified bloggers, and push it into a message queue.

[0056] 3. Private Data Connectors: Responsible for securely and pluggably accessing non-public enterprise data sources. For example, a Zendesk connector can periodically pull new work orders via API; an IMAP connector can monitor specific internal email accounts; and a database connector can poll customer feedback tables in the CRM. All data is uniformly formatted and pushed into a message queue.

[0057] IV. Processing Workers: During OnMessage, they retrieve raw data (such as URLs) from the queue; during Fetch & Clean, they scrape the URL content and extract the body text using BeautifulSoup or readability-lxml; during GetThemes, they query the dynamic topic library for a list of all topics with is_active=true (including IDs and descriptions); during BuildPrompt, they build a Prompt for a classification task, in the format described above; during CallLLM, they call Ollam's API via an HTTP request and obtain the response (e.g., "T102"); during Save, they store the body text, metadata, and the response "T102" into a structured information repository.

[0058] 5. Ollam Local Model Service: Runs the qwen2.5:32b model. It provides a standard API endpoint to receive prompts from processing workers and perform inference. Due to local calls, network latency is extremely low, and all data (including highly sensitive internal data) remains within the intranet.

[0059] VI. Push Scheduler: Its OnTimer (every 5 minutes) is responsible for querying SELECT * FROM ProcessedDBWHERE timestamp > (NOW() - 5min); FindSubscribers is responsible for finding all users who have subscribed to the topic and whose validity period is "5 minutes" for each piece of information retrieved, based on its topic ID; TriggerPush is responsible for delivering the information content and the target user list to module 10. In addition, there is another Aggregate summary task (for 30-minute intervals), with similar logic, but it will package the information into a summary before pushing it.

[0060] VII. Configuration Management Interface: A web-based (such as React or Vue) management backend allows administrators to add, delete, modify, and query "dynamic theme library" and "user / rule library" to enable agile adjustments to business operations.

[0061] Since the system described in Embodiment 2 of this invention is a system used to implement the method of Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this system based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All systems used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.

[0062] The technical solutions provided in the embodiments of the present invention have at least the following technical effects: 1. Achieve unified and secure analysis of internal and external data: Employing an architecture combining a "private data connector" and a "localized large-scale model," it enables the analysis of highly sensitive, non-public data such as CRM, work orders, emails, and reports. Because the LLM is deployed locally, sensitive data remains 100% within the internal network, completely resolving the fundamental technical and compliance challenges of SaaS platforms in handling internal data. It breaks down data silos between "external public opinion" and "internal feedback." Enterprise managers can obtain a complete, interconnected global view, such as automatically linking "negative complaints on social media" with "internal channel messages," thereby achieving earlier and more accurate crisis warnings and root cause analysis.

[0063] 2. Balancing extremely low cost with extremely high data security: By deploying a localized large-scale model, all computation and data processing are completed on the user's local (or private cloud) server. The benefits are twofold: Firstly, (cost) it avoids pay-per-use fees for expensive third-party LLM APIs (such as OpenAI, Google), with marginal costs approaching zero after the initial hardware investment. Secondly, (security) sensitive data such as monitored topics, keywords, and acquired public opinion information are 100% kept within the internal network, resolving the data privacy and trade secret leakage risks associated with SaaS platforms.

[0064] 3. Achieving extremely high monitoring accuracy and consistency: Through an innovative "topic-matching" model, open-ended tagging is transformed into a closed-set classification task. The benefits are twofold: Firstly, (accuracy): Compared to keyword matching, LLM can deeply understand context, metaphors, and irony, accurately determining whether an article is related to the "topic description," significantly reducing false positives and false negatives. Secondly, (consistency): Compared to open-ended AI tagging, the output of this invention is a fixed topic ID, rather than unpredictable natural language tags. This ensures that downstream automated push rules ("When T102 occurs, immediately notify Zhang San") can be executed with 100% reliability.

[0065] 4. Unparalleled Business Agility and Flexibility: By decoupling core logic from the "dynamic topic library," zero-shot classification is achieved using LLM. The benefits are: when business needs change (e.g., to start monitoring a new competitor or a new regulation), users simply add a new topic description in the management interface. The system requires no code modification or model retraining; in the next minute of polling, the new topic is automatically included in the LLM's classification task, enabling instant monitoring of new events.

[0066] 5. Lightweight and Easy-to-Deploy System: A microservices and containerization (Docker) framework can be adopted (all functional modules except the local main model can run as lightweight containers (such as Docker) on the same server's CPU or distributed across multiple virtual machines), focusing on the core "early warning" function. The benefits are twofold: Firstly, (lightweight) it is designed specifically for small and medium-sized enterprises, eliminating the bloated functions of large SaaS platforms, resulting in low system resource consumption and high operating efficiency. Secondly, (easy deployment) the entire system can be packaged as a docker-compose file, enabling "one-click startup" on the customer's server, greatly reducing the complexity of local deployment and maintenance.

[0067] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0069] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0071] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An agile public opinion early warning method based on a local large-scale model, characterized in that, include: Multi-source heterogeneous information collection process: acquiring public data sources and non-public enterprise data sources; This involves using a private data connector to securely and pluggably access non-public enterprise data sources; and then sending the acquired data sources, after labeling their origins, into a message queue. Business logic configuration process: Build a dynamic topic library and deploy a localized large language model; the dynamic topic library includes user-defined topics, each topic including a topic ID and a natural language description; configure user subscription topics and corresponding push rules; Data processing: Consume raw information from the message queue, then construct a classification task based on all active topics in the dynamic topic library, send it to the localized large model for classification reasoning, and obtain one or more matching topic IDs; store the article text, metadata, and matching topic IDs in the structured information database; The message push process involves querying newly added information in the structured information database, matching it with the user's subscribed topics, and then pushing the matching results according to the corresponding push rules.

2. The method according to claim 1, characterized in that: In the process of collecting multi-source heterogeneous information, publicly available structured information sources are obtained by subscribing to polling machines, while unstructured and dynamic information sources are obtained by distributed crawler clusters.

3. The method according to claim 1, characterized in that: The business logic configuration process also includes adding, deleting, and modifying topics in the dynamic topic library, as well as managing topic status.

4. The method according to claim 1, characterized in that: During the configuration of the business logic, the push rules include push channels, timeliness levels, and silent periods.

5. The method according to claim 4, characterized in that: The timeliness levels include instant push and summary push. When configured as summary push, multiple pieces of information are aggregated into a summary through a localized large language model and then sent.

6. An agile public opinion early warning system based on a local large model, characterized in that, include: A multi-source heterogeneous information acquisition module is used to acquire public data sources and non-public enterprise data sources; This involves using a private data connector to securely and pluggably access non-public enterprise data sources; and then sending the acquired data sources, after labeling their origins, into a message queue. The business logic configuration module is used to build a dynamic topic library and deploy a localized large language model; the dynamic topic library includes user-defined topics, each topic including a topic ID and a natural language description; and configures user subscription topics and corresponding push rules. The data processing module consumes raw information from the message queue, then constructs a classification task based on all active topics in the dynamic topic library, sends it to the localized large model for classification reasoning, and obtains one or more matching topic IDs; it then stores the article text, metadata, and matching topic IDs into a structured information database. The message push process involves querying newly added information in the structured information database, matching it with the user's subscribed topics, and then pushing the matching results according to the corresponding push rules.

7. The system according to claim 6, characterized in that: In the multi-source heterogeneous information acquisition module, publicly available structured information sources are obtained through a subscription poller, and unstructured and dynamic information sources are obtained through a distributed crawler cluster.

8. The system according to claim 6, characterized in that: The business logic configuration module is also used to add, delete, and modify topics in the dynamic topic library and manage topic status.

9. The system according to claim 6, characterized in that: In the business logic configuration module, the push rules include push channels, timeliness levels, and silent periods.

10. The system according to claim 9, characterized in that: The timeliness levels include instant push and summary push. When configured as summary push, multiple pieces of information are aggregated into a summary through a localized large language model and then sent.

Images