Intelligent task management system and operation method thereof

Abstract

The invention provides an intelligent task management system and an operation method thereof, and the system comprises a task scheduling center which is used for receiving a collection task submitted by a user, and obtaining the task data of the collection task through a collection engine; the task analysis module is used for performing security analysis on the task data and processing a security analysis result according to a preset processing strategy to obtain a task processing result; the security analysis comprises sensitive information identification and compliance check, and the disposal strategy comprises abnormal data interception, data desensitization, data annotation and manual auditing; and the management module is used for executing query, configuration and monitoring operations on the collection task according to a management instruction input by the user, the security analysis result and the task processing result, and generating a management report. According to the system and the operation method thereof, the safety of data acquisition is improved, and the data management efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of information security technology, and in particular to an intelligent task management system and its operation method. Background Technology

[0002] With the explosive growth of internet information, enterprises and institutions have an increasingly urgent need to efficiently and accurately obtain specific information from massive amounts of online data (web crawlers).

[0003] In related technologies, when using traditional web scraping tools (such as web crawler scripts) to collect web data, the collected content may contain sensitive information, illegal text or malicious code. Without an effective content security filtering mechanism, directly storing or using such data will bring huge compliance and security risks. Moreover, existing data management platforms have limited functionality and require the integration of multiple functional units to perform different data collection and analysis, resulting in low data management efficiency and high maintenance costs. Summary of the Invention

[0004] This invention provides an intelligent task management system and its operating method, addressing the shortcomings of existing technologies that rely on web crawler scripts for data collection, such as low security and limited functionality of data management platforms, leading to low data management efficiency and high maintenance costs. The system described in this invention improves both the security of data collection and the efficiency of data management.

[0005] This invention provides an intelligent task management system, comprising: The task scheduling center is used to receive collection tasks submitted by users, and the collection task simultaneously uses the collection engine to obtain the task data of the collection task. The task analysis module is used to perform security analysis on the task data and process the security analysis results according to a preset handling strategy to obtain the task processing result. The security analysis includes sensitive information identification and compliance checks, and the handling strategy includes abnormal data interception, data anonymization, data labeling, and manual review. The management module is used to perform query, configuration, and monitoring operations on the data collection task based on the management instructions input by the user, the security analysis results, and the task processing results, and to generate a management report.

[0006] According to the intelligent task management system provided by the present invention, the acquisition engine adopts a headless browser driver to simulate the behavior of a browser, and is used to execute JavaScript code in a webpage and obtain a complete DOM tree structure.

[0007] According to an intelligent task management system provided by the present invention, the task analysis module includes: The sensitive information identification unit is used to match sensitive information in task data using a sensitive word library and a regular expression engine; The compliance check unit is used to identify content in the task data that does not conform to text specifications and to detect malicious code in the task data through natural language processing or third-party content security APIs. The disposal strategy execution unit is used to determine the risk level of the data collection task based on the matching results, identification results, and detection results, and to process the task data according to the disposal strategy associated with the risk level.

[0008] According to an intelligent task management system provided by the present invention, the task scheduling center supports distributed deployment and elastic scaling, and the acquisition engine is used to distribute multiple acquisition tasks to corresponding multiple distributed nodes for execution.

[0009] According to an intelligent task management system provided by the present invention, the system further includes: The storage module is used to store the acquisition task and its data, the task processing results, and the management report; The storage module also supports a variety of storage media, including relational databases, key-value store databases, and NoSQL databases.

[0010] According to the present invention, an intelligent task management system is provided, wherein the management module is built based on the Java SpringBoot Web framework; The management module is also configured to perform at least one of the following based on user instructions: Query the management report and system operating status; Edit the sensitive word database, configure security rules and handling strategies; Query the logs and statistical reports generated by the task analysis module.

[0011] The present invention also provides a method for operating an intelligent task management system, comprising: The task scheduling center receives user-submitted data collection tasks and uses the data collection engine to obtain the task data of the data collection tasks. The task data is subjected to security analysis by the task analysis module, and the security analysis results are processed according to the preset handling strategy to obtain the task processing result; the security analysis includes sensitive information identification and compliance check, and the handling strategy includes abnormal data interception, data desensitization, data labeling and manual review; The management module performs query, configuration, and monitoring operations on the data collection task based on the management instructions input by the user, the security analysis results, and the task processing results, and generates a management report.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the operation method of the intelligent task management system as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the operation method of the intelligent task management system as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the operation method of the intelligent task management system as described above.

[0015] The intelligent task management system and its operation method provided by this invention receive user-submitted data collection tasks through a task scheduling center and acquire task data using a data collection engine; perform security analysis on the task data through a task analysis module and process the security analysis results according to a preset handling strategy; and perform query, configuration, and monitoring operations on the data collection tasks through a management module based on user-input management instructions, security analysis results, and task processing results, and generate management reports, thereby improving the security of data collection and increasing data management efficiency. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is one of the structural diagrams of the intelligent task management system provided by the present invention.

[0018] Figure 2 This is the second structural schematic diagram of the intelligent task management system provided by the present invention.

[0019] Figure 3 This is one of the flowcharts illustrating the operation method of the intelligent task management system provided by the present invention.

[0020] Figure 4 This is the second flowchart illustrating the operation method of the intelligent task management system provided by the present invention.

[0021] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0022] Figure label: 100: Task scheduling center; 200: Task analysis module; 210: Sensitive information identification unit; 220: Compliance Inspection Unit; 230: Disposal Strategy Execution Unit; 300: Management module; 400: Storage module. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] The following is combined with Figures 1-4 The present invention describes the intelligent task management system and its operation method.

[0025] Figure 1 This is one of the structural schematic diagrams of the intelligent task management system provided by the present invention, such as... Figure 1 As shown, the system includes the following: a task scheduling center 100, a task analysis module 200, and a management module 300.

[0026] The task scheduling center 100 is used to receive user-submitted collection tasks and simultaneously use the collection engine to obtain task data for the collection tasks.

[0027] In this embodiment, different scenarios correspond to different data acquisition tasks and task data; for example: (1) For e-commerce price monitoring scenarios, the data collection task can be to periodically collect price information of specified products on a specific e-commerce platform. The corresponding task data includes, but is not limited to, product name, product link, price, discount information, sales volume, number of comments, etc. (2) For social media sentiment analysis scenarios, the data collection task can be to monitor posts or comments related to specific keywords on social media platforms in real time. The corresponding task data can be post content, posting time, poster ID, number of likes, number of comments, number of reposts, keywords, sentiment, etc. (3) For news media content aggregation scenarios, the collection task can be to automatically crawl the latest news titles and summaries from multiple news websites. The corresponding task data includes news titles, news summaries, news links, publication time, source websites, keywords, etc.

[0028] (4) For recruitment information collection scenarios, the collection task can be to collect job information from multiple recruitment websites on a regular basis. The corresponding task data can be job title, company name, salary range, work location, job description, posting time, etc.

[0029] In this embodiment, the task scheduling center 100 is used to receive, parse, manage, and schedule network acquisition tasks submitted by users.

[0030] Specifically, the scheduling center receives collection tasks set by users through a unified management console (e.g., a web interface built on Spring Boot + Vue.js). The tasks include the URL of the target website, the collection frequency, the parsing rules, etc. The scheduling center parses the task instructions and converts them into tasks that the system can execute.

[0031] In this embodiment, the parsed task is placed in a scheduling queue, and the scheduling center starts the collection task at an appropriate time according to a preset scheduling strategy (e.g., a Cron expression based on Quartz).

[0032] In this embodiment, the scheduling center calls the acquisition engine to perform the actual data acquisition work and passes the acquired raw data to the subsequent task analysis module 200.

[0033] The task analysis module 200 is used to perform security analysis on task data and process the security analysis results according to the preset handling strategy to obtain the task processing result. The security analysis includes sensitive information identification and compliance check, and the handling strategy includes abnormal data interception, data desensitization, data labeling and manual review.

[0034] In this embodiment, the task analysis module 200 can perform security scanning and analysis on the task data collected by the acquisition engine to identify potential risks.

[0035] In this embodiment, high-risk data (abnormal data) is directly intercepted and not entered into the database; sensitive information is anonymized (for example, by using special characters such as asterisks); and data whose risk level cannot be determined is labeled "pending review" and submitted for manual review.

[0036] In this embodiment, users can choose to perform security analysis on task data according to their needs, such as only identifying sensitive information, only performing compliance checks, or both.

[0037] In this embodiment, the task analysis module 200 includes: a sensitive information identification unit 210, used to match sensitive information in task data using a sensitive word library and a regular expression engine; a compliance check unit 220, used to identify content in task data that does not conform to text specifications and to detect malicious code in task data using natural language processing or a third-party content security API; and a handling strategy execution unit 230, used to determine the risk level of the collection task based on the matching results, identification results, and detection results, and to process the task data according to the handling strategy associated with the risk level.

[0038] In this embodiment, the sensitive word database contains a variety of sensitive words, such as ID card numbers, bank card numbers, passwords, addresses, etc.; the sensitive word database can be updated regularly to accommodate the emergence of new sensitive words.

[0039] In this embodiment, the sensitive information identification unit 210 identifies potential sensitive information by matching task data with a sensitive word library and regular expressions.

[0040] In this embodiment, the regular expression engine is a tool for matching text patterns. For example, regular expressions can be used to match text in specific formats such as ID card numbers, mobile phone numbers, and email addresses.

[0041] In this embodiment, the task analysis module 200 can also use natural language processing technology or third-party content security APIs (Application Programming Interfaces) (e.g., cloud service content security APIs) to perform compliance checks on the text and identify illegal content (e.g., political content).

[0042] In this embodiment, the compliance inspection unit 220 can also scan code snippets, file links, etc. in the task data to prevent the introduction of attack scripts such as XSS and SQL injection. For example, the compliance inspection unit 220 can also use the DFA (Deterministic Finite Automaton) algorithm to quickly match sensitive words and integrate the Java SDK of the cloud platform content security API to perform more complex compliance checks.

[0043] In this embodiment, the user can preset processing strategies for different risk levels, such as directly intercepting high-risk data, de-identifying medium-risk data, and storing low-risk data normally; then the processing strategy execution unit 230 will process the data according to the preset processing strategy; the processing strategy may include abnormal data interception, data de-identification, data labeling, and manual review.

[0044] The management module 300 is used to perform query, configuration and monitoring operations on the collection tasks based on the management instructions, security analysis results and task processing results input by the user, and to generate management reports.

[0045] Specifically, users can query the execution status, collection results, and security analysis results of tasks; users can configure task parameters (e.g., collection frequency, parsing rules, and handling strategies); users can monitor the running status of tasks, such as CPU (Central Processing Unit) utilization, memory utilization, and network traffic; the management module 300 can also generate various management reports, such as task execution reports, security analysis reports, and system operation reports.

[0046] In a feasible embodiment, assume the data collection task involves crawling job information from a recruitment website. The collected job information may include company names, job descriptions, salary and benefits, etc. Sensitive information identification unit 210 identifies job descriptions containing the phrase "requires a copy of ID card," and compliance check unit 220 identifies salary and benefits containing false advertising information such as "monthly salary of over 100,000." Based on this information, the handling strategy execution unit 230 can determine that the risk level of the task is medium risk. For example, a predefined medium-risk handling strategy includes: deleting job information containing "requires a copy of ID card" and modifying salary and benefits containing false advertising information. The handling strategy execution unit 230 will automatically delete job information containing "requires a copy of ID card" and modify the salary and benefits containing false advertising information to "negotiable" according to the handling strategy. Finally, the administrator can view the security analysis report of a task to understand whether there is sensitive information or illegal content in the data collected by the task. If there is a security risk, the administrator can adjust the task configuration or handling strategy according to the recommendations in the report, such as increasing the sensitivity of the sensitive word library or adjusting the data anonymization rules.

[0047] The intelligent task management system provided in this embodiment of the invention receives user-submitted data collection tasks through a task scheduling center and acquires task data using a data collection engine. A task analysis module performs security analysis on the task data and processes the security analysis results according to a preset handling strategy. A management module performs query, configuration, and monitoring operations on the data collection tasks based on user-input management commands, security analysis results, and task processing results, and generates management reports. This improves the security of data collection and enhances data management efficiency.

[0048] In some embodiments, the data collection engine employs a headless browser driver to simulate browser behavior, execute JavaScript code in the webpage, and obtain the complete DOM tree structure.

[0049] In this embodiment, a headless browser refers to a browser without a graphical user interface that can run in the background to simulate the browser behavior of a real user.

[0050] For example, headless browsers include Headless Chrome, Puppeteer, Selenium WebDriver+PhantomJS, etc.

[0051] In this embodiment, the data collection engine can access the website like a real user, execute JavaScript code, and process cookies, headers, etc., thereby obtaining dynamically loaded content.

[0052] In this embodiment, the data collection engine controls the browser's behavior by driving a program that runs the headless browser, such as opening web pages, clicking buttons, and filling out forms.

[0053] In this embodiment, the DOM (Document Object Model) tree is a structured representation of a webpage, containing all the elements and attributes of the webpage. The headless browser in this embodiment can parse the webpage and generate a complete DOM tree structure, facilitating subsequent data extraction.

[0054] The intelligent task management system provided in this embodiment of the invention simulates browser behavior by setting the collection engine to use a headless browser driver to execute JavaScript code in web pages and obtain a complete DOM tree structure, thereby improving the accuracy and completeness of data collection. At the same time, it can simulate the browser behavior of real users and avoid being blocked by the anti-crawler mechanism of websites.

[0055] In some embodiments, the task scheduling center 100 supports distributed deployment and elastic scaling, and the collection engine is used to distribute multiple collection tasks to corresponding distributed nodes for execution.

[0056] In this embodiment, the task scheduling center 100 is built on Java and mature scheduling frameworks (such as Quartz, XXL-Job or Elastic-Job), and can provide rich RESTful APIs to receive user-submitted collection tasks and parse, manage and schedule them.

[0057] In this embodiment, the task scheduling center 100 also supports multiple triggering strategies (such as simple triggering, Cron expression, dependency triggering), task priority division, failure retry and alarm mechanisms.

[0058] In this embodiment, the data acquisition engine is a distributed data acquisition engine built on the Java Spring Cloud distributed system. This engine has the following capabilities: (1) Intelligent parsing and extraction: It integrates the Jsoup HTML parsing library and supports configurable parsing rules (such as XPath, CSS Selector, and regular expressions), which can accurately extract target data fields from complex HTML structures.

[0059] (2) Anti-crawler strategy: Built-in IP proxy pool polling, request header simulation, and access frequency control mechanism effectively improve the stability and success rate of data collection tasks.

[0060] (3) Distributed execution: Supports the distribution of collection tasks to multiple collection nodes for execution, realizing horizontal scaling and improving the efficiency of large-scale collection.

[0061] In this embodiment, the data collection engine can use Selenium WebDriver (Java API) to drive the headless Chrome browser and use Jsoup for HTML parsing. The proxy pool is managed and maintained using Redis.

[0062] The intelligent task management system provided in this embodiment of the invention supports distributed deployment and elastic expansion by setting up a task scheduling center 100. The acquisition engine is used to distribute multiple acquisition tasks to multiple corresponding distributed nodes for execution, which can effectively cope with large-scale, high-concurrency acquisition scenarios and improve the overall performance, stability and acquisition efficiency of the system.

[0063] In some embodiments, the system further includes: a storage module 400 for storing acquisition tasks and their task data, task processing results and management reports; the storage module 400 also supports a variety of storage media, including relational databases, key-value storage type databases and NoSQL databases.

[0064] In this embodiment, the storage module 400 can select a suitable storage medium for storage based on different data types and access requirements. For example, it can select a relational database (MySQL, TiDB), a key-value store (Redis), or a NoSQL database (Elasticsearch for full-text search, MongoDB for storing unstructured data) to provide a unified API structure for downstream systems to consume data.

[0065] For example, storage module 400 can use MySQL to store task metadata and structured results, and use Elasticsearch to provide retrieval services.

[0066] The intelligent task management system provided in this embodiment of the invention can effectively reduce storage costs and improve the overall performance of the system by setting up a storage module 400 to store the collected tasks and their task data, task processing results and management reports, and supporting multiple storage media.

[0067] In some embodiments, the management module 300 is built on the Java SpringBoot Web framework; the management module 300 is also used to perform at least one of the following according to user instructions: query management reports and system running status; edit sensitive word library, configure security rules and handling strategies; query logs and statistical reports generated by the task analysis module 200.

[0068] In this embodiment, the management module 300 uses Spring Boot to provide a REST API backend, and the frontend uses Vue.js to build a single-page application (SPA).

[0069] In this embodiment, the administrator can view various management reports through the management interface, such as task execution reports and security analysis reports. They can also view the system's operating status, such as CPU usage, memory usage, and network traffic.

[0070] In this embodiment, the administrator can also add, delete, or modify words in the sensitive word database to adapt to new security requirements.

[0071] In this embodiment, the administrator can also configure various security rules according to actual needs, such as IP blacklists and access frequency limits. Various handling strategies can also be configured, such as anonymizing sensitive data and blocking illegal content.

[0072] In this embodiment, the administrator can also view the logs generated by the task analysis module 200 to understand the detailed process of task analysis. They can also view various statistical reports, such as the number of sensitive information identified and the number of violations identified.

[0073] The intelligent task management system provided in this embodiment of the invention is built on the Java Spring Boot Web framework by setting up a management module. The management module is also used to query management reports and system running status according to user instructions; edit sensitive word library, configure security rules and handling strategies; and query logs and statistical reports generated by the task analysis module. It provides a user-friendly management interface, which makes it convenient for administrators to manage and monitor the system, and supports a variety of management functions, further improving the management efficiency of intelligent tasks.

[0074] Figure 2 This is the second structural schematic diagram of the intelligent task management system provided by the present invention. Figure 2In the illustrated embodiment, the user inputs task instructions through the operation entry of the management console. The task scheduling center schedules tasks according to the task instructions and obtains task data (corresponding to data extraction) through the acquisition engine. Then, the task data is security scanned / processed through the security control module (corresponding to the task analysis module), and the obtained data is input into the data storage module (corresponding to the storage module) to store the "clean" data (corresponding to the task processing result). Finally, the data is made available for downstream systems to consume through the API.

[0075] Specifically, after the intelligent task management system in this embodiment is developed, it can be deployed to a Linux server cluster. Users access the management console through a browser, configure multiple scheduled tasks that need to be monitored and managed, and set them to execute at regular intervals. The task scheduling center triggers tasks according to the plan, collecting information and conditions related to the elements under the task into the database. Before the data is entered into the database, the task analysis module automatically identifies data that does not meet the requirements and marks it as "pending review" according to the rules. At the same time, it informs the administrator of the corresponding management user. Finally, the data that meets the requirements is stored in the database for use by the business system.

[0076] The operation method of the intelligent task management system provided by the present invention will be described below. The operation method of the intelligent task management system described below can be referred to in correspondence with the intelligent task management system described above.

[0077] Figure 3 This is one of the flowcharts illustrating the operation method of the intelligent task management system provided by the present invention. Figure 3 This is a flowchart illustrating the operation method of the intelligent task management system provided by the present invention, as shown below. Figure 3 As shown, the operation method of this intelligent task management system includes the following steps: Step 310: Receive the collection tasks submitted by the user through the task scheduling center, and at the same time use the collection engine to obtain the task data of the collection tasks.

[0078] In this step, different scenarios correspond to different data collection tasks and task data; for example: (1) For e-commerce price monitoring scenarios, the data collection task can be to periodically collect price information of specified products on a specific e-commerce platform. The corresponding task data includes, but is not limited to, product name, product link, price, discount information, sales volume, number of comments, etc. (2) For social media sentiment analysis scenarios, the data collection task can be to monitor posts or comments related to specific keywords on social media platforms in real time. The corresponding task data can be post content, posting time, poster ID, number of likes, number of comments, number of reposts, keywords, sentiment, etc. (3) For news media content aggregation scenarios, the collection task can be to automatically crawl the latest news titles and summaries from multiple news websites. The corresponding task data includes news titles, news summaries, news links, publication time, source websites, keywords, etc.

[0079] (4) For recruitment information collection scenarios, the collection task can be to collect job information from multiple recruitment websites on a regular basis. The corresponding task data can be job title, company name, salary range, work location, job description, posting time, etc.

[0080] In this embodiment, the task scheduling center is used to receive, parse, manage, and schedule network acquisition tasks submitted by users.

[0081] Specifically, the scheduling center receives collection tasks set by users through a unified management console (e.g., a web interface built on Spring Boot + Vue.js). The tasks include the URL of the target website, the collection frequency, the parsing rules, etc. The scheduling center parses the task instructions and converts them into tasks that the system can execute.

[0082] In this embodiment, the parsed task is placed in a scheduling queue, and the scheduling center starts the collection task at an appropriate time according to a preset scheduling strategy (e.g., a Cron expression based on Quartz).

[0083] In this embodiment, the scheduling center calls the acquisition engine to perform the actual data acquisition work and passes the acquired raw data to the subsequent task analysis module.

[0084] Step 320: Perform security analysis on the task data through the task analysis module, and process the security analysis results according to the preset handling strategy to obtain the task processing result; the security analysis includes sensitive information identification and compliance check, and the handling strategy includes abnormal data interception, data desensitization, data labeling and manual review.

[0085] In this step, the task analysis module can perform security scanning and analysis on the task data collected by the acquisition engine to identify potential risks.

[0086] In this embodiment, high-risk data (abnormal data) is directly intercepted and not entered into the database; sensitive information is anonymized (for example, by using special characters such as asterisks); and data whose risk level cannot be determined is labeled "pending review" and submitted for manual review.

[0087] In this embodiment, users can choose to perform security analysis on task data according to their needs, such as only identifying sensitive information, only performing compliance checks, or both.

[0088] In this embodiment, the task analysis module includes: a sensitive information identification unit, used to match sensitive information in the task data using a sensitive word library and a regular expression engine; a compliance check unit, used to identify content in the task data that does not conform to text specifications and to detect malicious code in the task data using natural language processing or a third-party content security API; and a handling strategy execution unit, used to determine the risk level of the collection task based on the matching results, identification results, and detection results, and to process the task data according to the handling strategy associated with the risk level.

[0089] In this embodiment, the sensitive word database contains a variety of sensitive words, such as ID card numbers, bank card numbers, passwords, addresses, etc.; the sensitive word database can be updated regularly to accommodate the emergence of new sensitive words.

[0090] In this embodiment, the sensitive information identification unit identifies potential sensitive information by matching task data with a sensitive word library and regular expressions.

[0091] In this embodiment, the task analysis module can also use natural language processing technology or third-party content security APIs (e.g., cloud service content security APIs) to perform compliance checks on the text and identify illegal content (e.g., political content).

[0092] In this embodiment, the compliance inspection unit can also scan code snippets, file links, etc. in the task data to prevent the introduction of attack scripts such as XSS and SQL injection.

[0093] In this embodiment, the compliance check unit can also perform rapid matching of sensitive words by introducing the DFA algorithm and integrate the Java SDK of the cloud platform content security API to perform more complex compliance checks.

[0094] In this embodiment, the user can preset processing strategies for different risk levels, such as directly intercepting high-risk data, de-identifying medium-risk data, and storing low-risk data normally. The processing strategy execution unit will then process the data according to the preset processing strategy. The processing strategy may include abnormal data interception, data de-identification, data labeling, and manual review.

[0095] Step 330: The management module performs query, configuration, and monitoring operations on the data collection task based on the management instructions input by the user, the security analysis results, and the task processing results, and generates a management report.

[0096] In this step, the management module can manage the data collection task based on the user's input instructions and the output results of the task analysis module.

[0097] Specifically, users can query the execution status, collection results, and security analysis results of tasks; users can also configure task parameters (e.g., collection frequency, parsing rules, and handling strategies); and users can monitor the running status of tasks (e.g., CPU usage, memory usage, and network traffic); the management module can also generate various management reports, such as task execution reports, security analysis reports, and system operation reports.

[0098] The intelligent task management system provided in this invention operates by receiving user-submitted data collection tasks through a task scheduling center and acquiring task data using a data collection engine; performing security analysis on the task data through a task analysis module and processing the security analysis results according to a preset handling strategy; and performing query, configuration, and monitoring operations on the data collection tasks through a management module based on user-input management instructions, security analysis results, and task processing results, and generating management reports, thereby improving the security of data collection and increasing data management efficiency.

[0099] Figure 4 This is the second flowchart illustrating the operation method of the intelligent task management system provided by this invention. Figure 4 In the illustrated embodiment, the operation method of the intelligent task management system further includes the following steps: task configuration and extraction, task scheduling, data acquisition, content security analysis, and security processing and storage.

[0100] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute the operation method of the intelligent task management system. This method includes: parsing user-inputted task instructions through a task scheduling center and scheduling acquisition tasks based on the parsing results, while simultaneously acquiring task data using an acquisition engine; performing security analysis on the task data through a task analysis module and processing the security analysis results according to a preset handling strategy to obtain task processing results; the security analysis includes sensitive information identification and compliance checks, and the handling strategies include abnormal data interception, data desensitization, data labeling, and manual review; and performing query, configuration, and monitoring operations on the acquisition tasks through a management module based on user-inputted management instructions, security analysis results, and task processing results, and generating management reports.

[0101] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the operation method of the intelligent task management system provided by the above methods. The method includes: parsing the task instructions input by the user through a task scheduling center, scheduling the collection task according to the parsing result, and simultaneously using a collection engine to obtain the task data of the collection task; performing security analysis on the task data through a task analysis module, and processing the security analysis results according to a preset handling strategy to obtain the task processing result; the security analysis includes sensitive information identification and compliance check, and the handling strategy includes abnormal data interception, data desensitization, data labeling, and manual review; and performing query, configuration, and monitoring operations on the collection task through a management module according to the management instructions input by the user, the security analysis result, and the task processing result, and generating a management report.

[0103] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements an operation method for the intelligent task management system provided by the above methods. The method includes: parsing user-inputted task instructions through a task scheduling center, scheduling acquisition tasks according to the parsing results, and simultaneously acquiring task data of the acquisition tasks using an acquisition engine; performing security analysis on the task data through a task analysis module, and processing the security analysis results according to a preset handling strategy to obtain task processing results; the security analysis includes sensitive information identification and compliance checks, and the handling strategy includes abnormal data interception, data desensitization, data labeling, and manual review; and performing query, configuration, and monitoring operations on the acquisition tasks through a management module based on user-inputted management instructions, security analysis results, and task processing results, and generating a management report.

[0104] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent task management system, characterized in that, include: The task scheduling center is used to receive collection tasks submitted by users and to obtain the task data of the collection tasks using the collection engine. The task analysis module is used to perform security analysis on the task data and process the security analysis results according to the preset handling strategy to obtain the task processing result; the security analysis includes sensitive information identification and compliance check, and the handling strategy includes abnormal data interception, data desensitization, data labeling and manual review; The management module is used to perform query, configuration, and monitoring operations on the data collection task based on the management instructions input by the user, the security analysis results, and the task processing results, and to generate a management report.

2. The intelligent task management system according to claim 1, characterized in that, The data collection engine integrates a headless browser driver to simulate browser behavior, execute JavaScript code in web pages, and obtain the complete DOM tree structure.

3. The intelligent task management system according to claim 1, characterized in that, The task analysis module includes: The sensitive information identification unit is used to match sensitive information in task data using a sensitive word library and a regular expression engine; The compliance check unit is used to identify content in the task data that does not conform to text specifications and to detect malicious code in the task data through natural language processing or third-party content security APIs. The disposal strategy execution unit is used to determine the risk level of the data collection task based on the matching results, identification results, and detection results, and to process the task data according to the disposal strategy associated with the risk level.

4. The intelligent task management system according to claim 1 or 2, characterized in that, The task scheduling center supports distributed deployment and elastic scaling, and the collection engine is used to distribute multiple collection tasks to multiple corresponding distributed nodes for execution.

5. The intelligent task management system according to claim 1, characterized in that, The system also includes: The storage module is used to store the acquisition task and its data, the task processing results, and the management report; The storage module also supports a variety of storage media, including relational databases, key-value store databases, and NoSQL databases.

6. The intelligent task management system according to claim 1, characterized in that, The management module is built on the Java Spring Boot Web framework; The management module is also configured to perform at least one of the following based on user instructions: Query the management report and system operating status; Edit the sensitive word database, configure security rules and handling strategies; Query the logs and statistical reports generated by the task analysis module.

7. A method for operating an intelligent task management system, characterized in that, include: The task scheduling center receives user-submitted data collection tasks and simultaneously uses the data collection engine to obtain the task data of the data collection tasks. The task data is subjected to security analysis by the task analysis module, and the results are processed according to the preset handling strategy to obtain the task processing result. The security analysis includes sensitive information identification and compliance check, and the handling strategy includes abnormal data interception, data desensitization, data labeling and manual review. The management module performs query, configuration, and monitoring operations on the data collection task based on the management instructions input by the user, the security analysis results, and the task processing results, and generates a management report.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the operation method of the intelligent task management system as described in claim 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the operation method of the intelligent task management system as described in claim 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the operation method of the intelligent task management system as described in claim 7.

Images