Method and system for intelligently extracting and optimizing webpage content
Through intelligent algorithms to generate and optimize regular expressions, combined with exception capture and retry mechanisms, the problems of high labor costs and structural changes in web page content extraction are solved, and efficient and stable web page content extraction is achieved.
Patent Information
- Application Number
- CN202410934428.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-22
AI Technical Summary
Existing web page content extraction methods rely on manual writing of regular expressions, resulting in high cost and low efficiency, and changes in website structure lead to the failure of regular expressions and the data cannot be extracted correctly.
Intelligent algorithms are used to generate and optimize regular expressions, and the design interface supports addition, deletion, modification and query, combining exception capture and retry mechanisms, automating web page content extraction and matching, and providing a visual monitoring interface.
It reduces manual investment, improves extraction efficiency, flexibly responds to website structure changes, ensures the stability and continuity of data extraction, and enhances the flexibility and reliability of the system.
Smart Images

Figure CN119003904A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of network information crawling, and in particular to a method and system for intelligently extracting and optimizing webpage content. Background Art
[0002] In today's digital age, the amount of information on the Internet is growing exponentially. The effective extraction and analysis of web page content has become a key link in many business scenarios, such as big data analysis, competitive intelligence collection, content aggregation services, etc.
[0003] The rapid development of artificial intelligence technology, especially breakthroughs in natural language processing (NLP) and machine learning, has provided new possibilities for solving the above problems. However, the market currently lacks a solution that efficiently integrates artificial intelligence with traditional technologies (such as regular expressions) to intelligently automate the process of extracting web page content, especially for complex and changeable web page structures.
[0004] At present, there are the following technical problems for intelligent extraction of web page content:
[0005] (1) Web page content extraction methods rely on the generation of regular expressions. However, in the process of generating and optimizing regular expressions, traditional methods require a lot of manual participation in writing, which not only increases labor costs but also may lead to inefficiency. The update and expansion of regular expression libraries often rely on manual collection and verification, which is inefficient and prone to errors.
[0006] (2) Website structure often changes, such as page layout adjustment, element ID change, etc. These changes may cause the original regular expression to become invalid, making it impossible to correctly extract the required data. Summary of the invention
[0007] The purpose of the present invention is to provide a method and system for intelligent extraction and optimization of web page content, so as to solve the problems mentioned in the above background technology that the existing extraction method increases labor costs, is inefficient, and the original regular expression becomes invalid due to changes in network structure.
[0008] To achieve the above object, the present invention adopts the following technical solutions:
[0009] According to one aspect of the present invention, a method for intelligently extracting and optimizing webpage content is provided, the method comprising:
[0010] Create an array of expressions for storing matching web page content; use a database and file system to persistently store the expressions, and design an interface to support add, delete, modify, and query operations;
[0011] Crawl web page content, set the URL of the target website and crawling rules, start the crawling program to obtain web page content and preprocess it to obtain preprocessed data;
[0012] Determine whether the expression array corresponding to the webpage content is empty, if it is empty, call the intelligent algorithm to generate an expression, if it is not empty, traverse the expression array to match the preprocessed data;
[0013] If the match is successful, relevant content is extracted and the corresponding expression is stored in the expression array; if the match fails, an intelligent algorithm is called to generate a new expression;
[0014] Deploy exception capture and logging mechanisms, and perform parameterized configuration of retry decisions.
[0015] Based on the above solution, the preprocessing includes removing HTML tags from the web page source data to obtain preprocessed data.
[0016] Based on the above scheme, the calling of the intelligent algorithm to generate a new expression includes calling the intelligent algorithm, passing in the preprocessed data and the target content indication, and returning a matching expression.
[0017] Based on the above solution, a parser is written to extract the expression generated by the intelligent algorithm and verify it. If the verification is successful, it is added to the corresponding expression array.
[0018] Based on the above-mentioned scheme, the exception capture mechanism and logging mechanism include recording exception data when an exception is captured, and the logging mechanism automatically records log data; the log data includes detailed information of key operations and the exception data; the exception data includes but is not limited to error type, timestamp, stack trace, operator ID, task ID and processing duration.
[0019] Based on the above solution, an intelligent analysis engine is developed to automatically parse the log data and predict anomalies to prompt warnings.
[0020] Based on the above scheme, the parameterized configuration retry decision includes configuring the maximum number of retries and the retry interval; the retry decision includes determining whether to retry and the timing of the retry based on the exception type and the historical retry success rate.
[0021] Based on the above solution, a configuration interface is also provided to customize the monitoring strategy, including setting error thresholds, logging options and warning conditions.
[0022] Based on the above solution, it also includes providing a visual monitoring interface to display key indicators, which include but are not limited to system operation status, success rate, and error distribution.
[0023] According to another aspect of the present invention, there is provided a system for intelligent extraction and optimization of web page content, the system comprising an expression management module, a web page content crawling module, a web page content matching module, an expression generation module, an exception capture and retry module, and a monitoring and statistics module;
[0024] The expression management module manages and maintains the expression array;
[0025] The webpage content crawling module obtains webpage source data and preprocesses it to obtain preprocessed data;
[0026] The webpage content matching module traverses the expression array to match the pre-processed data;
[0027] The expression generation module calls an intelligent algorithm to generate a new expression;
[0028] The exception capture and retry module captures exceptions and records exception data, as well as parameterizes and configures retry decisions;
[0029] The monitoring and statistics module records and automatically parses log data, provides a visual monitoring interface and displays key indicators.
[0030] It can be seen from the above technical solutions that the present invention has at least the following advantages and positive effects compared with the prior art:
[0031] (1) Use intelligent algorithms to automatically generate and optimize regular expressions, reduce manual input, and improve work efficiency;
[0032] (2) Parameterized intelligent retry logic allows users to configure the maximum number of retries and retry intervals according to their needs. These parameters can be easily adjusted to adapt to different network conditions or server response times, improving the flexibility and configurability of the system as well as the stability and reliability of the system.
[0033] (3) The regular expression is persistently stored, and the database and file system are updated after a new regular expression is generated. The present invention can continuously enrich and improve the regular expression library as the frequency of use increases, and can flexibly respond to changes in different website structures, reduce data extraction failures caused by web page revisions, and ensure the stability and continuity of data extraction.
[0034] The present invention should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] The drawings herein are incorporated into and constitute a part of the specification, showing embodiments consistent with the present invention, and together with the specification, are used to explain the principles of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0036] Figure 1 A schematic diagram of a method for intelligently extracting and optimizing webpage content according to the present invention;
[0037] Figure 2 A schematic diagram of the present invention calling a large model to generate a regular expression;
[0038] Figure 3 A schematic diagram of matching web page source data with expression arrays of the present invention;
[0039] Figure 4 A schematic diagram of a system for intelligent extraction and optimization of web page content according to the present invention. DETAILED DESCRIPTION
[0040] In order to more clearly explain the purpose, technical solutions and advantages of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The example implementation methods can be implemented in various forms and should not be understood as being limited to the examples described herein. On the contrary, these implementation methods are provided to make the present invention more comprehensive and complete, and to fully convey the concepts of the example implementation methods to those skilled in the art.
[0041] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.
[0042] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0043] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.
[0044] The present invention is described in detail below in conjunction with specific embodiments:
[0045] Example 1
[0046] like Figure 1 As shown, this embodiment provides a method for intelligent extraction and optimization of web page content, and the specific steps of the method are as follows:
[0047] S1: Create an expression array for storing matching web page content; use a database and a file system to persistently store the expression, and design an interface to support add, delete, modify, and query operations;
[0048] In this embodiment, two arrays are created, namely: detail_page_regex_array and list_page_regex_array. detail_page_regex_array is used to store expressions of detail pages, and list_page_regex_array is used to store expressions of list pages.
[0049] Furthermore, select a suitable database and file system to persistently store expressions to ensure that historical data can still be effectively used after the system is restarted; design the database table structure to store expressions and related information (such as creation time, update time, source, etc.); implement the operation interface (such as add, delete, modify and query) to synchronize with the expression array.
[0050] Furthermore, a RESTful API or command line interface is designed to allow users to manually add, edit, or delete expressions; the interface backend logic is implemented to process user requests and update the expression array.
[0051] S2: crawling web page content, setting the URL of the target website and crawling rules, starting the crawling program to obtain web page source data and preprocessing to obtain preprocessed data;
[0052] In this embodiment, Python's scrapy framework is used to crawl web page content, the target web page URL is input, the crawling program is started, the web page content is obtained, that is, the web page source data, and the web page source data is preprocessed to remove HTML tags to obtain preprocessed data.
[0053] S3: determining whether the expression array corresponding to the webpage source data is empty, if it is empty, calling an intelligent algorithm to generate an expression, if it is not empty, traversing the expression array to match the preprocessed data;
[0054] In this embodiment, the pre-processed data is firstly judged whether the array for storing expressions is empty. If it is empty, it is necessary to call an intelligent algorithm, that is, an external service or an internal algorithm. In this embodiment, a large model is called to generate a corresponding regular expression according to a given input and add it to the expression array; if it is not empty, each element in the expression array is traversed, and for each regular expression in the array, it is attempted to be applied to the web page content to be processed, a matching operation is performed, and the web page content is matched, and the matching result of each regular expression is recorded or processed;
[0055] Furthermore, according to business needs, the matched results are further processed or analyzed; the results may need to be saved in a database, file, or sent to other systems via the network; if it is found that no suitable regular expression can match certain key parts of the web page content, it may be necessary to try to regenerate or adjust the regular expression.
[0056] Specifically, in this embodiment, Figure 2 As shown, calling the large model to generate a regular expression includes:
[0057] Select or train a general large model (such as a Transformer-based model) that can generate regular expressions; use the preprocessed data and target content indication as input, and call the large model to generate regular expressions. Furthermore, write a parser to extract the regular expressions generated by the large model, and verify the generated regular expressions to ensure their validity and accuracy, add the verified regular expressions to the corresponding expression array, and synchronize them to the database and file system.
[0058] S4: If the match is successful, relevant content is extracted and the corresponding expression is stored in the expression array; if the match fails, an intelligent algorithm is called to generate a new expression;
[0059] Combination Figure 3 After the matching operation is performed in this embodiment, for each web page to be extracted, it is determined whether the match is successful. If the match is successful, its type is determined to be a list page or a detail page; when the web page source data is a list page and the match is successful, it is skipped; when the web page source data is a detail page and the match is successful, the relevant content is extracted and saved; for the web page source data that fails to match, the large model is called to generate a new regular expression;
[0060] Specifically, if it is a list page, the expression array is traversed and a matching operation is performed on each regular expression; if any regular expression matches successfully, further processing of the current web page source data is skipped (because list pages usually do not need to extract specific content); if all regular expressions fail to match, you can choose to record this situation, but usually no immediate action is required because the main purpose of the list page may be just navigation or indexing.
[0061] If it is a details page, the expression array is also traversed to perform a matching operation on each regular expression; if any regular expression matches successfully, the relevant content is extracted according to the rules of the regular expression and stored in the database; if all regular expressions fail to match, the next step is executed.
[0062] For web page source data that failed to match, in this embodiment, the large model is called to generate a new regular expression, and the generated new regular expression is added to the regular expression array. Some strategies may be needed to decide whether to replace or supplement the existing regular expression. Optionally, the newly generated regular expression is immediately tried to re-match the web page source data that failed to match, or this task is postponed to subsequent batch processing; when all web page source data have been processed, this step ends.
[0063] Furthermore, when calling the big model to generate a new regular expression, it may be necessary to provide additional information about the page where the match failed (such as the URL of the page, partial content samples, etc.) so that the big model can more accurately generate applicable regular expressions; certain verification or testing should be performed on the newly generated regular expressions to ensure their correctness and effectiveness; before storing the content in the database, data cleaning and formatting may be required to ensure data consistency and readability; if processing a large amount of web page source data, it may be necessary to consider performance optimization and error handling mechanisms to ensure the stability and reliability of the system.
[0064] S5: Deploy exception capture mechanism and logging mechanism, and perform parameterized configuration retry decision;
[0065] In this embodiment, an exception capture mechanism and a log recording mechanism are also set. In key operation links, such as content matching and calling intelligent algorithms, an exception capture logic is added. When an exception is captured, the log recording mechanism is started to record the exception data. The log recording mechanism automatically records the log data; the log data includes detailed information of key operations and the exception data; the exception data includes but is not limited to error type, timestamp, stack trace, operator ID, task ID and processing duration. Preferably, in this embodiment, an intelligent analysis engine is also developed to automatically parse the log data and predict the exception to prompt an early warning.
[0066] Furthermore, the parameterized configuration retry decision includes configuring the maximum number of retries and the retry interval; the retry decision includes deciding whether to retry and when to retry based on the exception type and the historical retry success rate. Implement the retry logic, set the maximum number of retries, set a short delay before each retry to avoid frequent calls, and when the retry limit is reached, record the failure details and execute the skip strategy. Specifically, based on the exception type and the historical retry success rate, intelligently decide whether to retry and when to retry; for some temporary problems with higher certainty, the system may choose to retry immediately; and for problems that may require manual intervention, record detailed information and skip. Furthermore, when the set maximum number of retries is reached, the system will perform corresponding failure handling operations according to the configured strategy, including but not limited to recording detailed failure logs, sending alarm notifications, executing backup processes, or simply skipping the current task.
[0067] Preferably, the exception capture mechanism and parameterized configuration retry decision are an independent module that can be easily integrated into other parts of the system, or replaced and upgraded as needed; this parameterized and modular design not only improves the flexibility and configurability of the system, but also enhances the stability and reliability of the system, ensuring that the system can make reasonable responses when faced with various abnormal situations.
[0068] Preferably, in this embodiment, a log framework is selected and log data is configured, such as the logging module in the Python standard library or a third-party library such as Loguru, and the log data includes but is not limited to error type, timestamp, and stack trace. Furthermore, an intelligent analysis engine is developed to automatically parse the log data, identify abnormal patterns and potential problems in system operation, and predict abnormalities to prompt warnings; using machine learning algorithms, the intelligent analysis engine can predict the trend of fault occurrence and issue warnings in advance.
[0069] Furthermore, this embodiment provides a configuration interface for customizing monitoring strategies, allowing users to customize monitoring strategies according to specific needs, including setting error thresholds, logging options, and warning conditions. By providing a user-friendly configuration interface, users can easily set and manage these parameters.
[0070] Furthermore, this embodiment integrates a closed-loop feedback mechanism to ensure that monitored problems can be fed back to system maintenance personnel in a timely manner, supports automated workflows, and can automatically trigger problem-solving processes based on monitoring results, including notifying relevant personnel and recording problem-solving status.
[0071] Furthermore, this embodiment also provides a visual monitoring interface that can display key performance indicators in real time, supports dynamic data updates and multi-dimensional data analysis, and enables users to understand the operating status of the system from different angles; the key performance indicators include but are not limited to system operating status, success rate, and error distribution.
[0072] Furthermore, this embodiment also has the function of generating periodic data reports, which can summarize and analyze system performance data over a period of time, including but not limited to charts, statistics and trend analysis, to provide decision support for users. It also has strict security and access control mechanisms to ensure the protection of sensitive data; it implements role-based access control to ensure that only authorized users can access or modify monitoring configurations and log data.
[0073] Example 2
[0074] like Figure 4 As shown, this embodiment exemplarily presents a system for intelligent extraction and optimization of web page content, including an expression management module, a web page content crawling module, a web page content matching module, an expression generation module, an exception capture and retry module, and a monitoring and statistics module;
[0075] The expression management module manages and maintains the expression array. In this embodiment, two expression arrays are created, which are respectively used to store regular expressions matching the details page and the list page; the web page content crawling module obtains the web page content and performs preprocessing to remove HTML tags to obtain preprocessed data; the web page content matching module traverses the expression array and matches the preprocessed data; the expression generation module calls the intelligent algorithm to generate a new expression; the exception capture and retry module captures exceptions and records exception data, and parameterizes the configuration of retry decisions; the monitoring and statistics module records and automatically parses log data, provides a visual monitoring interface and displays key indicators.
[0076] Specifically, in this embodiment, two arrays are created in the expression management module, namely: detail_page_regex_array and list_page_regex_array. detail_page_regex_array is used to store expressions for detail pages, and list_page_regex_array is used to store expressions for list pages. The database and file system are used to persistently store expressions and implement the operation interface. Furthermore, a database table structure is designed to store regular expressions and related information (such as creation time, update time, source, etc.); an operation interface (such as add, delete, modify, and query) is implemented to synchronize with the expression array. A RESTful API or command line interface is designed to allow users to manually add, edit, or delete expressions; the interface backend logic is implemented to process user requests and update the regular expression array.
[0077] The webpage content crawling module inputs the target webpage URL, starts the crawling program, obtains the webpage source data, performs preprocessing to remove HTML tags, and obtains preprocessed data.
[0078] The web page content matching module first makes a preliminary judgment on the web page source data to see whether the corresponding expression array is empty. If it is empty, it is transmitted to the expression generation module. The expression generation module in this embodiment calls the large model to generate a corresponding regular expression based on the given input and adds it to the expression array; if it is not empty, each element in the expression array is traversed, and for each regular expression in the array, it is attempted to be applied to the web page content to be processed, a matching operation is performed, the web page content is matched, and the matching results of each regular expression are recorded or processed.
[0079] Furthermore, in the webpage content matching module, for each webpage to be extracted, first determine its type as a list page or a detail page; if it is a list page, traverse the expression array and perform a matching operation on each regular expression; if any regular expression is matched successfully, skip further processing of the current webpage source data. If it is a detail page, traverse the expression array and perform a matching operation on each regular expression; if any regular expression is matched successfully, extract relevant content according to the rules of the regular expression.
[0080] For web page source data that fails to match, it is transmitted to the expression generation module, and the large model is called to generate a new regular expression and add it to the expression array.
[0081] An exception capture and retry mechanism is set up in the exception capture and retry module, and exception capture logic is added in key links. When an exception is captured, an error log is recorded; parameterized configuration of retry decisions is performed, and the maximum number of retries and retry interval are set. According to the exception type and the historical retry success rate, an intelligent decision is made whether to retry and the timing of the retry; for some temporary problems with higher certainty, the system may choose to retry immediately; and for problems that may require manual intervention, detailed information is recorded and skipped. Furthermore, when the set maximum number of retries is reached, the system will perform corresponding failure processing operations according to the configured strategy, including but not limited to recording detailed failure logs, sending alarm notifications, executing backup processes, or simply skipping the current task. Preferably, the exception capture and retry module is an independent module that can be easily integrated into other parts of the system, or replaced and upgraded as needed.
[0082] A logging mechanism is set in the monitoring and statistics module to automatically record log data, which includes detailed information of key operations and abnormal data; abnormal data includes but is not limited to error type, timestamp, stack trace, operator identification, task identification and processing duration. Further, an intelligent analysis engine is developed to automatically parse the log data and predict abnormalities to prompt warnings. Further, a configuration interface is provided to customize monitoring strategies, including setting error thresholds, logging options and warning conditions. Further, a closed-loop feedback mechanism is integrated to ensure that monitored problems can be fed back to system maintenance personnel in a timely manner, support automated workflows, and automatically trigger problem solving processes based on monitoring results, including notifying relevant personnel and recording problem solving status. Further, a visual monitoring interface is also provided to display key indicators, support dynamic data updates and multi-dimensional data analysis, so that users can understand the operating status of the system from different angles; the key indicators include but are not limited to system operating status, success rate, and error distribution. Further, it has the function of generating periodic data reports, which can summarize and analyze system performance data over a period of time, including but not limited to charts, statistics and trend analysis, to provide decision support for users. Furthermore, it also has strict security and access control mechanisms to ensure the protection of sensitive data; role-based access control is implemented to ensure that only authorized users can access or modify monitoring configurations and log data.
[0083] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present invention, which follows the general principles of the present invention and includes common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present invention are indicated by the claims. It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is limited only by the appended claims.
Claims
1. A method for intelligent extraction and optimization of web page content, It is characterized in that The method comprises: Create an array of expressions to store matching web page content; use databases and file systems to persistently store expressions, and design interfaces to support add, delete, modify, and query operations; Crawl web page content, set the URL of the target website and crawling rules, start the crawling program to obtain web page source data and preprocess it to obtain preprocessed data; Determine whether the expression array corresponding to the webpage source data is empty, if it is empty, call the intelligent algorithm to generate an expression, if it is not empty, traverse the expression array to match the preprocessed data; If the match is successful, relevant content is extracted and the corresponding expression is stored in the expression array; if the match fails, an intelligent algorithm is called to generate a new expression; Deploy exception capture and logging mechanisms, and perform parameterized configuration of retry decisions.
2. According to the method for intelligent extraction and optimization of web page content as claimed in claim 1, It is characterized in that The preprocessing includes removing HTML tags from the web page source data to obtain preprocessed data.
3. According to the method for intelligent extraction and optimization of web page content as claimed in claim 1, It is characterized in that The calling intelligent algorithm generates a new expression, inputs the preprocessed data and target content indication, and returns the corresponding expression.
4. According to the method for intelligent extraction and optimization of web page content as claimed in claim 3, It is characterized in that Write a parser to extract the expression generated by the intelligent algorithm and verify it. If the verification is successful, add it to the corresponding expression array.
5. According to the method for intelligent extraction and optimization of web page content as claimed in claim 1, It is characterized in that The exception capture mechanism and log recording mechanism include recording exception data when an exception is captured, and the log recording mechanism automatically records log data; the log data includes detailed information of key operations and the exception data; the exception data includes but is not limited to error type, timestamp, stack trace, operator ID, task ID and processing duration.
6. A method for intelligently extracting and optimizing webpage content according to claim 5, It is characterized in that Develop an intelligent analysis engine to automatically parse the log data and predict anomalies to prompt early warning.
7. A method for intelligently extracting and optimizing webpage content according to claim 1, It is characterized in that The parameterized configuration retry decision includes configuring the maximum number of retries and the retry interval; the retry decision includes determining whether to retry and the timing of the retry based on the exception type and the historical retry success rate.
8. According to the method of intelligent extraction and optimization of web page content as described in claim 1, it also includes providing a configuration interface to customize monitoring strategies, including setting error thresholds, logging options and warning conditions.
9. According to claim 1, a method for intelligent extraction and optimization of web page content also includes providing a visual monitoring interface to display key performance indicators, wherein the key performance indicators include but are not limited to system operation status, success rate, and error distribution.
10. A system for intelligent extraction and optimization of web page content, It is characterized in that It includes expression management module, web content crawling module, web content matching module, expression generation module, exception capture and retry module, monitoring and statistics module; The expression management module manages and maintains the expression array; The webpage content crawling module obtains webpage source data and preprocesses it to obtain preprocessed data; The webpage content matching module traverses the expression array to match the pre-processed data; The expression generation module calls an intelligent algorithm to generate a new expression; The exception capture and retry module captures exceptions and records exception data, as well as parameterizes and configures retry decisions; The monitoring and statistics module records and automatically parses log data, provides a visual monitoring interface and displays key performance indicators.
Citation Information
Patent Citations
Intelligent extraction system and intelligent extraction method for article type web pages
CN102567530A
Text matching method and device, computer system and readable storage medium
CN110674250A
Webpage information extraction method and system and electronic equipment
CN111966881A
Text classification method of regular expression generated based on large language model
CN117556049A
Method and apparatus for web information extraction service
EP1069515A1
Cited By
Log analysis method using AI large model
CN119848046A
