Method, device and server for obtaining target content information in webpage

Abstract

The specification provides a method and device for obtaining target content information in a webpage and a server, and is applied to the field of artificial intelligence. Based on the method, the page screenshot of the target website webpage can be processed by using a preset image processing model based on the visual dimension to screen out the page screenshots of the first parent webpage and the first child webpage; the page screenshots of the first parent webpage and the first child webpage are processed by using a preset text recognition model based on the text dimension to determine the target webpage containing the target content information to be extracted; and based on the structural dimension, the target content information is extracted from the target webpage according to the page link of the target webpage and the matching page structure rule. Thus, the target content information required is accurately extracted from a large number of webpage pages contained in the target website by fusing and using the features of the webpage page in multiple dimensions such as vision, text and structure, and the target content information is comprehensively and efficiently extracted.

Description

Technical Field

[0001] This manual belongs to the field of artificial intelligence technology, and in particular relates to methods, devices and servers for obtaining target content information from web pages. Background Technology

[0002] With the development and popularization of Internet technology, information about many important conferences (such as cutting-edge academic conferences) is published on specific web pages on relevant websites.

[0003] However, each website usually contains a large number of web pages; and information about many important meetings is often published on different web pages on different websites.

[0004] Based on existing methods, if users want to understand or collect meeting information for all important meetings within a certain time period, they often need to manually browse and search each webpage on every relevant website. This results in a large workload for users, a tedious and complex process, low processing efficiency, and a high risk of omissions.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This specification provides a method, apparatus, and server for obtaining target content information from web pages. By effectively integrating and utilizing the visual, textual, and structural features of web pages, it is possible to comprehensively and efficiently extract the required target content information accurately and automatically from a large number of web pages contained in a target website.

[0007] This specification provides a method for obtaining target content information from a webpage, including:

[0008] Obtain seed data for target websites containing the target content;

[0009] Based on the target website's seed data, obtain screenshots and links of multiple web pages on the target website; each screenshot corresponds to a specific web page link.

[0010] The image processing model is used to process screenshots of multiple web pages, and the screenshots of the first parent web page and the first child web page corresponding to the first parent web page are selected.

[0011] The screenshots of the first parent webpage and the first child webpage are processed using a preset text recognition model to obtain the corresponding text recognition results of the first parent webpage and the first child webpage.

[0012] Based on the text recognition results of the first parent webpage and the text recognition results of the first child webpage, the target webpage containing the target content information to be extracted is determined.

[0013] Based on the page links of the target webpage and the page structure rules that match the target webpage, the target content information is extracted from the target webpage.

[0014] In one embodiment, before processing screenshots of multiple web pages using a preset image processing model, the method further includes:

[0015] Based on a preset set of reference image features, image feature matching is performed on the screenshots of the multiple web pages to obtain the corresponding image feature matching results.

[0016] Based on the image feature matching results, screenshots of web pages that are not related to the target content are filtered out from the screenshots of the multiple web pages.

[0017] In one embodiment, a preset image processing model is used to process screenshots of multiple web pages, and a screenshot of a first parent web page and a screenshot of a first child web page corresponding to the first parent web page are selected, including:

[0018] The image processing model is used to process screenshots of multiple web pages to obtain visual similarity parameters of the screenshots of multiple web pages.

[0019] Using a preset image processing model, an intermediate image group of web pages with a subordinate relationship is selected based on the visual similarity parameters of screenshots from multiple web pages; wherein, the intermediate image group contains screenshots of at least two web pages.

[0020] Using a preset image processing model, and based on the common visual features of the webpage screenshots in the same intermediate image group, the screenshot of the first parent webpage and the screenshot of the first child webpage corresponding to the first parent webpage are determined from the intermediate image group.

[0021] In one embodiment, the target content includes at least one of the following: academic conferences, business conferences, and administrative conferences.

[0022] In one embodiment, when the target content includes an academic conference, the preset text recognition model includes: a preset conference title recognition model and a preset conference content recognition model.

[0023] In one embodiment, a preset text recognition model is used to process screenshots of the first parent webpage and the first child webpage to obtain corresponding text recognition results for the first parent webpage and the first child webpage, including:

[0024] The screenshots of the first parent webpage and the first child webpage are processed using a pre-defined meeting title recognition model to obtain the meeting title recognition results of the first parent webpage and the first child webpage.

[0025] The screenshots of the first parent webpage and the first child webpage are processed using a pre-defined meeting content recognition model to obtain the meeting content recognition results of the first parent webpage and the first child webpage.

[0026] The first parent webpage's conference title recognition result and the first parent webpage's conference content recognition result are combined to obtain the first parent webpage's text recognition result; the first child webpage's conference title recognition result and the first child webpage's conference content recognition result are combined to obtain the first child webpage's text recognition result.

[0027] In one embodiment, based on the text recognition results of the first parent webpage and the text recognition results of the first child webpage, a target webpage containing the target content information to be extracted is determined, including:

[0028] Based on the text recognition results of the first parent webpage and the first child webpage, webpages with a probability value greater than a preset first probability threshold and a probability value greater than a preset second probability threshold containing meeting title text are selected from the parent webpage and child webpages with a subordinate relationship, and are selected as the target webpages.

[0029] In one embodiment, target content information is extracted from the target webpage based on the page links of the target webpage and the page structure rules that match the target webpage, including:

[0030] Determine the webpage data of the target webpage based on the page links of the target webpage;

[0031] Based on the page structure rules that match the target webpage, the DOM tree data structure of the target webpage is determined from the webpage data of the target webpage;

[0032] Search for the specified tag element in the DOM tree data structure of the target webpage; and extract the target content information based on the searched specified tag element.

[0033] In one embodiment, target content information is extracted based on the searched specified tag elements, including:

[0034] By combining the searched specified tag elements with specified keywords related to the target content, the DOM tree data structure of the target webpage is retrieved to extract the target content information.

[0035] In one embodiment, after extracting the target content information from the target webpage, the method further includes:

[0036] The target content information is processed using a pre-defined semantic recognition model to obtain the corresponding target semantic classification result;

[0037] Based on the target semantic classification results, determine whether the extracted target content information meets the quality requirements.

[0038] In one embodiment, after determining that the extracted target content information meets the quality requirements based on the target semantic classification result, the method further includes:

[0039] Obtain the text recognition results of the target webpage, as well as the text recognition results of related webpages that are subordinate to the target webpage;

[0040] Based on the text recognition results of the target webpage and the text recognition results of the associated webpage, related content information related to the target content information is extracted from the associated webpage.

[0041] Based on the target content information and related content information, generate a summary report on the target content information of the target website.

[0042] This specification also provides a device for obtaining target content information from a webpage, including:

[0043] The first acquisition module is used to acquire website seed data of target websites that involve target content;

[0044] The second acquisition module is used to obtain screenshots and links of multiple web pages in the target website based on the website seed data of the target website; wherein, the page screenshots and page links correspond one-to-one.

[0045] The image processing module is used to process screenshots of multiple web pages using a preset image processing model, and to filter out the screenshot of the first parent web page and the screenshot of the first child web page corresponding to the first parent web page.

[0046] The text recognition module is used to process the screenshots of the first parent webpage and the first child webpage using a preset text recognition model, and to obtain the corresponding text recognition results of the first parent webpage and the first child webpage.

[0047] The determination module is used to determine the target webpage containing the target content information to be extracted based on the text recognition results of the first parent webpage and the text recognition results of the first child webpage.

[0048] The extraction module is used to extract target content information from the target webpage based on the page links of the target webpage and the page structure rules that match the target webpage.

[0049] This specification also provides a server, including a processor and a memory for storing processor-executable instructions, wherein the processor executes the instructions to implement the relevant steps of the method for obtaining target content information in the webpage.

[0050] This specification also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, perform the following steps: acquiring website seed data of a target website involving target content; acquiring screenshots and links of multiple web pages in the target website based on the website seed data, wherein each screenshot corresponds to a page link; processing the screenshots of multiple web pages using a preset image processing model to filter out the screenshot of a first parent web page and the screenshot of a first child web page corresponding to the first parent web page; processing the screenshots of the first parent web page and the first child web page respectively using a preset text recognition model to obtain the corresponding text recognition results of the first parent web page and the first child web page; determining the target web page containing the target content information to be extracted based on the text recognition results of the first parent web page and the first child web page; and extracting the target content information from the target web page based on the page links of the target web page and the page structure rules matching the target web page.

[0051] This specification also provides a computer program product comprising a computer program that, when executed by a processor, implements the steps related to the method for obtaining target content information from the webpage.

[0052] Based on the method, apparatus, and server for obtaining target content information from web pages provided in this specification, the following steps can be taken: First, based on the website seed data of the target website, screenshots and links of each web page in the target website can be obtained. Then, based on the visual dimension, the screenshots of multiple web pages in the target website are processed using a preset image processing model to filter out the screenshots of the first parent web page and the first child web page, which have a high probability of containing target content information and are visually related. Further, based on the text dimension, the screenshots of the first parent web page and the first child web page are processed using a preset text recognition model to determine the target web page that truly contains the target content information to be extracted in terms of text semantics. Finally, based on the structural dimension, the target content information is extracted from the target web page in a targeted manner according to the page links of the target web page and the page structure rules that match the target web page. This allows for the effective integration and utilization of multiple dimensions of webpage features, including visual, textual, and structural aspects, to comprehensively and efficiently extract the target content information needed by the user from a large number of webpages on the target website. This effectively reduces the user's workload, shortens processing time, and avoids omissions when extracting content information. Attached Figure Description

[0053] To more clearly illustrate the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart illustrating a method for obtaining target content information from a webpage, provided in one embodiment of this specification.

[0055] Figure 2 This is a schematic diagram illustrating an embodiment of the method for obtaining target content information in a webpage provided in this specification, within a specific scenario example.

[0056] Figure 3 This is a schematic diagram illustrating an embodiment of the method for obtaining target content information in a webpage provided in this specification, within a specific scenario example.

[0057] Figure 4 This is a schematic diagram illustrating an embodiment of the method for obtaining target content information in a webpage provided in this specification, within a specific scenario example.

[0058] Figure 5 This is a schematic diagram illustrating an embodiment of the method for obtaining target content information in a webpage provided in this specification, within a specific scenario example.

[0059] Figure 6 This is a schematic diagram illustrating an embodiment of the method for obtaining target content information in a webpage provided in this specification, within a specific scenario example.

[0060] Figure 7 This is a schematic diagram illustrating an embodiment of the method for obtaining target content information in a webpage provided in this specification, within a specific scenario example.

[0061] Figure 8 This is a schematic diagram of the structural composition of a server provided in one embodiment of this specification;

[0062] Figure 9 This is a schematic diagram of the structure of a device for obtaining target content information from a webpage, provided in one embodiment of this specification. Detailed Implementation

[0063] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0064] See Figure 1 As shown in the embodiments of this specification, a method for obtaining target content information from a webpage is provided, wherein the method is specifically applied on the server side. In specific implementation, the method may include the following:

[0065] S101: Obtain the website seed data of the target website involving the target content;

[0066] S102: Based on the target website's seed data, obtain screenshots and links of multiple web pages on the target website; wherein, each screenshot corresponds to a page link.

[0067] S103: Use a preset image processing model to process screenshots of multiple web pages, and filter out the screenshot of the first parent web page and the screenshot of the first child web page corresponding to the first parent web page.

[0068] S104: Use the preset text recognition model to process the screenshots of the first parent webpage and the first child webpage respectively, and obtain the corresponding text recognition results of the first parent webpage and the first child webpage.

[0069] S105: Based on the text recognition results of the first parent webpage and the text recognition results of the first child webpage, determine the target webpage containing the target content information to be extracted;

[0070] S106: Extract target content information from the target webpage based on the page links of the target webpage and the page structure rules that match the target webpage.

[0071] Based on the above embodiments, by fully integrating and utilizing the visual, semantic text, structural and other features of website web pages, the target content information required by the user can be accurately and automatically extracted from a large number of web pages contained in the target website in a comprehensive and efficient manner. This can effectively reduce the user's workload, shorten processing time, improve processing efficiency, and avoid omissions when extracting content information.

[0072] In some embodiments, the method for obtaining target content information in the above-mentioned webpage can be specifically applied to the server side.

[0073] Among them, see Figure 2 As shown, the aforementioned server may specifically include a backend server applied to one side of a cloud service platform, capable of data transmission, data processing, and other functions. Specifically, the server may be, for example, an electronic device with data processing, storage, and network interaction capabilities. Alternatively, the server may be a software program running on the electronic device, providing support for data processing, storage, and network interaction. In this embodiment, the number of servers is not specifically limited. The server may be a single server, several servers, or a server cluster formed by several servers.

[0074] In practice, when a user wants to search for a summary of academic conferences of interest in the current time period, they can use their client terminal to generate a conference information summary request. This request can carry the time information of the current time period specified by the user and be sent to the server through the client terminal.

[0075] Specifically, the aforementioned client terminal may include a front-end applied to the user side, capable of data collection, data transmission, and other functions. Specifically, the client terminal may be an electronic device such as a desktop computer, tablet computer, laptop computer, or smartphone. Alternatively, the client terminal may also be a software application capable of running on the aforementioned electronic device.

[0076] The server receives and, based on the aforementioned conference information aggregation request, invokes a search engine to initially filter websites on the internet that may contain information about academic conferences of interest to the user, such as university websites and academic institution websites, as target websites. It then obtains the homepage links of these target websites as seed data. Based on this seed data, it retrieves screenshots and links of multiple web pages within the target website. Further, the server can process these screenshots using a pre-defined image processing model to filter out the first parent and child web pages that have a high probability of containing the target content (i.e., conference information) and exhibit a visual hierarchy. Next, it processes these screenshots using a pre-defined text recognition model, identifying the target web pages that truly contain the target content based on text semantics. Finally, based on the page links and matching page structure rules, it accurately locates and extracts the required target content from the target web pages. Next, the server can organize and summarize the target content information extracted from each target webpage on different target websites; retain the target content information belonging to the current time period, and generate a summary report of the corresponding target content information (i.e., a summary report of academic conference information); and then send the above summary report of target content information to the user's client terminal.

[0077] The client terminal receives and displays a summary report of the aforementioned target content information to the user. This allows users to conveniently and efficiently obtain a summary of academic conference information for the current time period automatically.

[0078] In some embodiments, the target website may be a single website involving the target content, or multiple websites involving the target content.

[0079] The aforementioned seed data for a target website can be understood as data that allows access to all web pages within that website. Specifically, the seed data could be a link to the target website's homepage. Based on this seed data, the server can obtain the links to each web page on the target website, and then, based on these links, browse to all the web pages on the target website.

[0080] Specifically, the server can hold and maintain a website seed dataset; this dataset can store the seed data for each website, as well as the topic content description information for each website. The server can receive and, based on a user's request to obtain target content information, filter out websites from the aforementioned website seed dataset whose topic content description information matches the request—that is, websites involving the target content—as target websites; and then obtain the website seed data for those target websites.

[0081] In some embodiments, the target content may specifically include at least one of the following: academic conferences, business conferences, administrative meetings, etc.

[0082] Of course, the target content listed above is only an illustrative example. In actual implementation, depending on the specific application scenario and processing requirements, the target content may also include other types of content such as news reports and notices.

[0083] Based on the above embodiments, different types of content information can be obtained using the method for obtaining target content information in web pages provided in this specification in different application scenarios to meet diverse scenario requirements.

[0084] In some embodiments, the server can use a crawling framework, such as Scrapy (a fast, high-level screen crawling and web scraping framework used to crawl websites and extract structured data from pages), to automatically crawl each webpage of the target website based on the target website's seed data. It will then take screenshots of the crawled webpages and collect their links; save the screenshots and links of the same webpage. This allows the server to obtain screenshots and links of multiple webpages from the target website.

[0085] When obtaining screenshots of web pages from a target website, pre-defined constraints and page links can be used to group screenshots from the same domain into the same target website. Simultaneously, screenshots for each web page on the target website are retrieved, and duplicate screenshots are removed. This process yields relatively complete screenshots and links of the target website without duplication.

[0086] In some embodiments, the aforementioned preset image processing model can be understood as a pre-trained neural network model capable of identifying screenshots of web pages with hierarchical relationships based on visual features in an image, and further intelligently distinguishing between parent and child web pages within these hierarchical relationships. The training method for the aforementioned preset image processing model will be explained in detail later.

[0087] In this context, a subpage can be understood as a page containing detailed information about the target content. A parent page can be understood as an introductory page containing information about the target content, and is located as a one-level index page on the subpages. Typically, users can directly access the corresponding subpage through relevant links on the parent page. Therefore, a subpage can be understood as belonging to the parent page.

[0088] Specifically, for example, an academic conference's lecture directory page contains links to individual lecture content pages within that conference. Each lecture content page contains detailed information about the lecture. In this case, the lecture directory page can be understood as a parent page; correspondingly, the lecture content pages can be understood as child pages.

[0089] In some embodiments, before processing screenshots of multiple web pages using a preset image processing model, the method may further include the following:

[0090] S1: Based on a preset reference image feature set, perform image feature matching on the screenshots of the multiple web pages to obtain the corresponding image feature matching results;

[0091] S2: Based on the image feature matching results, filter out the screenshots of web pages that are not related to the target content from the screenshots of the multiple web pages.

[0092] Specifically, the aforementioned preset reference image feature set stores common image features of screenshots of web pages containing target content. This preset reference image feature set can be obtained by pre-clustering the image features of a large number of screenshots of web pages containing target content.

[0093] Based on the above embodiments, by performing preliminary image feature matching on screenshots of multiple web pages according to a preset reference image feature set, screenshots of web pages that can be clearly identified as unrelated to the target content can be filtered out first, thereby reducing the amount of subsequent data processing and improving the overall data processing efficiency.

[0094] In some embodiments, see Figure 3 As shown, the above-described method uses a preset image processing model to process screenshots of multiple web pages, selecting the screenshot of the first parent web page and the screenshot of the first child web page corresponding to the first parent web page. In specific implementation, this may include the following:

[0095] S1: Use a preset image processing model to process screenshots of multiple web pages to obtain visual similarity parameters of the screenshots of multiple web pages;

[0096] S2: Using a preset image processing model, based on the visual similarity parameters of screenshots from multiple web pages, filter out intermediate image groups of web pages that have a subordinate relationship; wherein, the intermediate image group contains screenshots of at least two web pages.

[0097] S3: Using a preset image processing model, obtain and determine the screenshot of the first parent webpage and the screenshot of the first child webpage corresponding to the first parent webpage from the intermediate image group based on the common visual features of the webpage screenshots in the same intermediate image group.

[0098] Based on the above embodiments, a preset image processing model can be used to accurately and efficiently automatically identify parent and child webpage screenshots that correspond to each other in a subordinate relationship from multiple webpage screenshots of a target website, based on the visual dimension.

[0099] In some embodiments, specifically in implementation, firstly, considering that parent and child web pages with a subordinate relationship will have a certain degree of visual similarity and continuity, a preset image processing model can be used to process web page screenshots and filter out screenshots of web pages with a subordinate relationship based on visual similarity parameters to construct an intermediate image group. This intermediate image group contains screenshots of at least two web pages. Typically, this intermediate image group contains screenshots of more than two web pages.

[0100] Secondly, considering that for a webpage corresponding to the same intermediate image group, there is usually only one parent webpage and multiple child webpages, and these child webpages typically share some common visual features; furthermore, these common visual features also contain visual features that distinguish them from the parent webpage. Therefore, a pre-defined image processing model can be used, with an intermediate image group as the processing unit, to first identify the visual features of each page screenshot contained within the intermediate image group; then, by comparing the visual features of different page screenshots pairwise, the common visual features that only one page screenshot in the intermediate image group lacks, while the remaining page screenshots all possess, can be identified. Based on these common visual features, the webpage screenshots that do not possess these common visual features can be identified as the page screenshots of the parent webpage, i.e., the page screenshots of the first parent webpage; simultaneously, the remaining page screenshots in the intermediate image group can be identified as the page screenshots of the first child webpages belonging to the first parent webpage.

[0101] In some embodiments, see Figure 4 As shown, before implementation, the above-mentioned preset image processing model can be trained in the following way:

[0102] S1: Collect screenshots of sample web pages from the sample website as sample data;

[0103] S2: Identify the screenshots of the parent webpage and child webpages of the samples that have a subordinate relationship in the sample data, and mark them accordingly to obtain the labeled sample data;

[0104] S3: Using the ResNet structure, construct an initial image processing model based on a convolutional neural network; and construct a loss function based on triplet error that matches this initial image processing model;

[0105] S4: Train the initial image processing model using the loss function and labeled sample data to obtain a preset image processing model with the required accuracy.

[0106] Specifically, the loss function based on triplet error can be understood as a loss function constructed based on anchor samples (A), positive samples (P), and negative samples (N). Training the model based on this triplet error loss function allows the model to converge more quickly in a direction that makes the anchor samples and positive samples as close as possible, forming the Siamese network model architecture, thereby improving training efficiency and model accuracy. The Siamese network can learn a similarity metric from the data and use this learned similarity metric to compare and match new, unknown category samples.

[0107] Specifically, based on the labeled sample data, further steps can be taken to select screenshots of the parent webpage as anchor samples, screenshots of child webpages that are subordinate to the anchor sample as positive samples, and randomly selected screenshots that are not subordinate to the anchor sample as negative samples. These anchor samples, along with their corresponding positive and negative samples, are then combined into a single sample data set. Following this method, multiple sample data sets can be constructed based on the labeled sample data.

[0108] Meanwhile, based on the similarity distance between the anchor sample and the positive sample, and the feature distance between the anchor sample and the negative sample, a corresponding loss function based on triplet error can be constructed. The specific formula is as follows:

[0109] L(A,P,N)=max(||f(A)-f(P)|| 2 -||f(A)-f(N)|| 2 +a, 0)

[0110] Where L represents the loss value, A, P, and N represent the anchor sample, positive sample, and negative sample in the same sample data group, respectively, f represents the model output, and a represents the margin parameter. The margin parameter is used to increase ||f(A)-f(P)|| 2 With ||f(A)-f(N)|| 2 The gap between them.

[0111] Furthermore, during specific training, a loss function and multiple sample data sets can be used to train the model through gradient descent, so as to efficiently train and obtain a preset image processing model with better performance.

[0112] In some embodiments, considering that among the first parent webpage and first child webpages with a subordinate relationship selected based on visual dimensions, there may still be pages that do not actually contain the target content information of interest, a preset text recognition model can be used to process the screenshots of the aforementioned webpages based on the text dimension to further filter out the pages that actually contain the required target content information as the target webpages.

[0113] In some embodiments, the aforementioned preset text recognition model can be understood as a pre-trained algorithm model that can detect and recognize the probability of an image containing text of interest based on text features in the image.

[0114] When the target content includes academic conferences, the preset text recognition model may specifically include: a preset conference title recognition model and a preset conference content recognition model.

[0115] Specifically, the aforementioned preset conference title recognition model can be understood as an algorithmic model capable of detecting and recognizing the probability value of an image containing conference title text (e.g., academic conference title). Similarly, the aforementioned preset conference content recognition model can be understood as an algorithmic model capable of detecting and recognizing the probability value of an image containing conference content text (e.g., academic conference content).

[0116] Based on the above embodiments, for cases where the target content includes academic conferences, a preset conference title recognition model and a preset conference content recognition model can be used as preset text recognition models. Based on the text dimension, more targeted and effective text processing can be performed on the screenshots of the first parent webpage and the first child webpage.

[0117] In some embodiments, the aforementioned preset meeting title recognition model may specifically include an algorithm model based on TF-IDF and XGBoost, trained in advance using screenshots of web pages containing and not containing meeting titles. Similarly, the aforementioned preset meeting content recognition model may specifically include an algorithm model based on TF-IDF and XGBoost, trained in advance using screenshots of web pages containing and not containing meeting content.

[0118] In some embodiments, see Figure 5 As shown, the above-mentioned text recognition model is used to process the screenshots of the first parent webpage and the first child webpage respectively to obtain the corresponding text recognition results of the first parent webpage and the first child webpage. In specific implementation, the following may be included:

[0119] S1: Use the preset meeting title recognition model to process the screenshots of the first parent webpage and the first child webpage respectively, and obtain the meeting title recognition results of the first parent webpage and the meeting title recognition results of the first child webpage.

[0120] S2: Use the preset meeting content recognition model to process the screenshots of the first parent webpage and the first child webpage respectively, and obtain the meeting content recognition results of the first parent webpage and the meeting content recognition results of the first child webpage.

[0121] S3: Combine the recognition results of the first parent webpage conference title and the first parent webpage conference content to obtain the recognition result of the first parent webpage text; combine the recognition results of the first child webpage conference title and the first child webpage conference content to obtain the recognition result of the first child webpage text.

[0122] Specifically, the meeting title recognition result can be the probability value that the screenshot contains the meeting title text. The meeting content recognition result can be the probability value that the screenshot contains the meeting content text.

[0123] Accordingly, the aforementioned text recognition results for the first parent webpage indicate the probability values ​​that the corresponding screenshot of the first parent webpage contains both the meeting title text and the meeting content text. Similarly, the aforementioned text recognition results for the first child webpage indicate the probability values ​​that the corresponding screenshot of the first child webpage contains both the meeting title text and the meeting content text.

[0124] Based on the above embodiments, a preset text recognition model can be used to process the screenshots of the first parent webpage and the first child webpage respectively to obtain the text recognition results of the first parent webpage and the first child webpage that can better reflect the text dimension features in the screenshots.

[0125] In some embodiments, it is also considered that a webpage containing information about an academic conference of interest to be extracted will in most cases include both the conference title text and the conference content text related to that academic conference.

[0126] Therefore, based on the text recognition results of the first parent webpage and the text recognition results of the first child webpage, the target webpage containing the target content information to be extracted is determined. In specific implementation, this may include the following:

[0127] Based on the text recognition results of the first parent webpage and the first child webpage, webpages with a probability value greater than a preset first probability threshold and a probability value greater than a preset second probability threshold containing meeting title text are selected from the parent and child webpages with a subordinate relationship. These webpages are then selected as the target webpages. The preset second probability threshold is greater than the preset first probability threshold.

[0128] Based on the above embodiments, the target webpage that truly contains the target content information to be extracted can be accurately identified based on the text dimension, according to the text recognition results of the first parent webpage and the first child webpage.

[0129] In some embodiments, when specifically determining the target webpage containing the target content information to be extracted, the following may also be included:

[0130] S1: Process the screenshots of the first parent webpage and the first child webpage using a pre-defined text feature extraction model to obtain the text features of the first parent webpage and the first child webpage.

[0131] S2: Based on the preset reference text feature set, perform text feature matching on the text features of the first parent webpage and the text features of the first child webpage to obtain the corresponding text feature matching results.

[0132] S3: Based on the text feature matching results, filter out the target webpage from the first parent webpage and the first child webpage that have a subordinate relationship.

[0133] Specifically, the preset reference text feature set stores the common text features of screenshots of web pages containing the target content. This preset reference text feature set can be obtained by clustering the text features of a large number of screenshots of web pages containing the target content.

[0134] In some embodiments, see Figure 6 As shown, the above method of extracting target content information from a target webpage based on its page links and matching page structure rules can, in practice, include the following:

[0135] S1: Determine the webpage data of the target webpage based on the page links of the target webpage;

[0136] S2: Based on the page structure rules that match the target webpage, determine the DOM tree data structure of the target webpage from the webpage data of the target webpage;

[0137] S3: Search for the specified tag element in the DOM tree data structure of the target webpage; and extract the target content information based on the searched specified tag element.

[0138] Specifically, the DOM (Document Object Model) tree refers to the Document Object Model. When a user interacts with a server to access a webpage, the server sends the webpage's HTML code to the client. The client's browser first parses this HTML code and generates the DOM tree. Based on this tree structure, the browser can render and display the corresponding webpage.

[0139] Based on the above embodiments, the structural features of the target webpage can be utilized to efficiently and accurately extract the required target content information from the target webpage.

[0140] In some embodiments, during implementation, the webpage data of the target webpage can be queried first based on the page links of the target webpage to obtain the page structure characteristics of the target webpage; then, based on the page structure characteristics of the target webpage, a preset set of page structure rules is queried to determine the page structure rules that match the target webpage. Specifically, the page structure rules that match the target webpage may include data structure rules for webpage data of webpages with the same page structure type as the target webpage.

[0141] Specifically, the webpage data of the target webpage can be the HTML code of the target webpage.

[0142] In practice, the server can intelligently identify the DOM tree data structure of the target webpage by crawling the DOM tree of the target webpage, based on the page structure rules that match the target webpage; then, it can search for information such as "..." within that DOM tree data structure. <title>”、"< / title> The system uses specified tag elements to locate the target data area containing the required target content information; it then extracts the text characters from the target data area to obtain the required target content information.

[0143] When locating the target data area containing the required target content information and extracting the text characters in the target data area, semantic judgment can also be performed on the text characters. Based on the semantic judgment results, only text characters related to the target content can be selectively extracted from the text characters in the target data area, thereby making the extracted target content information more accurate.

[0144] In some embodiments, the extraction of target content information based on the searched specified tag elements may include the following:

[0145] By combining the searched specified tag elements with specified keywords related to the target content, the DOM tree data structure of the target webpage is retrieved to extract the target content information.

[0146] Specifically, the designated keywords mentioned above can be key characters related to the target content.

[0147] In practice, the target data area containing the target content information can be located first in the DOM tree data structure of the target webpage based on the specified tag element; then, the target data area can be further searched based on the specified keyword to find the area containing the specified keyword and / or synonyms of the specified keyword, as well as the area adjacent to the area containing the specified keyword and / or synonyms of the specified keyword, as the core data area; and then the required target content information can be extracted from the core data area.

[0148] Based on the above embodiments, by introducing and utilizing specified keywords, and combining semantic-based keywords with structural-based tag elements, the required target content information can be extracted from the target webpage more accurately and quickly.

[0149] In some embodiments, after extracting the target content information from the target webpage, the method may further include the following:

[0150] S1: Process the target content information using a pre-defined semantic recognition model to obtain the corresponding target semantic classification result;

[0151] S2: Based on the target semantic classification results, determine whether the extracted target content information meets the quality requirements.

[0152] Specifically, the aforementioned preset semantic recognition model can be a neural network model that is trained in advance using sample data containing target content information and can automatically detect and recognize whether the input text content belongs to the target content information.

[0153] In practice, if the target semantic classification results confirm that the extracted target content information is indeed the required target content information (e.g., academic conference information), then the extracted target content information meets the quality requirements. Conversely, if the target semantic classification results determine that the extracted target content information is not actually the required target content information, then the extracted target content information meets the quality requirements. This can then trigger the re-extraction of the target content information from the target webpage.

[0154] Based on the above embodiments, by introducing and utilizing a preset semantic recognition model, the quality of the target content information extracted from the target webpage can be detected based on the features of the semantic dimension, so as to ensure that the required target content information can be truly extracted in the end.

[0155] In some embodiments, see Figure 7 As shown, after determining that the extracted target content information meets the quality requirements based on the target semantic classification results, the method may further include the following in its specific implementation:

[0156] S1: Obtain the text recognition results of the target webpage and the text recognition results of related webpages that are subordinate to the target webpage;

[0157] S2: Based on the text recognition results of the target webpage and the text recognition results of the associated webpage, extract related content information related to the target content information from the associated webpage;

[0158] S3: Generate a summary report on the target content information of the target website based on the target content information and related content information.

[0159] Specifically, the aforementioned related web pages can be the parent or child pages of the target web page.

[0160] In some cases, the aforementioned related web pages often also contain related content information relevant to the target content, such as introductions to academic conferences or external evaluations of those conferences. This content not only supplements the target content but is also among the information that users are most interested in.

[0161] Based on the above embodiments, by extracting the associated content information of the target webpage's associated webpages and using the target content information and associated content information together, a relatively comprehensive, complete, user-friendly, and high-quality summary report can be generated.

[0162] In some embodiments, during specific implementation, the correlation between the content information in the associated webpage and the target content information in the target webpage can be determined based on the text recognition results of the target webpage and the associated webpage. Then, based on this correlation, related content information relevant to the target content information can be extracted from the associated webpage in a targeted manner. The specific process of extracting related content information from the associated webpage can be found in the embodiment for extracting target content information from the target webpage. It will not be elaborated upon here.

[0163] In some embodiments, the server can compile and summarize target content information from different target websites into a single summary report covering multiple target websites and their target content. This summary report is then sent to the user's client terminal. The client terminal receives and displays the summary report, enabling the user to conveniently and comprehensively understand the target content information they are interested in. For example, it can allow users to easily and comprehensively understand the conference information and specific content of relevant academic conferences within the current time period, thus keeping them abreast of the latest academic developments.

[0164] As can be seen from the above, the method for obtaining target content information in a webpage provided in this specification first obtains screenshots and links of various webpages on the target website based on the website seed data of the target website; then, based on the visual dimension, it processes the screenshots of multiple webpages on the target website using a preset image processing model to filter out the screenshots of the first parent webpage and the first child webpage that have a high probability of containing target content information and have a visually subordinate relationship; further, based on the text dimension, it processes the screenshots of the first parent webpage and the first child webpage respectively using a preset text recognition model to determine the target webpage that truly contains the target content information to be extracted in terms of text semantics; finally, based on the structural dimension, it extracts the target content information from the target webpage in a targeted manner according to the page links of the target webpage and the page structure rules that match the target webpage. Thus, by effectively integrating and utilizing the features of multiple dimensions such as visual, text, and structure of webpages, the required target content information can be extracted accurately and comprehensively from a large number of webpages contained in the target website, effectively reducing the user's workload, shortening processing time, and avoiding the omission of content information.

[0165] In a specific scenario example, the method for obtaining target content information from web pages provided in this manual was applied to process approximately 400 web pages related to academic conferences from 100 university and institutional websites to automatically extract the required academic conference information. On average, each website contained four web pages related to academic conferences. Testing revealed that the accuracy rate of the academic conference information extracted using this method reached 86%.

[0166] Based on the above scenario examples, the method for obtaining target content information from web pages provided in this manual has been verified. By fusing semantic, visual, and structural features, web page information is identified and extracted efficiently without relying on numerous external interfaces. Furthermore, it can quickly extract and summarize relevant academic conference information from massive amounts of web page data, solving problems such as limited channels, cumbersome processes, and low efficiency for technical personnel in obtaining academic conference information. It also enables real-time monitoring of cutting-edge scientific and technological developments, helping enterprises enhance their overall technological strength.

[0167] See Figure 8 As shown in the figure, this specification provides a specific server, which includes a network communication port 801, a processor 802, and a memory 803. The above structures are connected by internal cables so that the various structures can perform specific data interaction.

[0168] Specifically, the network communication port 801 can be used to obtain the website seed data of the target website involving the target content.

[0169] The processor 802 is specifically configured to: acquire screenshots and links of multiple web pages from a target website based on the website seed data of the target website; wherein each screenshot corresponds to a specific web page link; process the screenshots of multiple web pages using a preset image processing model to select the screenshot of the first parent web page and the screenshot of the first child web page corresponding to the first parent web page; process the screenshots of the first parent web page and the first child web page respectively using a preset text recognition model to obtain the corresponding text recognition results of the first parent web page and the first child web page; determine the target web page containing the target content information to be extracted based on the text recognition results of the first parent web page and the first child web page; and extract the target content information from the target web page based on the page links of the target web page and the page structure rules that match the target web page.

[0170] The memory 803 can be used to store the corresponding instruction program.

[0171] In this embodiment, the network communication port 801 can be a virtual port bound to different communication protocols, thereby enabling the sending or receiving of different data. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. Furthermore, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM or CDMA; it can also be a Wi-Fi chip; or it can be a Bluetooth chip.

[0172] In this embodiment, the processor 802 can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. This specification is not limiting.

[0173] In this embodiment, the memory 803 may include multiple layers. In a digital system, anything that can store binary data can be a memory. In an integrated circuit, a circuit with storage function but no physical form is also called a memory, such as RAM, FIFO, etc. In a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0174] This specification also provides a computer-readable storage medium for a method of obtaining target content information from the aforementioned webpage. The computer-readable storage medium stores computer program instructions that, when executed, implement the following: obtaining website seed data of a target website involving the target content; obtaining screenshots and links of multiple webpages on the target website based on the website seed data; wherein each screenshot corresponds to a specific page link; processing the screenshots of the multiple webpages using a preset image processing model to filter out the screenshot of a first parent webpage and the screenshot of a first child webpage corresponding to the first parent webpage; processing the screenshots of the first parent webpage and the first child webpage respectively using a preset text recognition model to obtain corresponding text recognition results for the first parent webpage and the first child webpage; determining the target webpage containing the target content information to be extracted based on the text recognition results of the first parent webpage and the first child webpage; and extracting the target content information from the target webpage based on the page links of the target webpage and the page structure rules matching the target webpage.

[0175] In this embodiment, the storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), cache, hard disk drive (HDD), or memory card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured according to standards specified in the communication protocol for network connection communication.

[0176] In this embodiment, the specific functions and effects implemented by the program instructions stored in the computer-readable storage medium can be explained in comparison with other embodiments, and will not be repeated here.

[0177] This specification also provides a computer program product comprising a computer program that, when executed by a processor, performs the following steps: acquiring website seed data of a target website involving target content; acquiring screenshots and links of multiple web pages in the target website based on the website seed data, wherein each screenshot corresponds to a specific page link; processing the screenshots of the multiple web pages using a preset image processing model to filter out the screenshot of a first parent web page and the screenshot of a first child web page corresponding to the first parent web page; processing the screenshots of the first parent web page and the first child web page respectively using a preset text recognition model to obtain corresponding text recognition results for the first parent web page and the first child web page; determining the target web page containing the target content information to be extracted based on the text recognition results of the first parent web page and the first child web page; and extracting the target content information from the target web page based on the page links of the target web page and the page structure rules matching the target web page.

[0178] See Figure 9 As shown, at the software level, this specification also provides a device for obtaining target content information from a webpage, which may specifically include the following structural modules:

[0179] The first acquisition module 901 can be used to acquire website seed data of target websites involving target content;

[0180] The second acquisition module 902 can be used to acquire screenshots and links of multiple web pages in the target website based on the website seed data of the target website; wherein, the page screenshots and page links correspond one-to-one.

[0181] The image processing module 903 can be used to process screenshots of multiple web pages using a preset image processing model, and filter out the screenshot of the first parent web page and the screenshot of the first child web page corresponding to the first parent web page.

[0182] The text recognition module 904 can be used to process the screenshots of the first parent webpage and the first child webpage using a preset text recognition model, and obtain the corresponding text recognition results of the first parent webpage and the first child webpage.

[0183] The determination module 905 can be used to determine the target webpage containing the target content information to be extracted based on the recognition results of the first parent webpage text and the recognition results of the first child webpage text.

[0184] The extraction module 906 is specifically used to extract target content information from the target webpage based on the page links of the target webpage and the page structure rules that match the target webpage.

[0185] In some embodiments, before processing the screenshots of multiple web pages using a preset image processing model, the device may also be used to perform image feature matching on the screenshots of the multiple web pages according to a preset reference image feature set to obtain corresponding image feature matching results; and based on the image feature matching results, to filter out the screenshots of web pages that are not related to the target content from the screenshots of the multiple web pages.

[0186] In some embodiments, when the image processing module 903 is specifically implemented, it can process screenshots of multiple web pages using a preset image processing model in the following manner to filter out the screenshot of a first parent web page and the screenshot of a first child web page corresponding to the first parent web page: The preset image processing model is used to process the screenshots of multiple web pages to obtain visual similarity parameters of the screenshots of multiple web pages; the preset image processing model is used to filter out an intermediate image group of web pages with a subordinate relationship based on the visual similarity parameters of the screenshots of multiple web pages; wherein the intermediate image group contains screenshots of at least two web pages; the preset image processing model is used to obtain and, based on the common visual features of the screenshots of web pages in the same intermediate image group, the screenshot of the first parent web page and the screenshot of the first child web page corresponding to the first parent web page are determined from the intermediate image group.

[0187] In some embodiments, the target content may specifically include at least one of the following: academic conferences, business conferences, administrative meetings, etc.

[0188] In some embodiments, when the target content includes an academic conference, the preset text recognition model may specifically include: a preset conference title recognition model and a preset conference content recognition model.

[0189] In some embodiments, when the text recognition processing module 904 is specifically implemented, it can process the screenshots of the first parent webpage and the first child webpage using a preset text recognition model in the following manner to obtain the corresponding text recognition results of the first parent webpage and the first child webpage: Processing the screenshots of the first parent webpage and the first child webpage using a preset meeting title recognition model to obtain the meeting title recognition results of the first parent webpage and the meeting title recognition results of the first child webpage; Processing the screenshots of the first parent webpage and the first child webpage using a preset meeting content recognition model to obtain the meeting content recognition results of the first parent webpage and the meeting content recognition results of the first child webpage; Combining the meeting title recognition results of the first parent webpage and the meeting content recognition results of the first parent webpage to obtain the text recognition result of the first parent webpage; Combining the meeting title recognition results of the first child webpage and the meeting content recognition results of the first child webpage to obtain the text recognition result of the first child webpage.

[0190] In some embodiments, when the determination module 905 is specifically implemented, it can determine the target webpage containing the target content information to be extracted based on the first parent webpage text recognition result and the first child webpage text recognition result in the following manner: Based on the first parent webpage text recognition result and the first child webpage text recognition result, select webpages from the first parent webpage and the first child webpage that have a subordinate relationship, where the probability value of containing the meeting title text is greater than a preset first probability threshold and the probability value of containing the meeting content text is greater than a preset second probability threshold, and use these webpages as the target webpages.

[0191] In some embodiments, when the extraction module 906 is specifically implemented, it can extract target content information from the target webpage according to the page links of the target webpage and the page structure rules that match the target webpage in the following manner: determine the webpage data of the target webpage according to the page links of the target webpage; determine the DOM tree data structure of the target webpage from the webpage data of the target webpage according to the page structure rules that match the target webpage; search for specified tag elements in the DOM tree data structure of the target webpage; and extract the target content information according to the searched specified tag elements.

[0192] In some embodiments, when the extraction module 906 is specifically implemented, the target content information can be extracted based on the searched specified tag elements in the following manner: by combining the searched specified tag elements and specified keywords related to the target content, the DOM tree data structure of the target webpage is retrieved to extract the target content information.

[0193] In some embodiments, after the extraction module 906 extracts the target content information from the target webpage, it can also be used to: process the target content information using a preset semantic recognition model to obtain the corresponding target semantic classification result; and determine whether the extracted target content information meets the quality requirements based on the target semantic classification result.

[0194] In some embodiments, after determining that the extracted target content information meets the quality requirements based on the target semantic classification results, the device may further be used to obtain the text recognition results of the target webpage and the text recognition results of related webpages that have a subordinate relationship with the target webpage; extract related content information related to the target content information from the related webpages based on the text recognition results of the target webpage and the text recognition results of the related webpages; and generate a summary report on the target content information of the target website based on the target content information and the related content information.

[0195] It should be noted that the units, devices, or modules described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above devices are described by dividing them into various modules according to their functions. Of course, in implementing this specification, the functions of each module can be implemented in one or more software and / or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection between the devices or units shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0196] As can be seen from the above, the device for obtaining target content information in web pages provided in the embodiments of this specification can effectively integrate and utilize the features of multiple dimensions such as visual, text, and structure of web pages to comprehensively and efficiently extract the required target content information from a large number of web pages contained in the target website, effectively reducing the user's workload, shortening processing time, and avoiding the omission of content information.

[0197] While this specification provides the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or client terminal product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0198] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0199] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer-readable storage media, including storage devices.

[0200] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. This specification can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0201] Although this specification has been described by way of examples, those skilled in the art will recognize that many variations and modifications are possible without departing from the spirit of this specification, and it is intended that the appended claims cover such variations and modifications without departing from the spirit of this specification.

Claims

1. A method for obtaining target content information from a webpage, characterized in that, include: Obtain seed data for target websites containing the target content; Based on the target website's seed data, obtain screenshots and links of multiple web pages on the target website; each screenshot corresponds to a specific web page link. The image processing model is used to process screenshots of multiple web pages, and the screenshots of the first parent web page and the first child web page corresponding to the first parent web page are selected. The screenshots of the first parent webpage and the first child webpage are processed using a preset text recognition model to obtain the corresponding text recognition results of the first parent webpage and the first child webpage. Based on the text recognition results of the first parent webpage and the text recognition results of the first child webpage, the target webpage containing the target content information to be extracted is determined. Based on the page links of the target webpage and the page structure rules that match the target webpage, the target content information is extracted from the target webpage. The target content information is processed using a pre-defined semantic recognition model to obtain the corresponding target semantic classification result; Based on the target semantic classification results, determine whether the extracted target content information meets the quality requirements; If it is determined that the extracted target content information meets the quality requirements, then obtain the text recognition results of the target webpage and the text recognition results of related webpages that have a subordinate relationship with the target webpage. Based on the text recognition results of the target webpage and the text recognition results of the associated webpage, related content information related to the target content information is extracted from the associated webpage. Based on the target content information and related content information, generate a summary report on the target content information of the target website.

2. The method according to claim 1, characterized in that, Before processing screenshots of multiple web pages using a preset image processing model, the method further includes: Based on a preset set of reference image features, image feature matching is performed on the screenshots of the multiple web pages to obtain the corresponding image feature matching results. Based on the image feature matching results, screenshots of web pages that are not related to the target content are filtered out from the screenshots of the multiple web pages.

3. The method according to claim 1, characterized in that, The image processing model is used to process screenshots of multiple web pages, and the screenshots of the first parent web page and the first child web page corresponding to the first parent web page are selected, including: The image processing model is used to process screenshots of multiple web pages to obtain visual similarity parameters of the screenshots of multiple web pages. Using a preset image processing model, an intermediate image group of web pages with a subordinate relationship is selected based on the visual similarity parameters of screenshots from multiple web pages; wherein, the intermediate image group contains screenshots of at least two web pages. Using a preset image processing model, and based on the common visual features of the webpage screenshots in the same intermediate image group, the screenshot of the first parent webpage and the screenshot of the first child webpage corresponding to the first parent webpage are determined from the intermediate image group.

4. The method according to claim 1, characterized in that, The target content includes at least one of the following: academic conferences, business conferences, and administrative conferences.

5. The method according to claim 4, characterized in that, When the target content includes academic conferences, the preset text recognition model includes: a preset conference title recognition model and a preset conference content recognition model.

6. The method according to claim 5, characterized in that, Using a pre-defined text recognition model, the screenshots of the first parent webpage and the first child webpage are processed respectively to obtain the corresponding text recognition results for the first parent webpage and the first child webpage, including: The screenshots of the first parent webpage and the first child webpage are processed using a pre-defined meeting title recognition model to obtain the meeting title recognition results of the first parent webpage and the first child webpage. The screenshots of the first parent webpage and the first child webpage are processed using a pre-defined meeting content recognition model to obtain the meeting content recognition results of the first parent webpage and the first child webpage. The first parent webpage's conference title recognition result and the first parent webpage's conference content recognition result are combined to obtain the first parent webpage's text recognition result; the first child webpage's conference title recognition result and the first child webpage's conference content recognition result are combined to obtain the first child webpage's text recognition result.

7. The method according to claim 6, characterized in that, Based on the text recognition results of the first parent webpage and the first child webpage, the target webpage containing the target content information to be extracted is determined, including: Based on the text recognition results of the first parent webpage and the first child webpage, webpages with a probability value greater than a preset first probability threshold and a probability value greater than a preset second probability threshold containing meeting title text are selected from the parent webpage and child webpages with a subordinate relationship, and are selected as the target webpages.

8. The method according to claim 1, characterized in that, Based on the page links of the target webpage and the page structure rules that match the target webpage, the target content information is extracted from the target webpage, including: Determine the webpage data of the target webpage based on the page links of the target webpage; Based on the page structure rules that match the target webpage, the DOM tree data structure of the target webpage is determined from the webpage data of the target webpage; Search for the specified tag element in the DOM tree data structure of the target webpage; and extract the target content information based on the searched specified tag element.

9. The method according to claim 8, characterized in that, Based on the searched specified tag elements, extract the target content information, including: By combining the searched specified tag elements with specified keywords related to the target content, the DOM tree data structure of the target webpage is retrieved to extract the target content information.

10. A device for acquiring target content information from a webpage, characterized in that, include: The first acquisition module is used to acquire website seed data of target websites that involve target content; The second acquisition module is used to obtain screenshots and links of multiple web pages in the target website based on the website seed data of the target website; wherein, the page screenshots and page links correspond one-to-one. The image processing module is used to process screenshots of multiple web pages using a preset image processing model, and to filter out the screenshot of the first parent web page and the screenshot of the first child web page corresponding to the first parent web page. The text recognition module is used to process the screenshots of the first parent webpage and the first child webpage using a preset text recognition model, and to obtain the corresponding text recognition results of the first parent webpage and the first child webpage. The determination module is used to determine the target webpage containing the target content information to be extracted based on the text recognition results of the first parent webpage and the text recognition results of the first child webpage. The extraction module is used to extract target content information from the target webpage based on the page links of the target webpage and the page structure rules that match the target webpage. The extraction module is further configured to process target content information using a preset semantic recognition model to obtain corresponding target semantic classification results; determine whether the extracted target content information meets quality requirements based on the target semantic classification results; if the extracted target content information meets quality requirements, obtain the text recognition results of the target webpage and the text recognition results of related webpages that are subordinate to the target webpage; extract related content information related to the target content information from the related webpages based on the text recognition results of the target webpage and the text recognition results of the related webpages; and generate a summary report of target content information of the target website based on the target content information and the related content information.

11. A server, characterized in that, It includes a processor and a memory for storing processor-executable instructions, wherein the processor, when executing the instructions, implements the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 9.

13. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 9.

Images