A method, device and equipment for generating PDF based on webpage and storage medium

Abstract

The application provides a method, device and equipment for generating a PDF based on a webpage, and a storage medium. The method comprises the following steps: obtaining a webpage access path and an Xpath path of core content of a webpage to be accessed; sending an access request to the webpage based on the webpage access path, and obtaining feedback webpage data; determining a webpage element node tree of the webpage data; determining non-core data that needs to be deleted in the webpage data based on content in the Xpath path and the webpage element node tree, wherein the non-core data is data other than core data and display-related data in the webpage data; deleting the non-core data in the webpage; and generating a corresponding PDF file based on the core data displayed in the webpage data. The method for generating a PDF based on a webpage can directly convert webpage content into a PDF file, and the layout is normal, and the file content can be enlarged without distortion.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for generating PDFs from web pages. Background Technology

[0002] PDF (Portable Document Format) is a file format developed by Adobe Systems for exchanging files in a way that is independent of applications, operating systems, and hardware. Based on the PostScript language's image model, PDF files guarantee accurate colors and print quality on any printer. For PDF files directly converted from web pages, in addition to enlarging the content without loss of quality, it also preserves links within the web page and supports text selection.

[0003] Currently, there are two main approaches to converting web pages to PDF:

[0004] The approach involves two steps: first, scanning the webpage and taking a screenshot, then converting the screenshot to PDF.

[0005] 2. Use a dedicated PDF conversion tool to directly convert web pages to PDF in one step, such as the browser's "print" function, Adobe, or wkhtmltopdf.

[0006] Specifically, existing solution 1 generates an HTML file by executing PhantomJS and using a preset access path to the target page, and saves the HTML file in a specified directory on the server; the file path of the HTML file is then passed to wkhtmltopdf; a PDF file is generated using wkhtmltopdf, the file path, the preset file save address, and file generation parameters. The implementation process is as follows: Figure 1 As shown.

[0007] Figure 1 "URL①": This URL is the access path of the webpage that needs to be converted;

[0008] Figure 1 "Path ②": This path is the access path for HTML files in a specified directory on the server;

[0009] Solution 1 focuses on the use of PhantomJS and wkhtmltopdf, with the server playing a crucial role as a process hub and storage intermediary. A detailed analysis follows:

[0010] 1) PhantomJS simulates a browser sending requests and processing response content, making the generated HTML file consistent with what you actually see. Its headless approach makes it more efficient.

[0011] 2) The introduction of the server not only supports saving previously generated HTML files at a higher capacity, but also supports PDF conversion from multiple devices. When different devices have different conversion needs, they can be met simultaneously; when all connected devices process a task at the same time, the overall conversion efficiency can be improved.

[0012] 3) wkhtmltopdf is a cross-platform, mainstream PDF conversion tool that supports multiple programming languages. It supports two methods: direct URL to PDF and HTML file to PDF. The second method makes the entire system more flexible and efficient, avoiding the problems of slow file generation and excessively large file size that occur when directly using wkhtmltopdf to convert URLs to PDF files.

[0013] However, directly using conversion tools like wkhtmltopdf for PDF conversion can lead to layout misalignment issues when converting web pages with special tags such as nav, aside, header, and footer tags. Examples of layout misalignment can be found in [link to example]. Figure 2 and Figure 3 As shown.

[0014] The above technical solution requires PhantomJS processing before storing the webpage to a PDF file and generating an HTML file. While this somewhat mitigates the shortcomings of wkhtmltopdf, the process is still quite redundant.

[0015] For the existing Solution 2, the heights of the webpage to be processed and the PDF file are determined based on the PDF generation instructions. The page elements within the webpage to be processed and their corresponding heights are also determined. This allows for the determination of the screenshot area to obtain the image to be processed. Finally, a PDF file is generated based on the image. The implementation process can be found in [reference needed]. Figure 4 As shown.

[0016] in, Figure 4 Step 2: Because the web page elements have a nested structure, it is necessary to obtain the information of each level of web page elements and their height when traversing the page.

[0017] Figure 4Step 3: Nested elements on a webpage are characterized by a parent element potentially having multiple child elements, and the parent element's page height being the sum of the heights of all its child elements. When taking screenshots, to ensure the display of a complete webpage element isn't affected during the capture, any extra complete child elements can be placed on the next page, or the entire parent element that cannot be fully captured can be placed on the next page. This avoids splitting a child element in two across two pages, affecting the final conversion effect.

[0018] Furthermore, the key aspects of the conventional conversion process (as shown in Figure 5) lie in the hierarchical traversal of webpage elements and the calculation of their heights. A detailed analysis of the technical solution is as follows:

[0019] 1) By traversing the elements at all levels of a webpage, one can understand the micro-composition of the webpage and customize the retention and transformation of content of interest (e.g., Figure 5-1 You can keep only page element 1 and page element 3); by traversing the web page elements, you can obtain the existence relationships between elements: parallel (sibling) relationship, subordinate (parent-child) relationship, etc.; by traversing the web page elements, you can obtain the height of each level of web page elements. When multiple elements are distributed on a page, web page elements that exceed the page height can be placed on the next page (e.g., ...). Figure 5-1 When an element that exceeds the page height has multiple child nodes, and some of these child nodes can be completely preserved on the current page, the remaining portion can be moved to the next page (e.g., ...). Figure 5-2 ).

[0020] for Figure 4 In steps 4 and 5: the html2canvas plugin was used for taking screenshots. For the image to PDF conversion part, the user wanted the final tool to support offline conversion, so jsPDF was used instead of tools like wkhtmltopdf and iText that require online conversion and are time-consuming and resource-intensive.

[0021] However, while the above approach of taking screenshots first and then converting them to PDF files offers some flexibility and preserves the original page layout relatively well, the resulting PDF files cannot be scaled without loss of quality, and the links on the webpage are not clickable, and the text content on the webpage cannot be selected with the mouse. Summary of the Invention

[0022] This application provides a method, apparatus, device, and storage medium for directly converting web page content into PDF files with proper layout and supporting undistorted enlargement of file content.

[0023] To address the aforementioned technical problems, embodiments of this application provide a method for generating PDFs based on web pages, including:

[0024] Obtain the webpage access path and the XPath path of the core content of the webpage to be accessed;

[0025] Based on the webpage access path, an access request is sent to the webpage to obtain the feedback webpage data;

[0026] Determine the webpage element node tree of the webpage data;

[0027] Based on the content in the XPath path and the web page element node tree, non-core data that needs to be deleted from the web page data is determined. The non-core data is the data in the web page data other than the core data and its display-related data.

[0028] Delete non-core data from the webpage;

[0029] Generate a corresponding PDF file based on the core data displayed in the webpage data.

[0030] As an optional embodiment, the step of sending an access request to a webpage based on the webpage access path and obtaining the feedback webpage data includes:

[0031] The virtual browser sends an access request to the webpage based on the webpage access path and obtains the webpage data in response.

[0032] As an optional embodiment, the webpage data obtained as feedback includes:

[0033] After the target time period of sending the access request, retrieve the webpage data as feedback;

[0034] Simulates human behavior when browsing web pages, enabling the web page data to be loaded and fully presented.

[0035] As an optional embodiment, the step of determining the non-core data to be deleted from the webpage data based on the content and webpage element node tree in the XPath path includes:

[0036] The core content of the webpage data is located based on the XPath path;

[0037] Based on the core content and the webpage element node tree, non-core data that needs to be deleted from the webpage data is determined.

[0038] As an optional embodiment, determining the non-core data to be deleted from the webpage data based on the core content and the webpage element node tree includes:

[0039] Traverse the webpage element node tree to obtain the complete set of webpage element nodes;

[0040] Based on the core content, determine the ancestor node and subtree node corresponding to the core content in the complete set of web page element nodes;

[0041] The complete set of webpage element nodes, the ancestor nodes and subtree nodes corresponding to the core content determine the non-core data that needs to be deleted from the webpage data.

[0042] As an optional embodiment, generating a corresponding PDF file based on the core data displayed in the webpage data includes:

[0043] Based on the core data displayed in the webpage, a corresponding PDF file is generated in a non-image format.

[0044] Another embodiment of this application also provides an apparatus for generating PDFs based on web pages, comprising:

[0045] The module is used to obtain the webpage access path and the XPath path of the core content of the webpage to be accessed;

[0046] The access module is used to send an access request to the webpage based on the webpage access path and obtain the webpage data in response.

[0047] The first determining module is used to determine the webpage element node tree of the webpage data;

[0048] The second determining module is used to determine the non-core data that needs to be deleted from the web page data based on the content in the XPath path and the web page element node tree. The non-core data is the data in the web page data other than the core data and its display-related data.

[0049] The deletion module is used to delete non-core data from the webpage.

[0050] The generation module is used to generate corresponding PDF files based on the core data displayed in the web page data.

[0051] As an optional embodiment, the step of sending an access request to a webpage based on the webpage access path and obtaining the feedback webpage data includes:

[0052] The virtual browser sends an access request to the webpage based on the webpage access path and obtains the webpage data in response.

[0053] As an optional embodiment, the webpage data obtained as feedback includes:

[0054] After the target time period of sending the access request, retrieve the webpage data as feedback;

[0055] Simulates human behavior when browsing web pages, enabling the web page data to be loaded and fully presented.

[0056] As an optional embodiment, the step of determining the non-core data to be deleted from the webpage data based on the content and webpage element node tree in the XPath path includes:

[0057] The core content of the webpage data is located based on the XPath path;

[0058] Based on the core content and the webpage element node tree, non-core data that needs to be deleted from the webpage data is determined.

[0059] Based on the above embodiments, it can be understood that the beneficial effects of this application include: by sending an access request to a given webpage, obtaining the webpage content in response, and then selectively filtering out useless content (i.e., non-core content), only converting the main content (i.e., core content) to PDF, the purpose of directly converting webpage content to PDF is achieved. Simultaneously, it solves the technical problems that the converted PDF file, being in image format, cannot achieve distortion-free scaling, valid webpage links, or arbitrary text selection by the mouse. It also solves the technical problem that, when converting all webpage content, some webpage elements affect the overall conversion effect, leading to misaligned webpage layout. The PDF file converted using the method in this embodiment ensures the indiscriminate conversion of core content, has high conversion efficiency, and is more usable and of greater reference value.

[0060] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0061] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0062] The accompanying drawings are provided to further illustrate the present application and form part of the specification. They are used together with the embodiments of the present application to explain the application and do not constitute a limitation thereof. In the drawings:

[0063] Figure 1 This is a flowchart illustrating the standard conversion process between web pages and PDFs in existing technologies.

[0064] Figure 2 This is a screenshot of the webpage to be converted using existing technology.

[0065] Figure 3 This is a screenshot of the webpage after conversion using existing technology.

[0066] Figure 4This is a flowchart illustrating another common conversion process between web pages and PDFs in existing technologies.

[0067] Figure 5-1 This is a screenshot of the result of a webpage conversion using existing technology.

[0068] Figure 5-2 This is another rendering of a webpage after conversion using existing technology.

[0069] Figure 6 This is a flowchart of a method for generating PDF based on web pages in an embodiment of this application.

[0070] Figure 7 This is an application flowchart in an embodiment of this application.

[0071] Figure 8 This is a schematic diagram of the structure of the webpage element node tree in the embodiments of this application.

[0072] Figure 9 This is a schematic diagram illustrating the core and non-core content of a webpage in an embodiment of this application.

[0073] Figure 10 This is a diagram showing the effect of converting the core content of the webpage in the embodiments of this application into PDF.

[0074] Figure 11 This is an application flowchart of a method for generating PDF based on web pages in another embodiment of this application.

[0075] Figure 12 This is a screenshot showing the result of using other methods to achieve the same effect as the PDF conversion.

[0076] Figure 13 This is a structural block diagram of the apparatus for generating PDFs based on web pages in the embodiments of this application. Detailed Implementation

[0077] The specific embodiments of this application will now be described in detail with reference to the accompanying drawings, but these are not intended to limit the scope of this application.

[0078] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the following description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.

[0079] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0080] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0081] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application, which have the features described in the claims and are therefore all within the scope of protection defined herein.

[0082] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0083] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.

[0084] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.

[0085] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0086] like Figure 6 As shown in the figure, this application provides a method for generating PDF based on web pages, including:

[0087] Obtain the webpage access path and the XPath path of the core content of the webpage to be accessed;

[0088] Send an access request to the webpage based on the webpage access path and obtain the webpage data in response;

[0089] Determine the node tree of web page elements in the web page data;

[0090] Based on the content in the XPath path and the node tree of web page elements, non-core data that needs to be deleted from the web page data is identified. Non-core data refers to data in the web page data other than core data and its display-related data.

[0091] Remove non-essential data from the webpage;

[0092] Generate corresponding PDF files based on the core data displayed in the webpage.

[0093] Based on the above embodiments, it can be understood that the beneficial effects of this embodiment include: by sending an access request to a given webpage, obtaining the webpage content in response, and then selectively filtering out useless content (i.e., non-core content), only converting the main content (i.e., core content) to PDF, the purpose of directly converting webpage content to PDF is achieved. Simultaneously, it solves the technical problems of the converted PDF file being in image format, which prevents distortion-free zooming, ensures the validity of webpage links, and allows arbitrary text selection by the mouse. It also solves the technical problem of some webpage elements affecting the overall conversion effect and causing misalignment of the webpage layout when converting all webpage content in the past. The PDF file converted by the method in this embodiment can ensure the indiscriminate conversion of core content, has high conversion efficiency, and is more usable and of greater reference value.

[0094] Furthermore, based on the webpage access path, an access request is sent to the webpage to obtain the returned webpage data, including:

[0095] The system uses a virtual browser to send access requests to web pages based on the web page access path, and then obtains the web page data in response.

[0096] The webpage data from which feedback was received includes:

[0097] Retrieve the feedback webpage data after the target time period of sending the access request;

[0098] It simulates the behavior of a human browsing a webpage, enabling the webpage data to load and be fully displayed.

[0099] For example, such as Figure 7 As shown, the above steps can be implemented using a third-party tool in the Python programming language, Pyppteer (similar to a headless browser). This tool sends a webpage access request to the server using a given URL. Compared to a regular web crawler, this tool constructs a virtual browser, simulating the browsing process of a human. Next, a waiting time can be set, i.e., a target time period (the specific duration is variable). After this period, the tool retrieves the raw response data from the server, i.e., the webpage data, completes the rendering of the webpage, and then uses the JavaScript programming language to simulate human page-turning behavior, ensuring the previously retrieved webpage content is fully loaded and presented correctly.

[0100] Furthermore, based on the content in the XPath path and the webpage element node tree, non-core data that needs to be deleted from the webpage data is identified, including:

[0101] Locating the core content of a webpage based on XPath;

[0102] Based on the core content and the node tree of web page elements, identify the non-core data in the web page data that needs to be deleted.

[0103] Among them, non-core data that needs to be deleted from the webpage data is determined based on the core content and the webpage element node tree, including:

[0104] Traverse the node tree of web page elements to obtain the complete set of web page element nodes;

[0105] Based on the core content, determine the ancestor node and subtree node of the corresponding core content in the complete set of web page element nodes;

[0106] The complete set of webpage element nodes, the ancestor nodes of the corresponding core content, and the subtree nodes determine the non-core data that needs to be deleted from the webpage data.

[0107] For example, when converting web pages to PDF, many unimportant web page elements can affect the conversion result and cause layout misalignment. This embodiment addresses this by using the JavaScript programming language to process all web page elements. Specifically, it involves pre-determining the XPath path of the core content within the web page elements, locating the core content to be converted using the XPath path, obtaining the web page element node tree based on the web page content, and then traversing and extracting from the node tree to obtain two parts: the complete set of web page element nodes and a subset consisting of the ancestor nodes and subtree nodes of the core content. Finally, all nodes outside the intersection of these two subsets are removed, i.e., all non-core content is removed. The algorithm is intuitively presented as follows: Figure 8 As shown in the diagram, the `article` section represents the core content. The ancestor path of the `article` (HTML-body), the `article` tag (HTML) and its subtrees (div, p), and the `head` tag (HTML) and its subtrees (scritp) are all content that needs to be retained. The `head` section contains content related to the display of the `article`, ensuring its style and layout are correct and preventing the loss of the original webpage style after conversion. The remaining parts are nodes that need to be removed; these are all non-core content. Corresponding to an actual webpage, such as... Figure 9 As shown, the framed parts are non-core content, while the unframed parts are core content.

[0108] Generate corresponding PDF files based on the core data displayed in the webpage data, including:

[0109] Based on the core data displayed in the webpage, a corresponding PDF file is generated in a non-image format.

[0110] In other words, after Pyppteer completes the above steps and obtains the processing result, it can convert the core content of the webpage into PDF, rather than taking screenshots of the core content and then converting them as images. Therefore, in this embodiment, the converted PDF file supports free selection and copying using mouse or other input methods, and webpage links in the file are also clickable. Furthermore, since the converted PDF file is not in image format, users can zoom in freely without any distortion. A screenshot of the converted PDF file can be found here. Figure 10 As shown, it is compared to Figure 9 The webpage content shown has had unnecessary elements such as cookie queries, sidebars, and headers removed, which negatively impact the layout of the converted result. This perfectly restores the original webpage's main text, i.e., the core content. In contrast, PDF conversions achieved using other methods or tools would result in... Figure 11 As shown, the core content is obscured by other non-core content, affecting the user's viewing and usage.

[0111] Specifically, the method in this embodiment can be applied to saving PDFs of large-scale automated focused web crawler results. For example... Figure 12 As shown, in practical applications, the first step is to determine the webpage path information for all focused crawlers, set up the crawler environment, and specify the XPath for the content to be crawled on all designated webpages. Next, the path information and XPath for the content to be crawled are obtained from a webpage. A browser is simulated to send a webpage access request to the server, and the response data is waited for to be fully loaded. Once the webpage content is received, the webpage elements and resources that need to be crawled and transformed are filtered based on the XPath. Useless webpage elements are deleted, the crawling results are saved, and a PDF file of the crawling results is generated. This process is repeated until all webpages have been crawled using focused web crawlers.

[0112] Based on the above, this example focuses on personalized conversion of webpage content, utilizing XPath, a common tool in web crawlers, when specifying the content to be converted. Pyppteer, used to dynamically simulate webpage loading and browsing, is also a common tool in web crawlers. This embodiment combines resource and tool sharing with the ability of web crawlers to automate PDF conversion, while the PDF conversion itself provides additional results from the web crawler. This better adapts to the personalized needs of focused web crawlers and enhances the user experience.

[0113] like Figure 13 As shown, another embodiment of this application also provides an apparatus for generating PDFs based on web pages, including:

[0114] The module is used to obtain the webpage access path and the XPath path of the core content of the webpage to be accessed;

[0115] The access module is used to send an access request to the webpage based on the webpage access path and obtain the webpage data in response.

[0116] The first determining module is used to determine the webpage element node tree of the webpage data;

[0117] The second determining module is used to determine the non-core data that needs to be deleted from the web page data based on the content in the XPath path and the web page element node tree. The non-core data is the data in the web page data other than the core data and its display-related data.

[0118] The deletion module is used to delete non-core data from the webpage.

[0119] The generation module is used to generate corresponding PDF files based on the core data displayed in the web page data.

[0120] As an optional embodiment, the step of sending an access request to a webpage based on the webpage access path and obtaining the feedback webpage data includes:

[0121] The virtual browser sends an access request to the webpage based on the webpage access path and obtains the webpage data in response.

[0122] As an optional embodiment, the webpage data obtained as feedback includes:

[0123] After the target time period of sending the access request, retrieve the webpage data as feedback;

[0124] Simulates human behavior when browsing web pages, enabling the web page data to be loaded and fully presented.

[0125] As an optional embodiment, the step of determining the non-core data to be deleted from the webpage data based on the content and webpage element node tree in the XPath path includes:

[0126] The core content of the webpage data is located based on the XPath path;

[0127] Based on the core content and the webpage element node tree, non-core data that needs to be deleted from the webpage data is determined.

[0128] As an optional embodiment, determining the non-core data to be deleted from the webpage data based on the core content and the webpage element node tree includes:

[0129] Traverse the webpage element node tree to obtain the complete set of webpage element nodes;

[0130] Based on the core content, determine the ancestor node and subtree node corresponding to the core content in the complete set of web page element nodes;

[0131] The complete set of webpage element nodes, the ancestor nodes and subtree nodes corresponding to the core content determine the non-core data that needs to be deleted from the webpage data.

[0132] As an optional embodiment, generating a corresponding PDF file based on the core data displayed in the webpage data includes:

[0133] Based on the core data displayed in the webpage, a corresponding PDF file is generated in a non-image format.

[0134] Another embodiment of this application also provides an electronic device, including:

[0135] One or more processors;

[0136] Memory, configured to store one or more programs;

[0137] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-described method for generating PDFs based on web pages.

[0138] Furthermore, one embodiment of this application also provides a storage medium storing a computer program that, when executed by a processor, implements the method for generating PDFs based on web pages as described above. It should be understood that the various solutions in this embodiment have the corresponding technical effects in the above-described method embodiments, and will not be repeated here.

[0139] Furthermore, embodiments of this application also provide a computer program product, which is tangibly stored on a computer-readable medium and includes computer-readable instructions that, when executed, cause at least one processor to perform a method for generating PDF based on web pages, such as the embodiments described above.

[0140] It should be noted that the computer storage medium of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access storage media (RAM), read-only storage media (ROM), erasable programmable read-only storage media (EPROM or flash memory), optical fibers, portable compact disk read-only storage media (CD-ROM), optical storage media, magnetic storage media, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program configured for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, antenna, optical fiber, RF, etc., or any suitable combination thereof.

[0141] Furthermore, those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0142] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0143] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0144] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0145] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

[0146] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. A method for generating a PDF based on a web page, characterized by, The method comprises the following steps: Firstly, obtaining a webpage access path and an Xpath path of core content of a webpage element of a webpage to be accessed; Based on the webpage access path, sending an access request to the webpage to obtain feedback webpage data; Then, determining a webpage element node tree of the webpage data; Based on the Xpath path, screening out the required and converted webpage element and resource content and the webpage element node tree to determine non-core data required to be deleted in the webpage data, wherein the non-core data is data other than the core data and its display related data in the webpage data; Deleting the non-core data in the webpage, comprising: respectively acquiring a webpage element node full set, a subset composed of an ancestor node and a subtree node of core content, removing all nodes outside the intersection of the two, the core content including an article part, and the retained content including an article corresponding ancestor path, an article tag and its subtree, and a head tag and its subtree, the head part being display related content of the article; Based on the displayed core data in the webpage data, generating a corresponding PDF file.

2. The method for generating a PDF based on a webpage according to claim 1, wherein, The method comprises the following steps: Based on a virtual browser, sending an access request to the webpage based on the webpage access path to obtain feedback webpage data.

3. The method for generating a PDF based on a web page according to claim 1, wherein, The method comprises the following steps: After a target time period of sending the access request, obtaining the feedback webpage data; Simulating human behavior of browsing the webpage to enable the webpage data to complete loading and complete presentation.

4. The method for generating a PDF based on a web page according to claim 1, wherein, The method comprises the following steps: Based on the Xpath path, locating the core content in the webpage data; Based on the core content and the webpage element node tree, determining the non-core data required to be deleted in the webpage data.

5. The method for generating a PDF based on a web page according to claim 1, wherein, The method comprises the following steps: Based on the displayed core data in the webpage data, generating a corresponding PDF file in a non-picture form.

6. An apparatus for generating a PDF based on a web page, the apparatus comprising: The device is used for the method of claim 1, and the device comprises: An obtaining module, configured to obtain a webpage access path and an Xpath path of core content of a webpage to be accessed; An access module, configured to send an access request to the webpage based on the webpage access path to obtain feedback webpage data; A first determining module, configured to determine a webpage element node tree of the webpage data; A second determining module, configured to determine non-core data required to be deleted in the webpage data based on the Xpath path and the webpage element node tree, wherein the non-core data is data other than the core data and its display related data in the webpage data; A deleting module, configured to delete the non-core data in the webpage; A generating module, configured to generate a corresponding PDF file based on the displayed core data in the webpage data.

7. The apparatus for generating a PDF based on a web page according to claim 6, wherein, The method comprises the following steps: Based on a virtual browser, an access request is sent to a webpage based on the webpage access path, and feedback webpage data is obtained.

8. The apparatus for generating a PDF based on a web page according to claim 6, wherein, The obtained feedback webpage data includes: After a target time period for sending the access request, the feedback webpage data is obtained; The webpage data is loaded and completely presented by simulating human browsing behavior.

9. The apparatus for generating a PDF based on a web page according to claim 6, wherein, The non-core data to be deleted in the webpage data is determined based on the content in the Xpath path and the webpage element node tree, including: The core content in the webpage data is located based on the Xpath path; The non-core data to be deleted in the webpage data is determined based on the core content and the webpage element node tree.

10. An electronic device, comprising: The method comprises: One or more processors; Memory configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for generating a PDF based on a webpage according to any one of claims 1-5.

11. A storage medium, characterized by A computer program is stored thereon, which is executed by a processor to implement the method for generating a PDF based on a webpage according to any one of claims 1-5.

Images