Method, system and equipment for efficiently converting HTML into PDF in batches

Abstract

The invention discloses a method, a system and equipment for efficiently converting HTML (Hypertext Markup Language) into PDF (Portable Document Format) in batches, and the method comprises the following steps: obtaining a unique identifier of each file in a to-be-converted HTML file and a first feature of the file corresponding to the unique identifier according to the to-be-converted HTML file; respectively adding the HTML file to be converted into at least one task sequence according to the first feature of the HTML file to be converted; obtaining a first available resource, and configuring a first available process number according to the first available resource; configuring a host process and at least one sub-process according to the first available resource and the first available process number; and executing the host process to complete the HTML file conversion task. According to the method, the problems of how to efficiently convert mass HTML (Hypertext Markup Language) files at present, how to utilize server resources to the maximum extent and how to avoid abnormal conditions such as deadlock caused by overload of the server resources are solved.

Description

Technical Field

[0001] The present application relates to the field of software technology, and in particular to a method, system and device for efficiently converting HTML to PDF in batches. Background Art

[0002] With the development of the Internet, HTML (Hypertext Markup Language) has become the standard format for web content display and information transmission. PDF (Portable Document Format) is widely used in document storage, transmission and printing due to its good compatibility, stability and easy printing on different devices and platforms. In order to facilitate users to generate PDF files in batches, especially in scenarios such as content management systems (CMS), enterprise document management and e-commerce platforms, batch conversion from HTML to PDF has become an urgent need. At present, there are many technologies and tools on the market that can realize the function of converting HTML to PDF. Common technical means include: JavaScript libraries, server-side tools, third-party API services and programming language libraries, such as Python's pdfkit, Java's iText, PHP's TCPDF, etc. Users can call these libraries in their own applications for batch conversion, which is highly flexible.

[0003] However, when implementing batch conversion of HTML to PDF, existing solutions need to solve the problem of how to efficiently convert a large number of HTML files, maximize the use of server resources, and avoid abnormal situations such as deadlock caused by overload of server resources. Summary of the invention

[0004] The present invention provides a method, system and device for efficiently converting HTML to PDF in batches, which at least solves the problem of how to efficiently convert a large number of HTML files, maximize the use of server resources, and avoid abnormal situations such as deadlock caused by overload of server resources.

[0005] The present invention provides a method for efficiently converting HTML to PDF in batches, comprising the following steps: In response to receiving the HTML file conversion task, obtaining the HTML file to be converted; According to the HTML file to be converted, obtaining a unique identifier of each file in the HTML file to be converted and a first feature of the file corresponding to the unique identifier, wherein the first feature is configured to characterize resources consumed in a process of converting the HTML file to PDF; According to the first feature of the HTML file to be converted, respectively add the HTML file to be converted to at least one task sequence; Acquire a first available resource, and configure a first number of available processes according to the first available resource; According to the first available resource and the first number of available processes, configure a main process and at least one sub-process; The main process is configured to specify a target task sequence for the second sub-process according to the working status of the first sub-process, the first sub-process is configured as a sub-process that is executing an HTML file conversion task, and the second sub-process is configured as an idle sub-process; The subprocess is configured to read an HTML file in a target task sequence specified by the main process, convert the HTML file into a PDF file, and save the converted PDF file to a specified output directory; The main process is executed to complete the HTML file conversion task.

[0006] Optionally, obtaining, according to the HTML file to be converted, a unique identifier of each file in the HTML file to be converted and a first feature of the file corresponding to the unique identifier includes: According to the HTML file to be converted, obtaining the absolute address and file name of each file in the HTML file to be converted; According to the absolute address and file name of each file, obtain the unique identifier of each file; According to the size of each file, obtain the first feature of each file; According to the unique identifier of each file and the first feature of each file, an association between the unique identifier and the first feature is established.

[0007] Optionally, the acquiring the first available resource and configuring the first number of available processes according to the first available resource includes: Obtaining at least one of the number of available threads of a processor, single-thread processing efficiency, cache size, read bandwidth, and write bandwidth, wherein the number of available threads, single-thread processing efficiency, cache size, read bandwidth, and write bandwidth belong to the first available resource; The first number of available processes is configured according to at least one of the number of available threads of the processor, single-thread processing efficiency, cache size, read bandwidth, and write bandwidth, and the first number of available processes is configured to be not less than the number of available threads of the processor.

[0008] Optionally, the main process is further configured to start a monitoring process, and the monitoring process monitors the tasks or states of the child processes through the monitoring dictionary; The subprocess is further configured to synchronize the task sequence to which the HTML file being converted belongs and the working status of the subprocess into the monitoring dictionary: The step of specifying a target task sequence for the second sub-process according to the working condition of the first sub-process includes: Acquire the task sequence to which the HTML file being converted by the first sub-process belongs through the monitoring process; According to the task sequence to which the HTML file being converted by the first sub-process belongs, a target task sequence is specified for the second sub-process.

[0009] Optionally, specifying a task sequence for the second sub-process according to the task sequence to which the HTML file being converted by the first sub-process belongs includes: According to the task sequence to which the HTML file being converted by the first sub-process belongs, obtaining the system resources occupied by the first sub-process; Obtaining second available resources according to the system resources occupied by the first sub-process and the first available resources; A target task sequence is specified for a second sub-process according to the second available resource and the at least one task sequence.

[0010] Optionally, before the step of adding the HTML files to be converted to at least one task sequence according to the first feature of the HTML files to be converted, the step further includes: Clustering the HTML files according to the first feature of the HTML files to be converted to obtain clustering results; At least one task sequence matching the clustering result is created according to the clustering result.

[0011] Optionally, creating at least one task sequence matching the clustering result according to the clustering result includes: According to the clustering result, creating at least one empty task sequence matching the clustering result; According to the clustering result, adding the HTML file to the at least one empty task sequence to obtain at least one task sequence; According to the HTML file in the at least one task sequence, obtaining a first feature of the HTML file in each task sequence; According to the first feature of the HTML file in each task sequence, a resource consumption tag is marked for each task sequence, and the resource consumption tag is used to represent the resources consumed in the process of converting the HTML file in the task sequence into PDF.

[0012] In another aspect, a system for efficiently batch converting HTML to PDF includes a conversion platform and a data management platform; The conversion platform is configured to: In response to receiving the HTML file conversion task, obtaining the HTML file to be converted through the data management platform; According to the HTML file to be converted, obtaining a unique identifier of each file in the HTML file to be converted and a first feature of the file corresponding to the unique identifier, wherein the first feature is configured to characterize resources consumed in a process of converting the HTML file to PDF; According to the first feature of the HTML file to be converted, respectively add the HTML file to be converted to at least one task sequence; Acquire a first available resource, and configure a first number of available processes according to the first available resource; According to the first available resource and the first number of available processes, configure a main process and at least one sub-process; The main process is configured to specify a target task sequence for the second sub-process according to the working status of the first sub-process, the first sub-process is configured as a sub-process that is executing an HTML file conversion task, and the second sub-process is configured as an idle sub-process; The subprocess is configured to read an HTML file in a target task sequence specified by the main process, convert the HTML file into a PDF file, and save the converted PDF file to a specified output directory; Execute the main process to complete the HTML file conversion task, and save the converted file to the data management platform; The data management platform is configured to: Providing data storage and / or data management services for the conversion platform.

[0013] Optionally, before the step of adding the HTML files to be converted to at least one task sequence according to the first feature of the HTML files to be converted, the step further includes: Clustering the HTML files according to the first feature of the HTML files to be converted to obtain clustering results; According to the clustering result, creating at least one task sequence matching the clustering result; The step of creating at least one task sequence matching the clustering result according to the clustering result includes: According to the clustering result, creating at least one empty task sequence matching the clustering result; According to the clustering result, adding the HTML file to the at least one empty task sequence to obtain at least one task sequence; According to the HTML file in the at least one task sequence, obtaining a first feature of the HTML file in each task sequence; According to the first feature of the HTML file in each task sequence, a resource consumption tag is marked for each task sequence, and the resource consumption tag is used to represent the resources consumed in the process of converting the HTML file in the task sequence into PDF.

[0014] On the other hand, a device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the above methods.

[0015] On the other hand, a computer-readable storage medium is provided. The device includes a memory and a processor. The memory stores a computer program. The processor executes the computer program to implement any of the above methods.

[0016] Compared with the prior art, the present invention has the following advantages and beneficial effects: The present invention discloses a method, system and device for efficiently converting HTML to PDF in batches, comprising: in response to receiving an HTML file conversion task, obtaining an HTML file to be converted; according to the HTML file to be converted, obtaining a unique identifier of each file in the HTML file to be converted and a first feature of the file corresponding to the unique identifier, wherein the first feature is configured to characterize the resources consumed by the HTML file in the process of converting the HTML file to PDF; according to the first feature of the HTML file to be converted, adding the HTML file to be converted to at least one task sequence respectively; obtaining a first available resource, and configuring a first available process number according to the first available resource; configuring a main process and at least one sub-process according to the first available resource and the first available process number; the main process is configured to specify a target task sequence for a second sub-process according to the working condition of the first sub-process, the first sub-process is configured as a sub-process that is executing the HTML file conversion task, and the second sub-process is configured as an idle sub-process; the sub-process is configured to read the HTML file in the target task sequence specified by the main process, convert the HTML file into a PDF file, and save the converted PDF file to a specified output directory; and executing the main process to complete the HTML file conversion task. At least it solves the problem of how to efficiently convert large quantities of HTML files, maximize the use of server resources, and avoid abnormal situations such as deadlock due to overloading of server resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation of the present application or the technical solution in the prior art, the following is a brief introduction to the drawings required for the specific implementation or the prior art description. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual scale.

[0018] Figure 1 A flowchart of the method for efficiently converting HTML to PDF in batches in this application; Figure 2 This is a schematic diagram of the structure of a computer device in this application.

[0019] Markings in the figure: 101 - processor, 102 - communication bus, 103 - network interface, 104 - user interface, 105 - memory.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the scheme of the present disclosure, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present disclosure.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0023] Example 1 like Figure 1 As shown, a method for efficiently converting HTML to PDF in batches includes the following steps: S1. In response to receiving an HTML file conversion task, obtaining an HTML file to be converted.

[0024] Optionally, the information of the HTML file conversion task includes the storage address of the HTML file to be converted or the folder including the HTML file, and may also include information or parameters related to the HTML to PDF conversion task, such as system resources allocated to the task, the time to start executing the task, etc.

[0025] Optionally, there can be one or more HTML files or folders to be converted.

[0026] S2. According to the HTML file to be converted, obtain a unique identifier of each file in the HTML file to be converted and a first feature of the file corresponding to the unique identifier.

[0027] Optionally, the unique identifier of each file may be the name of each file. When there are multiple folders, the unique identifier of each file may be the folder name plus the file name or the storage absolute address plus the file name.

[0028] Specifically, the first feature is configured to characterize the resources consumed in the process of converting the HTML file into a PDF file.

[0029] Optionally, the first feature can be obtained according to the size of the corresponding HTML file. Specifically, it also includes a pre-configured mapping table or mapping relationship, which is used to obtain the feature value of the first feature of the HTML file according to the size of the HTML file.

[0030] S3. According to the first feature of the HTML file to be converted, add the HTML file to be converted into at least one task sequence respectively.

[0031] Optionally, according to the first feature of the HTML file to be converted, adding the HTML file to be converted to at least one task sequence respectively includes: The HTML files to be converted are clustered according to the first features of the HTML files to be converted, and the HTML files to be converted of the same category are grouped into the same task sequence based on the clustering result; wherein one task sequence may correspond to one category or multiple categories.

[0032] Optionally, according to the first feature of the HTML file to be converted, adding the HTML file to be converted to at least one task sequence respectively includes: A first available resource is obtained, at least one task sequence is preset according to the first available resource, and a first task sequence tag is generated for each task sequence, wherein the first task sequence tag is used to characterize the resources consumed in the process of converting the HTML file in the corresponding task sequence into PDF.

[0033] S4. Obtain a first available resource, and configure a first number of available processes according to the first available resource.

[0034] In order to achieve efficient conversion of large quantities of HTML files, it is necessary to execute multiple conversion processes simultaneously. The number of configured processes is closely related to the tasks to be performed and system resources.

[0035] S5. Configure a main process and at least one sub-process according to the first available resource and the first number of available processes.

[0036] Specifically, the main process is configured to specify a target task sequence for the second sub-process according to the working status of the first sub-process.

[0037] Specifically, the first sub-process is configured as a sub-process that is executing an HTML file conversion task, and the second sub-process is configured as an idle sub-process.

[0038] Specifically, the subprocess is configured to read the HTML file in the target task sequence specified by the main process, convert the HTML file into a PDF file, and save the converted PDF file to a specified output directory.

[0039] S6. Execute the main process to complete the HTML file conversion task.

[0040] In the existing solutions, there is generally only one task sequence, and all child processes obtain HTML files from the task sequence in sequence. The problem with this method is that it is impossible to adjust the HTML files obtained by the child processes according to the system resources consumed by the tasks being executed. For example, in the traditional method of generating a task sequence by sorting HTML files from small to large, all child processes are converting smaller files at the beginning of processing. At this time, the utilization rate of the allocated system resources may be low. When the small files are converted, all child processes are converting larger files. At this time, the demand for system resources is large, and it is easy to cause abnormal situations such as deadlock due to overload of server resources.

[0041] By adopting the method described in this embodiment, HTML files are first clustered according to size and then added to at least one task sequence. When a sub-process needs to obtain an HTML file for conversion, the main process selects a suitable task sequence from multiple task sequences and assigns it to the sub-process based on the system resources occupied by conversion tasks being executed by other sub-processes and the total available system resources, so as to ensure the highest utilization rate of system resources and less fluctuation in demand for system resources. This solves the problem of how to efficiently convert large quantities of HTML files, maximize the use of server resources, and avoid abnormal situations such as deadlock due to overload of server resources.

[0042] Example 2 This embodiment is based on the first embodiment, and provides a method for efficiently converting HTML to PDF in batches, including the following steps: S1. In response to receiving an HTML file conversion task, obtaining an HTML file to be converted.

[0043] Optionally, the information of the HTML file conversion task includes the storage address of the HTML file to be converted or the folder including the HTML file, and may also include information or parameters related to the HTML to PDF conversion task, such as system resources allocated to the task, the time to start executing the task, etc.

[0044] Optionally, there can be one or more HTML files or folders to be converted.

[0045] S2. According to the HTML file to be converted, obtain a unique identifier of each file in the HTML file to be converted and a first feature of the file corresponding to the unique identifier.

[0046] Optionally, the unique identifier of each file may be the name of each file. When there are multiple folders, the unique identifier of each file may be the folder name plus the file name or the storage absolute address plus the file name.

[0047] Specifically, the first feature is configured to characterize the resources consumed in the process of converting the HTML file into a PDF file.

[0048] Optionally, the first feature can be obtained according to the size of the corresponding HTML file. Specifically, it also includes a pre-configured mapping table or mapping relationship, which is used to obtain the feature value of the first feature of the HTML file according to the size of the HTML file.

[0049] Optionally, obtaining, according to the HTML file to be converted, a unique identifier of each file in the HTML file to be converted and a first feature of the file corresponding to the unique identifier includes: According to the HTML file to be converted, the absolute address and file name of each file in the HTML file to be converted are obtained; According to the absolute address and file name of each file, obtain the unique identifier of each file; According to the size of each file, obtain the first feature of each file; According to the unique identifier of each file and the first feature of each file, an association between the unique identifier and the first feature is established.

[0050] S3. According to the first feature of the HTML file to be converted, add the HTML file to be converted into at least one task sequence respectively.

[0051] Optionally, according to the first feature of the HTML file to be converted, adding the HTML file to be converted to at least one task sequence respectively includes: The HTML files to be converted are clustered according to the first features of the HTML files to be converted, and the HTML files to be converted of the same category are grouped into the same task sequence based on the clustering result; wherein one task sequence may correspond to one category or multiple categories.

[0052] Optionally, according to the first feature of the HTML file to be converted, adding the HTML file to be converted to at least one task sequence respectively includes: A first available resource is obtained, at least one task sequence is preset according to the first available resource, and a first task sequence tag is generated for each task sequence, wherein the first task sequence tag is used to characterize the resources consumed in the process of converting the HTML file in the corresponding task sequence into PDF.

[0053] S4. Obtain a first available resource, and configure a first number of available processes according to the first available resource.

[0054] In order to achieve efficient conversion of large quantities of HTML files, it is necessary to execute multiple conversion processes simultaneously. The number of configured processes is closely related to the tasks to be performed and system resources.

[0055] Optionally, obtaining a first available resource and configuring a first available number of processes according to the first available resource includes: Obtaining at least one of the number of available threads, single-thread processing efficiency, cache size, read bandwidth, and write bandwidth of a processor, where the number of available threads, single-thread processing efficiency, cache size, read bandwidth, and write bandwidth belong to the first available resource; The first number of available processes is configured according to at least one of the number of available threads of the processor, single-thread processing efficiency, cache size, read bandwidth, and write bandwidth, and the first number of available processes is configured to be not less than the number of available threads of the processor.

[0056] Generally speaking, you can use the same number of processes as the number of available CPU cores or threads to maximize CPU utilization. If your tasks rely more on I / O operations (such as disk reading, writing, or network operations) than on the CPU, you can create more processes than the number of cores, because I / O operations cause waiting time during which the CPU can switch to other processes for execution.

[0057] S5. Configure a main process and at least one sub-process according to the first available resource and the first number of available processes.

[0058] Specifically, the main process is configured to specify a target task sequence for the second sub-process according to the working status of the first sub-process.

[0059] Specifically, the first sub-process is configured as a sub-process that is executing an HTML file conversion task, and the second sub-process is configured as an idle sub-process.

[0060] Specifically, the subprocess is configured to read the HTML file in the target task sequence specified by the main process, convert the HTML file into a PDF file, and save the converted PDF file to a specified output directory.

[0061] Optionally, the main process is also configured to start a monitoring process, which monitors the tasks or status of the child process through the monitoring dictionary; The child process is also configured to synchronize the task sequence to which the HTML file being converted belongs and the working status of the child process into the monitoring dictionary: According to the working status of the first sub-process, a target task sequence is specified for the second sub-process, including: By monitoring the process, obtaining the task sequence to which the HTML file being converted by the first sub-process belongs; According to the task sequence to which the HTML file being converted by the first sub-process belongs, a target task sequence is specified for the second sub-process.

[0062] Optionally, specifying a task sequence for the second sub-process according to the task sequence to which the HTML file being converted by the first sub-process belongs includes: According to the task sequence to which the HTML file being converted by the first sub-process belongs, obtaining the system resources occupied by the first sub-process; Obtaining a second available resource according to the system resources occupied by the first subprocess and the first available resource; A target task sequence is specified for the second sub-process according to the second available resource and the at least one task sequence.

[0063] Optionally, according to the first feature of the HTML file to be converted, before the step of adding the HTML file to be converted to at least one task sequence, the method further includes: Clustering the HTML files according to the first feature of the HTML files to be converted to obtain clustering results; According to the clustering result, at least one task sequence matching the clustering result is created.

[0064] Specifically, a pseudo code to implement this step uses the K-Means clustering algorithm with k=3: def main(directory, k=3): # 1. Get the HTML file html_files = get_html_files(directory) # 2. Get the HTML file size file_sizes = [get_file_size(file) for file in html_files] # 3. Clustering based on file size labels = cluster_files_by_size(file_sizes, k) # 4. Create a task sequence based on clustering results task_sequences = create_task_sequences(html_files, labels) # Output task sequence for task_id, files in task_sequences.items(): print(f"Task {task_id}: {files}") # implement directory = "path_to_html_files" main(directory, k=3).

[0065] Optionally, the number of clusters of the clustering result is not greater than the number of subprocesses to avoid excessive number of clustering categories and increase of system burden.

[0066] Optionally, based on the clustering result, at least one task sequence matching the clustering result is created, including: According to the clustering result, at least one empty task sequence matching the clustering result is created; According to the clustering result, the HTML file is added to at least one empty task sequence to obtain at least one task sequence; According to the HTML file in at least one task sequence, obtaining a first feature of the HTML file in each task sequence; According to the first feature of the HTML file in each task sequence, a resource consumption tag is marked for each task sequence, and the resource consumption tag is used to represent the resources consumed in the process of converting the HTML file in the task sequence into PDF.

[0067] Optionally, the resource consumption tag is used to indicate the maximum value of resources consumed during the conversion of HTML files in the task sequence to PDF.

[0068] Optionally, obtaining a first available resource and configuring a first available number of processes according to the first available resource includes: A first available resource is obtained, and a first available number of processes is configured according to the first available resource, a resource consumption tag of each task sequence, and the number of HTML files in each task sequence.

[0069] Specifically, ; in, n is the first available process number, P is the first available resource, N is the total number of HTML files, K i It is a task sequence i Resource consumption tags, N i It is a task sequence i The total number of HTML files in .

[0070] Specifically, an example, including 100 HTML files, is divided into 5 categories by the K-Means clustering algorithm with k=5. The number of files in each category is shown in Table 1 below; Table 1: Number of documents in each category

[0071] According to the first feature of the HTML file in each task sequence, the HTML file is added to 5 task sequences and the resource consumption label is marked for each task sequence: 0-500kb corresponds to a resource consumption label of 5 units of system resources, 500-1000kb corresponds to a resource consumption label of 10 units of system resources, 1000-2000kb corresponds to a resource consumption label of 20 units of system resources, 2000-2500kb corresponds to a resource consumption label of 25 units of system resources, and 2500-3000kb corresponds to a resource consumption label of 30 units of system resources; the first available resources are 250 units of system resources; the first available process number is 14, including 1 main process, 1 monitoring process and 12 child processes. After the main process, monitoring process and sub-process are configured, the main process is executed to start the conversion. During the execution process, when a sub-process completes the conversion of the previous HTML file, the process serves as the second sub-process. The main process reads the system resources occupied by the conversion tasks being executed by the other 11 sub-processes through the monitoring process, that is, the system resources occupied by the first sub-process, and then subtracts the system resources occupied by the first sub-process from the first available resources to obtain the second available resources. For example, if the system resources occupied by the first sub-process are 240 units, the second available resources are 10 units. Based on the second available resources and 5 task sequences, the target task sequence is specified for the second sub-process as the task sequence corresponding to 10 units of system resources. Here, the method of specifying the target task sequence for the second sub-process is to specify the task sequence with the largest resource consumption label among the task sequences that do not exceed the second available resources for the second sub-process.

[0072] Optionally, in the above method, the method for obtaining the second available resource according to the system resources occupied by the first sub-process and the first available resource, that is, the method in which the main process reads the system resources occupied by the conversion tasks being executed by the other 11 sub-processes through the monitoring process, includes: Read each task sequence corresponding to each first sub-process and the resource consumption tag corresponding to each task sequence through the monitoring process; The system resources occupied by the first subprocess are obtained by summing up the task sequences corresponding to the first subprocesses and the resource consumption tags corresponding to the task sequences; The second available resource is obtained by subtracting the system resources occupied by the first subprocess from the first available resource.

[0073] The above method does not need to obtain the specific system resources occupied by the HTML file converted by the first sub-process, but only needs to obtain the resource consumption label of the task sequence corresponding to the HTML, and the storage amount that the monitoring process needs to obtain and record is relatively small. Correspondingly, the first feature of the HTML file being converted by the first sub-process can also be directly obtained, and the system resources occupied by the first sub-process can be calculated by the first feature, which has higher accuracy and higher utilization of system resources.

[0074] In the above method, the main process is configured to be responsible for the overall task scheduling and process management, which may include: initializing and setting the process pool, configuring the multi-process environment; assigning HTML files to child processes for processing; starting a monitoring process to track and report the progress and status of the entire task; waiting for all child process tasks to be completed, ending the monitoring process and cleaning up resources.

[0075] Each child process is assigned one or more HTML files and performs the following tasks: read the specified HTML file; use WeasyPrint or other conversion libraries to convert the HTML content to a PDF file; save the converted PDF file to the specified output directory; during the conversion process, any success or failure results will be updated to a shared status dictionary, which is managed by the manager of the main process.

[0076] The monitoring process is independent of the main process and the sub-process, and is mainly responsible for the following tasks: regularly checking the processing status of each file in the shared status dictionary, such as in progress, completed or failed); counting and reporting the number of completed and failed tasks, as well as the overall progress; monitoring the entire conversion process until all files are processed.

[0077] Specifically, in the above method, multi-process parallel processing is implemented through Python's multiprocessing module. This method uses multiple cores of the CPU to process the conversion of multiple HTML files at the same time, thereby improving the processing speed. An exemplary code is as follows: pool = multiprocessing.Pool(processes=multiprocessing.cpu_count()) for html_file in html_files: pool.apply_async(convert_to_pdf, args=(html_file, output_folder)) pool.close() pool.join(); In the above code, multiprocessing.Pool(processes=multiprocessing.cpu_count()) is used to create a process pool. The processes parameter specifies the number of processes, and multiprocessing.cpu_count() automatically allocates an appropriate number of processes based on the number of CPU cores on the current device to take advantage of multiple cores. Once all tasks are submitted to the process pool, pool.close() will close the process pool and prevent any new tasks from being submitted. pool.join() will block the main process until all child processes finish executing.

[0078] S6. Execute the main process to complete the HTML file conversion task.

[0079] Example 3 An efficient system for batch conversion of HTML to PDF, including a conversion platform and a data management platform; The conversion platform is configured as: In response to receiving the HTML file conversion task, obtaining the HTML file to be converted through the data management platform; According to the HTML file to be converted, obtaining a unique identifier of each file in the HTML file to be converted and a first feature of the file corresponding to the unique identifier, wherein the first feature is configured to characterize resources consumed in a process of converting the HTML file to PDF; According to the first feature of the HTML file to be converted, respectively add the HTML file to be converted to at least one task sequence; Acquire a first available resource, and configure a first available number of processes according to the first available resource; According to the first available resource and the first number of available processes, configure a main process and at least one sub-process; The main process is configured to specify a target task sequence for the second sub-process according to the working status of the first sub-process, the first sub-process is configured as a sub-process that is executing the HTML file conversion task, and the second sub-process is configured as an idle sub-process; The child process is configured to read the HTML file in the target task sequence specified by the main process, convert the HTML file into a PDF file, and save the converted PDF file to the specified output directory; Execute the main process to complete the HTML file conversion task and save the converted file to the data management platform; The data management platform is configured as: Providing data storage and / or data management services for the conversion platform.

[0080] Optionally, according to the first feature of the HTML file to be converted, before the step of adding the HTML file to be converted to at least one task sequence, the method further includes: Clustering the HTML files according to the first feature of the HTML files to be converted to obtain clustering results; According to the clustering result, at least one task sequence matching the clustering result is created; According to the clustering result, at least one task sequence matching the clustering result is created, including: According to the clustering result, at least one empty task sequence matching the clustering result is created; According to the clustering result, the HTML file is added to at least one empty task sequence to obtain at least one task sequence; According to the HTML file in at least one task sequence, obtaining a first feature of the HTML file in each task sequence; According to the first feature of the HTML file in each task sequence, a resource consumption tag is marked for each task sequence, and the resource consumption tag is used to represent the resources consumed in the process of converting the HTML file in the task sequence into PDF.

[0081] Optionally, obtaining, according to the HTML file to be converted, a unique identifier of each file in the HTML file to be converted and a first feature of the file corresponding to the unique identifier includes: According to the HTML file to be converted, the absolute address and file name of each file in the HTML file to be converted are obtained; According to the absolute address and file name of each file, obtain the unique identifier of each file; According to the size of each file, obtain the first feature of each file; According to the unique identifier of each file and the first feature of each file, an association between the unique identifier and the first feature is established.

[0082] Optionally, obtaining a first available resource and configuring a first available number of processes according to the first available resource includes: Obtaining at least one of the number of available threads, single-thread processing efficiency, cache size, read bandwidth, and write bandwidth of a processor, where the number of available threads, single-thread processing efficiency, cache size, read bandwidth, and write bandwidth belong to the first available resource; The first number of available processes is configured according to at least one of the number of available threads of the processor, single-thread processing efficiency, cache size, read bandwidth, and write bandwidth, and the first number of available processes is configured to be not less than the number of available threads of the processor.

[0083] Optionally, the main process is also configured to start a monitoring process, which monitors the tasks or status of the child process through the monitoring dictionary; The child process is also configured to synchronize the task sequence to which the HTML file being converted belongs and the working status of the child process into the monitoring dictionary: According to the working status of the first sub-process, a target task sequence is specified for the second sub-process, including: By monitoring the process, obtaining the task sequence to which the HTML file being converted by the first sub-process belongs; According to the task sequence to which the HTML file being converted by the first sub-process belongs, a target task sequence is specified for the second sub-process.

[0084] Optionally, specifying a task sequence for the second sub-process according to the task sequence to which the HTML file being converted by the first sub-process belongs includes: According to the task sequence to which the HTML file being converted by the first sub-process belongs, obtaining the system resources occupied by the first sub-process; Obtaining a second available resource according to the system resources occupied by the first subprocess and the first available resource; A target task sequence is specified for the second sub-process according to the second available resource and the at least one task sequence.

[0085] Optionally, according to the first feature of the HTML file to be converted, before the step of adding the HTML file to be converted to at least one task sequence, the method further includes: Clustering the HTML files according to the first feature of the HTML files to be converted to obtain clustering results; According to the clustering result, at least one task sequence matching the clustering result is created.

[0086] Example 4 This embodiment provides a device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement any of the above methods.

[0087] Specifically, Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of an electronic device of the hardware operating environment involved in the embodiment of the present application. The electronic device may include: a processor 101, such as a central processing unit (CPU), a communication bus 102, a user interface 104, a network interface 103, and a memory 105. Among them, the communication bus 102 is used to realize the connection and communication between these components. The user interface 104 may include a display (Display), an input unit such as a keyboard (Keyboard), and the user interface 104 may also include a standard wired interface and a wireless interface. The network interface 103 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 105 may optionally be a storage device independent of the aforementioned processor 101. The memory 105 may be a high-speed random access memory (RAM) memory, or it may be a stable non-volatile memory (NVM), such as at least one disk storage. The processor 101 may be a general-purpose processor, including a central processing unit, a network processor, etc., or it may be a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0088] Those skilled in the art will appreciate that Figure 2 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0089] like Figure 2 As shown, the memory 105 as a storage medium may include an operating system, a network communication module, a user interface module, and an application program for implementing a method for efficiently converting HTML to PDF in batches.

[0090] exist Figure 2In the electronic device shown, the network interface 103 is mainly used for data communication with the network server; the user interface 104 is mainly used for data interaction with the user; the processor 101 and the memory 105 in the present application can be set in the electronic device, and the electronic device calls the application program stored in the memory 105 for implementing a method for efficiently converting HTML to PDF in batches through the processor 101 to implement the above method.

[0091] Example 5 This embodiment provides a computer-readable storage medium, on which a computer program is stored, and a processor executes the computer program to implement any of the above methods.

[0092] In some embodiments, the computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.

[0093] In the above embodiments of the present disclosure, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0094] In several embodiments provided in the present application, resources and system resources refer to hardware resources or other resources that a device such as a computer or server used to process conversion work can provide for a conversion task.

[0095] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0096] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0097] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a non-volatile storage medium, including a number of instructions to enable a device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present disclosure. The aforementioned non-volatile storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0099] The above are only preferred embodiments of the present disclosure. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present disclosure. These improvements and modifications should also be regarded as the protection scope of the present disclosure.

Claims

1. An efficient method for batch converting HTML to PDF, characterized in that: The following steps are involved: In response to receiving the HTML file conversion task, obtaining the HTML file to be converted; According to the HTML file to be converted, obtaining a unique identifier of each file in the HTML file to be converted and a first feature of the file corresponding to the unique identifier, wherein the first feature is configured to characterize resources consumed in a process of converting the HTML file to PDF; According to the first feature of the HTML file to be converted, respectively add the HTML file to be converted to at least one task sequence; Acquire a first available resource, and configure a first number of available processes according to the first available resource; According to the first available resource and the first number of available processes, configure a main process and at least one sub-process; The main process is configured to specify a target task sequence for the second sub-process according to the working status of the first sub-process, the first sub-process is configured as a sub-process that is executing an HTML file conversion task, and the second sub-process is configured as an idle sub-process; The subprocess is configured to read an HTML file in a target task sequence specified by the main process, convert the HTML file into a PDF file, and save the converted PDF file to a specified output directory; The main process is executed to complete the HTML file conversion task.

2. The method for efficiently converting HTML to PDF in batches as claimed in claim 1, characterized in that: The step of obtaining, according to the HTML file to be converted, a unique identifier of each file in the HTML file to be converted and a first feature of the file corresponding to the unique identifier comprises: According to the HTML file to be converted, obtaining the absolute address and file name of each file in the HTML file to be converted; According to the absolute address and file name of each file, obtain the unique identifier of each file; According to the size of each file, obtain the first feature of each file; According to the unique identifier of each file and the first feature of each file, an association between the unique identifier and the first feature is established.

3. The method for efficiently converting HTML to PDF in batches as claimed in claim 1, characterized in that: The obtaining of the first available resource and configuring the first number of available processes according to the first available resource includes: Obtaining at least one of the number of available threads of a processor, single-thread processing efficiency, cache size, read bandwidth, and write bandwidth, wherein the number of available threads, single-thread processing efficiency, cache size, read bandwidth, and write bandwidth belong to the first available resource; The first number of available processes is configured according to at least one of the number of available threads of the processor, single-thread processing efficiency, cache size, read bandwidth, and write bandwidth, and the first number of available processes is configured to be not less than the number of available threads of the processor.

4. The method for efficiently converting HTML to PDF in batches as claimed in claim 1, characterized in that: The main process is further configured to start a monitoring process, and the monitoring process monitors the tasks or states of the subprocesses through a monitoring dictionary; The subprocess is further configured to synchronize the task sequence to which the HTML file being converted belongs and the working status of the subprocess into the monitoring dictionary: The step of specifying a target task sequence for the second sub-process according to the working condition of the first sub-process includes: Acquire the task sequence to which the HTML file being converted by the first sub-process belongs through the monitoring process; According to the task sequence to which the HTML file being converted by the first sub-process belongs, a target task sequence is specified for the second sub-process.

5. The method for efficiently converting HTML to PDF in batches as claimed in claim 4, characterized in that: The step of specifying a task sequence for the second sub-process according to the task sequence to which the HTML file being converted by the first sub-process belongs includes: According to the task sequence to which the HTML file being converted by the first sub-process belongs, obtaining the system resources occupied by the first sub-process; Obtaining second available resources according to the system resources occupied by the first sub-process and the first available resources; A target task sequence is specified for a second sub-process according to the second available resource and the at least one task sequence.

6. The method for efficiently converting HTML to PDF in batches as claimed in claim 1, characterized in that: Before the step of adding the HTML files to be converted into at least one task sequence according to the first feature of the HTML files to be converted, the method further includes: Clustering the HTML files according to the first feature of the HTML files to be converted to obtain clustering results; At least one task sequence matching the clustering result is created according to the clustering result.

7. The method for efficiently converting HTML to PDF in batches as claimed in claim 6, characterized in that: The step of creating at least one task sequence matching the clustering result according to the clustering result includes: According to the clustering result, creating at least one empty task sequence matching the clustering result; According to the clustering result, adding the HTML file to the at least one empty task sequence to obtain at least one task sequence; According to the HTML file in the at least one task sequence, obtaining a first feature of the HTML file in each task sequence; According to the first feature of the HTML file in each task sequence, a resource consumption tag is marked for each task sequence, and the resource consumption tag is used to represent the resources consumed in the process of converting the HTML file in the task sequence into PDF.

8. An efficient system for batch conversion of HTML to PDF, characterized by: Includes conversion platform and data management platform; The conversion platform is configured to: In response to receiving the HTML file conversion task, obtaining the HTML file to be converted through the data management platform; According to the HTML file to be converted, obtaining a unique identifier of each file in the HTML file to be converted and a first feature of the file corresponding to the unique identifier, wherein the first feature is configured to characterize resources consumed in a process of converting the HTML file to PDF; According to the first feature of the HTML file to be converted, respectively add the HTML file to be converted to at least one task sequence; Acquire a first available resource, and configure a first number of available processes according to the first available resource; According to the first available resource and the first number of available processes, configure a main process and at least one sub-process; The main process is configured to specify a target task sequence for the second sub-process according to the working status of the first sub-process, the first sub-process is configured as a sub-process that is executing an HTML file conversion task, and the second sub-process is configured as an idle sub-process; The subprocess is configured to read an HTML file in a target task sequence specified by the main process, convert the HTML file into a PDF file, and save the converted PDF file to a specified output directory; Execute the main process to complete the HTML file conversion task, and save the converted file to the data management platform; The data management platform is configured to: Providing data storage and / or data management services for the conversion platform.

9. The system for efficiently converting HTML to PDF in batches as claimed in claim 8, characterized in that: Before the step of adding the HTML files to be converted into at least one task sequence according to the first feature of the HTML files to be converted, the method further includes: Clustering the HTML files according to the first feature of the HTML files to be converted to obtain clustering results; According to the clustering result, creating at least one task sequence matching the clustering result; The step of creating at least one task sequence matching the clustering result according to the clustering result includes: According to the clustering result, creating at least one empty task sequence matching the clustering result; According to the clustering result, adding the HTML file to the at least one empty task sequence to obtain at least one task sequence; According to the HTML file in the at least one task sequence, obtaining a first feature of the HTML file in each task sequence; According to the first feature of the HTML file in each task sequence, a resource consumption tag is marked for each task sequence, and the resource consumption tag is used to represent the resources consumed in the process of converting the HTML file in the task sequence into PDF.

10. A device, characterized in that: The device comprises a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the method according to any one of claims 1 to 7.

Images