A large PDF file intelligent generation method with front-end and back-end cooperation

Abstract

The application discloses a front-rear end cooperative large PDF file intelligent generation method, relates to the technical field of computer software, and comprises the following steps: front-end rendering: the front end renders the work order data page by combining a headless browser with a Vue framework and an ElementUI component library, and dynamically displays structured work order details through a responsive data binding mechanism; data fragmentation: large work order data sets are split into N subtasks based on a dynamic fragmentation strategy; and small PDF files are generated in parallel: a containerized rendering cluster based on Kubernetes deployment management is used to allocate rendering tasks to each container; the small PDF files generated by the application through careful design with the help of the Vue and ElementUI frameworks are more attractive in terms of text layout, color matching and graphic display, and the large file after merging is more beautiful than the file generated by a traditional back end, thereby improving user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer software, in particular to a large PDF file intelligent generation method based on front-end and back-end cooperation. BACKGROUND

[0002] In the process of digital business operation, the ticket detail data presents an explosive growth. For example, in the scenarios of financial transaction settlement, e-commerce order management, online education homework statistics, etc., the number of tickets involved in a single business often reaches hundreds of thousands, and the data size is increasing exponentially. Especially in the settlement stage, the questionnaire task contains a large amount of structured and unstructured data, with an average of more than 2000 words per questionnaire. The three core demands of data integrity, generation efficiency and visual performance of file generation technology need to achieve 100% coverage of millions of records, minute-level response and compliance with enterprise brand specifications. Therefore, the PDF file generation technology needs to be continuously innovated to adapt to the business needs of high concurrency, large volume and strong experience.

[0003] The traditional large PDF file generation method mainly includes two modes of front-end and back-end separate work, but both modes have obvious limitations and essential defects:

[0004] For example, the front-end generation scheme is based on the browser environment to directly generate PDF, which has technical bottlenecks such as memory limitation, CPU overload and size limit. For example, when generating PDF in the front-end, it is severely limited by browser resources. The running environment resources of the browser are limited, and when processing the massive text data contained in the questionnaire, the page rendering and PDF generation process will quickly consume the memory of the browser, causing memory overflow error. At the same time, the complex PDF generation algorithm will cause the CPU to be in a high load running state, causing the browser to respond slowly or even crash, and finally unable to complete the generation task of large files. The size of the generated file is limited by the maximum number of threads of WebWorkers, and it is unable to handle super large tasks in parallel.

[0005] The back-end generation scheme uses JavaText, ApachePDFBox and other back-end libraries, which can handle GB-level data, but has structural shortcomings such as lack of style rendering capability, weak support for visual elements and lack of dynamic components. For example, although the back-end can handle large-scale data, the focus of back-end development is mainly on data storage and business logic processing, lacking professional front-end visual design tools and technologies. Back-end developers are difficult to use rich CSS styles and flexible layout techniques to optimize the layout of text as front-end personnel do, and cannot easily realize color matching and graphic drawing, resulting in a single style of the generated PDF file, harsh text layout, and monotonous color matching, which greatly affects the readability and aesthetics of the file, and it is difficult to meet the user's expectations for file aesthetics.

[0006] The prior art has the following disadvantages: the selection of the front-end scheme sacrifices the file size and stability, and the selection of the back-end scheme gives up the visual performance and user experience, so there is an urgent need for a large PDF file intelligent generation method that takes into account the efficiency, size and aesthetics of the front-end and back-end cooperation.

[0007] The above information disclosed in the background section is only used to strengthen the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0008] The purpose of the present application is to provide a front-end and back-end cooperative large PDF file intelligent generation method, which is carefully designed by the front-end with Vue and ElementUI framework, and the evidence chain generation service is decoupled from the business system by RocketMQ, to solve the problems in the background technology.

[0009] In order to achieve the above purpose, the present application provides the following technical scheme: a front-end and back-end cooperative large PDF file intelligent generation method, comprising the following steps:

[0010] S1, front-end rendering: the front-end renders the work order data page by combining the Vue framework and the ElementUI component library with the headless browser, and dynamically displays the structured work order details through the responsive data binding mechanism;

[0011] S2, data sharding: based on a dynamic sharding strategy, large work order data sets are divided into N sub-tasks;

[0012] S3, parallel generation of small PDF files: based on the Kubernetes deployment management containerized rendering cluster, each container is assigned a rendering task, and the number of threads in each container is configured, and the headless browser is called to generate a sharded PDF file;

[0013] S4, asynchronous data transmission: the sharded PDF file and the work order unit data are encapsulated as a message body, and the back-end receives the message body asynchronously through the RocketMQ message queue and transmits it to the back-end service asynchronously;

[0014] S5, multi-threaded file merging: using multi-threaded file stream technology to merge PDF files in work order logical order, back-end merging thread pool configuration, that is, the back-end service sorts the sharded PDF files according to work order number and timestamp, and selects a merging strategy according to the sharding size;

[0015] S6, two-stage merging of parallel / merging: based on ForkJoinPool, a two-stage file merging strategy of four-way parallel and binary tree recursive merging is implemented, and the final PDF is output;

[0016] S7, dynamic memory management: set the dynamic memory management module, trigger garbage collection GC when the system free memory is less than 20%, and reduce the merging parallelism to half of the original thread number synchronously;

[0017] S8, fault isolation: implement isolation mechanism for single shard fault, mark failed shard and rejoin task queue, continue to process the remaining shard merging operation, and limit the upper limit of fault shard retry times to 3 times;

[0018] S9, large-scale file processing: when generating a PDF file larger than 500MB, the front-end rendering cluster and the back-end merging service are physically separated, the intermediate file is stored using SSD disk, and CRC32 check mechanism is used to ensure file integrity.

[0019] Preferably, the headless browser uses Puppeteer, and the rendering steps of the work order data page are as follows:

[0020] Inject Echarts script chart library to generate data visualization SVG graphics;

[0021] Set PDF parameters through Puppeteer API, including margin ≤10mm, header / footer height ≤15mm;

[0022] Execute JavaScript code to preprocess DOM elements.

[0023] Preferably, the segmentation step of the subtask in the dynamic shard strategy is as follows:

[0024] According to the total number of large work order data sets, the total number of work orders is obtained, and the total number of work orders is segmented by using a shard algorithm to obtain N subtasks, wherein the calculation formula of the shard algorithm is , and , wherein represents the number of subtasks, represents the shard function of rounding up, represents the total number of work orders to be processed, represents the number of work orders that can be dynamically processed by each partition.

[0025] Preferably, the extension strategy of the containerized rendering cluster is as follows:

[0026] Real-time monitoring of request rate;

[0027] When the request rate is >1000 times / s, a weighted round robin load balancing algorithm is used, and the calculation formula of the weight is , wherein represents the weight of the i th node, represents the weight of the i th node, a weight coefficient of a percentage of available CPU resources of the i-th node, a percentage of available CPU resources of the i-th node, a percentage of available CPU resources of the i-th node, a percentage of available CPU resources of the i-th node, a weight coefficient of a percentage of available memory resources of the i-th node, a percentage of available memory resources of the i-th node, a percentage of available memory resources of the i-th node;

[0028] when the request rate ≤ 1000 times / s, the minimum connection number load balancing algorithm is adopted, and the expression of the minimum connection number algorithm is , wherein, is a minimum value function, is the current active connection number of the node; is a list containing all available nodes.

[0029] Preferably, the configuration step of the number of threads in each container is as follows:

[0030] The number of CPU cores of the computer device performing the task is counted, and the maximum available thread number in the system is obtained;

[0031] The number of threads in each container is calculated using the thread number configuration algorithm, wherein the calculation formula of the number of threads in each container is , wherein, is the configured number of threads in each container, is a minimum value function, is the number of CPU cores, is the maximum available thread number in the system.

[0032] Preferably, the fragmentation merging strategy of the multi-thread file is as follows:

[0033] Memory flow direct splicing is used for <100MB fragments;

[0034] Disk file merging is implemented for ≥100MB fragments.

[0035] Preferably, the dynamic management step of the backend merging thread pool is as follows:

[0036] The core thread number is initialized, wherein the expression of the core thread number is , wherein, is the core thread number of the thread pool, the threads in the thread pool are created and reserved in the thread pool at the initialization of the thread pool, and the core thread number is dynamically calculated according to the CPU core number of the system and is used to process the tasks in the task queue, is expressed as a method call in Java code, which returns the number of processor cores that the current Java virtual machine can use;

[0037] The maximum number of threads is set, and the expression of the maximum number of threads is , wherein is expressed as the maximum number of threads allowed to be created by the thread pool;

[0038] An exponential backoff strategy is adopted for the task retry mechanism, and when the task fails, it is retried according to the exponential backoff, wherein the delay time of the nth task retry is , and , wherein is expressed as the delay time of the nth task retry, is expressed as the base delay time being two hundred milliseconds, is expressed as the number of times of task retries.

[0039] Preferably, the execution steps of the two-stage file merging strategy are as follows:

[0040] Primary stage: Start four parallel threads to perform primary merging to generate intermediate files to , wherein the calculation formula of the number of intermediate files is , wherein is expressed as the first intermediate file, is expressed as the nth intermediate file, is expressed as the number of intermediate files generated in the primary merging stage;

[0041] Final stage: The intermediate files are merged into the final PDF file according to the number of recursive calls using the binary tree recursive merging algorithm, wherein the number of recursive calls is expressed as the recursion depth, and the calculation formula of the recursion depth is , wherein is expressed as the recursion depth of the binary tree recursive merging algorithm, is expressed as the logarithm with base 4 .

[0042] Preferably, the steps of processing the shard failure are as follows:

[0043] Set a state flag bit for each shard, including a to-be-processed shard flag of 0, a processing shard flag of 1, a successful shard flag of 2, and a failed shard flag of 3;

[0044] When the flag bit is 3, the shard fails, and the shard is re-added to the tail of the queue;

[0045] Skip the failed shard and continue merging other shards;

[0046] When the cumulative failure number is greater than 3, the shard is abandoned and an alarm is triggered.

[0047] A computer device comprises a memory, a processor and a readable storage medium having a computer program stored thereon, the computer program being executed by the processor to implement the steps of the large PDF file intelligent generation method of the front-end and back-end cooperation.

[0048] In the above technical solution, the technical effects and advantages provided by the present application are as follows:

[0049] The small PDF files generated by the present application through careful design by the front end with the Vue and ElementUI frameworks are more attractive in terms of text layout, color matching and graphic display, and the large file after merging is much more beautiful than the file generated by the traditional back end, thus improving the user experience; the decoupling of the evidence chain generation service and the business system by RocketMQ avoids the occupation of the hardware resources of the business system by the evidence chain generation task, ensures the stable and reliable operation of the business system, reduces the memory leakage rate and improves the task success rate when processing large-scale documents, and supports single shard fault isolation, further enhancing the system stability and significantly improving the task efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only represent some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0051] Figure 1 The flowchart of the large PDF file intelligent generation method of the front-end and back-end cooperation of the present application.

[0052] Figure 2 The system architecture diagram of the computer device of the present application.

[0053] Figure 3 The flowchart of the shard merging of the present application.

[0054] Figure 4 The data interaction flowchart of the large PDF file intelligent generation architecture of the front-end and back-end cooperation of the present application. DETAILED DESCRIPTION

[0055] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these example implementations are provided so that this disclosure will be thorough and complete, and will fully convey the inventive gist to those skilled in the art.

[0056] Embodiments

[0057] The present application provides a large PDF file intelligent generation method as shown in the front and rear end cooperation, comprising the following steps: Figures 1-4

[0058] S1, front-end rendering: the front-end renders the work order data page through a headless browser combined with a Vue framework and an ElementUI component library, and dynamically displays the structured work order details through a responsive data binding mechanism;

[0059] Specifically, the headless browser uses Puppeteer, and the rendering steps of the work order data page are as follows:

[0060] Injecting Echarts script chart library to generate data visualization SVG graphics;

[0061] Setting PDF parameters through Puppeteer API, including margin ≤10mm, header / footer height ≤15mm;

[0062] Executing JavaScript code to preprocess DOM elements to ensure that dynamic components are correctly rendered.

[0063] S2, data slicing: based on a dynamic slicing strategy, a large work order data set is divided into N subtasks;

[0064] Specifically, the segmentation steps of the subtasks in the dynamic slicing strategy are as follows:

[0065] According to the total number of work orders obtained from the large work order data set, the total number of work orders is segmented using a slicing algorithm to obtain N subtasks, wherein the calculation formula of the slicing algorithm is , and , wherein represents the number of subtasks, represents the upward rounding slicing function, represents the total number of work orders to be processed, represents the number of work orders that can be dynamically processed by each partition, and the number of work orders is determined by experimentally testing the work order processing time in different business scenarios The optimal value is 20.

[0066] S3, parallel generation of small PDF files: based on a Kubernetes deployment management containerized rendering cluster, each container is assigned a rendering task, and the number of threads in each container is configured, and a headless browser is called to generate sliced PDF files;

[0067] Specifically, the expansion strategy of the containerized rendering cluster is as follows: ​

[0068] monitoring the request rate in real time;

[0069] When the request rate > 1000 times / s, a weighted round-robin load balancing algorithm is adopted, and the calculation formula of the weight is represents the weight of the i-th node, and the weight is used to determine the priority of each node in receiving requests, and the higher the weight of the node, the greater the probability of being allocated to the request, represents the weight coefficient of the percentage of available CPU resources of the i-th node, represents the percentage of available CPU resources of the i-th node, reflecting the degree of freedom of the node CPU, and the higher the value, the more idle the CPU, which can handle more requests, represents the weight coefficient of the percentage of available memory resources of the i-th node, represents the percentage of available memory resources of the i-th node, reflecting the idle condition of the node memory, and the higher the value, the more sufficient the memory for processing new requests;

[0070] When the request rate ≤ 1000 times / s, a minimum connection number load balancing algorithm is adopted, and the expression of the minimum connection number algorithm is

[0071] Specifically, the configuration steps of the number of threads in each container are as follows:

[0072] The number of CPU cores of the computer device executing the task is counted, and the maximum available number of threads in the system is obtained;

[0073] The number of threads in each container is calculated using a thread number configuration algorithm, wherein the calculation formula of the number of threads in each container is

[0074] ​​​​​​​​​​​​​​S4, data asynchronous transmission: encapsulate the fragmented PDF file and the work unit data as a message body, and the backend receives the message body through the RocketMQ message queue and transmits it to the backend service asynchronously;

[0075] Specifically, the business decoupling process steps are as follows:

[0076] The business system writes the new work order data into the business database and the RocketMQ message queue at the same time;

[0077] The evidence chain generation service listens to the message queue and independently executes the PDF generation from the business system;

[0078] The business system only responds to the generation completion state notification.

[0079] S5, multi-thread file merging: use multi-thread file stream technology to merge PDF files according to the work order logical sequence, and the backend merging thread pool configuration, that is, the backend service sorts the fragmented PDF files according to the work order number and timestamp, and selects the merging strategy according to the size of the fragments;

[0080] Specifically, the multi-thread file merging strategy is as follows:

[0081] For <100MB fragments, use memory stream direct splicing to improve the merging speed of small files;

[0082] For ≥100MB fragments, implement disk file merging to avoid memory pressure during large file merging;

[0083] Specifically, the dynamic management steps of the backend merging thread pool are as follows:

[0084] Initialize the core thread number, wherein the expression of the core thread number is , wherein represents the core thread number of the thread pool, and the thread pool creates and retains the thread in the thread pool during initialization, which is dynamically calculated according to the CPU core number of the system, and is used to process the tasks in the task queue, represents a method call in Java code, which returns the number of processor cores that the current Java virtual machine can use;

[0085] Set the maximum thread number, wherein the expression of the maximum thread number is , wherein represents the maximum number of threads that the thread pool allows to create;

[0086] Use an exponential backoff strategy as a task retry mechanism, and retry when the task fails, wherein the nth task retry delay time is , and , wherein Represented as the first Delay time when retrying the next task This indicates a base delay of 200 milliseconds. This represents the number of task retries. The configuration of the backend merge thread pool and the task retry mechanism ensure the efficiency and reliability of file merging.

[0087] S6, Parallel / Merge Two-Phase Merging: Implement a two-phase file merging strategy based on ForkJoinPool, combining four-way parallelism and binary tree recursive merging, and output the final PDF;

[0088] Specifically, the execution steps of the two-stage file merging strategy are as follows:

[0089] Initial stage: Start four parallel threads to perform initial merging and generate intermediate files. to This is used to fully utilize the computing resources of multi-core CPUs and improve merging efficiency. The formula for calculating the number of intermediate files is: In the formula, This is indicated as the first intermediate file. Represented as the first An intermediate file This represents the number of intermediate files generated during the initial merging phase.

[0090] Final stage: A binary tree recursive merge algorithm is used to merge intermediate files into the final PDF file based on the number of recursive calls. The number of recursive calls is represented by the recursion depth, which is calculated using the following formula: In the formula, This represents the recursion depth of the binary tree recursive merge algorithm. Represented as base 4 The logarithm of .

[0091] S7. Dynamic Memory Management: Set up a dynamic memory management module to trigger garbage collection (GC) when the system's free memory is less than 20%, and simultaneously reduce the parallelism of the merged threads to half of the original number.

[0092] Specifically, the dynamic memory management steps are as follows:

[0093] Real-time monitoring of JVM heap memory usage, where JVM refers to the Java Virtual Machine, and JVM heap memory is the largest area of ​​memory managed by the Java Virtual Machine. It is a runtime memory area shared by all threads and is created when the virtual machine starts.

[0094] When memory utilization exceeds 80%, garbage collection (GC) is triggered to release memory, and the number of merged threads is reduced to 50% of the current value.

[0095] When the memory usage is less than 50%, the original thread number is restored.

[0096] S8, fault isolation: implement isolation mechanism for single shard fault, mark failed shard and rejoin task queue, continue to process the remaining shard merge operation, and limit the upper limit of fault shard retry times to 3 times;

[0097] Specifically, the shard fault processing steps are as follows:

[0098] Set a state flag bit for each shard, including 0 for a to-be-processed shard, 1 for a processing shard, 2 for a successful shard, and 3 for a failed shard;

[0099] When the flag bit is 3, the shard fails, and the shard is re-added to the tail of the queue;

[0100] Skip the fault shard and continue merging other shards;

[0101] When the cumulative failure times are greater than 3, give up the shard and trigger an alarm.

[0102] S9, large-scale file processing: when generating a PDF file larger than 500MB, the front-end rendering cluster and the back-end merge service are physically separated, the intermediate file is stored in an SSD disk, and the CRC32 check mechanism is used to ensure file integrity.

[0103] Specifically, the large-scale file processing steps are as follows:

[0104] Enable enhanced mode to deploy the front-end rendering cluster and the back-end merge service on different physical hosts;

[0105] Shard files are stored in an SSD disk array with a read-write bandwidth of 1GB / s;

[0106] After merging, perform CRC32 check mechanism to ensure file integrity, wherein the calculation formula of CRC32 check is , wherein represents the CRC32 check value, represents the final PDF file, represents the cyclic redundancy check algorithm.

[0107] It is supplemented that the Vue framework is a progressive JavaScript framework for building user interfaces, which can be integrated with other libraries or existing projects through bottom-up incremental development;

[0108] The ElementUI component library is a desktop component library based on Vue2.0, which can provide UI components such as buttons, input boxes, tables, and pop-up windows, to facilitate the rapid construction of beautiful and easy-to-use Web interfaces, reducing development cost and time;

[0109] RocketMQ is a distributed message middleware open-sourced by Alibaba, with high performance, high reliability, high scalability, etc. It can be used in asynchronous decoupling, traffic peak shaving, message communication, etc. It supports synchronous message, asynchronous message, sequential message and other message modes;

[0110] Puppeteer is a Node.js library that provides a high-level API to control Chrome or Chromium browsers. Through Puppeteer, you can achieve web screenshot, PDF generation, user simulation, crawler and other automated operations;

[0111] API is a set of rules and protocols that define the interaction between software components, i.e. Application Programming Interface;

[0112] PDF is the abbreviation of Portable Document Format, which is used to present and exchange documents. It can preserve the original format and layout of the document, and the appearance of the document will remain consistent on any operating system and device. PDF files can contain text, images, tables, charts and other elements;

[0113] GC is an automatic memory management mechanism, i.e. garbage collection, which is used to detect and recycle the memory occupied by objects that are no longer in use, thereby avoiding memory leaks and improving memory utilization;

[0114] SVG is an XML-based image format, i.e. Scalable Vector Graphics, which is used to create two-dimensional graphics and animations on web pages;

[0115] Echarts is a JavaScript-based visualization chart library open-sourced by Baidu, which is used to create interactive statistical charts such as line charts, column charts and pie charts;

[0116] JavaScript is a scripting language widely used in web development, which is used to add interactivity and dynamic functionality to web pages;

[0117] DOM is the programming interface of HTML and XML documents, which represents the document as a tree structure, and each node is an object, i.e. Document Object Model;

[0118] Kubernetes is an open-source system for automating deployment, scaling and management of containerized applications;

[0119] ForkJoinPool is a thread pool in Java, which is used to execute tasks in the Fork / Join framework, and the Fork / Join framework is a mechanism for parallel execution of tasks, which splits large tasks into small tasks fork, and then combines the results of small tasks join;

[0120] The SSD is a flash-based storage device, i.e., a solid state disk.

[0121] The CRC32 is a cyclic redundancy check algorithm using a 32-bit check value.

[0122] The above formulas are all dimensionless values calculated, and the formulas are obtained by collecting a large amount of data to simulate a formula of the most recent real situation, and the preset parameters in the formula are set by the person skilled in the art according to the actual situation.

[0123] The above embodiments can be realized wholly or partially by software, hardware, firmware or any other combination. When realized by software, the above embodiments can be realized wholly or partially in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center and the like containing one or more available medium sets. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD) or a semiconductor medium. The semiconductor medium can be a solid state disk.

[0124] It should be understood that in various embodiments of the present application, the size of the sequence number of each process described above does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0125] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0126] The above description is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for intelligently generating large PDF files through front-end and back-end collaboration, characterized in that: Includes the following steps: S1. Front-end rendering: The front-end renders the work order data page through a headless browser combined with the Vue framework and ElementUI component library, and dynamically displays the structured work order details through a responsive data binding mechanism. S2. Data Sharding: Based on a dynamic sharding strategy, a large work order dataset is divided into N sub-tasks; S3, Parallel generation of small PDF files: Based on a containerized rendering cluster deployed and managed by Kubernetes, it assigns rendering tasks to each container, configures the number of threads in each container, and calls a headless browser to generate fragmented PDF files; S4. Asynchronous Data Transmission: The fragmented PDF file and the work unit data are encapsulated into a message body, and the backend asynchronously receives the message body through the RocketMQ message queue and asynchronously transmits it to the backend service. S5. Multi-threaded file merging: Employs multi-threaded file stream technology to merge PDF files according to the logical order of the work order. The backend merge thread pool is configured so that the backend service sorts the fragmented PDF files by work order number and timestamp, and selects the merging strategy based on the fragment size. S6, Parallel / Merge Two-Phase Merging: Implement a two-phase file merging strategy based on ForkJoinPool, combining four-way parallelism and binary tree recursive merging, and output the final PDF; S7. Dynamic Memory Management: Set up a dynamic memory management module to trigger garbage collection (GC) when the system's free memory is less than 20%, and simultaneously reduce the parallelism of the merged threads to half of the original number. S8. Fault Isolation: Implement an isolation mechanism for single-shard failures, mark the failed shards and re-add them to the task queue, continue processing the remaining shard merging operations, and limit the maximum number of retries for the failed shards to 3. S9. Large-scale file processing: When generating PDF files larger than 500MB, the front-end rendering cluster and the back-end merging service are physically separated, intermediate files are stored on SSD disks, and the CRC32 check mechanism is used to ensure file integrity.

2. The intelligent generation method for large PDF files with front-end and back-end collaboration according to claim 1, characterized in that, The headless browser uses Puppeteer, and the rendering steps on the work order data page are as follows: Inject Echarts script chart library to generate data visualization SVG graphics; Set PDF parameters via PuppeteerAPI, including margins ≤ 10mm and header / footer height ≤ 15mm; Execute JavaScript code to preprocess DOM elements.

3. The intelligent generation method for large PDF files with front-end and back-end collaboration according to claim 2, characterized in that, The subtask segmentation steps in the dynamic segmentation strategy are as follows: The total number of work orders is obtained from a large work order dataset. A sharding algorithm is then used to divide this total into N subtasks. The calculation formula for the sharding algorithm is as follows: ,and In the formula, This represents the number of subtasks. This is represented as a slice function that rounds up. This represents the total number of work orders that need to be processed. This represents the number of work orders that can be dynamically processed for each partition.

4. The intelligent generation method for large PDF files with front-end and back-end collaboration according to claim 3, characterized in that, The scaling strategy for the containerized rendering cluster is as follows: Real-time monitoring of request rate; When the request rate exceeds 1000 requests / second, a weighted round-robin load balancing algorithm is used, and the weight is calculated using the following formula: In the formula, Let represent the weight of the i-th node. Represented as the first The weighting factor of the percentage of available CPU resources of each node. Represented as the first The percentage of available CPU resources for each node. Represented as the first The weighting factor for the percentage of available memory resources of each node. Represented as the first The percentage of available memory resources for each node; When the request rate is ≤1000 requests / second, the least connections load balancing algorithm is used, and the expression for the least connections algorithm is: In the formula, Represented as a function that takes the minimum value, This represents the current number of active connections to the node. It is represented as a list containing all available nodes.

5. The intelligent generation method for large PDF files with front-end and back-end collaboration according to claim 4, characterized in that, The steps for configuring the number of threads within each container are as follows: Count the number of CPU cores in the computer device executing the task, and simultaneously obtain the maximum number of available threads in the system; The number of threads in each container is calculated using a thread count configuration algorithm, where the formula for calculating the number of threads in each container is as follows: In the formula, This represents the number of threads configured per container. Represented as a function that takes the minimum value, This is expressed as the number of CPU cores. This represents the maximum number of available threads preset in the system.

6. The intelligent generation method for large PDF files with front-end and back-end collaboration according to claim 5, characterized in that, The file sharding and merging strategy for multi-threaded files is as follows: For fragments <100MB, use memory streams to directly concatenate them; Perform disk file merging on fragments ≥100MB.

7. The intelligent generation method for large PDF files with front-end and back-end collaboration according to claim 6, characterized in that, The dynamic management steps of the backend merged thread pool are as follows: Initialize the core thread count, where the expression for the core thread count is: In the formula, This represents the core thread count of the thread pool. The threads created and retained in the thread pool during initialization are dynamically calculated based on the number of CPU cores in the system and are used to process tasks in the task queue. Represented as a method call in Java code, it returns the number of processor cores currently available to the Java Virtual Machine; Set the maximum number of threads, where the expression for the maximum number of threads is: In the formula, This represents the maximum number of threads that the thread pool is allowed to create. An exponential backoff strategy is adopted as the task retry mechanism, and when a task fails, it is retried exponentially. The delay time for the nth retry is... ,and In the formula, Represented as the first Delay time when retrying the next task This indicates a base delay of 200 milliseconds. This represents the number of times the task was retried.

8. The intelligent generation method for large PDF files with front-end and back-end collaboration according to claim 7, characterized in that, The execution steps of the two-stage file merging strategy are as follows: Initial stage: Start four parallel threads to perform initial merging and generate intermediate files. to The formula for calculating the number of intermediate files is as follows: In the formula, This is indicated as the first intermediate file. Represented as the first An intermediate file This represents the number of intermediate files generated during the initial merging phase. Final stage: A binary tree recursive merge algorithm is used to merge intermediate files into the final PDF file based on the number of recursive calls. The number of recursive calls is represented by the recursion depth, which is calculated using the following formula: In the formula, This represents the recursion depth of the binary tree recursive merge algorithm. Represented as base 4 The logarithm of .

9. The intelligent generation method for large PDF files with front-end and back-end collaboration according to claim 8, characterized in that, The fragmentation fault handling steps are as follows: Set a status flag for each fragment, including 0 for fragments to be processed, 1 for fragments in process, 2 for successful fragments, and 3 for failed fragments; When the flag is 3, the sharding fails and the shard is added back to the tail of the queue. Skip the faulty fragment and continue merging other fragments; If the cumulative number of failures exceeds 3, the fragment is abandoned and an alarm is triggered.

10. A computer device, comprising: A memory, a processor, and a readable storage medium having a computer program stored thereon, characterized in that: when the computer program is executed by the processor, it implements the steps of the intelligent generation method for large PDF files with front-end and back-end collaboration as described in any one of claims 1 to 9.

Images