Large PDF (Portable Document Format) loading method and system based on image caching and storage medium thereof

Abstract

The invention belongs to the technical field of electric digital data processing, and particularly relates to a large-scale PDF loading method and system based on image caching and a storage medium thereof.The large-scale PDF loading system based on image caching can deconstruct a large-scale PDF document through a PDF analyzer, a PDF renderer and a PDF editor and render the content into a visual image file; the visual image file is compressed, then subjected to resource stream coding and stored in cache middleware, a resource stream code and equipment information are sent to a server through a client, and the server adjusts the resolution of the visual image file according to the equipment information of the client and outputs a resource stream file corresponding to the resource stream code. According to the method and the device, the page in the PDF file can be browsed by quickly turning the page through the resource flow file in the cache middleware, so that the time for repositioning and loading single-page resources of the large PDF file is saved, more accurate loading and rendering are realized, the time for PDF plug-in and large PDF cold reading is saved, and the global availability of PDF loading is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of electronic digital data processing, and in particular relates to a large-scale PDF loading method and system based on image cache and a storage medium thereof. Background Art

[0002] Common methods of loading PDF on mobile devices include full-text loading, page loading, and progressive loading. Full-text loading downloads the entire PDF file to the local computer, and then loads and browses it. Page loading divides the PDF file into multiple small files, which are then loaded and browsed in segments. Progressive loading first loads a low-resolution version or preview image of the PDF file, allowing users to quickly view and navigate the structure of the document. As the user browses, high-resolution page content is gradually loaded to provide clearer details and content display.

[0003] However, the existing technology has the following disadvantages: Full-text loading: The disadvantage is that it takes up a lot of bandwidth and storage space, which puts a burden on the user's network traffic and device storage. Paging loading: Each page needs to be reloaded. When the user turns the page quickly, it will cause freezes and unresponsiveness, and the user experience is not friendly enough. Progressive loading: Progressive loading is carried out step by step. The user may only see partial content or low-resolution preview images in the initial stage, which may easily lead to the user's inability to immediately obtain complete information and details, resulting in a poor user experience. The above loading method generally also needs to cooperate with third-party plug-ins, consuming some performance and traffic in third-party plug-in loading and PDF loading. In order to increase the loading speed of large PDF documents, a large PDF loading method that can avoid browsing freezes and loading failures is needed. Summary of the invention

[0004] In response to the above problems, the present invention proposes a large PDF loading method, system and storage medium based on image caching, which effectively solves the problem in the prior art that loading large PDF documents places a heavy burden on device storage, easily causing crashes, freezes and poor user experience.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows: a large PDF loading method based on image cache, comprising:

[0006] Step 1: Load and deconstruct large PDF files through the PDF parser combined with asynchronous non-blocking mechanism, and pre-process the deconstructed large PDF files into object models according to the standard format;

[0007] Step 2: Render the deconstructed large PDF document content into a visual image file through a PDF renderer;

[0008] Step 3: Perform pixel-adjustable lossless compression on the visualized image file in step 3 through a PDF compressor;

[0009] Step 4: Use the PDF operator to arrange the visual image file in step 2 into a resource stream file and perform resource stream encoding.

[0010] Step 5: Transfer the resource stream file after resource stream encoding to the cache middleware for cache processing;

[0011] Step 6: The client sends the resource stream code and device information in step 5 to the server;

[0012] Step 7: The server adjusts the resolution of the visualization image file according to the client device information and outputs a resource stream file corresponding to the resource stream encoding.

[0013] Furthermore, the preprocessing of deconstructing the large PDF file in step 1 includes but is not limited to text recognition, image optimization, and format standardization.

[0014] Furthermore, the asynchronous non-blocking mechanism includes but is not limited to creating a ThreadPoolTaskExecutor thread pool and configuring reasonable parameters such as the number of core threads, the maximum number of threads, and the buffer execution queue task capacity, and placing the time-consuming task of loading large PDF file resources into the thread pool with configured parameters.

[0015] Furthermore, the text recognition method includes but is not limited to using LSTM long short-term memory network, projection analysis and N-gram model.

[0016] Furthermore, the content rendering of the large PDF document in step 2 includes but is not limited to the Otsu method, the image binarization denoising method with adaptive threshold, the threshold offset adjustment method and the weighted processing method.

[0017] Furthermore, the encoding method of the resource stream file in step 4 includes but is not limited to counter generation and hash encoding.

[0018] Furthermore, in step 5, the resource stream files in the cache middleware will be cleaned up regularly, and the cleaning method includes but is not limited to setting expiration time and activity elimination.

[0019] Furthermore, the server can monitor the client's actions. When the PDF is scrolling, the client loads a default blank image. When the PDF stops scrolling, the current page is loaded. When the client stops at a certain page, the pages before and after the current page are loaded.

[0020] A large-scale PDF loading system based on image caching, characterized in that it includes a server and a client connected to the server, the server includes a remote processor and a PDF parser, a PDF renderer, a PDF compressor, a PDF operator, a cache middleware and a memory connected to the remote processor, the memory is used to store instructions for executing claims 1-8; the remote processor is used to run the instructions stored in the memory; the client includes a terminal processor and a display device connected to the terminal processor.

[0021] A storage medium storing a program for executing steps 1-8 described in claim 1, characterized in that it also includes an installation package and an upgrade program for installing the program.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] The present invention pre-processes large PDF documents through an asynchronous non-blocking mechanism, so that the server resource utilization rate is higher, the server's ability to handle multiple requests is greatly improved, the response speed is faster, and the customer waiting time is reduced;

[0024] At the same time, the present invention forms a resource stream file by deconstructing and rendering a large PDF file, so that the client can quickly call the resource stream file in the cache middleware, thereby realizing fast page turning to browse the precisely located pages in the large PDF file, eliminating the time for re-positioning and loading single-page resources of large PDF files, and realizing more accurate loading and rendering. It is suitable for situations where network bandwidth is limited or user device performance is limited, improves user experience and global availability of functions, eliminates the time for PDF plug-ins and large PDF cold reading, and improves the global availability of PDF loading. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of a large PDF loading method based on image caching of the present invention;

[0026] Figure 2 This is a structural block diagram of a large-scale PDF loading system based on image caching according to the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] like Figure 1-Figure 2As shown, a large PDF loading method based on image caching includes:

[0029] Step 1: Load and deconstruct large PDF files through the PDF parser combined with asynchronous non-blocking mechanism, and pre-process the deconstructed large PDF files into object models according to the standard format;

[0030] Step 2: Render the deconstructed large PDF document content into a visual image file through a PDF renderer;

[0031] Step 3: Perform pixel-adjustable lossless compression on the visualized image file in step 3 through a PDF compressor;

[0032] Step 4: Use the PDF operator to arrange the visual image file in step 2 into a resource stream file and perform resource stream encoding.

[0033] Step 5: Transfer the resource stream file after resource stream encoding to the cache middleware for cache processing;

[0034] Step 6: The client sends the resource stream code and device information in step 5 to the server;

[0035] Step 7: The server adjusts the resolution of the visualization image file according to the client device information and outputs a resource stream file corresponding to the resource stream encoding.

[0036] In this embodiment, the preprocessing of deconstructing the large PDF file in step 1 includes but is not limited to text recognition, image optimization, and format standardization. The image optimization includes locating the image object in the byte stream of the large PDF file and associating the overall structure in the large PDF file, identifying the type of the image object, extracting the original data of the image, and then parsing the size, color and resolution of the image. The format standardization can be performed using the PDF / A standard in the international standard. The format conversion can be set using the PdfAConformanceLevel of the open source technology component iText. The code is as follows: PdfAConformanceLevelconformanc eLevel=PdfAConformanceLevel.PDF_A_1B. If other formats are required, they can be set to other standards as required. This step is used to keep the preprocessed large PDF files consistent.

[0037] In this embodiment, the asynchronous non-blocking mechanism includes but is not limited to creating a ThreadPoolTaskExecutor thread pool, and configuring reasonable parameters such as the number of core threads, the maximum number of threads, and the buffer execution queue task capacity, and placing the time-consuming task of loading large PDF file resources into the thread pool with configured parameters. The number of core threads, the maximum number of threads, and the buffer execution queue task capacity are set according to the core and thread number of the processor. When the PDF parser loads and deconstructs large PDF files in combination with the asynchronous non-blocking mechanism, due to the existence of the thread pool, this process will not block other operations. For example, a thread is processing the deconstruction of a large PDF file. When it reads the content of the PDF file, recognizes text, optimizes images, and other operations, other threads can simultaneously process the loading tasks of other PDF files or execute other operations in the system. Even if the deconstruction process of a PDF file takes a long time because the file is too large or complex, it will not affect the response performance of the entire system. At the same time, by reasonably configuring the thread pool parameters, it can be ensured that when the system processes multiple PDF file loading tasks, it will not cause excessive consumption of system resources due to the creation of too many threads, nor will it cause too long response time due to task backlog.

[0038] In this embodiment, the text recognition method includes but is not limited to the use of LSTM long short-term memory network, projection analysis and N-gram model. The LSTM long short-term memory network and projection analysis are mainly used to classify the text in the document, such as distinguishing the table of contents, main text, references and other parts. The N-gram model is used to assist the LSTM long short-term memory network and projection analysis to correct the spelling and grammar of the text, thereby improving the accuracy of keyword matching.

[0039] In this embodiment, the content rendering of the large PDF document in step 2 includes but is not limited to the Otsu method, the image binarization denoising method with an adaptive threshold, the threshold offset adjustment method and the weighted processing method. The Otsu method includes a grayscale histogram of a statistical image, and then traverses all possible thresholds to calculate the inter-class variance under each threshold. The calculation formula of the inter-class variance is based on the pixel ratio of the foreground and the background and their respective average grayscale values. By finding the threshold that maximizes the inter-class variance, the pixels in the image are divided into two categories, the pixels greater than the threshold are set as the foreground, and the pixels less than the threshold are set as the background, thereby achieving binarization, dividing the image into small rectangular areas, and for each small area, calculating its own threshold. This threshold can be determined by statistical information such as the grayscale mean and standard deviation of the area. In this way, better binarization effects can be obtained in areas with different lighting conditions, reducing the noise caused by the overall threshold being not applicable to local areas. The threshold offset adjustment method is used to manually adjust the image effect after manual observation of binarization. The weighted processing method is used to assign different weights to different colors. By comparing the color weights of different types of areas, such as mixed areas of icons and text, the visual effects of different areas are improved, avoiding the impact of binarization on document rendering.

[0040] In this embodiment, the resource stream file encoding method in step 4 includes but is not limited to counter generation and hash coding. When the scene PDF data volume is large and the pressure of parsing resources needs to be reduced, hash coding is used. Hash coding includes hashing the resources, such as MD5 calculation, and taking the result as part of the resource. In different documents, the same resource stream file also has the same name. The stored hash code is managed through a dictionary, and the mapping relationship between the hash code and the resource stream file. Each time a new resource is added, first check whether the same hash code already exists in the dictionary. If it does, use the existing name, otherwise generate a new hash code and add it to the dictionary. Since the same image resource has the same name, when the system parses the PDF, for the same resource that has been processed, there is no need to perform complex parsing and processing again.

[0041] In this embodiment, the resource flow files in the cache middleware in step 5 are provided to subsequent clients in the form of KEY-VALUE for precise scheduling and use. At the same time, the resource flow files in the cache middleware in step 5 will be cleaned up regularly, and the cleaning method includes but is not limited to setting expiration time and activity elimination.

[0042] In this embodiment, the PDF compressor in step 3 adopts compression algorithms including but not limited to Flate and Run-Length encoding. In most scenarios of the present invention, the Flate lossless compression algorithm is used, which has the characteristics of lossless compression, high compression rate, and moderate computational complexity, and is more suitable for displaying high-quality image resources on low-end devices; Run-Length is mainly used in the case of simple image resources and repeated character texts.

[0043] In this embodiment, the server can monitor the client's actions. When the PDF is scrolling, the client loads a default blank image. When the PDF stops scrolling, it loads the current page. When the client stops on a certain page, it loads the pages before and after the current page. For example, in the case of web browsing, it can be monitored through JavaScript event monitoring, including scrolling event monitoring and stop scrolling event judgment. The time the current page of the PDF file stays on is judged by the timestamps of the two stop scrolling events. The server can collect relevant residence time information and preload individual pages with longer residence times to facilitate browsing of large PDF documents.

[0044] A large-scale PDF loading system based on image cache comprises a server and a client connected to the server, wherein the connection between the server and the client includes but is not limited to connection via a bus and connection via a mobile communication network, the server comprises a remote processor and a PDF parser, a PDF renderer, a PDF compressor, a PDF operator, a cache middleware and a memory connected to the remote processor, wherein the memory is used to store instructions for executing claims 1-8; the remote processor is used to run the instructions stored in the memory; the client comprises a terminal processor and a display device connected to the terminal processor.

[0045] A storage medium stores a program for executing steps 1-8 described in claim 1, and also includes an installation package and an upgrade program for installing the program.

[0046] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A large PDF loading method based on image caching, characterized in that: include: Step 1: Load and deconstruct large PDF files through the PDF parser combined with asynchronous non-blocking mechanism, and pre-process the deconstructed large PDF files into object models according to the standard format; Step 2: Render the deconstructed large PDF document content into a visual image file through a PDF renderer; Step 3: Perform pixel-adjustable lossless compression on the visualized image file in step 3 through a PDF compressor. Step 4: Use the PDF operator to arrange the visual image file in step 2 into a resource stream file and perform resource stream encoding. Step 5: Transfer the resource stream file after resource stream encoding to the cache middleware for cache processing; Step 6: The client sends the resource stream code and device information in step 5 to the server; Step 7: The server adjusts the resolution of the visualization image file according to the client device information and outputs a resource stream file corresponding to the resource stream encoding.

2. The large PDF loading method based on image caching according to claim 1 is characterized in that: The preprocessing of deconstructing the large PDF file in step 1 includes but is not limited to text recognition, image optimization, and format standardization.

3. The large PDF loading method based on image caching according to claim 1 is characterized in that: The asynchronous non-blocking mechanism includes but is not limited to creating a ThreadPoolTaskExecutor thread pool, and configuring reasonable parameters such as the number of core threads, the maximum number of threads, and the buffer execution queue task capacity, and placing the time-consuming task of loading large PDF file resources into the thread pool with configured parameters.

4. The large PDF loading method based on image caching according to claim 2 is characterized in that: The text recognition method includes but is not limited to using LSTM long short-term memory network, projection analysis and N-gram model.

5. The large PDF loading method based on image caching according to claim 1 is characterized in that: The content rendering of the large PDF document in step 2 includes but is not limited to the Otsu method, the image binarization denoising method with an adaptive threshold, the threshold offset adjustment method and the weighted processing method.

6. The large PDF loading method based on image caching according to claim 1 is characterized in that: The method of encoding the resource stream file in step 4 includes but is not limited to counter generation and hash encoding.

7. The large PDF loading method based on image caching according to claim 1 is characterized in that: In step 5, the resource flow files in the cache middleware will be cleaned up regularly, and the cleaning method is not limited to setting expiration time and activity elimination.

8. The large PDF loading method based on image caching according to claim 1 is characterized in that: Therefore, the server can monitor the client's actions. When the PDF is scrolling, the client loads the default blank image. When the PDF stops scrolling, it loads the current page. When the client stops on a certain page, it loads the pages before and after the current page.

9. A large-scale PDF loading system based on image caching, characterized in that: The invention comprises a server and a client connected to the server, wherein the server comprises a remote processor and a PDF parser, a PDF renderer, a PDF compressor, a PDF operator, a cache middleware and a memory connected to the remote processor, wherein the memory is used to store instructions for executing claims 1-8; the remote processor is used to run the instructions stored in the memory; and the client comprises a terminal processor and a display device connected to the terminal processor.

10. A storage medium storing a program for executing steps 1 to 8 of claim 1, characterized in that: Also included are installation packages and upgrade programs for installing the program.

Images