A method for extracting text content of Chinese PDF files for network streaming

Abstract

The invention belongs to the technical field of network information processing, in particular to a method for extracting text content of a Chinese PDF file oriented to network stream transmission. The invention includes: step A: providing an interface to the application layer logic; step B: extracting the text content of the Chinese PDF document from the network data analyzed by the application layer logic. The present invention analyzes and optimizes each step of extracting the content of the Chinese PDF file transmitted by the network data stream, and achieves the greatest possible optimization of the entire system in terms of time efficiency and space efficiency, and at the same time, avoids the generation of such harmful phenomena as time delay jitter, This enables the method to operate in a large-flow supervision system without affecting the overall operating efficiency of the system.

Description

technical field [0001] The invention belongs to the technical field of network information processing, in particular to a method for extracting text content of a Chinese PDF file oriented to network stream transmission. Background technique [0002] Because the PDF file format has the advantages of platform independence, complete standards, and high-quality output format, PDF files have become the main form of information sharing and transmission on the Internet, but there will also be content in PDF files that is not conducive to the healthy development of the Internet. The need to inspect the content of PDF files in network data streams. There have been many researches on PDF document text content extraction at home and abroad. In the research on the generation of PDF documents and their content extraction, Liu Ping and others introduced in detail the generation standards of PDF documents in ScienceWord software, and the extraction of text content after reverse analysis o...

AI Technical Summary

Problems solved by technology

Liu Lirong’s research on key technologies for formatted file content extraction and filtering mentioned key technologies such as formatted file content extraction and filtering in network traffic. It is a big deficiency, and it cannot solve the real-time supervision of the Chinese PDF text content transmitted by the network data stream

[0003] According to the above description, when the current method is aimed at extracting the text content of the PDF file transmitted by the network data stream, it lacks support for real-time or Chinese text content extraction, and the text content extraction of the Chinese PDF file transmitted by the network data stream is harmful to the system and the entire system. The processing flow has extremely high requirements, and its requirements are mainly reflected in:

[0004] ① A large amount of PDF file text content will be transmitted concurrently in the real-time network data stream, so the extra overhead generated for the content extraction of PDF files in the data stream will seriously affect the efficiency of real-time text content extraction

[0006] ③The encoding of Chinese PDF files is relatively complicated. Each PDF document has its own specific Chinese CID encoding, and the mapping relationship between CID encoding and Unicode encoding is not fixed in the storage location in the file, so a large number of CIDs that appear before the mapping file need to be cached Encoding and caching the mapping relationship will bring a large memory overhead

Images