Standardized processing method and system for multi-format files

Through the standardized processing method and system of multi-format files, the problem of insufficient processing capabilities of various file formats in the prior art is solved, and standardized processing of file formats and efficient integration and exchange of data are realized.

CN119025480BActive Publication Date: 2025-02-11AACAT TECHNOLOGY LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411514646.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-10-29
Publication Date
2025-02-11
Estimated Expiration
2044-10-29

AI Technical Summary

Technical Problem

The prior art has problems such as insufficient automated identification and conversion capabilities, format compatibility issues, difficult data integration and summary, and inefficient processing efficiency when processing multiple file formats.

Method used

Provide a standardized processing method and system for multi-format files. By obtaining file information, classifying and circulating, determining whether replacement processing is needed, and uniformly converting and saving, it supports multiple file formats such as pdf, word, excel, json and html. Automatic identification and processing.

Benefits of technology

It realizes standardized processing between different file formats, reduces data errors, improves processing accuracy and consistency, reduces the difficulty of data integration and summary, and improves the efficiency of cross-platform data exchange and processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025480B_ABST
    Figure CN119025480B_ABST
Patent Text Reader

Abstract

The present invention provides a standardized processing method and system for multi-format files, including: step S1: obtaining file information; classifying and circulating the files through the file information to obtain classified attachments; step S2: judging whether the classified attachments need to be replaced based on the type of the classified attachments; if the result is yes, replacing the file with an html file, formatting the html file, and saving it to a local file; if the result is no, converting the file into a file stream, performing a unified conversion, and saving it to a local file. The present invention is compatible with different data formats through a standardized processing method, reduces the influence of file data errors, improves correctness, and ensures consistency between different files, so that the files are easier to understand and analyze.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer information processing, and in particular, relates to a standardized processing method and system for multi-format files. Background Art

[0002] For the automated processing and conversion of different file formats, such as PDF, Word, Excel, HTML, and JSON, office file format conversion includes the following key steps: First, specialized software or algorithms are needed to parse the original file and understand its structure and content. This includes identifying elements such as text, images, tables, and their attributes in the file, such as fonts, sizes, colors, etc. Next, the content is extracted from the original file. This may include text, images, and other non-text elements. For text, it needs to be extracted from the encoding and may require character recognition, i.e., OCR, to be converted into an editable text format. For images, it may need to be converted into editable graphic objects or OCR processed to identify text in the image. The extracted content is then reconstructed into the target format. This may involve rearranging elements as required by the target format, applying appropriate formatting, and ensuring that the converted file retains the same amount of information and accuracy as the original file. Finally, the converted file needs to be optimized to ensure file quality and compatibility. This may include removing redundant information, adjusting the layout, and performing necessary verification to ensure that the converted file can be correctly displayed and used on different software or platforms.

[0003] At present, traditional file format conversion technology often relies on specific applications or manual operations, and has the following technical defects: first, it relies on manual operations or multiple software applications, which is time-consuming and prone to errors; second, it lacks recognition and processing capabilities and cannot automatically adapt to updates and changes in different formats; finally, there are format compatibility issues, which may cause the converted files to lose format or data, affecting the user experience and data integrity.

[0004] In short, different file formats may lead to differences in data representation and structure, which may introduce data entry errors or system deviations, resulting in low accuracy, readability and consistency of file system data.

[0005] Solve the problem of difficulty in data integration and aggregation: In the process of data analysis and storage, there will always be data from different sources and in different formats. When analyzing related file data, the difference in file formats weakens the file relevance, and increases the resistance in the reading, processing, and storage process. There is a problem of difficulty in data integration and aggregation.

[0006] When file data of different formats are transmitted and exchanged between different systems, due to the diversity and non-universality of the formats, errors, errors and incompatibility occur in the processing process, which greatly reduces the processing efficiency. There is a problem of low data processing efficiency.

[0007] Patent document CN111105481A provides a method based on automatically reading, identifying, and formatting json files and reorganizing, identifying, and saving them. This solution has the following major defects: First, the prior patent can only read and identify a single json format file, and cannot identify and process data files in other formats, such as html, pdf, excel, etc. This limits its application scenarios and makes it inflexible in dealing with the needs of multiple data formats. Secondly, the solution does not provide clear specifications and standards for the output format, which may lead to inconsistencies in different output results and affect the integration and use efficiency of the data. The lack of a fixed output format may lead to compatibility issues with other systems or applications, thereby affecting the subsequent processing and utilization of the data. Summary of the invention

[0008] In view of the defects in the prior art, an object of the present invention is to provide a standardized processing method and system for multi-format files.

[0009] A standardized processing method for multi-format files provided by the present invention includes:

[0010] Step S1: Obtain file information; classify and circulate the files according to the file information to obtain classified attachments;

[0011] Step S2: Based on the type of the classified attachment, determine whether the classified attachment needs to be replaced; if the result is yes, replace the file with an html file, format the html file, and save it to a local file; if the result is no, convert the file into a file stream, perform a unified conversion, and save it to a local file.

[0012] Preferably, in the step S1, the file format is obtained through the file information, and then the files are classified and circulated according to the file format to obtain the classified attachments;

[0013] The file formats include: pdf, word, excel, json and html;

[0014] The classified attachments include: pdf files, word files, excel files, json files and html files;

[0015] The file information includes: file extension, file header and file path.

[0016] Preferably, in step S2, the basis for determining whether the classified attachments need to be replaced is that the pdf file, word file and excel file do not need to be replaced; while the json file and html file need to be replaced.

[0017] Preferably, the processing of the pdf file, word file or excel file includes:

[0018] Step A1: Get the file content and return the file stream;

[0019] Step A2: determine whether the file stream meets a preset threshold; if the result is yes, report an error; if the result is no, save the file stream to the database;

[0020] The processing of HTML files includes:

[0021] Step B1: Obtain a file stream; convert the file stream into the file content of an HTML file;

[0022] Step B2: Format the file content and remove useless tags; the useless tags include comment tags and style tags;

[0023] Step B3: Obtain the a tag and img tag of the html file, and determine whether the a tag and img tag comply with the configuration rules of the database. If yes, delete them; if no, upload the contents of the a tag and img tag, and fill the uploaded paths back into the a tag and img tag;

[0024] Step B4: Remove special characters and meaningless tags from the HTML file, use the h2 tag to add title information to the text of the HTML file, and then convert it uniformly through the UTF-8 encoding format, compress it with zlib and save it to a local file.

[0025] The meaningless tags include: <script>与<style>;

[0026] 所述特殊字符包括:换行符、缩进符、meta标签与head标签。

[0027] 优选地,对于json文件的处理,包括:

[0028] 步骤C1:获取文件流;将文件流转化为文本内容;

[0029] 步骤C2:根据数据库的配置规则,判断所述文本内容是否需要替换成html文件,结果为是,则将json文件中的key值和其对应的文本内容映射关系存放到数据库中;

[0030] 步骤C3:定制freemaker模板,根据json文件的信息,将所述freemaker模板配置为横排或者竖排展示的html文本;

[0031] 步骤C4:组合替换所述key值,并且对时间进行日期格式化;

[0032] 步骤C6:将组合替换的所述key值和所述freemaker模板进行数据填充,得到标准化的html文本,继而将所述标准化的html文本转化为文件流,进而通过utf-8编码格式进行统一转化,由zlib压缩并且保存到本地文件;

[0033] 在所述步骤C4中,根据数据库的枚举值表组合替换所述key值;

[0034] 所述数据库为mysql数据库。

[0035] 根据本发明提供的一种多格式文件的标准化处理系统,包括:

[0036] 获取单元:获取文件信息;

[0037] 分类单元:通过所述文件信息对文件进行分类流转,得到分类后附件;

[0038] 判断单元:基于分类后附件的类型,判断分类后附件是否需要替换处理;结果为是,则将所述文件替换为html文件,格式化处理所述html文件,保存至本地文件;结果为否,则将所述文件转化为文件流,进行统一转化,保存至本地文件。

[0039] 优选地,所述分类单元通过文件信息获取文件格式,进而通过所述文件格式对文件进行分类流转,得到分类后附件;

[0040] 所述文件格式包括:pdf、word、excel、json与html;

[0041] 所述分类后附件包括:pdf文件、word文件、excel文件、json文件与html文件;

[0042] 所述文件信息,包括:文件扩展名、文件头与文件路径。

[0043] 优选地,在判断单元中,所述分类后附件是否需要替换处理的依据是,所述pdf文件、word文件与excel文件不需要替换处理;所述json文件与html文件需要替换处理。

[0044] 优选地,对于pdf文件、word文件或excel文件的处理,包括:

[0045] 模块A1:获取文件内容,返回文件流;

[0046] 模块A2:判断文件流是否符合预设阈值;结果为是,则报错;结果为否,则将所述文件流保存到数据库;

[0047] 对于html文件的处理,包括:

[0048] 模块B1:获取文件流;将文件流转化为html文件的文件内容;

[0049] 模块B2:格式化处理所述文件内容,去除无用标签;所述无用标签,包括:注释标签与style标签;

[0050] 模块B3:获取html文件的a标签和img标签,判断所述a标签和img标签是否符合数据库的配置规则,结果为是,则删除;结果为否,则将所述a标签和img标签的内容上传,并且将上传后的路径回填至所述a标签和img标签;

[0051] 模块B4:去除html文件的特殊字符与无意义的标签,使用h2标签对html文件的文本添加标题信息,进而通过utf-8编码格式进行统一转化后,由zlib压缩并且保存到本地文件。

[0052] 所述无意义的标签,包括:<script>与<style>;

[0053] 所述特殊字符包括:换行符、缩进符、meta标签与head标签。

[0054] 优选地,对于json文件的处理,包括:

[0055] 模块C1:获取文件流;将文件流转化为文本内容;

[0056] 模块C2:根据数据库的配置规则,判断所述文本内容是否需要替换成html文件,结果为是,则将json文件中的key值和其对应的文本内容映射关系存放到数据库中;

[0057] 模块C3:定制freemaker模板,根据json文件的信息,将所述freemaker模板配置为横排或者竖排展示的html文本;

[0058] 模块C4:组合替换所述key值,并且对时间进行日期格式化;

[0059] 模块C5:将组合替换的所述key值和所述freemaker模板进行数据填充,得到标准化的html文本,继而将所述标准化的html文本转化为文件流,进而通过utf-8编码格式进行统一转化,由zlib压缩并且保存到本地文件;

[0060] 在所述模块C4中,根据数据库的枚举值表组合替换所述key值;

[0061] 所述数据库为mysql数据库。

[0062] 与现有技术相比,本发明具有如下的有益效果:

[0063] 1、本发明通过标准化处理方法兼容不同的数据格式,减少了文件数据误差的影响,提高了正确性,并且确保不同文件之间的一致性,使得文件更容易被理解和分析处理。

[0064] 2、本发明提供的标准化处理能够帮助消除数据格式之间的差异,相较于数据分析和存储过程中将来自不同来源和格式的数据进行整合的步骤,降低了数据整合和汇总的难度。

[0065] 3、本发明在不同的系统和平台之间交换数据时,采用一致的文件数据格式和通用的文件处理,减小了文件读取的难度和文件处理的时间,从而提高数据跨平台交换和处理的速度和效率。

[0066] 4、本发明支持读取和识别多种文件格式,包括pdf、word、excel、json和html等,通过分析文件扩展名和文件头信息,自动判断并处理相应的文件类型。

[0067] 5、本发明采用固定的输出格式,通过使用zlib压缩工具对文件进行压缩,为业务方提供统一的附件内容和附件类型,从而保证文件系统的高可用性和一致性。附图说明

[0068] 通过阅读参照以下附图对非限制性实施例所作的详细描述,本发明的其它特征、目的和优点将会变得更明显:

[0069] 图1为本发明提供的文件标准化流程示意图;

[0070] 图2为本发明提供的多格式文件标准化文件格式对应图;

[0071] 图3为本发明提供的文件流转示意图;

[0072] 图4为本发明提供的html文件处理流程示意图;

[0073] 图5为本发明提供的json文件转换流程示意图。具体实施方式

[0074] 下面结合具体实施例对本发明进行详细说明。以下实施例将有助于本领域的技术人员进一步理解本发明,但不以任何形式限制本发明。应当指出的是,对本领域的普通技术人员来说,在不脱离本发明构思的前提下,还可以做出若干变化和改进。这些都属于本发明的保护范围。

[0075] 本方法提供的多格式文件标准化处理方法旨在提高跨平台和跨应用程序的文件互操作性,自动识别和转换多种格式,从而简化文件管理和分享。在收到文件消息之后,首先使用正则表达式对文件进行检验,将无用字符和不需要的字符进行剔除,后将附件和图片路径根据业务需求进行删除或保留,最后使用zlib压缩工具对文件进行压缩,提供给业务方统一的附件内容格式和附件类型,保证了文件系统的高可用性。

[0076] 本发明的主要目的在于提出一种多格式文本处理的方法,旨在解决现有各种文本格式附件上传无法统一的问题,统一文件格式,提高用户使用的便利性。

[0077] 为实现上述目的,根据本发明提供的一种文件标准化的系统,包括:文件分类和内容获取模块;所述文件分类和内容获取模块,包括:

[0078] 判断单元:通过文件扩展名、文件头信息等方法判断对应的文件格式。

[0079] 分类单元:根据上述判断单元获取文件格式进行pdf、word、excel、json和html等文件格式的分类。

[0080] 获取单元:根据文件路径和文件格式获取对应的文件流,支持file,即本地文件、ftp、http,即云端文件和hdfs,即一种文件储存方式,等文件路径。

[0081] 检验单元:对所获取的文件流进行检验;

[0082] 具体地,对md5值进行校验,将原文md5值和获取文件md5值进行校验,若完全一致,则进行下一步检验;若不一致,则在数据库中记录校验失败的文件流,告警推送给相关人员。

[0083] 具体地,大小判断,如果文件流的大小符合既定规则,则将文件进行下一步进行解析;若不符合既定规则,则在数据库中记录判断失败的文件流,告警推送给相关人员。

[0084] 所述既定规则,指的是校验所需校验文件流的大小是否超过事先设定的阈值。

[0085] 文件内容解析模块;

[0086] 步骤1:获取文件的分类结果,根据不同的文件分类则执行以下解析方法;文件是pdf、excel和word类型的文档,则不对文件流进行处理,保留文件流。

[0087] 文件是html文件,则进行以下处理:

[0088] 第一步:使用正则表达式去除无用标签;

[0089] 第二步:获取上述精简过的html内容中body标签中的文本内容;

[0090] 第三步:根据要求是否保留或者替换对应img标签和a标签;若替换,则将标签中的路径替换为已上传的图片或者文件;若保留,则不替换路径。

[0091] 第四步:去除特殊标签,去除缩进和换行符;添加h2标签,对文本进行标题添加;

[0092] 具体地,文件是json文件,则进行以下处理:

[0093] 判断文件是否需要替换操作,若json文件不需额外操作,只需原文上传则保留所述步骤1中的文件流;若文件需要转化为对应的html文件,则进行下述操作:

[0094] 第一步:将json中字段的映射关系配置到数据库中;

[0095] 第二步:支持枚举值映射;将json文件对应的freemaker模板保存在方法程序中,并将模板路径和json的对应关系配置到数据库中;

[0096] 第三步:对上述配置信息进行校验,无问题则将json中键值对应的值和配置信息中的映射进行匹配和组合,将组合后的信息写进freemaker模板中,返回组合后的文件流;若存在问题,则直接返回原文件流;未识别的文件类型则不对该文件内容处理,直接返回文件流。

[0097] 所述freemaker是一款模板引擎:即一种基于模板和要改变的数据,并用来生成输出文本的通用工具;

[0098] 所述输出文本,包括:html网页,电子邮件,配置文件,源代码。

[0099] 所述校验配置信息的方法是通过代码检查freemaker模板格式是否正确;所述问题,具体指的是组合后的信息无法正确写入freemaker模板中。

[0100] 具体地,还包括文件保存模块;

[0101] 将不需要处理的pdf、excel和word等类型的文档直接返回原文件路径,不进行保存;

[0102] 步骤2:将需转换的json文本和html文本的文件流统一转换为统一编码,即UTF-8;

[0103] 步骤3:将文件流通过zlib压缩算法压缩成统一格式文件;在本地文件或共享文件空间中创建对应格式的文件,将上述文件流通过异步输入输出的方式写入到新建的文件中。

[0104] 具体地,监控文件处理过程中的异常情况,包括:

[0105] 其一,对上述步骤执行过程中进行埋点,在处理过程中出现问题的地方进行记录。

[0106] 其二,将记录的信息报错在数据库中。

[0107] 其三,及时对错误的点进行重试、排查和告警。

[0108] 实施例1,为详细说明多格式文件标准化处理方法,如图1所示,以下表达了一种具体实施方法,包括:

[0109] 步骤S1:获取文件路径和类型;

[0110] 步骤S2:根据所述文件路径和类型,对文件进行分类流转,得到分类后的附件;

[0111] 具体地,分类流转如图2所示,在所述步骤S2中

[0112] 步骤S3:根据分类后的附件类型进行标准化处理,不同类型进行不同的标准化处理,得到标准化文件。

[0113] 具体地,所述标准化文件经UTF-8格式化后保存。

[0114] 具体地,处理不需要转换替换的文件,例如pdf,excel,如图3所示,包括:

[0115] 步骤A1:获取文件内容,返回文件流;

[0116] 步骤A2:判断文件流大小,符合指定大小,将文件进行报错,若不符合,将文件处理信息保存到数据库中,以供查询。所述指定大小,即事先设定的阈值。

[0117] 处理需要加工的文件,即html文本文件,如图4所示,包括:

[0118] 步骤B1:获取文件流,将文件流转化为html文件的文件内容;

[0119] 步骤B2:将html文件内容进行格式化处理,去除无用标签;所述无用标签,包括:注释标签、style标签;

[0120] 步骤B3:获取html文本中的a标签和img标签,根据判断,若不需要改标签,将标签直接删除;若需要对标签进行替换,则将该标签中的内容进行获取上传,将上传后的路径回填到该标签中;所述判断依据是基于文件的业务模块,从数据库中读取配置规则,判断是否需要更改。

[0121] 步骤B4:去除html文本中的特殊字符,如换行符、缩进符、meta标签、head标签等,去除无意义的标签。所述无意义的标签,即不影响页面显示或功能的标签,如<script>、<style>等。

[0122] 步骤B5:使用h2标签对html文本添加标题信息。

[0123] 具体地,处理需加工的json文件标准化处理步骤,如图5所示,包括:

[0124] 步骤C1:获取文件流,将文件流转化为文本内容;

[0125] 步骤C2:判断文本内容是否需要转换成html文件,若不需要,直接返回文件流;若需要则继续进行以下步骤;所述判断依据是基于文件的业务模块,从数据库中读取配置规则,判断是否需要更改。

[0126] 步骤C3:将json文件中的key值和其对应的中文文本映射关系存放到数据库,即mysql数据库中;

[0127] 步骤C4:定制freemaker模板,根据json文件的信息,将模板配置为横排或者竖排展示的html格式的文本;

[0128] 步骤C5:组合json文件的键值,即key值对数据,对需要枚举值的key值进行替换组合;对时间需要格式化的字段进行日期格式化;所述枚举值判断依据是读取数据库枚举值表,判断key值是否存在;若存在,则替换组合;若不存在,则直接使用原key值;所述时间格式化判断依据是检查字段是否是时间戳格式;若是,则进行日期格式化;若不是,则保持原内容;

[0129] 步骤C6:将组合的key值和freemaker模板进行数据填充,组装成一个标准化的html文本

[0130] 步骤C7:将标准化的html文本转化为文件流进行返回。

[0131] 步骤S4:将步骤S3得到的文件流使用utf-8编码格式进行统一转化。

[0132] 步骤S5:对html文件流和json文件流,使用zlib压缩工具,对其进行压缩

[0133] 步骤S6:保存文件流到本地文件。

[0134] 具体地,在所述步骤S6中,PDF、excel等不需转化的文件直接使用其文件格式进行报错;对转化的json和html进行以zlib格式进行保存。

[0135] 步骤S7:错误监测和重试机制;

[0136] 具体地,将上述过程进行关键节点埋点;所述关键节点,即上述步骤中每个步骤的执行结果,无论成功还是失败,均日志记录并存储;

[0137] 具体地,对处理文件过程中出现错误的情况进行记录,并将记录保存到数据库中。

[0138] 具体地,定时将该错误部分数据进行告警和重试。

[0139] 因此,多格式文件标准化处理方法通过识别文件类型和自动化处理,有效地解决了传统技术的这些缺陷,提升了文件转换的效率和准确性,并且增强了多格式文件的互操作性和长期保存的可靠性。

[0140] 本发明还提供一种多格式文件的标准化处理系统,所述多格式文件的标准化处理系统可以通过执行所述多格式文件的标准化处理方法的流程步骤予以实现,即本领域技术人员可以将所述多格式文件的标准化处理方法理解为所述多格式文件的标准化处理系统的优选实施方式。

[0141] 根据本发明提供的一种多格式文件的标准化处理系统,包括:

[0142] 获取单元:获取文件信息;

[0143] 分类单元:通过所述文件信息对文件进行分类流转,得到分类后附件;

[0144] 判断单元:基于分类后附件的类型,判断分类后附件是否需要替换处理;结果为是,则将所述文件替换为html文件,格式化处理所述html文件,保存至本地文件;结果为否,则将所述文件转化为文件流,进行统一转化,保存至本地文件。

[0145] 本领域技术人员知道,除了以纯计算机可读程序代码方式实现本发明提供的系统及其各个装置、模块、单元以外,完全可以通过将方法步骤进行逻辑编程来使得本发明提供的系统及其各个装置、模块、单元以逻辑门、开关、专用集成电路、可编程逻辑控制器以及嵌入式微控制器等的形式来实现相同功能。所以,本发明提供的系统及其各项装置、模块、单元可以被认为是一种硬件部件,而对其内包括的用于实现各种功能的装置、模块、单元也可以视为硬件部件内的结构;也可以将用于实现各种功能的装置、模块、单元视为既可以是实现方法的软件模块又可以是硬件部件内的结构。

[0146] 以上对本发明的具体实施例进行了描述。需要理解的是,本发明并不局限于上述特定实施方式,本领域技术人员可以在权利要求的范围内做出各种变化或修改,这并不影响本发明的实质内容。在不冲突的情况下,本申请的实施例和实施例中的特征可以任意相互组合。< / script>

Claims

1. A standardized processing method for multi-format files, characterized in that: include: Step S1: Obtain file information; classify and circulate the files according to the file information to obtain classified attachments; Step S2: Based on the type of the classified attachment, determine whether the classified attachment needs to be replaced; if the result is yes, replace the file with an HTML file, format the HTML file, and save it to a local file; if the result is no, convert the file into a file stream, perform a unified conversion, and save it to a local file; In the step S1, the file format is obtained through the file information, and then the files are classified and circulated according to the file format to obtain the classified attachments; The file format includes: json; the classified attachment includes: json file; the json file needs to be replaced; The processing of json files includes: Step C1: Obtain a file stream; convert the file stream into text content; Step C2: According to the configuration rules of the database, determine whether the text content needs to be replaced with an HTML file. If yes, store the mapping relationship between the key value in the JSON file and its corresponding text content in the database; Step C3: Customize the freemaker template, and configure the freemaker template as an HTML text displayed in horizontal or vertical rows according to the information in the json file; Step C4: Combining and replacing the key value, and formatting the time; Step C5: Fill the key value and the freemaker template with data to obtain a standardized HTML text, and then convert the standardized HTML text into a file stream, and then convert it uniformly through the UTF-8 encoding format, compress it by zlib and save it to a local file; In step C4, the key value is replaced according to the enumeration value table combination of the database.

2. The method for standardizing multi-format files according to claim 1, characterized in that: The file formats include: pdf, word, excel and html; The classified attachments include: pdf files, word files, excel files and html files; The file information includes: file extension, file header and file path.

3. The method for standardizing multi-format files according to claim 2, characterized in that: In step S2, whether the classified attachment needs to be replaced is based on the fact that the pdf file, word file and excel file do not need to be replaced; and the html file needs to be replaced.

4. The method for standardizing multi-format files according to claim 3, characterized in that: Processing of PDF files, Word files or Excel files, including: Step A1: Get the file content and return the file stream; Step A2: determine whether the file stream meets a preset threshold; if the result is yes, report an error; if the result is no, save the file stream to the database; The processing of HTML files includes: Step B1: Obtain a file stream; convert the file stream into the file content of an HTML file; Step B2: Format the file content and remove useless tags; the useless tags include comment tags and style tags; Step B3: Obtain the a tag and img tag of the html file, and determine whether the a tag and img tag comply with the configuration rules of the database. If yes, delete them; if no, upload the contents of the a tag and img tag, and fill the uploaded paths back into the a tag and img tag; Step B4: remove special characters and meaningless tags from the HTML file, use the h2 tag to add title information to the text of the HTML file, and then convert it uniformly through the UTF-8 encoding format, compress it with zlib and save it to a local file; The meaningless tags include: <script>与<style>;所述特殊字符包括:换行符、缩进符、meta标签与head标签。5.一种多格式文件的标准化处理系统,其特征在于,包括:获取单元:获取文件信息;分类单元:通过所述文件信息对文件进行分类流转,得到分类后附件;判断单元:基于分类后附件的类型,判断分类后附件是否需要替换处理;结果为是,则将所述文件替换为html文件,格式化处理所述html文件,保存至本地文件;结果为否,则将所述文件转化为文件流,进行统一转化,保存至本地文件;所述分类单元通过文件信息获取文件格式,进而通过所述文件格式对文件进行分类流转,得到分类后附件;所述文件格式包括:json;所述分类后附件包括:json文件;所述json文件需要替换处理;对于json文件的处理,包括:模块C1:获取文件流;将文件流转化为文本内容;模块C2:根据数据库的配置规则,判断所述文本内容是否需要替换成html文件,结果为是,则将json文件中的key值和其对应的文本内容映射关系存放到数据库中;模块C3:定制freemaker模板,根据json文件的信息,将所述freemaker模板配置为横排或者竖排展示的html文本;模块C4:组合替换所述key值,并且对时间进行日期格式化;模块C5:将组合替换的所述key值和所述freemaker模板进行数据填充,得到标准化的html文本,继而将所述标准化的html文本转化为文件流,进而通过utf-8编码格式进行统一转化,由zlib压缩并且保存到本地文件;在所述模块C4中,根据数据库的枚举值表组合替换所述key值。6.根据权利要求5所述的多格式文件的标准化处理系统,其特征在于,所述文件格式包括:pdf、word、excel与html;所述分类后附件包括:pdf文件、word文件、excel文件与html文件;所述文件信息,包括:文件扩展名、文件头与文件路径。7.根据权利要求6所述的多格式文件的标准化处理系统,其特征在于,在判断单元中,所述分类后附件是否需要替换处理的依据是,所述pdf文件、word文件与excel文件不需要替换处理;所述html文件需要替换处理。8.根据权利要求7所述的多格式文件的标准化处理系统,其特征在于,对于pdf文件、word文件或excel文件的处理,包括:模块A1:获取文件内容,返回文件流;模块A2:判断文件流是否符合预设阈值;结果为是,则报错;结果为否,则将所述文件流保存到数据库;对于html文件的处理,包括:模块B1:获取文件流;将文件流转化为html文件的文件内容;模块B2:格式化处理所述文件内容,去除无用标签;所述无用标签,包括:注释标签与style标签;模块B3:获取html文件的a标签和img标签,判断所述a标签和img标签是否符合数据库的配置规则,结果为是,则删除;结果为否,则将所述a标签和img标签的内容上传,并且将上传后的路径回填至所述a标签和img标签;模块B4:去除html文件的特殊字符与无意义的标签,使用h2标签对html文件的文本添加标题信息,进而通过utf-8编码格式进行统一转化后,由zlib压缩并且保存到本地文件;所述无意义的标签,包括:<script>与<style>;所述特殊字符包括:换行符、缩进符、meta标签与head标签。< / script>

Citation Information

Patent Citations

  • Method for automatically reading, identifying and formatting json file and recombining, identifying and storing json file

    CN111105481A

  • File analysis processing method and device, storage medium and electronic equipment

    CN114490848A

  • JSON data display method and device, equipment and storage medium

    CN116362200A

  • Data format conversion method and device, storage medium and electronic equipment

    CN117573613A