Document processing methods, electronic devices and computer program products

Abstract

This application discloses a document processing method, electronic device, and computer program product, relating to the field of data processing technology. The method includes: crawling source documents from a preset site based on a preset period and preset search terms to obtain source documents; parsing the source documents to obtain a document layout, and adjusting the layout of the source documents based on the document layout to obtain a target document; extracting identification information based on the source documents, and associating the identification information with the target document input and saving it to a preset database. This application achieves automated document collection and structured processing, reduces the time cost of manual operations, avoids omissions and structural misalignments during format conversion, ensures the timeliness and comprehensiveness of document acquisition, and improves the efficiency of compliance management.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to document processing methods, electronic devices, and computer program products. Background Technology

[0002] Against the backdrop of rapid development in the digital economy, data security has become a core issue for enterprise operations and social governance. Data security management documents, as crucial evidence for enterprise compliance, are directly related to compliance risk prevention and sustainable development. Enterprises and institutions need to regularly collect and organize domestic and international data security-related documents and archive them in standard formats to meet the needs of compliance audits, business adjustments, and risk assessments. The current data security management document management process mainly relies on manual operation. Relevant domestic and international management documents are manually searched through search engines; for foreign documents, professional translation agencies are commissioned for manual translation; subsequently, the document content is manually parsed, and key information is copied into a CSV (Comma-Separated Values) document in a fixed format for archiving. However, this manual management process has revealed many problems in practical application: First, facing a large number of documents, manual processing is insufficient to meet the high-frequency needs of compliance audits, and sampling reviews inevitably lead to omissions; second, the format conversion process is cumbersome, and with diverse document structures, manually extracting content and formatting it into a CSV document easily results in errors such as missing clause numbers and format misalignment. In other words, the existing manual document management process cannot efficiently collect and archive massive amounts of documents, resulting in low compliance management efficiency and increased risks.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a document processing method, electronic device, and computer program product, which aims to solve the technical problem of low efficiency in compliance management caused by manual document archiving.

[0005] To achieve the above objectives, this application proposes a document processing method, which includes: Source documents are crawled from preset sites based on preset periods and preset search terms; The source document is parsed to obtain the document layout, and the layout of the source document is adjusted based on the document layout to obtain the target document; Identification information is extracted from the source document, and the identification information and the target document are associated and saved to a preset database.

[0006] In one embodiment, the step of crawling source documents from a preset site based on a preset period and preset search terms to obtain source documents includes: The preset search terms are expanded using a knowledge graph, and optimized search terms are obtained by expanding synonyms based on a knowledge graph in the data security field. Based on the optimized search terms, incremental source documents are obtained by performing timed incremental crawling on domestic websites, and event-triggered source documents are obtained by performing event-triggered crawling on international websites. The incremental source documents or event source documents are used as source documents.

[0007] In one embodiment, the step of parsing the source document to obtain a document layout and adjusting the layout of the source document based on the document layout to obtain a target document includes: Multimodal recognition processing is performed on the source document to identify text, image, and table elements in the source document, and element recognition results are obtained; The element identification result is processed to obtain the parsed layout information. Based on the parsed layout information, the source document is processed to adjust the layout to obtain the target document.

[0008] In one embodiment, the step of performing layout parsing processing on the element identification result to obtain parsed layout information includes: Merge cells are marked on the table data using a dual marking method of row span and column span, resulting in the marked table data; The source document is subjected to structural association processing, which links footnotes to the main text by anchoring them and locates attachments by keywords while preserving hierarchical relationships, thus obtaining preliminary structured data; Perform text recognition enhancement processing on documents containing images to obtain recognizable text; By merging the preliminary structured data, the identifiable text, and the marked table data, structured layout information is obtained.

[0009] In one embodiment, after the steps of parsing the source document to obtain a document layout and adjusting the layout of the source document based on the document layout to obtain the target document, the method further includes: Determine the error rate of the target document, and when the error rate exceeds a preset threshold, perform a visual comparison process on the source document and the target document in the visualization interface, and receive user modification operations based on the visualization interface; The target document is adjusted according to the modification operation described above, and the adjusted target document is updated to the target document.

[0010] In one embodiment, prior to the step of parsing the source document to obtain the document layout, the method further includes: The source document and the preset terminology database are subjected to terminology matching and translation processing to obtain the initial translated text. The terminology database is obtained by extracting terms from national standards and industry white papers. The initial translation text is input into a preset translation model to obtain the translated source document, wherein the translation model is obtained through data augmentation and adversarial training to optimize the translation of nested sentences; The step of parsing the source document to obtain the document layout includes: The translated source document is parsed to obtain the document layout.

[0011] In one embodiment, before the step of parsing the translated source document to obtain the document layout, the method further includes: The translated source document is subjected to a terminology consistency scan to obtain a terminology verification result, and the translated source document is subjected to a semantic similarity verification to obtain a semantic verification result. If the terminology verification result and the semantic verification result are both verified, then the step of parsing the translated source document to obtain the document layout is executed; If the terminology verification result and / or the semantic verification result are not verified, then the user's document correction operation on the translated source document is received, the translated source document is updated based on the document correction operation, and the step of parsing the translated source document to obtain the document layout is executed.

[0012] In one embodiment, the step of extracting identifier information based on the source document includes: Based on the preset terminology database, the clause number, effective date, and table structure are parsed, and the source document is subjected to element parsing processing to obtain structured parsed data. Perform compliance verification on the structured parsed data, check the integrity and format compliance of the fields in the structured parsed data, and obtain the data verification result; If the data verification result is a verification failure, then the system receives the user's data correction operation on the structured parsed data, and adjusts the structured parsed data based on the data correction operation to obtain the identification information; If the data verification result is successful, then the structured parsed data will be used as the identification information.

[0013] In addition, to achieve the above objectives, this application also proposes an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the document processing method as described above.

[0014] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the document processing method described above.

[0015] This application addresses the inefficiency of manual retrieval through a combined crawling mechanism of preset cycles and search terms. The preset cycle ensures continuous tracking of document updates, while preset search terms guarantee accurate acquisition of target documents. This combination avoids the randomness of manual retrieval and mitigates data redundancy that may arise from simple timed crawling. Secondly, it employs document layout parsing and adaptive adjustment technology. During the parsing phase, the original structural features of the source documents are analyzed to provide data support for subsequent layout reconstruction. The dynamic adjustment mechanism based on the parsing results solves the archiving chaos caused by inconsistent formats across multiple source documents, particularly for documents containing complex tables or mixed text and images, maintaining the integrity of the document's logical structure. Finally, through a dual operation of identifier information extraction and associated storage, unstructured documents are transformed into searchable metadata during the extraction phase, and a mapping relationship between document entities and feature identifiers is established during the associated storage phase. This not only improves database retrieval efficiency but also provides a data foundation for subsequent version tracking and clause citation.

[0016] Thus, this application achieves automated document collection and structured processing, reduces the time cost of manual operation, avoids omissions of clauses and structural misalignments during format conversion, ensures the timeliness and comprehensiveness of document acquisition, and improves the management efficiency of compliance management. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an embodiment of the document processing method of this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the document processing method of this application; Figure 3 A schematic diagram of the module structure of a document processing method provided in an embodiment of this application; Figure 4A simplified flowchart illustrating a document processing method provided in an embodiment of this application; Figure 5 A schematic diagram of the multimodal recognition process of a document processing method provided in an embodiment of this application; Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the document processing method in this application embodiment.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] Current data security management document management processes rely heavily on manual operation. This involves manually searching for relevant domestic and international documents through search engines; for foreign documents, professional translation agencies are commissioned for manual translation; and then the document content is manually parsed, with key information copied into a CSV document for archiving. However, this manual management process has revealed several problems in practical application: First, with a massive number of documents, manual processing struggles to meet the high-frequency demands of compliance audits, and sampling methods inevitably lead to omissions; second, the format conversion process is cumbersome, and the diverse document structures make manually extracting content and formatting it into a CSV document prone to errors such as missing clause numbers and misaligned formats. In short, the existing manual document management process cannot efficiently collect and archive massive amounts of documents, resulting in low compliance management efficiency and increased risks.

[0024] This application provides a solution for full lifecycle management of documents by constructing an automated processing workflow. First, it addresses the inefficiency of manual retrieval through a combined crawling mechanism of preset cycles and search terms. The preset cycles ensure continuous tracking of document updates, while preset search terms guarantee accurate acquisition of target documents. This combination avoids the randomness of manual retrieval and mitigates data redundancy that may arise from simple timed crawling. Second, it employs document layout parsing and adaptive adjustment technology. During the parsing phase, the original structural features of the source documents are analyzed to provide data support for subsequent layout reconstruction. The dynamic adjustment mechanism based on the parsing results solves the problem of inconsistent archiving caused by multiple source document formats, especially for documents containing complex tables or mixed text and images, maintaining the integrity of the document's logical structure. Finally, through a dual operation of identifier information extraction and associated storage, unstructured documents are transformed into searchable metadata during the extraction phase, and a mapping relationship between document entities and feature identifiers is established during the associated storage phase. This not only improves database retrieval efficiency but also provides a data foundation for subsequent version tracking and clause citation.

[0025] It should be noted that the executing entity in this embodiment can be a computing service device with document processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions. The following description uses an electronic device as an example to illustrate this embodiment and the subsequent embodiments.

[0026] Based on this, embodiments of this application provide a document processing method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the document processing method of this application.

[0027] In this embodiment, the document processing method includes steps S10 to S30: Step S10: Crawling source documents from preset sites based on preset periods and preset search terms to obtain source documents; The preset period refers to a pre-defined document crawling time interval, which can be implemented using a scheduled task scheduler. For example, a crawling task can be set to run every day at midnight to ensure timely acquisition of the latest documents. This feature solves the problem of insufficient timeliness in manual retrieval. Preset search terms refer to a predefined set of keywords, which can be expanded using a knowledge graph to generate a list of synonyms. For example, "data security" can be expanded to related terms such as "information security" and "network security," improving the comprehensiveness of document retrieval.

[0028] This embodiment triggers crawling tasks at preset intervals and simultaneously retrieves documents from domestic and international websites by combining expanded search terms.

[0029] Step S20: Parse the source document to obtain the document layout, and adjust the layout of the source document based on the document layout to obtain the target document; Document layout parsing refers to identifying the arrangement structure of elements such as text, images, and tables in a document. This can be achieved using multimodal recognition algorithms, such as using convolutional neural networks to identify image regions and combining this with natural language processing techniques to locate chapter titles, providing data support for subsequent layout adjustments. Layout adjustment refers to reconstructing the document structure according to a standard format, which can be achieved using a dynamic layout engine. For example, it can automatically convert identified tables to a uniform column width format to maintain consistency across documents.

[0030] After multimodal parsing, the source document is analyzed to identify chapter structure, table positions, and image descriptions. For example, the column layout in a PDF (Portable Document Format) document is converted to a single-column layout. During the layout adjustment process, the dynamic engine rearranges the element positions based on the parsing results, such as merging multi-page tables into complete table cells.

[0031] Step S30: Extract identification information based on the source document, and save the identification information and target document input association to a preset database.

[0032] Identification information extraction refers to extracting key metadata such as clause numbers and effective dates from documents. This can be achieved using regular expression matching combined with semantic analysis, for example, extracting clause numbers in the format "Article X" through pattern recognition, ensuring the accuracy of structured data. Specifically, in this embodiment, during the identification information extraction stage, semantic analysis is used to locate clause numbers and effective dates, for example, extracting publication date information from the document header. Finally, the structured metadata is associated and stored with the standardized document, forming a database that can be quickly retrieved.

[0033] This embodiment realizes automated document acquisition and structured processing, reducing the time cost of manual operation, avoiding omissions of clauses and structural misalignments during format conversion, and ensuring the timeliness and comprehensiveness of document acquisition through preset cycle and intelligent search term combination. The layout parsing and adjustment technology solves the problem of archiving chaos caused by the differences in the formats of multi-source documents, while the extraction of identification information provides a data foundation for subsequent rapid retrieval and version management.

[0034] In one feasible embodiment, step S10: the step of crawling source documents from a preset site based on a preset period and preset search terms to obtain source documents, includes: Step S101: Perform knowledge graph expansion processing on the preset search terms, and obtain optimized search terms by expanding synonyms based on the knowledge graph in the data security field; Knowledge graph expansion processing refers to expanding the original search terms into a set of search terms that includes synonyms, near-synonyms, and variations of industry terms by utilizing the semantic relationships between entities in the knowledge graph of the data security field. Specifically, it can be implemented by using a node embedding algorithm based on graph neural networks, which expands the associated vocabulary by calculating the semantic similarity between nodes.

[0035] Specifically, in document acquisition scenarios, the original search terms are expanded through a knowledge graph to form a set of search terms that include variations of industry terms. For example, "cross-border data transfer" can be expanded into related terms such as "data export" and "cross-border data flow".

[0036] Step S102: Perform timed incremental crawling on domestic sites based on the optimized search terms to obtain incremental source documents, and perform event-triggered crawling on international sites based on the optimized search terms to obtain event source documents, and use the incremental source documents or event source documents as source documents.

[0037] Scheduled incremental crawling refers to collecting only newly added or updated documents from domestic websites at preset time intervals. This can be implemented using an incremental recognition algorithm based on hash value comparison, identifying updated content by comparing differences in the hash values ​​of webpage snapshots. Event-triggered crawling refers to performing immediate data collection after monitoring specific event trigger signals from international websites. This can be implemented using a webpage update listener based on keyword matching, determining document publication events by parsing preset keywords in site announcements.

[0038] For domestic government portals, a daily incremental crawling mechanism is employed. By comparing the hash values ​​of yesterday's and today's webpage snapshots, only newly released policy documents are downloaded. For the websites of international standardization organizations, event listeners are deployed to monitor keywords such as "released" and "updated" in webpage announcements in real time. When relevant events are detected, the crawling program is immediately triggered to obtain the latest version of the document. The two collection modes work together. Domestic documents maintain timeliness through periodic incremental updates, while international documents ensure rapid response to sudden policy changes through event-driven real-time collection.

[0039] Compared to traditional manual retrieval methods that rely on fixed keyword lists and cannot automatically expand semantically related vocabulary, easily overlooking similar documents using different terminology, and conventional web crawling techniques that use a uniform collection frequency, failing to differentiate between domestic and international site update characteristics, resulting in lag in international document collection, this embodiment uses knowledge graph-driven dynamic expansion of search terms to cover diverse terminology in the text. Through differentiated collection strategy design, it employs a timed incremental mechanism for the gradual updates of domestic documents and an event-triggered mechanism for the sudden releases of international documents, achieving a dual improvement in collection efficiency and timeliness. Therefore, this embodiment solves the problem of incomplete semantic coverage caused by manually setting search terms, ensuring that similar documents with different terminology can be collected; it overcomes the lag in international document updates caused by a uniform collection frequency, achieving differentiated and efficient acquisition of domestic and international documents; and ultimately forms a comprehensive and timely data security management document collection system, providing a complete and reliable source data foundation for subsequent document processing.

[0040] In one feasible embodiment, step S20: parsing the source document to obtain the document layout, and adjusting the layout of the source document based on the document layout to obtain the target document, includes: Step S201: Perform multimodal recognition processing on the source document to recognize text, image, and table elements to obtain element recognition results; Multimodal recognition processing refers to the simultaneous extraction of text, graphics, and table elements from a document using optical character recognition and image segmentation techniques. Specifically, it can be implemented using convolutional neural networks combined with region proposal networks to address the problem of incomplete recognition of mixed elements. In detail, after multimodal recognition processing, the source document is decomposed into text blocks, image regions, and table structures, with the position coordinates and content features of each element precisely recorded.

[0041] Step S201: Perform layout parsing processing on the element recognition result to obtain the parsed layout information, and perform layout adjustment processing on the source document based on the parsed layout information to obtain the target document.

[0042] Layout parsing refers to generating structured layout data by analyzing the spatial position and logical relationships of elements. This can be implemented using attention-based graph neural networks to establish hierarchical relationships between elements. Layout adjustment refers to standardizing and reorganizing document elements based on structured layout information. This can be achieved using dynamic template matching algorithms to eliminate differences in the original document format.

[0043] The layout parsing process calculates the relative positions and semantic relationships between elements to generate structured layout data that includes paragraph hierarchy, chart positioning, and cross-page links. When adjusting the layout based on this data, text content is rearranged according to semantic paragraphs, images are automatically embedded into corresponding areas based on their context, and table structures maintain data integrity through cell merging markers, ultimately forming a target document that conforms to a standard format.

[0044] Compared to traditional manual parsing that relies on visual judgment and manual copying and pasting, which struggles to handle complex layouts with mixed text, images, and tables, easily leading to lost clause numbers or damaged table structures, this embodiment, through automated element recognition and intelligent layout reconstruction, can accurately parse the nested relationships of multi-element documents. For example, in documents containing cross-page tables, it can automatically identify table header continuation marks and maintain the continuity of table data. Thus, this application achieves automated parsing and format conversion of complex layouts in data security management documents, avoiding element omissions and formatting errors that occur during manual processing. Especially when processing statistical tables with merged cells or clause lists with annotations, it can completely preserve the logical structure and presentation of the original data.

[0045] In one feasible embodiment, step S202: performing layout parsing processing on the element identification result to obtain parsed layout information includes: Step S2021: Perform merged cell marking processing on the table data, and use a dual marking method of row span and column span to obtain the marked table data; The dual-marking method for row span and column span refers to jointly marking the number of horizontal and vertical merges of recorded cells. This can be implemented using nested XML tags.

[0046]

[0047] Labels and set corresponding footnotes.

[0055]

[0056]

[0057]

[0058]

[0059]

[0060]

[0061]

[0062]

[0063]

[0064]

[0065]

[0066]

[0067]

[0068]

[0069]

[0070]

[0071]

[0072]

[0073]

[0074]

[0075]

[0076]

[0077]

[0078]

[0079]

[0080] Figure 3 Figure 3 Figure 4 Figure 4

[0081]

[0082]

[0083]

[0084] Figure 5

[0085]

[0086]

[0087]

[0088]

[0089]

[0090]

[0091]

[0092] Figure 6 Figure 6

[0093] Figure 6

[0094]

[0095]

[0096]

[0097]

[0098]

[0099]

[0100]

[0101]

[0102]

[0103]

[0104]

[0105]

[0106]

[0107]

[0108] Adding rowspan and colspan attribute values ​​to each cell within the tag is used to accurately reproduce the row and column relationships of complex tables. Specifically, when processing tables containing merged cells, recording both the horizontal and vertical merge counts of cells simultaneously avoids the table structure misalignment problem caused by traditional single-tag methods. Step S2022 involves performing structural association processing on the source document, linking footnotes to the main text position using anchor tags and locating attachments using keywords while preserving hierarchical relationships to obtain preliminary structured data; linking footnotes using anchor tags refers to establishing a hyperlink mapping between footnote numbers and their reference positions in the main text, which can be implemented using HTML anchor tags inserted at the reference points in the main text. Use jump links to ensure the document structure remains consistent.

[0048] When parsing the document structure, anchor point marking technology is used to establish bidirectional links between footnote content and the reference position in the main text. At the same time, a keyword matching algorithm is used to automatically identify the hierarchical identifiers in the attachment documents to maintain the subordinate relationship between the attachments and the main document.

[0049] Step S2023: Perform text recognition enhancement processing on the document containing images to obtain recognizable text; Text recognition enhancement processing refers to performing text recognition after denoising and sharpening low-resolution images. Specifically, convolutional neural networks can be used to perform super-resolution reconstruction of the image, and bicubic interpolation algorithms can be used to improve image clarity before OCR recognition, thereby enhancing the recognizability of text in the image.

[0050] For image documents, an image enhancement step is added before regular OCR recognition to solve the problem of text recognition errors caused by scanning quality.

[0051] Step S2024: Merge the preliminary structured data, identifiable text, and marked table data to obtain structured layout information.

[0052] The labeled tabular data, structured text, and enhanced recognition results are fused together to form complete structured layout information.

[0053] Compared to traditional table processing methods that rely on single-dimensional labeling, leading to distortion in the reconstruction of cells spanning multiple rows and columns, this embodiment employs a dual-labeling method to fully preserve the structural features of the table. It maintains structural relevance through anchor point labeling and keyword positioning. Therefore, this embodiment can accurately reconstruct the merged cell structure of complex tables, avoiding the content misalignment issues caused by traditional methods. It also fully preserves the positional relationships between footnotes and main text, as well as the hierarchical relationships of attachments, resolving the problem of fragmented document structural information and improving the accuracy of text recognition in low-quality scanned images.

[0054] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S20: After parsing the source document to obtain the document layout and adjusting the layout of the source document based on the document layout to obtain the target document, the following steps are included: Step S40: Determine the error rate of the target document, and when the error rate exceeds a preset threshold, perform visual comparison processing on the source document and the target document in the visualization interface, and receive user modification operations based on the visualization interface; The error rate refers to the degree of matching deviation between the elements in the target document after layout adjustment and the corresponding elements in the original document, which can be calculated using text similarity algorithms and image difference comparison algorithms, and is used to quantitatively evaluate the accuracy of the automated processing results. The preset threshold refers to the critical value that triggers manual intervention, which can be dynamically adjusted according to the acceptable error range in historical processing data, and is used to filter out high error rate documents that need manual correction. When the error rate of the target document exceeds the preset threshold, the system automatically starts the visualization comparison interface, and displays the layout differences between the source document and the target document in a split-screen mode. For example, the left side displays the original table structure of the source document, and the right side displays the numbering misalignment problem caused by merging cells in the target document. Step S50: Adjust the target document according to the modification operation, and update the adjusted target document as the target document. Visual comparison processing refers to presenting the layout differences between the source and target documents through highlighted annotations or split-screen display. This can be achieved through document structure tree comparison and element coordinate mapping techniques, used to intuitively expose formatting misalignments or missing clauses. User modification operations refer to directly editing or dragging and adjusting erroneous areas of the target document through an interactive interface. This can be achieved using graphical editing tools and version control mechanisms, used to specifically correct complex errors that automated processing cannot resolve. Users can access the editing panel by clicking on highlighted areas to manually correct table cross-row marking errors. The system records modifications in real time and updates the layout information of the target document synchronously. After correction, the updated target document re-enters the error rate detection process until the error rate falls below a threshold or the maximum number of iterations is reached. This embodiment triggers manual intervention by dynamically detecting the error rate threshold. While retaining the efficiency advantages of automated processing, it uses visual comparison to accurately locate local errors that need correction, avoiding the contradiction between the high cost of fully manual processing and the low reliability of purely automated processing. Therefore, this embodiment can automatically identify documents with excessive error rates during automated document processing, quickly locate the specific locations of format misalignment or missing clauses through visual comparison, and guide manual correction, thereby reducing the workload of manual processing throughout the entire process while improving the accuracy of the target document.In one feasible embodiment, before step S20: parsing the source document to obtain the document layout, the method further includes: step S01, performing terminology matching and translation processing on the source document and a preset terminology database to obtain initial translated text, wherein the terminology database is obtained by performing terminology extraction processing on national standards and industry white papers; terminology matching and translation processing refers to the process of standardized translation conversion based on a professional domain terminology database, specifically, terminology extraction algorithms in natural language processing technology can be used to extract professional terms from international document texts and establish a terminology lookup table to achieve consistent terminology mapping. When an international document is input into the system, the preset terminology database is first called for terminology matching and translation, for example, "data controller" is uniformly converted to "data controller" to eliminate terminology ambiguity caused by manual translation. Step S02, inputting the initial translated text into a preset translation model to obtain the translated source document, wherein the translation model is obtained through data augmentation and adversarial training to optimize the translation of nested sentences; the translation model can specifically adopt a bidirectional long short-term memory network model architecture, and generate augmented data containing nested sentences through adversarial training, so that the model learns the sentence structure and logical relationship unique to the text to be translated. The initial translated text is then input into an optimized translation model, which accurately handles the semantic relationships of complex sentences by analyzing special structures such as conditional clauses in the text. The double-processed translated document enters the layout parsing stage, where standardized terminology and precise sentence transformation avoid errors in clause number identification or table structure parsing due to translation mistakes. Step S20: The step of parsing the source document to obtain the document layout includes: Step S203, parsing the translated source document to obtain the document layout. The specific process of parsing the translated source document to obtain the document layout is not detailed here. It is understandable that compared to traditional human translation processes, which suffer from delayed terminology updates and misunderstandings of complex sentence structures, leading to mixed terminology or misaligned logical relationships in translated documents, this embodiment achieves terminology standardization by constructing a dynamically updated terminology database and combines it with a specially trained translation model to solve the problem of handling nested sentence structures in the text, forming a processing mechanism that deeply integrates machine translation with professional domain knowledge. Thus, this embodiment solves the problems of inconsistent terminology and inaccurate conversion of complex sentence structures in the process of translating international documents, reduces the probability of subsequent parsing errors such as missing clause numbers and misaligned table structures caused by translation errors, and provides a highly accurate automated document processing foundation for multinational companies' compliance management.In a feasible embodiment, before step S20: parsing the translated source document to obtain the document layout, the method further includes: step S03, performing a term consistency scanning process on the translated source document to obtain a term verification result, and performing a semantic similarity verification process on the translated source document to obtain a semantic verification result; the semantic similarity verification process refers to calculating the semantic correlation degree between the original text and the translation, and specifically can be implemented by using a neural network-based vector space model, which is used to identify logical breaks or syntactic errors generated during the translation process. The document correction operation refers to receiving a user's modification instruction for the translated document through a visual interface, and specifically can be implemented by using an interactive text editor, which is used to correct the translation content that fails the verification. The term consistency scanning process refers to matching professional terms in the translation text through a preset term library, and specifically can be implemented by using a regular expression-based term matching algorithm, which is used to detect the standardization degree of term usage in the translated document. Specifically, the translated document first undergoes a term consistency scanning process, and automatic matching is performed using the standardized terms in the preset term library to identify term deviations existing in the translation text. For example, when there is a standardized translation of "datasecurity" corresponding to "数据安全" in the term library, the system automatically detects whether non-standard expressions such as "数据安保" appear in the translation. At the same time, the semantic similarity verification process calculates the cosine similarity between the original text and the translation in the semantic space by converting them into high-dimensional semantic vectors, and determines that there is a semantic deviation when the similarity is lower than the preset threshold. Step S04, if the term verification result and the semantic verification result pass the verification, then execute the step of parsing the translated source document to obtain the document layout; if the term verification result and the semantic verification result pass the verification, then automatically enter the layout parsing link and subsequent steps. Step S05, if the term verification result and / or the semantic verification result fails the verification, then receive the user's document correction operation for the translated source document, update the translated source document based on the document correction operation, and execute the step of parsing the translated source document to obtain the document layout. If any verification fails, an artificial correction process is triggered, and the user can modify the abnormal positions marked by the system in the visual interface. The corrected document re-enters the verification process until the processing requirements are met. Compared with the prior art that relies on manual proofreading of translated documents, there are problems of low efficiency and easy omission of term errors. This application combines an automated verification mechanism with manual intervention. On the basis of retaining the rigor of manual proofreading, it uses machine processing to improve the verification efficiency, and at the same time ensures that the translated document meets the subsequent processing requirements in terms of term standardization and semantic integrity through dual verification dimensions. Thus, this embodiment solves the problem of layout parsing errors caused by term inconsistency or semantic deviation in the translated document, blocks the flow of incorrect data to subsequent processing links through a pre-verification mechanism, and ensures the accuracy and reliability of the translation link in the document processing process, providing high-quality data input for subsequent structured processing and archiving.In one feasible embodiment, step S30, the step of extracting identification information based on the source document, includes: Step S301, parsing the clause number, effective date, and table structure based on a preset terminology database, and performing element parsing processing on the source document to obtain structured parsing data; the preset terminology database refers to a database storing standardized terms and data structures in a preset field. For example, in the legal field, the terminology database can be implemented using a legal clause template matching algorithm combined with a regular expression rule library, used to identify legal elements in the document. The source document is structured parsed using the clause number rules and effective date format templates stored in the preset terminology database. For example, a regular expression matching algorithm is used to extract the clause number field, and a date recognition model is used to locate the effective date information. During the table structure parsing process, a row and column span marking method is used to identify merged cells, maintaining the logical relationship of the original table. Step S302, performing compliance verification processing on the structured parsing data, checking the completeness and format standardization of the fields in the structured parsing data, and obtaining data verification results; the compliance verification processing refers to the process of checking the parsing results using logical rules, which can be implemented using a field missing detection algorithm and a format standardization verification model to ensure the completeness and standardization of the extracted data. After initial parsing, a field integrity check algorithm is used to verify whether any required fields are missing, such as checking if clause numbers are consecutive and if the effective date is correct. During format compliance verification, a predefined XML Schema is used to validate the parsed data, such as verifying whether the table nesting level conforms to the standard template. In step S303, if the data verification result is a failure, the user's data correction operation for the structured parsed data is received, and the structured parsed data is adjusted based on the data correction operation to obtain identification information. The data correction operation refers to the interactive process of manually adjusting the parsing results, which can be implemented using a visual annotation tool combined with a version control mechanism to correct abnormal data that cannot be identified in automated processing. When a field is missing or the format is abnormal, the error location is highlighted through the visual interface, and the parsed data is updated after receiving a manual correction instruction, such as supplementing missing clause numbers or adjusting table column width parameters. In step S304, if the data verification result is a pass, the structured parsed data is used as the identification information. If the data verification result is a pass, the structured parsed data is used as the identification information and subsequent operations are performed. Compared to traditional manual parsing, which requires operators to copy each clause to a CSV file, leading to potential omissions of clause numbers due to visual fatigue and the potential for disrupting the original structure when manually adjusting table formatting, this embodiment uses an automated parsing algorithm to accurately locate text elements, avoiding formatting misalignments caused by manual copying and pasting. For example, it automatically records row and column span information when parsing merged cells.The dual verification mechanism replaces manual sampling inspection. For example, the algorithm batch detects the continuity of clause numbers. Compared with manual verification, it can cover all data fields. The human-machine collaborative verification mode ensures processing efficiency and solves the structural anomaly problem of complex tables through visual correction tools. For example, the manual only needs to correct the special merged cell format that the algorithm cannot recognize. Thus, this embodiment realizes the automatic extraction and standardized output of key document information and eliminates the risk of missing clause numbers caused by manual copying and pasting. For example, in order to help understand the implementation process of the document processing method obtained by combining this embodiment with the above embodiment one, please refer to the following, which provides a schematic diagram of the module structure of a document processing method, specifically including a document intelligent retrieval module, a multimodal document processing module, a multilingual translation module, a format automatic conversion module, and a compliance archiving module. Based on the above module structure, please refer to the following, which provides a brief flowchart of a document processing method, specifically: (1) Document intelligent retrieval module. This module mainly achieves accurate retrieval through multi-source data acquisition and knowledge organization. In terms of data crawling, a differentiated strategy is adopted: incremental crawling is carried out on domestic sites at a timed frequency of once a day to ensure timely updates of domestic information; for international sites, an event-triggered crawling mechanism is adopted to focus on information related to key events. At the same time, a knowledge graph in the field of data security is constructed, and the core keywords are expanded with synonyms, such as expanding "data cross-border" to "data export" and "cross-border transmission rules" to enhance the comprehensiveness of the search; and Boolean search is supported to improve the flexibility and accuracy of the search. (2) Multilingual translation module. This module mainly includes: construction and management of professional terminology database: in the terminology collection stage, core terms are extracted from national standards and industry white papers through multi-source integration to form a bilingual comparison database. For example, international document terms such as GDPR, HIPAA, and APEC CBPR can be collected; a terminology tree is established according to the subdivided dimensions of the data security field to ensure the consistency of terminology context; a dynamic update mechanism is implemented, scanning international dynamics every quarter, automatically crawling new terms through crawlers, and incorporating them into the terminology database after review by domain experts. During terminology consistency checks, the translation engine prioritizes searching the terminology database. If it detects a mismatch between "Data Subject" and other terms, it will trigger a warning and halt the translation process, requiring manual confirmation before continuing. When inputting terms like "cross-border transmission," the terminology database will automatically recommend related terms to reduce the terminology mismatch rate. Translation model optimization: In terms of data augmentation, a large number of documents are used for training, with a focus on optimizing nested sentence structures using long sentences. Fake translation samples created by domain experts are introduced for adversarial training, allowing the model to identify and correct ambiguities in specialized terminology through adversarial learning.Syntactic analysis is deeply integrated to construct a semantic tag system. For example, in the legal industry, it can include syntactic tags such as "obligatory clause", "prohibitory clause" and "authorizing clause". "Should take necessary measures" belongs to the obligatory clause, and "shall take necessary measures" is used in the translation. "May decide on its own" belongs to the authorizing clause, and is translated as "may decide on its own". At the same time, logical connectives such as "but", "in addition" and "unless" are parsed through the syntactic tree to ensure that the logical relationship of the translated clause is consistent with the original text. Translation quality double verification: In the terminology consistency verification stage, after the translation is completed, regular expressions are used to match the keywords in the terminology library to check the translation consistency of the same term in the document. For example, if "data classification and grading" appears in both "Data Classification" and "Data Grading" in the document, it will be automatically marked and prompted to merge. When comparing across documents, the translation versions of the same type of document are matched with the terminology library to ensure the uniformity of industry terminology. At the same time, semantic similarity verification is carried out to ensure that the translated content is consistent with the semantics of the original text. (3) Multimodal document processing module. This module takes PDF parsing and layout analysis as its core and constructs a hybrid document processing framework. Please refer to the following: The multimodal recognition process can be: The multimodal recognition process begins with inputting a PDF or other format document and first determines whether it is a scanned document. If it is not a scanned document, the text is extracted directly. If it is a scanned document, the image quality is optimized by OCR parsing and tilt correction, noise removal and text enhancement in sequence. The table structure is identified through the TableBank model, and finally the chapter structure analysis is carried out to complete the structured parsing and extraction of the document content. In terms of table recognition optimization, the "row span + column span" dual marking method is used for merged cells. The recognition accuracy is improved by training a large number of manually annotated table samples. In the complex structure processing strategy, footnotes are identified by anchor point marking and automatically associated with the corresponding position in the main text. Attachments are located by keywords such as "appendix" and "attachment". The hierarchical relationship is preserved during conversion. For PDFs containing images, PaddleOCR is used for super-resolution processing before recognition, and a preset domain character set is used to improve the recognition rate of special symbols. (4) Format automatic conversion module. This module mainly includes: CSV mapping and rule engine design: The preset mapping rule library achieves accurate correspondence between PDF structural features and CSV fields. For example, the preset mapping rule library can be shown in Table 1 below: Table 1 Preset Mapping Rule Library Format Accuracy Verification: The structural integrity check verifies whether the CSV contains all required fields. If any are missing, they are marked in red and a prompt to complete them is displayed. Regarding format consistency verification, the date field is validated using the regular expression "^\d {4}-\d{2}-\d {2}$", and the format is automatically converted when an error is found. The clause number checks whether the expression "Article X" is standardized; if it is not standardized, a correction suggestion is displayed.A cross-document format unification mechanism can automatically synchronize CSV format rules for different versions of the same document, ensuring that historical documents are consistent with the latest document format. Error handling and manual intervention: An abnormal scenario circuit breaker mechanism is set up. When the PDF parsing error rate exceeds 15%, a manual intervention process is automatically triggered, prompting the user to upload a high-definition version or manually complete the content. A manual verification workbench is provided, displaying the original PDF on the left and the converted CSV on the right. Users can directly drag and drop to correct field mapping relationships, and the modified records are automatically saved to the rule base for optimizing subsequent conversion processes. (V) Compliance archiving module: This module is used to extract identification information based on the source document and associate the identification information with the target document input to a preset database. It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the document processing method of this application. More simple transformations based on this technical concept are all within the protection scope of this application. This application also provides a document processing device. The document processing device provided by this application, using the document processing method in the above embodiments, can solve the technical problem of low efficiency in compliance management caused by manual document archiving. The beneficial effects of the document processing apparatus provided in this application are the same as those of the document processing method provided in the above embodiments, and other technical features in the document processing apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here. This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the document processing method in the first embodiment above. The following reference shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs, PADs, PMPs, vehicle terminals, etc., and fixed terminals such as digital TVs, desktop computers, etc. The illustrated electronic device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application. As shown, the electronic device may include a processing device 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. Random access memory 1004 also stores various programs and data required for the operation of the electronic device. Processing device 1001, read-only memory 1002, and random access memory 1004 are interconnected via bus 1005. Input / output interface 1006 is also connected to the bus.Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays, speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication devices 1009 allow electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although electronic devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented or possessed alternatively. In particular, according to embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by the processing device 1001, it performs the functions defined in the method of the embodiments disclosed in this application. The electronic device provided in this application, employing the document processing method in the above embodiments, can solve the technical problem of low compliance management efficiency caused by manual document archiving. The beneficial effects of the electronic device provided in this application are the same as those of the document processing method provided in the above embodiments, and other technical features in the electronic device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here. It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples. The above descriptions are only specific embodiments of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims. This application provides a computer-readable storage medium having computer-readable program instructions stored thereon, which are used to execute the document processing method in the above embodiments. The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system or device, or any combination thereof.More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fiber, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF, etc., or any suitable combination thereof. The aforementioned computer-readable storage medium may be included in an electronic device; or it may exist independently and not assembled into an electronic device. The aforementioned computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to implement various embodiments of the document processing method described above. Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer. The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram can represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively represented blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. The modules described in the embodiments of this application can be implemented in software or hardware.The name of a module does not necessarily limit the unit itself. The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions for executing the above-described document processing method, which can solve the technical problem of low efficiency in compliance management caused by manual document archiving. The beneficial effects of the computer-readable storage medium provided in this application are the same as those of the document processing method provided in the above embodiments, and will not be repeated here. This application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the document processing method described above. The computer program product provided in this application can solve the technical problem of low efficiency in compliance management caused by manual document archiving. The beneficial effects of the computer program product provided in this application are the same as those of the document processing method provided in the above embodiments, and will not be repeated here. The above descriptions are only some embodiments of this application and do not limit the patent scope of this application. Any equivalent structural transformations made using the content of this application's specification and drawings under the technical concept of this application, or direct / indirect applications in other related technical fields, are included within the patent protection scope of this application.

Claims

1. A document processing method, characterized in that, The document processing method includes: Source documents are crawled from preset sites based on preset periods and preset search terms; The source document is parsed to obtain the document layout, and the layout of the source document is adjusted based on the document layout to obtain the target document; Based on the source document, extract identification information, and associate the identification information with the target document input and save it to a preset database; The step of parsing the source document to obtain a document layout and adjusting the layout of the source document based on the document layout to obtain the target document includes: Multimodal recognition processing is performed on the source document to identify text, image, and table elements in the source document, and element recognition results are obtained; Merge cells are marked on the table data using a dual marking method of row span and column span, resulting in the marked table data; The source document is subjected to structural association processing, which links footnotes to the main text by anchoring them and locates attachments by keywords while preserving hierarchical relationships, thus obtaining preliminary structured data; Perform text recognition enhancement processing on documents containing images to obtain recognizable text; By merging the preliminary structured data, the identifiable text, and the marked table data, structured layout information is obtained; Based on the parsed layout information, the source document is subjected to layout adjustment processing to obtain the target document; The step of extracting identifier information based on the source document includes: Based on the preset terminology database, the clause number, effective date, and table structure are parsed, and the source document is subjected to element parsing processing to obtain structured parsed data. Perform compliance verification on the structured parsed data, check the integrity and format compliance of the fields in the structured parsed data, and obtain the data verification result; If the data verification result is a verification failure, then the system receives the user's data correction operation on the structured parsed data, and adjusts the structured parsed data based on the data correction operation to obtain the identification information; If the data verification result is successful, then the structured parsed data will be used as the identification information.

2. The document processing method as described in claim 1, characterized in that, The steps of crawling source documents from preset sites based on preset periods and preset search terms to obtain source documents include: The preset search terms are expanded using a knowledge graph, and optimized search terms are obtained by expanding synonyms based on a knowledge graph in the data security field. Based on the optimized search terms, incremental source documents are obtained by performing timed incremental crawling on domestic websites, and event-triggered source documents are obtained by performing event-triggered crawling on international websites. The incremental source documents or event source documents are used as source documents.

3. The document processing method as described in claim 1, characterized in that, After the steps of parsing the source document to obtain the document layout and adjusting the layout of the source document based on the document layout to obtain the target document, the method further includes: Determine the error rate of the target document, and when the error rate exceeds a preset threshold, perform a visual comparison process on the source document and the target document in the visualization interface, and receive user modification operations based on the visualization interface; The target document is adjusted according to the modification operation described above, and the adjusted target document is updated to the target document.

4. The document processing method as described in claim 1, characterized in that, Before the step of parsing the source document to obtain the document layout, the method further includes: The source document and the preset terminology database are subjected to terminology matching and translation processing to obtain the initial translated text. The terminology database is obtained by extracting terms from national standards and industry white papers. The initial translation text is input into a preset translation model to obtain the translated source document, wherein the translation model is obtained through data augmentation and adversarial training to optimize the translation of nested sentences; The step of parsing the source document to obtain the document layout includes: The translated source document is parsed to obtain the document layout.

5. The document processing method as described in claim 4, characterized in that, Before the step of parsing the translated source document to obtain the document layout, the method further includes: The translated source document is subjected to a terminology consistency scan to obtain a terminology verification result, and the translated source document is subjected to a semantic similarity verification to obtain a semantic verification result. If the terminology verification result and the semantic verification result are both verified, then the step of parsing the translated source document to obtain the document layout is executed; If the terminology verification result and / or the semantic verification result are not verified, then the user's document correction operation on the translated source document is received, the translated source document is updated based on the document correction operation, and the step of parsing the translated source document to obtain the document layout is executed.

6. An electronic device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the document processing method as described in any one of claims 1 to 5.

7. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the document processing method as described in any one of claims 1 to 5.

Images