File processing method and device

Abstract

The invention relates to the technical field of information processing, and discloses a file processing method and device, and the method comprises the steps: obtaining a target file; the target file comprises a plurality of content units; the types of the content units comprise structured data and unstructured data; respectively performing semantic extraction on the plurality of content units, and correspondingly generating association features among the plurality of content units; classifying the target file based on the associated features to generate at least one classification tag; extracting domain information from the associated features according to the classification labels; and generating an abstract of the target file based on the field information, the classification tag and the association features among the plurality of content units. The problems of fuzzy file classification, scattered information extraction and insufficient abstract flexibility of a traditional method are solved. For example, the method can be applied to bidding document processing, accurate identification of relevance between technical parameters and business terms and generation of structured abstracts, and is also suitable for scenes such as contract review and technical document analysis.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a file processing method and device. Background Art

[0002] In complex document processing, documents such as technical specifications and contract agreements usually contain a mixture of structured and unstructured data. For example, technical documents contain both technical parameters in tabular form and functional descriptions in text paragraphs; commercial contracts contain both clause lists and free text supplementary explanations. Existing technologies have certain limitations.

[0003] Traditional methods rely on keyword matching or fixed templates, and it is difficult to identify semantic associations across data types. For example, the technical parameters in the table need to be associated with the acceptance clauses in the text, and the existing system cannot effectively model such logical relationships. Multi-dimensional information such as technology, business, and cost in the document is scattered in different content units. For example, in a bidding scenario, equipment performance parameters (technical attributes) and itemized quotations (cost attributes) need to be associated across tables and texts, and traditional methods lack unified processing capabilities. Users need to dynamically adjust the summary content according to their needs. For example, a technical review requires a detailed parameter summary, while a business decision requires an overview of the key points of the clauses, and existing technologies cannot be flexibly adapted.

[0004] Therefore, there is an urgent need for a document processing method that can overcome the shortcomings of traditional methods in document classification, information extraction, and summary flexibility. Summary of the invention

[0005] In view of this, the present invention provides a file processing method and device to solve the problems of fuzzy file classification, scattered information extraction and insufficient flexibility of summary in traditional methods. The technical solution is as follows: In a first aspect, the present invention provides a file processing method, the method comprising: Acquire a target file; the target file includes a plurality of content units; types of the content units include structured data and unstructured data; Semantic extraction is performed on a plurality of content units respectively, and correlation features between the plurality of content units are correspondingly generated; Classifying the target file based on the associated features and generating at least one classification label; According to the classification label, extract domain information from the associated features; A summary of the target document is generated based on domain information, classification labels, and association features between multiple content units.

[0006] A file processing method provided by an embodiment of the present invention systematically solves the core problems in file processing, especially bidding documents, through the synergistic effect of multi-dimensional technical features. First, by dividing the content units into structured and unstructured data and generating associated features, the limitations of the semantic separation of tables and texts in traditional methods are broken, so that the logical dependencies of cross-modal data can be fully captured, thereby improving classification accuracy; secondly, a multi-level classification mechanism based on associated features realizes fine-grained recognition of file attributes, avoiding misjudgment caused by semantic isolation of traditional single classification models; finally, a dynamic summary is generated by combining domain information and associated features, and through the dual mechanisms of logical association and domain adaptation, the accuracy of key data is retained, and a readable expression that meets user needs is generated, thereby reducing the cost of manual processing while meeting the differentiated needs of multiple roles such as evaluation experts and project managers for summary content.

[0007] In an optional embodiment, the structured data includes at least one of a table, a list of terms, and a technical parameter table; the unstructured data includes at least one of a free text paragraph and a descriptive text in an image.

[0008] An embodiment of the present invention provides a file processing method, in which structured data includes tables, clause lists, and technical parameter tables (such as itemized quotations), and unstructured data includes free text paragraphs (such as technical solution descriptions) and descriptive text in images (such as drawing annotations in scanned copies). A structured parsing algorithm (such as OCR-based table restoration) is used for tables, and a BERT model is used to extract semantics for free text, which greatly improves efficiency compared to the hybrid processing mode. Clearly defining the scope of structured data (such as a technical parameter table) can accurately locate key fields (such as unit price and total price), avoiding omissions caused by fuzzy definitions in traditional methods (for example, misjudging the "auxiliary machine quotation" in a nested table as ordinary text).

[0009] In an optional implementation, semantic extraction is performed on a plurality of content units respectively, and correlation features between the plurality of content units are generated accordingly, including: Jointly encode structured data and unstructured data to generate an initial semantic vector; Acquire logical associations between different content units, where the logical associations include at least one of parameter reference relationships, clause dependency relationships, and context semantic connection; Based on the logical association relationship, the initial semantic vector is modified to obtain the association features between multiple content units.

[0010] The embodiment of the present invention provides a file processing method, which maps table values ​​and text descriptions to a unified vector space, and corrects semantic vectors based on parameter reference relationships and clause dependencies. Joint coding solves the problem of semantic separation between tables and text. Logical association correction can identify the dependencies between clauses, avoid manual review of missing key constraints, make the association features more consistent with the business logic of the bidding documents, and improve the accuracy of subsequent classification and summary.

[0011] In an optional implementation manner, before performing semantic extraction on a plurality of content units respectively, the method further includes: Perform OCR analysis, table structure restoration, and text cleaning on the target file to generate standardized structured data and unstructured data.

[0012] A file processing method provided by an embodiment of the present invention processes nested tables in scanned documents / PDFs through OCR parsing and table restoration, and removes garbled characters and irrelevant formats through text cleaning. OCR parsing supports character recognition of scanned documents (such as scanned copies of paper tender documents), which solves the limitation that traditional methods only support electronic documents and expands the coverage of bidding scenarios. Table structure restoration ensures the complete parsing of technical parameter tables (such as quotations with nested multi-level headers) by identifying the relationship between rows and columns of the table (such as restoration of merged cells), avoiding field misalignment caused by complex structures of traditional table parsing tools. Text cleaning removes irrelevant characters (such as headers and page numbers) and abnormal spaces, improves the input quality of the semantic extraction model, and reduces misjudgments caused by noise interference (such as misidentifying "Page 3" as text content).

[0013] In an optional implementation, classifying the target file based on the associated features to generate at least one classification label includes: Analyze the semantic information of associated features through the pre-trained language model to identify the attribute labels of the target file; According to the attribute labels, a multi-label classification model is used to map the attribute labels to domain labels.

[0014] A file processing method provided by an embodiment of the present invention utilizes the contextual understanding ability of a language model to identify implicit business terms in technical attributes, thereby solving the problem of misclassification caused by semantic ambiguity in traditional classification models. Multi-label classification supports accurate labeling of multi-domain cross-content in bidding documents, meeting the needs for multi-dimensional information retrieval in bid evaluation. Multi-label classification supports multiple label labeling of the same file, solving the problem of insufficient label capacity of a single classification model. Domain labels drive subsequent processing logic.

[0015] In an optional implementation, the target document includes a bidding document or a tender document, and the method further includes: The semantic information of the associated features is analyzed through a pre-trained language model to distinguish the target file as a bidding document or a tender document, and generate a first-level classification label; and the attributes of the target file are annotated to generate a second-level classification label; the second-level classification label includes technical attributes, business attributes or cost attributes; the pre-trained language model includes the BERT model; According to the attribute labels, a multi-label classification model is used to map the attribute labels to domain labels; domain labels include materials, auxiliary machines, equipment, desulfurization, denitrification, host, and network and information systems; multi-label classification models include RoBERTa models; If the semantic information of the associated feature contains the bidding content and the technical parameter is a range value, then the target document is confirmed to be a bidding document; If the semantic information of the associated feature contains bidding content and the technical parameters are definite values, the target document is confirmed to be a bidding document.

[0016] The present invention provides a file processing method that quickly screens target files through primary classification, automatically identifies bidding documents based on range value features, and avoids confusion in processing logic caused by manual labeling errors. By determining the value, the bid response content is quickly located, and the bid evaluation experts are assisted in comparing the bidding requirements with the bid commitments, thereby improving the bid evaluation efficiency.

[0017] In an optional implementation, extracting domain information from associated features includes: Generate an attention weight matrix based on the classification label; The weight of each feature vector is obtained by weighting the associated features through the multi-head self-attention mechanism; The weight of each feature vector is compared with the weight threshold, and the target feature vector is extracted from each feature vector as the domain information according to the comparison result.

[0018] The document processing method provided by the embodiment of the present invention dynamically adjusts the attention weight through classification labels, focuses on key content units, and suppresses the interference of irrelevant information. The multi-head self-attention mechanism captures the semantic associations of different dimensions, accurately extracts the core information of the field, and supports the generation of high-quality summaries.

[0019] In an optional implementation, generating a summary of the target file includes: Determine the summary template based on the domain information and classification labels; The template content is filled based on the associated features of the plurality of content units to generate a summary of the target document.

[0020] The present invention provides a method for processing documents, which automatically matches templates based on domain tags to ensure that the summary content is highly adapted to user needs (such as bid evaluation experts focusing on technical details). It integrates information across data units (such as parameters in a table + clauses in a text) into the same summary, avoiding the information fragmentation problem of traditional summary generation modules (such as only extracting table fields and ignoring associated text).

[0021] In an optional implementation, filling the template content based on the associated features of the plurality of content units includes: The TextRank algorithm is used to filter the correlation features between multiple content units, select high-weight text paragraphs and table fields, and generate extractive summaries; The extractive summary is fed into a Seq2Seq model to generate a natural language text summary.

[0022] The document processing method provided by the embodiment of the present invention ensures that the summary contains the core information of the bidding document (such as key performance indicators and quotation amount) by calculating the importance of content units (such as the weight of the technical parameter table is higher than that of the conventional description text), thereby solving the problem of missing key points caused by traditional random extraction. Seq2Seq generation converts the extractive summary into coherent natural language, improves the readability of the summary, and meets the quick reading needs of non-technical users.

[0023] In summary, the file processing method provided by the present invention, first, standardizes multi-format files through data preprocessing to provide high-quality input for subsequent processing; secondly, joint coding and logical association correction break through the semantic barriers between structured and unstructured data, so that the multi-level classification model can achieve accurate classification based on fine-grained semantics; further, the attention weight mechanism is combined with domain label mapping to ensure that information extraction focuses on the core content; finally, hybrid summary generation, through template filling and natural language conversion, not only retains the accuracy of key data, but also generates readable expressions that meet user scenarios. The synergistic effect of the above technical features enables the system to achieve breakthrough improvements in classification accuracy, information extraction completeness and summary generation efficiency, while being compatible with multi-format files and large-scale real-time processing requirements.

[0024] In a second aspect, the present invention provides a file processing device, the device comprising: An acquisition module is used to acquire a target file; the target file includes a plurality of content units; the types of the content units include structured data and unstructured data; A parsing module, used to extract semantics from the plurality of content units respectively, and to generate correlation features between the plurality of content units accordingly; A classification module, used for classifying the target file based on the associated feature and generating at least one classification label; An extraction module, used for extracting domain information from the associated features according to the classification label; The generation module is used to generate a summary of the target file based on the domain information, the classification label and the association features between the multiple content units.

[0025] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the file processing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0026] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the file processing method of the first aspect or any corresponding embodiment thereof.

[0027] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the file processing method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 is a flowchart of a file processing method according to an embodiment of the present invention; Figure 2 It is a flowchart of a bidding document processing method according to an embodiment of the present invention; Figure 3 is a structural block diagram of a file processing device according to an embodiment of the present invention; Figure 4 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0031] In the field of complex document processing technology, traditional file classification and information extraction methods have long faced multi-dimensional technical bottlenecks. Existing systems are usually based on single-modal data processing (such as processing only text or tables), resulting in a break in cross-modal logical associations. The core problem of existing technologies is that most solutions only support primary classification (such as distinguishing bidding / tender documents) and cannot achieve multi-level mapping of "file type-attribute category-domain label". Entity recognition methods based on keyword matching (such as regular expressions) are difficult to capture implicit relationships in nested structures. Existing summary algorithms (such as template-based filling) cannot dynamically adapt to user role requirements. Bidding evaluation experts need to focus on technical parameter comparisons, while financial personnel focus on cost structures, but traditional systems can only generate fixed-format summaries, resulting in information overload or omission of key data.

[0032] The above defects are particularly acute in the typical application scenario of bidding document processing. Bidding documents and tender documents usually contain a large amount of complex structured and unstructured information. Bidding documents usually include technical requirements, commercial terms and other instructions, while tender documents include commercial quotations, technical solutions and price lists. These documents are not only long in length, but also have diverse structures. The manual review and classification process is time-consuming and error-prone. Especially in large projects, the amount of documents submitted by multiple bidders is very large, which further increases the burden of manual processing.

[0033] The limitations of existing technologies are particularly significant in such bidding document processing scenarios, and they are usually unable to meet the specific needs of bidding documents. Specifically, it is difficult for existing methods to accurately classify documents based on the contextual semantics of the document content, especially when distinguishing between technical, commercial, and price documents. There are often confusions in the bidding and tender documents, such as supplier names, key technical parameters, quoted amounts, etc. This information is often nested in complex sentences or tables, and simple rule-based methods have difficulty extracting complete and accurate data. Traditional methods cannot effectively generate document summaries for specific needs, such as extracting only technical details related to a specific field (such as desulfurization equipment parameters).

[0034] Therefore, in order to solve the deficiencies of the prior art in complex document processing, an embodiment of the present invention provides a file processing method that can solve the problems of fuzzy file classification, scattered information extraction and insufficient flexibility of summarization in traditional methods.

[0035] The method flow of a file processing method provided by an embodiment of the present invention is as follows: Figure 1 As shown, the following steps are included.

[0036] S101, obtaining a target file; the target file includes a plurality of content units; the types of content units include structured data and unstructured data.

[0037] Specifically, the target file in S101 is the electronic document to be processed, including but not limited to PDF, Word, scanned images and other formats. Content unit is the smallest logical unit in the document that is segmented and processed, and is divided into structured data and unstructured data. Structured data is data with a fixed format, such as a table (a collection of data aligned in rows and columns), a list of terms (a collection of items with numbers / symbols), and a technical parameter table (a combination of parameter name-value-unit). Unstructured data is a free text paragraph without a fixed format or a descriptive text in an image (such as a text description in a scanned document).

[0038] S102: Perform semantic extraction on a plurality of content units respectively, and generate correlation features between the plurality of content units accordingly.

[0039] Specifically, in S102, semantic extraction is to perform semantic analysis on each content unit to extract its core meaning. For example, the table field "unit price: ￥100" is extracted as "price entity-value 100-currency unit ￥". The text paragraph "the supplier promises to deliver within 10 days" is extracted as "delivery terms-time 10 days-responsible entity supplier". The association feature refers to the logical or semantic association relationship between different content units. For example, the "total price = unit price × quantity" in the table and the "quotation includes taxes and fees" in the text form a numerical dependency relationship. The "payment terms: 30% advance payment" in the terms list forms a terms reference relationship with the "performance bond template" in the attachment.

[0040] S103: Classify the target file based on the associated features and generate at least one classification label.

[0041] Specifically, in S103, the classification label is a category identifier that describes the file attributes. For example, the first-level classification: file type label (such as "bidding document" and "bid document"). The second-level classification: file attribute label (such as "technical document", "business document" and "price document"). The third-level classification: field label (such as "equipment field" and "material field"). The classification basis is to determine the classification logic according to the type and distribution of the associated features. For example, if the technical parameters and clause reference relationship in the associated features are dominant, it is classified as "technical document".

[0042] S104. Extracting domain information from associated features based on classification labels.

[0043] Specifically, in S104, the domain information is the professional field or subject information to which the file content belongs. For example, if the classification label is "technical documents" and "desulfurization efficiency" and "sulfur capacity" appear frequently in the associated features, the domain information is "desulfurization equipment". The domain information is extracted by screening the associated features related to the classification label.

[0044] S105: Generate a summary of the target file based on the domain information, the classification tags, and the association features between the multiple content units.

[0045] Specifically, in S105, the domain information, classification labels and associated features are integrated to output a concise document summary. The focus of the summary (such as technical parameters, business terms) is determined based on the classification labels, key content is extracted through associated features, and sentences are organized according to natural language rules.

[0046] A file processing method provided by an embodiment of the present invention achieves a systematic improvement in the efficiency of complex document processing through the synergistic effect of progressive technical features. Firstly, by splitting the target file into structured data and unstructured data, a differentiated parsing logic is designed to solve the format conflict and information omission problems caused by mixed processing in traditional methods. Secondly, semantic extraction is performed on each content unit and cross-unit correlation features are constructed, which breaks the limitation of data islands in traditional methods and forms a semantic network for key information scattered in tables and texts, providing a multi-dimensional contextual basis for subsequent processing. On this basis, multi-level classification labels are generated based on the distribution law of correlation features, and fine-grained classification is achieved by dynamically perceiving contextual semantics to avoid the risk of misjudgment in traditional single-label classification. Furthermore, the classification labels are combined to filter the core information of the domain in the correlation features, and redundant data interference is accurately eliminated to ensure the targeting of information extraction. Finally, a dynamic summary is generated by integrating domain information, classification labels and correlation features, and the focus of the summary is adaptively adjusted according to the classification results. The correlation features are used to ensure logical coherence, so as to output a highly readable summary that meets the needs of multiple roles while retaining data accuracy, greatly reducing the time spent on manual review, and systematically solving the core problems of information fragmentation, extensive classification, and rigid summary in traditional document processing.

[0047] Optionally, in the above steps, structured data includes tables, clause lists, and technical parameter tables (such as itemized quotations), and unstructured data includes free text paragraphs (such as technical solution descriptions) and descriptive text in images (such as drawing annotations in scanned copies). Using structured parsing algorithms (such as OCR-based table restoration) for tables and using the BERT model to extract semantics for free text greatly improves efficiency compared to the hybrid processing mode. Clearly defining the scope of structured data (such as technical parameter tables) can accurately locate key fields (such as unit price and total price), avoiding omissions caused by fuzzy definitions in traditional methods (for example, misjudging the "auxiliary machine quotation" in a nested table as ordinary text).

[0048] Optionally, in the above step S102, semantic extraction is performed on several content units respectively, and the correlation features between the multiple content units are generated accordingly, specifically including: joint encoding of structured data and unstructured data to generate an initial semantic vector; obtaining the logical association relationship between different content units, the logical association relationship includes at least one of parameter reference relationship, clause dependency relationship and context semantic connection; based on the logical association relationship, the initial semantic vector is corrected to obtain the correlation features between the multiple content units. The table values ​​and text descriptions are mapped to a unified vector space, and the semantic vector is corrected based on the parameter reference relationship and clause dependency relationship. Joint encoding solves the problem of semantic separation between tables and texts. Logical association correction can identify the dependency relationship between clauses, avoid manual review from missing key constraints, make the correlation features more in line with the business logic of the bidding documents, and improve the accuracy of subsequent classification and summary.

[0049] Before semantic extraction of several content units, a preprocessing process is also included, in which the target file is subjected to OCR parsing, table structure restoration, and text cleaning to generate standardized structured data and unstructured data. Nested tables in scanned documents / PDFs are processed through OCR parsing and table restoration, and garbled characters and irrelevant formats are removed through text cleaning. OCR parsing supports character recognition of scanned documents (such as scanned copies of paper bidding documents), which solves the limitation that traditional methods only support electronic documents and expands the coverage of bidding scenarios. Table structure restoration ensures the complete parsing of technical parameter tables (such as quotations with nested multi-level headers) by identifying the relationship between table rows and columns (such as merging cell restoration), avoiding field misalignment caused by complex structures of traditional table parsing tools. Text cleaning removes irrelevant characters (such as header page numbers) and abnormal spaces, improves the input quality of semantic extraction models, and reduces misjudgments caused by noise interference (such as misidentifying "Page 3" as text content).

[0050] Optionally, in the above step S103, the target file is classified based on the associated features to generate at least one classification label, including: analyzing the semantic information of the associated features through a pre-trained language model to identify the attribute labels of the target file; and mapping the attribute labels to domain labels using a multi-label classification model based on the attribute labels.

[0051] Taking the target file as a bidding document including bidding documents and tender documents as an example, the pre-trained language model is used to analyze the semantic information of the associated features, distinguish the target file as a bidding document or a tender document, and generate a first-level classification label; and the attributes of the target file are annotated to generate a second-level classification label; the second-level classification label includes technical attributes, business attributes or cost attributes; the pre-trained language model includes the BERT model; according to the attribute label, a multi-label classification model is used to map the attribute label to the domain label; the domain label includes materials, auxiliary machines, equipment, desulfurization, denitrification, host, and network and information systems; the multi-label classification model includes the RoBERTa model; among them, if the semantic information of the associated features contains bidding content and the technical parameters are range values, the target file is confirmed to be a bidding document; if the semantic information of the associated features contains bidding content and the technical parameters are definite values, the target file is confirmed to be a tender document. The context understanding ability of the language model is used to identify the implicit business terms in the technical attributes, which solves the misclassification caused by semantic ambiguity in the traditional classification model. Through multi-label classification, the accurate annotation of multi-domain cross-content in the bidding documents is supported to meet the needs of multi-dimensional information retrieval in bid evaluation. Multi-label classification supports multiple label annotations for the same file, solving the problem of insufficient label capacity of a single classification model. Quickly screen target files through primary classification, automatically identify bidding documents based on range value features, and avoid confusion in processing logic caused by manual labeling errors. Quickly locate bid response content by determining values, assist bid evaluation experts in comparing bidding requirements with bid commitments, and improve bid evaluation efficiency.

[0052] Optionally, in the above step S104, domain information is extracted from associated features, specifically including: generating an attention weight matrix according to classification labels; performing weighted calculation on associated features through a multi-head self-attention mechanism to obtain the weight of each feature vector; comparing the weight of each feature vector with a weight threshold, and extracting a target feature vector from each feature vector as domain information according to the comparison result. Dynamically adjust the attention weight through classification labels to focus on key content units and suppress interference from irrelevant information. The multi-head self-attention mechanism captures semantic associations of different dimensions, accurately extracts core domain information, and supports the generation of high-quality summaries.

[0053] Optionally, in the above step S105, the summary template is determined according to the domain information and classification labels; the template content is filled based on the associated features of multiple content units to generate a summary of the target file. Automatically match the template based on the domain label to ensure that the summary content is highly adapted to user needs (such as the evaluation experts pay attention to technical details). Integrate information across data units (such as parameters in the table + clauses in the text) into the same summary to avoid the information fragmentation problem of the traditional summary generation module (such as only extracting table fields and ignoring associated text). Specifically, the TextRank algorithm is used to screen the associated features between multiple content units, filter out high-weight text paragraphs and table fields, and generate an extractive summary; the extractive summary is input into the Seq2Seq model to generate a natural language text summary. By calculating the importance of the content unit (such as the weight of the technical parameter table is higher than that of the conventional description text), it is ensured that the summary contains the core information of the bidding document (such as key performance indicators and quotation amount), solving the problem of missing key points caused by traditional random extraction. Seq2Seq generation converts the extractive summary into a coherent natural language, improves the readability of the summary, and meets the needs of non-technical users for fast reading.

[0054] In summary, the file processing method provided by the embodiment of the present invention, first, standardizes multi-format files through data preprocessing to provide high-quality input for subsequent processing; secondly, joint encoding and logical association correction break through the semantic barriers between structured and unstructured data, so that the multi-level classification model can achieve accurate classification based on fine-grained semantics; further, the attention weight mechanism is combined with domain label mapping to ensure that information extraction focuses on the core content; finally, hybrid summary generation, through template filling and natural language conversion, not only retains the accuracy of key data, but also generates readable expressions that meet user scenarios. The synergistic effect of the above technical features enables the system to achieve breakthrough improvements in classification accuracy, information extraction completeness and summary generation efficiency, while being compatible with multi-format files and large-scale real-time processing requirements.

[0055] By way of example, the following will use the typical application scenario of bidding document processing to explain in detail the file processing method provided in the above embodiment. Based on the file processing method provided in the above embodiment, a processing system based on bidding documents is constructed to classify bidding documents and extract summaries.

[0056] The bidding document processing system includes: a data preprocessing module, a classification module, an information extraction module, a summary generation module, a domain mapping module and a user interaction and visualization module.

[0057] The data preprocessing module is used for file parsing, text cleaning and generating structured data. It supports parsing of PDF, Word and other format files, and processes scanned documents in combination with OCR technology. It removes irrelevant characters, spaces and format anomalies, extracts plain text content, identifies tables and nested format content, and converts them into parsable structured data.

[0058] The classification module is used for preliminary classification of documents and secondary classification. It uses deep learning models (such as BERT or RoBERTa) to classify documents and distinguish between bidding documents and tender documents. Based on the preliminary classification, it is further subdivided into technical, commercial and price documents, and the corresponding subdivided modules (such as materials, auxiliary machines, etc.) are marked.

[0059] The information extraction module is used to extract key information such as supplier name, bid amount, technical indicators, etc. through the BERT-CRF model. Combined with the BiLSTM+Attention model, it can extract complex relationships such as payment terms and delivery time. Logically associate multiple pieces of information to generate a complete set of key information.

[0060] The summary generation module is used to calculate the importance of sentences and generate extractive summaries using the TextRank algorithm. It uses pre-trained generation models (such as T5 and GPT) to generate fluent and coherent abstract summaries. It also supports adjusting the summary length and granularity based on user needs to meet the needs of different scenarios.

[0061] The domain mapping module uses a multi-label classification model to map document content into seven major areas (materials, auxiliary equipment, equipment, desulfurization, denitrification, host, network and information system). It generates specific content summaries for each field and supports customized processing of bid evaluation requirements.

[0062] The user interaction and visualization module is used to provide an intuitive graphical interface, support file upload, classification results and summary viewing. It supports exporting results in formats such as JSON and Excel for subsequent evaluation and recording. It achieves efficient file classification and summary extraction through asynchronous processing and provides dynamic result display.

[0063] The processing flow of the bidding document processing system is as follows Figure 2 As shown, it includes the following processes: Users upload bidding documents or tender documents; the data preprocessing module performs format parsing and text cleaning on the documents; the classification module identifies document categories and further subdivides them; the information extraction module extracts key information and aggregates related content; the summary generation module generates high-quality summaries and supports user customization; the domain mapping module completes content classification and domain labeling; users view the results through the interface and export or adjust the summary as needed.

[0064] The functional implementation of each module will be described in detail below.

[0065] The functions of the data preprocessing module include OCR parsing, table parsing, and text cleaning.

[0066] OCR parsing uses the Tesseract OCR model for character recognition, combined with deep learning models (such as CRAFT) to improve the character recognition accuracy of scanned documents. The final output of OCR parsing is structured plain text data or table images.

[0067] Character recognition can be viewed as a classification problem. Assuming that a given image segment I, the OCR model outputs a probability distribution expressed as: ; Where c is the possible character category, I is the image to be recognized, the softmax function converts the result of the linear transformation into a probability distribution, f(I) is the image feature extractor (such as the feature vector extracted by CNN), and W and b are the weight matrix and bias of the model.

[0068] The CRAFT (Character Region Awareness for Text Detection) model achieves more accurate text extraction by detecting the boundaries of text regions. The loss function of the detection region is: ; Where Lregion is the text region detection loss, Laffinity is the character association loss, and λ1 and λ2 are weight hyperparameters.

[0069] Table parsing uses deep learning models such as TabularNet to identify complex table structures and generate parsable structured data. It recursively parses nested tables to extract the logical relationship between table headers and data.

[0070] Based on the TabularNet model, the rows, columns, and cell boundaries of the table are detected. The prediction process is as follows: ; Where S represents the boundary of the table unit (such as the coordinates of the upper left corner and the lower right corner), Ws and bs are the weight and bias of the table detection layer, respectively, and f(I) is the image feature.

[0071] For nested tables, a recursive parsing strategy is used to detect the nested area R from the main table and use R as the input of the subtable; the above table detection process is repeated for the subtable until the parsing is completed. Recursive formula for recursive parsing: ; In the formula, Rk represents the k-th nested region, f detect Detection function for the table.

[0072] Analyze the mapping relationship between header and data cell based on rule engine or neural network: ; Where M(h,d) represents the mapping relationship between the table header h and the data cell d, the argmax function returns the independent variable value corresponding to the maximum value of the sim function, H and D are the table header set and the data cell set respectively, and sim(h,d) is the similarity function (such as cosine similarity or deep matching model).

[0073] Text cleaning is a cleaning module based on regular expressions and rule engines. It removes irrelevant characters and unifies the text format. Use regular expression R1 to delete invalid characters. The expression is: ; In the formula, R1 is a regular expression that matches invalid characters, and Re.sub is a regular matching replacement function.

[0074] The date format is processed uniformly, the expression is: ; Where R2 is the regular expression matching rule for the date format, and YYYY-MM-DD is the universal date format of year, month, and day.

[0075] The classification module uses a pre-trained language model (such as BERT) for fine-tuning and classification. The first-level classification is to distinguish between bidding documents and tender documents. The second-level classification is further subdivided into technical, commercial and price documents based on the content.

[0076] The text input generates feature representation through the BERT model, the expression is: ; In the formula, x i represents the i-th sentence or paragraph of the input, h i Represents the context feature vector output by BERT.

[0077] Use the classifier to predict the feature vector, the expression is: ; Where W and b are the weight matrix and bias vector parameters of the classifier, and P(y|x) is the classification probability distribution.

[0078] Using the multi-task learning method, the primary classification and secondary classification models are jointly trained, and the expression is: ; In the formula, α and β are task weights, L 一级分类 and L 二级分类 is the cross entropy loss for classification tasks.

[0079] The attention mechanism is introduced to optimize the classification results, especially for documents with multiple topics. The important context is extracted through the attention mechanism, and the expression is: ; In the formula, is the weight matrix, for the eigenvector Perform a linear transformation.

[0080] The extracted feature vector can be expressed as: ; In the formula, is the attention weight, and H is the weighted feature representation.

[0081] The specific classification steps are as follows: First, the weighted feature representation H obtained by the attention mechanism has integrated the important information of each part of the document. The weighted feature representation H is input into the fully connected layer of the classifier, and the expression is: ; In the formula, is the weight matrix of the primary / secondary classifier, is the bias vector.

[0082] Then, the softmax function is used to convert the resulting vector into a probability, expressed as: ; Finally, according to the probability distribution output by the softmax function, the category with the largest probability is selected as the final classification result. The expression is: ; This classification process can be used to quickly classify bidding documents, for example, the technical proposal and quotation documents of the bidding documents are classified and then submitted to the review experts respectively. It can improve the classification accuracy for multi-topic scenarios of complex documents (such as those involving both technology and business).

[0083] The specific extraction process of the information extraction module is as follows: First, the BERT-CRF model is used in combination with the domain dictionary for entity recognition. The recognition categories include: supplier name, amount, date, technical parameters, etc.

[0084] Specifically, after using BERT to extract context features, the CRF layer is used to predict the entity label sequence, expressed as: ; In the formula, s t (y t ) is the BERT pair label y t The score, A[y t−1 ,y t ] is the transfer score between labels.

[0085] Subsequently, relationship extraction is performed based on the dual-tower architecture relationship extraction model (BiLSTM+Attention).

[0086] Specifically, assume that the input entity pair is: (e1, e2); use BiLSTM to obtain the contextual features of the entity, the expression is: ; In the formula, It is the feature representation of text words.

[0087] The context between entities is weighted through the attention mechanism, and the expression is: ; Using the classifier to predict the relationship, the expression is: ; In the formula, and are the weight matrix and bias vector of the classifier.

[0088] Finally, a graph neural network (GNN) is used to associate scattered information across segments to form a complete context.

[0089] Specifically, create a paragraph node and The relationship is expressed as: ; Where N(i) is the node The set of neighbor nodes of It means that after k+1 iterations, the node The characteristic vector of is the weight matrix. is the activation function, and the obtained result is transformed nonlinearly.

[0090] The summary generation module includes extractive summary generation and abstract summary generation, and supports dynamic summary adjustment.

[0091] Extractive summary generation uses the TextRank algorithm, combined with sentence vector similarity calculation, to filter out high-importance sentences. The expression is: ; In the formula, In( ) is the point to node The set of all nodes of Out( ) is a node The set of all nodes pointed to, For Node To Node The weight of the edge, For Node The weights of all node edges pointing to , and d is the damping coefficient.

[0092] Abstractive summary generation is based on Seq2Seq models (such as T5) to generate natural language summaries.

[0093] Specifically, the generation quality is optimized through reinforcement learning and trained in combination with the ROUGE indicator, expressed as: ; In the formula, T represents the length of the generated sequence, and t is a time step index, which takes values ​​from 1 to T. is the real word (or tag) generated at time step t. P represents the generated word sequence given the input sequence and the generated word sequence. probability.

[0094] Dynamic summaries are adjusted based on user needs and can generate long or short summaries.

[0095] The domain mapping module uses the RoBERTa model for multi-label classification and maps documents to seven domains. The expression of the document is: ; The domain mapping expression is: ; The RoBERTa model can also be fine-tuned using domain data to improve the classification accuracy in small sample domains. The fine-tuning expression is: ; In the formula, N represents the number of samples, represents the true label of the i-th sample. Represents the predicted probability of the model for the i-th sample.

[0096] The specific process of fine-tuning is as follows: Collect a large number of bidding documents from public resource trading platform channels and screen and mark them.

[0097] Select the RoBERTa model, load the weights of the pre-trained model, set relevant hyperparameters, select an appropriate learning rate range and dynamically monitor and adjust it, set the batch size reasonably based on computing resources and data volume, and determine the appropriate number of training rounds through cross-validation and other methods.

[0098] Introduce L1 or L2 regularization to constrain the parameters of the model, prevent overfitting, improve the generalization ability of the model, and adopt corresponding data enhancement methods for different types of data.

[0099] For small sample data, some layers are usually frozen first, and only the last few layers are fine-tuned. After the model is stable, more layers are gradually released for fine-tuning. The prepared domain data is input into the model for training, and the performance indicators of the model, such as accuracy, recall, F1 value, etc., are monitored to adjust the training strategy in time.

[0100] Use the test set to evaluate the trained model, calculate various performance indicators, and fully understand the model's classification ability in the field of small samples. Analyze the evaluation results, find out the problems with the model, and optimize it by adjusting hyperparameters, improving data enhancement methods, changing model structure, etc., and deploy the optimized model to the actual application environment to ensure that the model can run stably.

[0101] The user interaction and visualization module displays classification, information extraction and summary generation results through a visual interface, and supports export in multiple formats (such as JSON and Excel). Based on the asynchronous processing framework, users can view the classification results and summary generation progress in real time after uploading files.

[0102] In addition, the model parameters in each module can be adjusted based on user evaluation, especially the model used in the classification module, to continuously improve the classification and generation effects.

[0103] Specifically, if users report that the model's classification effect on new data fluctuates greatly, or the generated content is unstable, it may be that the learning rate is too large and needs to be appropriately reduced, such as from the current 0.001 to 0.0001. This will make the model take smaller steps when updating parameters, converge more robustly, and improve the stability of classification and generation. On the contrary, if the model learns slowly when faced with new and complex bidding document types, and it is difficult to quickly adapt and accurately classify or generate, you can try to increase the learning rate moderately, such as from 0.0001 to 0.0005, to speed up the model's learning of new features.

[0104] When users report that there are concentrated errors in the classification of bidding documents for specific large-scale projects and the training time is long, you can consider increasing the batch size. For example, increasing it from 32 to 64, so that the model can calculate based on more data each time the parameters are updated, reduce the variance of the gradient estimate, and improve training efficiency. If the classification results vary significantly between different batches, it indicates that the model is sensitive to data batches. The batch size can be reduced to 16 to increase the randomness of the training, so that the model can better learn the data features and improve the generalization ability.

[0105] If the user feedback model performs well on known data, but does not perform well in classifying or generating new bidding documents, there may be an overfitting problem. At this time, the regularization parameter should be increased, such as increasing the L2 regularization coefficient from 0.001 to 0.01, to impose stronger constraints on the model parameters and reduce the risk of overfitting. If the model performs poorly on all types of data, it may be underfitting, and the regularization parameter needs to be reduced, such as to 0.0001, so that the model has more degrees of freedom to learn data features.

[0106] If users provide feedback that the model does not accurately classify bidding documents with complex structures and numerous clauses, or that the generated summary fails to fully cover key information, you can try to increase the number of hidden layer neurons to give the model stronger feature learning and expression capabilities. For example, increase the number of hidden layer neurons from 128 to 256. If the model training time is too long and the classification and generation effects are not significantly improved, you can appropriately reduce the number of hidden layer neurons, such as from 256 to 192, to simplify the model structure and improve training efficiency.

[0107] In summary, the bidding document processing system provided in this example realizes the automatic classification of bidding documents and tender documents through deep learning technology, which significantly improves efficiency and reduces error rate compared with the traditional method that relies on manual screening. The system can subdivide and classify technical, commercial and price documents according to the content of the documents, providing accurate support for users to quickly locate key information. At the same time, the system further refines the classification into specific fields (such as materials, auxiliary machines, equipment, etc.), making the classification results more targeted and convenient for different review experts to handle the division of labor. This function provides a reliable solution for batch processing of large-scale bidding documents and greatly reduces the time cost. Accurate extraction of key information in the document (such as supplier name, amount, date, technical parameters, etc.) is achieved. With the help of cross-segment information aggregation technology, the system can integrate relevant data scattered in different paragraphs into logically consistent structured content to ensure the comprehensiveness and consistency of information extraction. Compared with traditional manual extraction, the present invention significantly reduces human omissions and provides high-quality data support for review experts, especially in the analysis of complex documents and tables. By combining extractive summarization with abstract summarization, the present invention provides users with flexible and diverse summary generation capabilities. Extractive summarization uses algorithms to automatically filter important sentences in documents and generate condensed versions of high-weight content, which is suitable for quick browsing. Abstractive summarization uses generative models to generate fluent and natural text, which is convenient for interpreting complex technical solutions or business terms. In addition, users can adjust the length and granularity of the summary according to their needs, and obtain content that fits the usage scenario in real time, thereby greatly improving decision-making efficiency and user experience. Through multi-label classification technology and domain mapping models, the document content is accurately divided into seven key areas (such as materials, auxiliary machines, equipment, etc.). This domain-specific support enables the system to automatically adapt to the needs of different industries and professions, enhancing the practicality of classification and summary results. The domain mapping module also combines adaptive optimization technology to improve the confidence and interpretability of classification, providing efficient auxiliary tools for domain experts to assign files, and significantly accelerating the review process. The system also sets up an intuitive user interface, allowing users to complete tasks such as file uploading, classification result viewing, and summary generation through simple operations. The system supports the export of results in multiple formats (such as JSON and Excel) to facilitate subsequent data processing. At the same time, the real-time feedback function allows users to dynamically adjust the classification and summary strategies during the operation to ensure that the output results match the actual needs. This user-friendly design significantly lowers the threshold for using the system, making it easy for both technical experts and business personnel to get started, greatly improving the overall user experience. It supports parsing of multiple file formats (such as PDF, Word, images, etc.), and is compatible with a variety of document types submitted by different industries and suppliers, including complex tables and nested structures.The modular design enables the system to be flexibly expanded according to user needs, such as adding new parsing rule bases or adjusting generation models to adapt to future diversified business needs. This feature not only ensures the stability of the system in the current application scenario, but also provides technical support for future industry expansion.

[0108] In the present embodiment, a file processing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0109] This embodiment provides a file processing device, such as Figure 3 As shown, including: The acquisition module 301 is used to acquire a target file; the target file includes a plurality of content units; the types of the content units include structured data and unstructured data; The parsing module 302 is used to extract semantic information from the plurality of content units and generate correlation features between the plurality of content units; A classification module 303, configured to classify the target file based on the associated feature and generate at least one classification label; An extraction module 304, configured to extract domain information from the associated features according to the classification label; The generating module 305 is used to generate a summary of the target file based on the domain information, the classification label and the association features between the multiple content units.

[0110] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0111] The file processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0112] The embodiment of the present invention also provides a computer device having the above Figure 3 The file processing device shown.

[0113] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0114] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0115] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0116] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0117] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0118] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0119] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device may be a touch screen.

[0120] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0121] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0122] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A file processing method, characterized in that: The method comprises: Acquire a target file; the target file includes a plurality of content units; the types of the content units include structured data and unstructured data; Performing semantic extraction on the plurality of content units respectively, and correspondingly generating association features between the plurality of content units; Classify the target file based on the associated features and generate at least one classification label; Extracting domain information from the associated features according to the classification labels; A summary of the target file is generated based on the domain information, the classification label, and the association features between the plurality of content units.

2. The method according to claim 1, characterized in that: The structured data includes at least one of a table, a list of terms and a technical parameter table; the unstructured data includes at least one of a free text paragraph and a descriptive text in an image.

3. The method according to claim 2, characterized in that The performing semantic extraction on the plurality of content units respectively and correspondingly generating association features between the plurality of content units includes: Jointly encode structured data and unstructured data to generate an initial semantic vector; Acquire a logical association relationship between different content units, wherein the logical association relationship includes at least one of a parameter reference relationship, a clause dependency relationship, and a context semantic connection; Based on the logical association relationship, the initial semantic vector is modified to obtain association features between multiple content units.

4. The method according to claim 3, characterized in that Before extracting the semantics of the plurality of content units respectively, the method further includes: The target file is subjected to OCR analysis, table structure restoration and text cleaning to generate standardized structured data and unstructured data.

5. The method according to claim 3, characterized in that: The step of classifying the target file based on the associated features to generate at least one classification label includes: Analyzing semantic information of the associated features through a pre-trained language model to identify attribute labels of the target file; According to the attribute labels, a multi-label classification model is used to map the attribute labels to domain labels.

6. The method according to claim 5, characterized in that The target document includes a bidding document or a tender document, and the method further includes: Analyze the semantic information of the associated features through a pre-trained language model, distinguish the target file as a bidding document or a tender document, and generate a primary classification label; and annotate the attributes of the target file to generate a secondary classification label; the secondary classification label includes a technical attribute, a business attribute or a cost attribute; the pre-trained language model includes a BERT model; According to the attribute labels, a multi-label classification model is used to map the attribute labels to domain labels; the domain labels include materials, auxiliary machines, equipment, desulfurization, denitrification, host, and network and information systems; the multi-label classification model includes a RoBERTa model; If the semantic information of the associated feature includes the bidding content and the technical parameter is a range value, then the target document is confirmed to be a bidding document; If the semantic information of the associated feature includes bidding content and the technical parameter is a definite value, the target document is confirmed to be a bidding document.

7. The method according to claim 6, characterized in that The extracting domain information from the associated features includes: Generate an attention weight matrix according to the classification label; The associated features are weighted by a multi-head self-attention mechanism to obtain the weight of each feature vector; The weight of each feature vector is compared with the weight threshold, and the target feature vector is extracted from each feature vector as the domain information according to the comparison result.

8. The method according to claim 7, characterized in that The generating a summary of the target file comprises: Determine a summary template according to the field information and classification labels; The template content is filled based on the associated features of the plurality of content units to generate a summary of the target file.

9. The method according to claim 8, characterized in that The filling of the template content based on the associated features of the plurality of content units includes: The TextRank algorithm is used to filter the correlation features between multiple content units, select high-weight text paragraphs and table fields, and generate extractive summaries; The extractive summary is fed into a Seq2Seq model to generate a natural language text summary.

10. A file processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a target file; the target file includes a plurality of content units; the types of the content units include structured data and unstructured data; A parsing module, used to perform semantic extraction on the plurality of content units respectively, and correspondingly generate correlation features between the plurality of content units; A classification module, used to classify the target file based on the associated features and generate at least one classification label; An extraction module, used to extract domain information from the associated features according to the classification labels; A generation module is used to generate a summary of the target file based on the domain information, the classification label and the association features between the multiple content units.

Images