File processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a file processing method and device, electronic equipment and a storage medium, and relates to the field of financial science and technology and natural language process.The method comprises the steps that in response to an obtained to-be-processed target file, the target file is preprocessed, and a non-text object in the target file is converted into a text object; obtaining key information according to a text object in the target file, and taking the key information as a cue word; wherein the key information comprises named entities, semantic relationships, semantic events and theme phrases; and sending the target file and the cue word to the at least one file analysis model to obtain an analysis result of the target file through the at least one file analysis model. According to the technical scheme, the named entities, the semantic relations, the semantic events and the theme phrases are added into the cue words, the problem that the analysis precision of a file analysis model is insufficient is solved, meanwhile, the analysis complexity of the file analysis model is reduced, and the file analysis efficiency is improved.

Description

Technical Field

[0001] This invention relates to the fields of financial technology and natural language processing, and more particularly to a document processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the continuous development of computer technology, the number and types of files generated in business systems are also increasing. How to effectively parse file content has become an important part of the business processing.

[0003] In the existing technology, due to the rapid development of Large Language Model (LLM) technology, parsing file content through LLM has become one of the commonly used parsing methods in the field of file parsing. Specifically, after sending the file to be parsed to the LLM, a fixed context window is configured to guide it to perform file parsing and obtain the file parsing results. Then, based on the file parsing results, a hierarchical content structure is generated by manually sorting out the content logic.

[0004] However, this method of file parsing not only results in poor parsing accuracy of large language models and the parsing results cannot accurately reproduce the original file content, requiring manual intervention to organize the parsing results, but also requires a long file processing time and has low parsing efficiency. Summary of the Invention

[0005] This invention provides a file processing method, apparatus, electronic device, and storage medium to solve the problems of poor parsing accuracy and low parsing efficiency of large language models.

[0006] According to another aspect of the present invention, a file processing method is provided, comprising: In response to obtaining the target file to be processed, the target file is preprocessed to convert non-text objects in the target file into text objects; Key information is obtained from the text objects in the target file and used as prompt words; wherein, the key information includes named entities, semantic relationships, semantic events and topic phrases; The target file and the prompt word are sent to at least one file parsing model to obtain the parsing result of the target file through the at least one file parsing model.

[0007] The preprocessing of the target file includes: obtaining structured metadata of the target file through regular expressions; obtaining the title level of the target file and constructing a chapter syntax tree of the target file based on the title level; obtaining keywords based on word frequency and inverse document frequency, and obtaining key sentences based on a graph-based sorting method, so as to use the keywords and key sentences as semantic metadata of the target file; the step of obtaining key information based on text objects in the target file includes: obtaining key information based on the structured metadata, the chapter syntax tree and the semantic metadata in the target file.

[0008] The text object includes the original text object; the preprocessing of the target file includes: obtaining multiple semantic blocks of the target file based on the syntactic hierarchical features, logical structure features and chapter hierarchical features of each of the original text objects.

[0009] The text object also includes image text objects and table text objects; after obtaining multiple semantic blocks of the target file based on the syntactic hierarchy features, logical structure features and chapter hierarchy features of each of the original text objects, the method further includes: adding the image text objects and the table text objects to the corresponding semantic blocks according to the semantic reference results and spatial coordinate relationships, and adjusting the semantic block results through domain rules and entity boundaries.

[0010] The step of sending the target file and the prompt word to at least one file parsing model to obtain the parsing result of the target file through the at least one file parsing model includes: updating the semantic block results of the target file so that adjacent semantic blocks have overlapping text; compressing each semantic block and configuring global position encoding for each semantic block; and sending the target file and the prompt word to at least one file parsing model so that the at least one file parsing model obtains the parsing result of the target file based on a hierarchical attention mechanism.

[0011] Sending the target file and the prompt word to at least one file parsing model to obtain the parsing result of the target file through the at least one file parsing model includes: obtaining multiple parsing subtasks according to the parsing task request of the target file, and constructing a task graph according to the dependency relationship between each of the parsing subtasks; and obtaining the parsing result of each of the parsing subtasks through the multiple file parsing models according to the task graph and task priority.

[0012] According to another aspect of the present invention, a document processing apparatus is provided, comprising: The preprocessing execution module is used to preprocess the target file in response to obtaining the target file to be processed, so as to convert the non-text objects in the target file into text objects; The prompt word acquisition module is used to obtain key information based on the text objects in the target file and use the key information as prompt words; wherein, the key information includes named entities, semantic relationships, semantic events and topic phrases; The parsing execution module is used to send the target file and the prompt words to at least one file parsing model, so as to obtain the parsing result of the target file through the at least one file parsing model.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the file processing method described in any embodiment of the present invention.

[0014] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the file processing method described in any embodiment of the present invention.

[0015] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the file processing method described in any embodiment of the present invention.

[0016] The technical solution of this invention, in response to obtaining a target file to be processed, preprocesses the target file to convert non-text objects in the target file into text objects; extracts key information based on the text objects in the target file and uses the key information as prompt words; and sends the target file and prompt words to at least one file parsing model to obtain the parsing result of the target file through at least one file parsing model. Thus, by adding named entities, semantic relationships, semantic events, and topic phrases to the prompt words, the key information of the target file is extracted, thereby guiding the file parsing model to perform deep parsing operations. This solves the problem of insufficient parsing accuracy of file parsing models, while reducing the parsing complexity of file parsing models and improving file parsing efficiency.

[0017] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a file processing method provided according to Embodiment 1 of the present invention; Figure 2 This is a flowchart of another file processing method provided according to Embodiment 2 of the present invention; Figure 3 This is a flowchart of yet another file processing method provided according to Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of a document processing device according to Embodiment 4 of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device that implements the file processing method of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] Example 1 Figure 1This is a flowchart of a file processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where a file parsing model is guided to parse a file by extracting named entities, semantic relationships, semantic events, and topic phrases. This method can be executed by a file processing device, which can be implemented in hardware and / or software. The file processing device can be configured in an electronic device. Figure 1 As shown, the method includes: S101. In response to obtaining the target file to be processed, preprocess the target file to convert non-text objects in the target file into text objects.

[0023] After obtaining the target file to be processed, the preprocessing process for the target file includes format parsing and text cleaning. Format parsing refers to calling different format conversion engines to perform format conversion for documents with different storage formats. At the same time, it is a process of extracting multimodal data (such as text, formulas, tables and images) in the document and encapsulating them into text objects to eliminate the interference of file storage format differences on semantic extraction and achieve the separation and extraction of original content and style information.

[0024] For text objects in the target file (i.e., original text objects), that is, information that is presented in text form in the target file itself, their text form is directly preserved; while for non-text objects in the target file (e.g., formulas, tables, and images), in order to ensure the continuity of their content, the formulas and tables are also treated as images and then the images, formulas, and tables in the document are extracted through image decoupling technology, and the text content, text style, and spatial coordinates are identified.

[0025] Text styles include attributes such as font and color; spatial coordinates include the coordinates of the current non-text object in the file page, and the coordinates of the identified text content in the current non-text object; then, the current image, formula, or table is replaced with the text content, and the text style and spatial coordinates are configured for the text content; in addition, for tables, it is also necessary to reconstruct the logical relationship of cells based on the text content in the row header and list header; thus, multimodal parsing and structured storage of the target file are achieved, solving the parsing obstacles of non-continuous text structures such as mixed text and images and table formulas in documents.

[0026] Text cleaning refers to filtering non-semantic noise in target files. It first processes the raw byte stream, identifying and filtering non-text elements (e.g., binary control characters, placeholders, and invalid bytes). Secondly, based on the parsed text of text objects and non-text objects, it converts full-width characters in the parsed text to half-width characters, and removes residual typesetting tags and garbled characters generated during the encoding process, such as garbled characters generated by mixing GBK (Chinese Internal Code Specification), UTF-8 (Unicode Transformation Format - 8 bit), and BIG5 (Big5 Code) encodings.

[0027] Then, the formula region is isolated by the boundary detection algorithm to avoid word segmentation errors in the subsequent word segmentation process. In this way, by constructing a three-level cleaning pipeline including binary cleaning, semantic cleaning and structural hierarchical cleaning, the text cleaning of the target file is completed, avoiding non-semantic noise from polluting the file parsing model. Based on this, through format parsing and text cleaning, the file parsing model is provided with high-purity (i.e., clean document after cleaning) and structure-preserving (i.e., retaining the original text logic and hierarchical structure) text raw materials.

[0028] S102. Obtain key information from the text objects in the target file and use the key information as prompt words; wherein, the key information includes named entities, semantic relationships, semantic events and topic phrases.

[0029] Named entities are entities in text that have specific meanings, including entity classes (e.g., names of people, organizations, places, and proper nouns), time classes, and number classes (e.g., dates, currencies, and percentages). Named entity recognition includes the identification of entity boundaries and the classification of entity categories. Named entities can be obtained through rule models, statistical models, or deep learning-based entity recognition models.

[0030] Semantic relations refer to the semantic relationships between entities. Based on the names of each entity and their semantic relations, the triple information of the knowledge graph can be constructed. For example, in the text "Institution A is located at location B", the subject (Institution A), the semantic relation (located at), and the object (location B) can be extracted. Semantic relations can be identified and obtained through rule models or joint learning models.

[0031] Semantic events refer to events that occur in text. They consist of event trigger words and event elements, which may include time, location, and participants. For example, if organization C releases a new product D, the corresponding event trigger word is "release," and the event elements include the participant (organization C). Semantic events can be identified and obtained through rule models or end-to-end models.

[0032] Topic phrases are phrases in a text that summarize the core theme of the article. They can be obtained by calculating the global importance of each word using Term Frequency-Inverse Document Frequency (TF-IDF) and extracting keywords describing the document's theme as topic phrases. Alternatively, they can be identified by sequence labeling models that combine contextual semantics.

[0033] Therefore, after converting non-text objects into text objects, all elements in the file are presented in text form. Named entities, semantic relationships, semantic events, and topic phrases are then identified and extracted from the complete text content and used as prompt words for the file parsing model. This not only avoids the loss of key information but also guides the file parsing model to complete the file parsing, simplifying the parsing complexity of the file parsing model.

[0034] S103. Send the target file and the prompt word to at least one file parsing model to obtain the parsing result of the target file through the at least one file parsing model.

[0035] The file parsing model is a large language model based on deep learning. It parses the deeper meaning of the input text by understanding the context and semantic relationships of the text. In this embodiment of the invention, a single file parsing model can sequentially complete various parsing operations on the target file, such as text content translation, content outline extraction, and mind map generation. Alternatively, file parsing models with different functions can be used to perform the above-mentioned parsing operations such as text content translation, content outline extraction, and mind map generation respectively.

[0036] Specifically, file parsing models with different functionalities can all be trained based on a general large language model, with different training samples configured for different functional requirements, thereby enhancing the targeted parsing capabilities of different file parsing models. Thus, by using named entities, semantic relationships, semantic events, and topic phrases as prompts, key information of the target file is extracted, avoiding the insufficient parsing depth caused by the syntactic complexity of the file parsing model, improving the parsing accuracy, reducing the parsing complexity of the file parsing model, and improving file parsing efficiency.

[0037] Optionally, in this embodiment of the invention, sending the target file and the prompt word to at least one file parsing model to obtain the parsing result of the target file through the at least one file parsing model includes: obtaining multiple parsing subtasks according to the parsing task request of the target file, and constructing a task graph according to the dependency relationship between each of the parsing subtasks; and obtaining the parsing result of each of the parsing subtasks through the multiple file parsing models according to the task graph and task priority.

[0038] Specifically, upon obtaining the target file to be processed, the parsing task request for the file is broken down into multiple subtasks, such as translating the third paragraph, extracting technical entities, and generating a chapter summary. The third paragraph corresponds to semantic block X, and translating the third paragraph is equivalent to translating semantic block X. Based on pre-configured task dependency rules, the dependencies between the above subtasks are identified (for example, generating a chapter summary requires waiting for the semantic block X to be translated, and a task graph of type Directed Acyclic Graph (DAG) is constructed based on this dependency relationship; where a DAG is a directed graph without loops, each edge of which has a clear direction, reflecting the unidirectional dependency and order relationship between vertices).

[0039] By using a task graph, multiple subtasks that do not have dependencies are configured to be executed in parallel, that is, executed separately using different file parsing models. Subtasks that have dependencies are configured to be executed serially, that is, the execution of downstream subtasks depends on the completion of upstream subtasks. At the same time, based on preset scheduling rules (e.g., user permissions and service level agreements) or real-time load, different execution priorities are assigned to different subtasks that are executed in parallel (e.g., priority is given to translation tasks), so that when task resources are insufficient, high-priority tasks can interrupt the execution of low-priority tasks.

[0040] Furthermore, during task execution, execution errors for each subtask (e.g., translation timeout) can be monitored synchronously. Upon detecting an error, a retry operation or backup solution (e.g., switching to a backup file parsing model) is triggered and recorded in the error log. Simultaneously, the solution for the error is communicated to the user, for example, by sending the message "Technical terminology translation failed, basic mode has been enabled." Thus, by using file parsing models with different functions, parallel processing of multiple subtasks such as text translation, structured data generation, and visualization output is achieved. This solves the problem of low efficiency in single-model task processing and avoids the reduction in model usability due to insufficient computing power or functional fragmentation.

[0041] In particular, the key information extracted from the target file, as well as the parsing results output by the text parsing model, can be output in various forms such as text, tables, charts, structured reports, and slides. For example, technical parameters can be converted into comparative bar charts, and layout elements (such as headers, footers, tables of contents, and references) can be automatically injected according to user-specified or system default format requirements to meet the needs of standardized document generation. At the same time, through locally integrated translation resources, while generating multilingual translations, the translations are localized and adapted, such as unit conversion, date format adjustment, and vocabulary filtering.

[0042] Furthermore, the output can be tailored to the contextual markers of the target file (e.g., a unified glossary) to ensure consistency in terminology and style throughout the text, particularly the logical coherence across chapters. Longer articles can also be streamed in a progressive manner. Interactive elements can be configured within the output, such as embedded question-and-answer anchors and detailed annotations that expand upon clicking, enhancing user reading convenience.

[0043] After obtaining the translation results, content outline, and mind map of the target file through the file parsing model, the parsing results are displayed to the user, who can add, delete, and modify them. Simultaneously, an export option is provided to save the parsing results in a specified format, meeting the format requirements of different business scenarios. Furthermore, the parsing results can be shared with a single click via the cloud service's Application Programming Interface (API), and real-time version updates ensure collaboration and data consistency during multi-user editing.

[0044] The technical solution of this invention, in response to obtaining a target file to be processed, preprocesses the target file to convert non-text objects in the target file into text objects; extracts key information based on the text objects in the target file and uses the key information as prompt words; and sends the target file and prompt words to at least one file parsing model to obtain the parsing result of the target file through at least one file parsing model. Thus, by adding named entities, semantic relationships, semantic events, and topic phrases to the prompt words, the key information of the target file is extracted, thereby guiding the file parsing model to perform deep parsing operations. This solves the problem of insufficient parsing accuracy of file parsing models, while reducing the parsing complexity of file parsing models and improving file parsing efficiency.

[0045] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant regions.

[0046] Example 2 Figure 2 This is a flowchart of a file processing method provided in Embodiment 2 of the present invention. The relationship between this embodiment and the above embodiments is that the preprocessing process further includes metadata extraction, such as... Figure 2 As shown, the method specifically includes: S201. In response to obtaining the target file to be processed, perform format parsing and text cleaning on the target file to convert non-text objects in the target file into text objects.

[0047] S202. Obtain the structured metadata of the target file using regular expressions.

[0048] Metadata extraction refers to the process of extracting descriptive information from a document, thereby guiding the semantic analysis of the document parsing model through structured features and providing anchor points for generating a hierarchical outline of the document. The aforementioned descriptive information includes basic attribute information such as title, author, creation date, and keywords, as well as structural information such as chapter titles, paragraph distribution, table positions, and image positions.

[0049] Firstly, for documents generated by business systems, especially those in financial systems, there are usually structured fields describing basic document information at the beginning, end, or other fixed locations (e.g., headers and footers). These structured fields typically have a fixed format, such as "Author A + Date B". The structured metadata can be obtained by matching the fixed pattern at the beginning and end of the document using regular expressions.

[0050] S203. Obtain the title level of the target file and construct the chapter syntax tree of the target file based on the title level of the target file.

[0051] Secondly, the title of the target file can be identified based on the trained Conditional Random Field (CRF) model to obtain the chapter identification results of the target file, and then a tree-structured chapter syntax tree can be constructed based on the chapter identification results. In particular, when training the Conditional Random Field model, each title is first labeled with a level label in the training samples. For example, the first chapter is configured as H1 (i.e., the first level label), the first section is configured as H2 (i.e., the second level label), and the first section is configured as H3 (i.e., the third level label).

[0052] Then, the input features of the Conditional Random Field (CRF) are configured, namely the contextual information of the title, including word features, format features, and position features. Word features can include the current word (e.g., "Chapter 1"), the previous word (e.g., "Table of Contents"), and the next word (e.g., "Introduction"). Format features can include whether it contains numbers (e.g., whether it contains "1.1") and whether it contains punctuation (e.g., whether it contains the colon ":"). Position features indicate whether it is located at a specific position in the document (e.g., the beginning or the middle). Finally, the training samples are input into the CRF model to train the model's title recognition function.

[0053] S204. Obtain keywords based on word frequency and inverse document frequency, and obtain key sentences based on graph-based sorting method, so as to use the keywords and key sentences as semantic metadata of the target file.

[0054] Finally, the global importance of each word is calculated using Term Frequency-Inverse Document Frequency (TF-IDF) to extract keywords describing the document's topic. Simultaneously, a graph-based ranking algorithm is used to calculate the local importance of each sentence, thereby extracting key sentences describing the document's main content. These include, for example, the main idea sentence summarizing the entire text, the concluding sentence deepening the theme, and high-information-density sentences containing core elements such as entities, actions, and conclusions. The graph-based ranking algorithm constructs a graph based on the similarity between sentences, extracting important sentences as the key sentence extraction method.

[0055] S205. Based on the structured metadata, chapter syntax tree, and semantic metadata in the target file, obtain key information and use the key information as prompt words; wherein, the key information includes named entities, semantic relationships, semantic events, and topic phrases.

[0056] After extracting the structured metadata, chapter syntax tree, and semantic metadata of the target file, these can be used as the basis for extraction. Key information such as named entities, semantic relationships, semantic events, and topic phrases can be directly extracted from them. Alternatively, they can be used as a verification basis. The obtained key information such as named entities, semantic relationships, semantic events, and topic phrases can be compared with the aforementioned structured metadata, chapter syntax tree, and semantic metadata to determine whether the key information completely covers the structured metadata, chapter syntax tree, and semantic metadata, thus verifying the accuracy of the extraction results.

[0057] S206. Send the target file and the prompt word to at least one file parsing model to obtain the parsing result of the target file through the at least one file parsing model.

[0058] The technical solution of this invention obtains structured metadata of a target file through regular expressions; obtains the title level of the target file and constructs a chapter syntax tree based on the title level; obtains keywords based on term frequency and inverse document frequency, and obtains key sentences based on a graph-based sorting method, using the keywords and key sentences as semantic metadata of the target file. Thus, through structured metadata, chapter syntax trees, and semantic metadata, complete metadata acquisition of the target file is achieved, providing sufficient semantic basis for subsequent semantic segmentation processing and ensuring the accuracy and completeness of the key information extraction results.

[0059] Example 3 Figure 3 This is a flowchart of a file processing method provided in Embodiment 3 of the present invention. The relationship between this embodiment and the above embodiments is that the preprocessing process includes semantic block processing; the text objects include original text objects, image text objects, and table text objects, such as... Figure 3 As shown, the method specifically includes: S301. In response to obtaining the target file to be processed, perform format parsing and text cleaning on the target file to convert non-text objects in the target file into text objects.

[0060] S302. Based on the syntactic hierarchical features, logical structure features, and chapter hierarchical features of each of the original text objects, obtain multiple semantic blocks of the target file.

[0061] Semantic chunking refers to dividing a target file into coherent units with independent semantics based on the inherent meaning of the text, ensuring that each chunk carries complete information logic, thereby reducing the parsing complexity of the text parsing model and the extraction complexity of key information; Original text objects refer to information in the target file that is originally presented in text form, which can be divided based on natural paragraphs to form multiple original text objects; Image text objects are text information converted from images, with each image being an image text object; Table text objects are text information converted from tables, with each table being a table text object.

[0062] To obtain grammatical hierarchical features, firstly, for each original text object, dependency syntax tree and constituent syntax analysis are used to identify the subject-verb-object structure of the original text object. Then, based on the subject-verb-object structure, the nesting depth of the clauses is determined. Compound sentences with high nesting degree are more suitable for whole preservation, while sentences with low nesting degree can be further split. Based on this, the grammatical hierarchical features of the current original text object can be determined by the number of nested clauses according to the syntactic analysis results, or the grammatical hierarchical features can be quantified into scores to obtain a grammatical hierarchical score, that is, the more nested clauses, the higher the grammatical hierarchical score.

[0063] Secondly, logical connectors representing logical relationships such as cause and effect, transition, and example are obtained from the original text object. Regions with dense logical connectors are saved into the same semantic block, while regions with sparse logical connectors can be split into different semantic blocks. Based on this, the logical structure features of the current original text object can be obtained according to the number or frequency of logical connectors in the original text object, or the logical structure features can be quantified into scores to obtain a logical structure score. That is, the larger the number or frequency of logical connectors, the higher the logical structure score.

[0064] Then, based on the chapter syntax tree obtained by the above technical solution, the chapter level of the current original text object is determined. The higher the chapter level, the more general content the current original text object may involve, which is more suitable for fine-grained semantic segmentation. Accordingly, based on the title level of the original text object, the chapter level feature of the current original text object is obtained, or the chapter level feature is quantified into a score to obtain a chapter level score. That is, the higher the chapter level of the original text object, the smaller the chapter level score.

[0065] Therefore, based on the grammatical hierarchy features, logical structure features, and chapter hierarchy features of the current original text object, the matching semantic block granularity can be obtained by querying the mapping table, or the grammatical hierarchy score, logical structure score, and chapter hierarchy score of the current original text object can be multiplied with their respective weights, and the sum of the results of each multiplication operation can be used as the semantic block granularity score of the current original text object.

[0066] The smaller the semantic chunk granularity score, the finer the granularity of the triggered segmentation method, which may involve splitting the current original text object, that is, splitting clauses into different semantic chunks; the larger the semantic chunk granularity score, the finer the granularity of the triggered segmentation method, which may involve merging the current original text object, that is, merging the current original text object with other original text objects into the same semantic chunk. In this way, each original text object in natural language is chunked according to semantic features, which not only ensures the logical coherence of each chunk result, but also performs fine-grained chunking of complex semantic text, further reducing the complexity of file parsing.

[0067] S303. Obtain key information according to each semantic block, and use the key information as prompt words; wherein, the key information includes named entities, semantic relationships, semantic events and topic phrases.

[0068] Dividing the target file into different semantic blocks not only reduces the complexity of extracting key information, but also further reduces the parsing complexity of the file parsing model.

[0069] S304. Send the target file and the prompt word to at least one file parsing model to obtain the parsing result of the target file through the at least one file parsing model.

[0070] Optionally, in this embodiment of the invention, after obtaining multiple semantic blocks of the target file, the method further includes: adding image text objects and table text objects to the corresponding semantic blocks according to semantic reference results and spatial coordinates, and adjusting the semantic block results through domain rules and entity boundaries. Specifically, the semantic reference results of text such as figure number, figure name, table number, and table name in the original text object are obtained, that is, the referent of the text, and the original text object is associated with the image text object or table text object according to the referent.

[0071] At the same time, extract the bounding box coordinates of all elements. If the vertical distance between the original text object and the image text object or table text object below it is less than the first distance threshold, or the horizontal distance between the original text object and the image text object or table text object to its side is less than the second distance threshold, and there is no other inserted content between the original text object and the image text object or table text object, then a spatial adjacency relationship can be established.

[0072] For example, "such as Figure 1 "shown" indicates the current original text object and Figure 1Related; "See the table below" indicates that the current original text object is related to the table with the closest spatial location; "The flowchart on the left shows the specific values" indicates that the image is related to the image whose horizontal axis is smaller than the current original text object and whose vertical axis falls within the coordinate range of the current original text object. This solves the problem of text being separated from images and tables in mixed media such as web page files and report files, and ensures the integrity and continuity of semantic text.

[0073] Domain rules represent a predefined set of rules for the current business domain. They explicitly define which text elements cannot be split, such as author information and publication date. Among the candidate split points, it is checked whether the candidate split point is inside a named entity. If the current split point is inside a named entity, the nearest safe breakpoint (e.g., comma and period) is searched backward to prevent the destruction of critical information and ensure the integrity of entities and structured content.

[0074] Optionally, in this embodiment of the invention, sending the target file and the prompt word to at least one file parsing model to obtain the parsing result of the target file through the at least one file parsing model includes: updating the semantic block results of the target file to make adjacent semantic blocks have overlapping text; compressing each semantic block and configuring global position encoding for each semantic block; and sending the target file and the prompt word to at least one file parsing model so that the at least one file parsing model obtains the parsing result of the target file based on a hierarchical attention mechanism.

[0075] Specifically, for each semantic block, a certain proportion of overlapping text is retained with both the preceding and following semantic blocks. For example, the current semantic block retains 10% overlapping text with the preceding semantic block, and also retains 10% overlapping text with the following semantic block. This establishes a positional mapping between semantic blocks through overlapping text. Then, each semantic block is compressed, especially the overlapping text, to avoid the overlapping text occupying too much storage resources. Finally, each semantic block is encoded according to its own global position to strengthen its position in the text.

[0076] Accordingly, after sending the target file and prompt words to the file parsing model, the model is guided to obtain the parsing results of the target file through a hierarchical attention mechanism. The hierarchical attention mechanism means that the file parsing model processes the details within semantic blocks through the low-level structure, the relationships between semantic blocks through the middle-level structure, and the global context through the high-level structure. This breaks through the limitation of the fixed context window of the file parsing model, guides the file parsing model to maintain the coherence of the global theme and logic during the parsing process, and improves the accuracy and completeness of the file parsing results.

[0077] The technical solution of this invention, after obtaining multiple semantic blocks of the target file, adds image text objects and table text objects to the corresponding semantic blocks according to semantic reference results and spatial coordinates, and adjusts the semantic block results through domain rules and entity boundaries. Then, it obtains the key information of the target file through each semantic block. In this way, each original text object in natural language is divided into blocks according to semantic features, which not only ensures the logical coherence of each semantic block result, but also performs fine-grained block division of complex semantic text, reducing the difficulty of file parsing.

[0078] Example 4 Figure 4 This is a structural block diagram of a document processing device provided in Embodiment 4 of the present invention. The device specifically includes: The preprocessing execution module 401 is used to preprocess the target file in response to obtaining the target file to be processed, so as to convert the non-text objects in the target file into text objects; The prompt word acquisition module 402 is used to acquire key information based on the text object in the target file and use the key information as prompt words; wherein, the key information includes named entities, semantic relationships, semantic events and topic phrases; The parsing execution module 403 is used to send the target file and the prompt word to at least one file parsing model so as to obtain the parsing result of the target file through the at least one file parsing model.

[0079] The technical solution of this invention, in response to obtaining a target file to be processed, preprocesses the target file to convert non-text objects in the target file into text objects; extracts key information based on the text objects in the target file and uses the key information as prompt words; and sends the target file and prompt words to at least one file parsing model to obtain the parsing result of the target file through at least one file parsing model. Thus, by adding named entities, semantic relationships, semantic events, and topic phrases to the prompt words, the key information of the target file is extracted, thereby guiding the file parsing model to perform deep parsing operations. This solves the problem of insufficient parsing accuracy of file parsing models, while reducing the parsing complexity of file parsing models and improving file parsing efficiency.

[0080] Optionally, the file processing device is further configured to: obtain the structured metadata of the target file through regular expressions; obtain the title level of the target file and construct the chapter syntax tree of the target file based on the title level; obtain keywords based on word frequency and inverse document frequency, and obtain key sentences based on a graph-based sorting method, so as to use the keywords and key sentences as semantic metadata of the target file; and obtain key information based on the structured metadata, the chapter syntax tree and the semantic metadata in the target file.

[0081] Optionally, the text object includes the original text object; the preprocessing execution module 401 is further configured to obtain multiple semantic blocks of the target file based on the syntactic level features, logical structure features and chapter level features of each of the original text objects.

[0082] Optionally, the text object also includes an image text object and a table text object; the preprocessing execution module 401 is further configured to add the image text object and the table text object to the corresponding semantic block according to the semantic reference result and the spatial coordinate relationship, and adjust the semantic block result through domain rules and entity boundaries.

[0083] Optionally, the parsing execution module 403 is further configured to update the semantic block results of the target file so that adjacent semantic blocks have overlapping text; compress each semantic block and configure global position encoding for each semantic block; and send the target file and the prompt words to at least one file parsing model so that the at least one file parsing model obtains the parsing results of the target file based on a hierarchical attention mechanism.

[0084] Optionally, the parsing execution module 403 is further configured to obtain multiple parsing subtasks according to the parsing task request of the target file, and construct a task graph according to the dependency relationship between each parsing subtask; and obtain the parsing results of each parsing subtask through multiple file parsing models according to the task graph and task priority.

[0085] The above-described apparatus can execute the file processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the file processing method provided in any embodiment of the present invention.

[0086] Example 5 Figure 5A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, electronic devices, blade electronic devices, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0087] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0088] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0089] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as file processing methods.

[0090] In some embodiments, the file processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on a heterogeneous hardware accelerator via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the file processing method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the file processing method by any other suitable means (e.g., by means of firmware).

[0091] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0092] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0093] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0094] To provide user interaction, the systems and techniques described herein can be implemented on a heterogeneous hardware accelerator, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the heterogeneous hardware accelerator. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or haptic feedback); and input from the user can be received in any form (including sound input, voice input, or haptic input).

[0095] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0096] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0097] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0098] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A file processing method, characterized in that, include: In response to obtaining the target file to be processed, the target file is preprocessed to convert non-text objects in the target file into text objects; Key information is obtained from the text objects in the target file and used as prompt words; wherein, the key information includes named entities, semantic relationships, semantic events and topic phrases; The target file and the prompt word are sent to at least one file parsing model to obtain the parsing result of the target file through the at least one file parsing model.

2. The file processing method according to claim 1, characterized in that, The preprocessing of the target file includes: The structured metadata of the target file is obtained using regular expressions; Obtain the heading level of the target file, and construct the chapter syntax tree of the target file based on the heading level; Keywords are obtained based on word frequency and inverse document frequency, and key sentences are obtained based on graph-based sorting methods, so that the keywords and key sentences are used as semantic metadata of the target file; The step of obtaining key information based on text objects in the target file includes: Key information is obtained based on the structured metadata, chapter syntax tree, and semantic metadata in the target file.

3. The document processing method according to claim 1 or 2, characterized in that, The text object includes the original text object; the preprocessing of the target file includes: Based on the syntactic hierarchical features, logical structure features, and chapter hierarchical features of each of the original text objects, multiple semantic blocks of the target file are obtained.

4. The file processing method according to claim 3, characterized in that, The text object also includes image text objects and table text objects; after obtaining multiple semantic blocks of the target file based on the syntactic hierarchy features, logical structure features, and chapter hierarchy features of each of the original text objects, the method further includes: Based on the semantic reference results and spatial coordinate relationships, the image text object and the table text object are added to the corresponding semantic blocks, and the semantic block results are adjusted through domain rules and entity boundaries.

5. The document processing method according to claim 3, characterized in that, Sending the target file and the prompt words to at least one file parsing model to obtain the parsing result of the target file through the at least one file parsing model includes: The semantic segmentation results of the target file are updated to ensure that adjacent semantic segments have overlapping text. Each semantic block is compressed, and a global positional encoding is configured for each semantic block. The target file and the prompt words are sent to at least one file parsing model, so that the at least one file parsing model obtains the parsing result of the target file based on a hierarchical attention mechanism.

6. The file processing method according to claim 1, characterized in that, Sending the target file and the prompt words to at least one file parsing model to obtain the parsing result of the target file through the at least one file parsing model includes: Multiple parsing subtasks are obtained based on the parsing task request of the target file, and a task graph is constructed based on the dependencies between the parsing subtasks. Based on the task graph and task priority, the parsing results of each of the parsing subtasks are obtained through multiple file parsing models.

7. A document processing device, characterized in that, include: The preprocessing execution module is used to preprocess the target file in response to obtaining the target file to be processed, so as to convert the non-text objects in the target file into text objects; The prompt word acquisition module is used to obtain key information based on the text objects in the target file and use the key information as prompt words; wherein, the key information includes named entities, semantic relationships, semantic events and topic phrases; The parsing execution module is used to send the target file and the prompt words to at least one file parsing model, so as to obtain the parsing result of the target file through the at least one file parsing model.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the file processing method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the file processing method according to any one of claims 1-6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the file processing method of any one of claims 1-6.

Images