Document processing method and device, electronic equipment and storage medium

Abstract

The invention relates to a document processing method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining a to-be-processed target document; identifying redundant contents from the paragraphs of the target document, wherein the redundant contents are paragraphs of which the element types are determined; performing document identification on non-redundant contents in the target document by using the document element identification model to obtain a model identification result of paragraphs in the non-redundant contents, the non-redundant contents being other contents except the redundant contents in the target document, the model identification result comprises the element type of the paragraph; and determining a final recognition result of the paragraph in the target document according to a model recognition result of the paragraph in the non-redundant content and the redundant content. Therefore, the processing capacity of the model is utilized more effectively, the model is prevented from processing unnecessary redundant content, the recognition efficiency and recognition accuracy of the model can be improved, and the operation burden of the model is relieved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a document processing method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of information technology, the application scope of electronic documents is becoming more and more extensive. At the same time, in order to efficiently process electronic documents, it has become a common practice to use models (such as machine learning models, deep learning models, etc.) to identify and analyze documents.

[0003] However, in actual application scenarios, the complexity and diversity of electronic documents pose many challenges to the recognition and analysis of the model. A typical electronic document may contain hundreds or thousands of pages with complex content, and not all characters constitute valid content required for the model to perform document processing tasks. The practice of directly feeding the content of the electronic document into the model for recognition and analysis indiscriminately in the related technology will cause the model to need to perform unnecessary recognition and analysis on these non-essential contents, which will not only waste the model's computing resources and reduce processing efficiency, but may also interfere with the model's recognition accuracy. Summary of the invention

[0004] The present application provides a document processing method, device, electronic device and storage medium to solve the technical problem in the related art that the practice of directly and indiscriminately sending the content of an electronic document to a model for recognition and analysis will cause the model to need to perform unnecessary recognition and analysis on part of the content, which not only wastes the model's computing resources and reduces processing efficiency, but may also interfere with the model's recognition accuracy.

[0005] In a first aspect, the present application provides a document processing method, the method comprising:

[0006] Get the target document to be processed;

[0007] Identifying redundant content from paragraphs of the target document, wherein the redundant content is a paragraph of a determined element type;

[0008] Using a document element recognition model to perform document recognition on non-redundant content in the target document, obtaining a model recognition result of a paragraph in the non-redundant content, wherein the non-redundant content refers to other content in the target document except the redundant content, and the model recognition result includes an element type of the paragraph;

[0009] A final recognition result of the paragraph in the target document is determined according to the model recognition result of the paragraph in the non-redundant content and the element type of the paragraph in the redundant content.

[0010] In a possible implementation manner, identifying redundant content from the paragraphs of the target document includes:

[0011] A first paragraph that meets a specific element format is identified from paragraphs of the target document, and the first paragraph is determined to be redundant content.

[0012] In a possible implementation manner, the using of the document element recognition model to perform document recognition on the non-redundant content in the target document to obtain a model recognition result of a paragraph in the non-redundant content includes:

[0013] Identifying content to be simplified from paragraphs of non-redundant content in the target document, and performing a simplification operation on the content to be simplified to obtain simplified non-redundant content;

[0014] The document element recognition model is used to perform document recognition on the simplified non-redundant content to obtain a model recognition result of the paragraphs in the non-redundant content of the target document.

[0015] In a possible implementation manner, identifying the content to be simplified from the paragraphs of the non-redundant content of the target document, and performing a simplification operation on the content to be simplified to obtain the simplified non-redundant content includes:

[0016] Identifying non-visual content from paragraphs of non-redundant content in the target document, and determining the non-visual content as content to be simplified;

[0017] The non-visual content is deleted from the paragraph of the non-redundant content of the target document to obtain simplified non-redundant content.

[0018] In a possible implementation manner, the identifying non-visual content from the paragraphs of non-redundant content of the target document and determining the non-visual content as content to be simplified includes:

[0019] Identify space characters from paragraphs of non-redundant content in the target document, and determine the space characters as content to be simplified; and / or,

[0020] A separator is identified from the target document, and the separator is determined as content to be simplified.

[0021] In a possible implementation manner, identifying the content to be simplified from the paragraphs of the non-redundant content of the target document, and performing a simplification operation on the content to be simplified to obtain the simplified non-redundant content includes:

[0022] Identifying intermediate content to be processed into final content from paragraphs of non-redundant content of the target document, and converting the intermediate content into final content; and / or,

[0023] An element of a preset type is identified from a paragraph of non-redundant content of the target document, and the element of the preset type in the paragraph is converted into a content indicator symbol, where the content indicator symbol is used to represent the element of the preset type.

[0024] In a possible implementation manner, identifying the content to be simplified from the paragraphs of the non-redundant content of the target document, and performing a simplification operation on the content to be simplified to obtain the simplified non-redundant content includes:

[0025] Identifying object elements from paragraphs of non-redundant content in the target document, determining the object elements as content to be streamlined, and converting the object elements into corresponding placeholders; and / or,

[0026] Identifying a table element from a paragraph of non-redundant content of the target document, determining the table element as content to be streamlined, and converting the table element into position information of the table element in the target document; and / or,

[0027] Identifying domain elements from paragraphs of non-redundant content of the target document, determining the domain elements as content to be streamlined, and converting the domain elements into corresponding domain results; and / or,

[0028] Revision contents with revision marks are identified from paragraphs of non-redundant contents of the target document, the revision contents are determined as contents to be simplified, and the revision contents are converted into corresponding revised contents.

[0029] In a possible implementation manner, determining a final recognition result of a paragraph in the target document according to the model recognition result of the paragraph in the non-redundant content and the element type of the paragraph in the redundant content includes:

[0030] A final recognition result of the paragraph in the target document is determined according to the model recognition result of the paragraph in the non-redundant content and the specific element format satisfied by the first paragraph.

[0031] In a possible implementation manner, obtaining the target document to be processed includes:

[0032] Create a copy of the original document;

[0033] The copy document is preprocessed according to a set preprocessing method to obtain a preprocessed document, and the preprocessed document is determined as a target document to be processed.

[0034] In a possible implementation manner, preprocessing the copy document according to a set preprocessing method to obtain a preprocessed document includes:

[0035] A non-standard typesetting style related to segmentation or paging is identified from the copy document, and the non-standard typesetting style is converted into a set standard typesetting style to obtain a pre-processed document.

[0036] In a possible implementation, the method further includes:

[0037] Based on the paragraph mapping relationship between the original document and the target document, and the final recognition result of the paragraph in the target document, the final recognition result of the paragraph in the original document is determined.

[0038] In a second aspect, the present application provides a document processing device, the device comprising:

[0039] A document acquisition module is used to acquire the target document to be processed;

[0040] A redundancy identification module, used to identify redundant content from paragraphs of the target document, wherein the redundant content is a paragraph of a determined element type;

[0041] A model recognition module, used to perform document recognition on the non-redundant content in the target document using a document element recognition model to obtain a model recognition result of a paragraph in the non-redundant content, wherein the non-redundant content refers to other content in the target document except the redundant content, and the model recognition result includes an element type of the paragraph;

[0042] The result determination module is used to determine the final recognition result of the paragraph in the target document according to the model recognition result of the paragraph in the non-redundant content and the element type of the paragraph in the redundant content.

[0043] In a possible implementation manner, the redundancy identification module is specifically used to:

[0044] A first paragraph that meets a specific element format is identified from paragraphs of the target document, and the first paragraph is determined to be redundant content.

[0045] In a possible implementation, the model identification module includes:

[0046] A simplification element is used to identify the content to be simplified from the paragraphs of the non-redundant content of the target document, and perform a simplification operation on the content to be simplified to obtain the simplified non-redundant content;

[0047] The recognition unit is used to perform document recognition on the simplified non-redundant content using a document element recognition model to obtain a model recognition result of a paragraph in the non-redundant content of the target document.

[0048] In a possible implementation, the simplified element includes:

[0049] A first identification subunit is used to identify non-visual content from the paragraphs of non-redundant content of the target document, and determine the non-visual content as content to be simplified;

[0050] The first simplification subunit is used to delete the non-visual content from the paragraph of the non-redundant content of the target document to obtain simplified non-redundant content.

[0051] In a possible implementation manner, the first identification subunit is specifically used to:

[0052] Identify space characters from paragraphs of non-redundant content in the target document, and determine the space characters as content to be simplified; and / or,

[0053] A separator is identified from the target document, and the separator is determined as content to be simplified.

[0054] In a possible implementation manner, the simplified element is specifically used for:

[0055] Identifying intermediate content to be processed into final content from paragraphs of non-redundant content of the target document, and converting the intermediate content into final content; and / or,

[0056] An element of a preset type is identified from a paragraph of non-redundant content of the target document, and the element of the preset type in the paragraph is converted into a content indicator symbol, where the content indicator symbol is used to represent the element of the preset type.

[0057] In a possible implementation manner, the simplified element is specifically used for:

[0058] Identifying object elements from paragraphs of non-redundant content in the target document, determining the object elements as content to be streamlined, and converting the object elements into corresponding placeholders; and / or,

[0059] Identifying a table element from a paragraph of non-redundant content of the target document, determining the table element as content to be streamlined, and converting the table element into position information of the table element in the target document; and / or,

[0060] Identifying domain elements from paragraphs of non-redundant content of the target document, determining the domain elements as content to be streamlined, and converting the domain elements into corresponding domain results; and / or,

[0061] Revision contents with revision marks are identified from paragraphs of non-redundant contents of the target document, the revision contents are determined as contents to be simplified, and the revision contents are converted into corresponding revised contents.

[0062] In a possible implementation manner, the result determination module is specifically used to:

[0063] A final recognition result of the paragraph in the target document is determined according to the model recognition result of the paragraph in the non-redundant content and the specific element format satisfied by the first paragraph.

[0064] In a possible implementation, the document acquisition module includes:

[0065] A copy creation unit, used for creating a copy document of the original document;

[0066] The copy processing unit is used to pre-process the copy document according to a set pre-processing method to obtain a pre-processed document, and determine the pre-processed document as a target document to be processed.

[0067] In a possible implementation manner, the replica processing unit is specifically used to:

[0068] A non-standard typesetting style related to segmentation or paging is identified from the copy document, and the non-standard typesetting style is converted into a set standard typesetting style to obtain a pre-processed document.

[0069] In a possible implementation manner, the result determination module is further used to:

[0070] Based on the paragraph mapping relationship between the original document and the target document, and the final recognition result of the paragraph in the target document, the final recognition result of the paragraph in the original document is determined.

[0071] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory, wherein the processor is used to execute a document processing program stored in the memory to implement the document processing method described in any one of the first aspects.

[0072] In a fourth aspect, the present application provides a storage medium storing one or more programs, wherein the one or more programs can be executed by one or more processors to implement the document processing method described in any one of the first aspects.

[0073] The above-mentioned technical scheme provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application, by acquiring the target document to be processed, identifies paragraphs of determined element types from the paragraphs of the target document as redundant content, and uses the document element recognition model to perform document recognition on the non-redundant content in the target document to obtain the model recognition result of the paragraph in the non-redundant content, where the non-redundant content refers to other content in the target document except the redundant content, and determines the final recognition result of the paragraph in the target document based on the model recognition result of the paragraph in the non-redundant content and the redundant content, thereby realizing the collaborative work between the server and the model. This collaborative work mechanism can more effectively utilize the processing power of the model and avoid the model from processing unnecessary redundant content, which can not only improve the recognition efficiency and accuracy of the model, but also reduce the burden on the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0075] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0076] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0077] Figure 1 A schematic diagram of a system architecture involved in an embodiment of the present application;

[0078] Figure 2 A flowchart of an embodiment of a document processing method provided in an embodiment of the present application;

[0079] Figure 3 A flowchart of another document processing method provided in an embodiment of the present application;

[0080] Figure 4 A diagram comparing the effects of using soft and hard carriage returns for text content.

[0081] Figure 5 A diagram showing the attachment of separators to content and the correct segmentation structure;

[0082] Figure 6This is a schematic diagram of the segmentation effect of a space character or a tab character;

[0083] Figure 7 This is a schematic diagram showing the effect of text content not being segmented correctly;

[0084] Figure 8 A schematic diagram of using consecutive blank lines to achieve paging effect;

[0085] Fig. 9 A block diagram of an embodiment of a document processing device provided in an embodiment of the present application;

[0086] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0087] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0088] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0089] In order to solve the technical problem that the prior art practice of directly feeding the contents of electronic documents into the model for recognition and analysis indiscriminately causes the model to perform unnecessary recognition and analysis on part of the contents, which not only wastes the computing resources of the model and reduces processing efficiency, but may also interfere with the recognition accuracy of the model, the present application provides a document processing method, device, electronic device and storage medium, which can achieve more effective utilization of the processing power of the model, avoid the model from processing unnecessary redundant content, improve the recognition efficiency and recognition accuracy of the model, and reduce the computing burden of the model.

[0090] Figure 1 Schematic diagram of a system architecture involved in an embodiment of the present application. Figure 1As shown, the system architecture 100 may include: a client 101, a server 102, a model 103, and a network 104. The network 104 is used to provide a communication link between the client 101, the server 102, and the model 103, and may include a wireless link, a wired link, or an optical fiber cable, etc., which is not limited in this application.

[0091] The client 101 may be a hardware device or software that supports network connection to provide document editing services and other network services. When the device is hardware, it may support various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, servers, etc. When the device is software, it may be installed in the electronic devices listed above.

[0092] The server 102 is the core processing unit of the entire system architecture 100, and can be deployed with a variety of algorithms and relatively powerful computing resources. Among them, the algorithm may involve multiple fields such as data processing, machine learning, and deep learning, and is used to process the data uploaded or the request sent by the client 101, and perform specific computing tasks, such as document content processing, document content analysis, etc. The computing resources may include high-performance computing clusters, GPU accelerator cards, etc., which are used to support the rapid execution and efficient processing of algorithms.

[0093] The model end 103 is a server or device deployed with one or more models, which are used to process specific computing tasks or provide specific services. For example, a document element recognition model, a natural language processing model, etc. are deployed on the model end 103. Among them, the document element recognition model is used to identify the element type (such as abstract, title, text, etc.) of the paragraph in the input document. When the server end 102 receives the request from the client 101 and needs to call the model, it forwards the request to the model end 103, and the model end 103 uses the deployed model to perform reasoning calculations and outputs the corresponding results. In addition, with the continuous advancement of algorithms and the continuous accumulation of data, the models deployed on the model end 103 can be updated or optimized regularly. Exemplarily, the model end 103 supports automatic updating and version control of the model to ensure that the model can always maintain the latest performance and accuracy. Exemplarily, the server end 102 is responsible for managing the model on the model end 103, including uploading, updating, deleting and version control of the model.

[0094] Understandably, Figure 1 The number of networks and devices in the system architecture 100 is only for illustration. According to actual needs, the system architecture 100 may include any number of networks and devices, which is not limited in the embodiment of the present application.

[0095] In some examples, at least two of the client 101, the server 102, and the model 103 may be the same device, and different terminals in the same device may also transmit data to each other via wireless or wired connections.

[0096] based on Figure 1 In the illustrated system architecture, in an exemplary application scenario, the client 101 sends a document automatic typesetting request to the server 102 and attaches the document to be typeset. The server 102 receives the request and document from the client 101, forwards the document to the model 103, and requests the model 103 to call the deployed document element recognition model to analyze and recognize the document. After receiving the document, the model 103 calls the deployed document element recognition model to analyze and recognize the document, determine the element type of the paragraph in the document, and then returns the recognition result to the server 102.

[0097] Furthermore, after receiving the model recognition result returned by the model end 103, the server end 102 can also use its own algorithms and computing resources to further analyze and verify the model recognition result returned by the model end 103. For example, the server end can check the accuracy of the model recognition result, or further adjust and optimize the model recognition result based on the document's typesetting style, paragraph format and other features. Finally, the server end 103 determines the element type of the paragraph in the document based on the model recognition result and the results of further analysis. Finally, the server end 103 automatically typesets the document according to the determined element type of the paragraph in the document, and returns the typeset document to the client end 101. Alternatively, the server end 103 sends the determined element type of the paragraph in the document to the client end 101, and the client end 101 automatically typesets the document according to the determined element type of the paragraph in the document to obtain the typeset document.

[0098] Through the above process, the server 102 and the model 103 work together. On the one hand, it can efficiently process the automatic typesetting request of the document. On the other hand, the server 102 can make up for the lack of capabilities of the model 103 in certain aspects, thereby improving the accuracy of the final element type recognition result, and further ensuring that the typesetting of the document meets specific standards or requirements.

[0099] In the above exemplary application scenario, the server 102 requests the model end 103 to call the deployed document element recognition model to identify the paragraph element type of the document. In actual application scenarios, an electronic document may contain hundreds or thousands of pages, and not all characters in the document constitute real and valid text content. The so-called "real and valid" is based on the final visual effect. Some characters in the document are used as placeholders, while others are used for typesetting purposes such as paragraph separation, word separation, section separation, column separation, and paging. Secondly, in the real and valid text content, there are also some paragraphs whose element types can be directly identified by the server. For example, the number of words required for different types of paragraphs is different, the character combination of a specific element type paragraph is fixed or can be enumerated; the structure of a specific paragraph interval is fixed or can be enumerated so that it is easy to identify, etc. These paragraphs whose element types can be directly identified by the server do not need to be identified by the model for element type. In view of this, in order to improve the recognition efficiency of the model and reduce its computational burden, the technical solution provided in the embodiment of the present application proposes: through the collaboration between the server end 102 and the model end 103, before submitting the document to the model, the document content can be filtered, streamlined, and other operations can be performed first, aiming to eliminate information in the document that is useless or unhelpful for the model to identify the element type, so as to improve the recognition efficiency of the model and reduce its computational burden.

[0100] It should be noted that the above division of the client 101, the server 102 and the model 103, as well as the functional division of the execution of each end, is only an example, and some functions can be executed by any end, and the embodiments of the present application do not limit this. For example, the above-mentioned typesetting function can be executed on the server 102, or on the client 101; for another example, the above-mentioned filtering, streamlining and other operations on the document content can be executed on the server 102, or on the model 103, or partially on the server 102 and partially on the model 103. In order to briefly explain and clarify, the embodiments of the present application only use some examples to illustrate the functional division of the execution of each end, but the embodiments of the present application do not limit this.

[0101] Figure 2 The flowchart of an embodiment of a document processing method provided in the embodiment of the present application is shown in FIG. In one embodiment, the method is applied to a server, for example Figure 1 The server 102 shown in the example. Figure 2 As shown, the process includes the following steps:

[0102] Step 201: Obtain the target document to be processed.

[0103] The target document to be processed refers to a document that needs to be submitted to the model for feature type recognition, but the document content needs to be filtered, streamlined, etc. before being submitted to the model.

[0104] In one embodiment, the target document refers to an original document, for example, a document that needs to be typeset and is attached when the client 101 sends a request for automatic typesetting of a document to the server 102 .

[0105] In another embodiment, the target document is not the original document, but a document obtained by preprocessing a copy of the original document. The key purpose of preprocessing the copy of the original document is to ensure that the content in the document can be correctly segmented, because correct segmentation is an important prerequisite for the subsequent model end to identify the element type of the paragraph in the document, and is also an important prerequisite for correctly typesetting the paragraph after the element type of the paragraph is determined.

[0106] Specifically, for the correct segmentation of the original document, the difficulty lies in that the original document is correctly segmented at the display level (visual level), but if the segmentation is performed based on the document data of the original document, it will lead to incorrect segmentation due to the different segmentation standards adopted. A simple example is that there are three strings in the document data of the original document: string 1, string 2 and string 3, where string 1 and string 2 are separated by a soft return, and string 2 and string 3 are separated by a hard return. From the visual level, it can be clearly determined that string 1, string 2 and string 3 belong to different paragraphs. However, from the document data level, the original document is segmented using the segmentation standard of the hard return character, and the result of the segmentation is: string 1 and string 2 belong to one paragraph, and string 3 belongs to another paragraph, that is, a segmentation error occurs. In this application, segmentation is performed based on the document data of the original document. In order to obtain correct segmentation, it is necessary to pre-process the copy document of the original document according to the segmentation standard.

[0107] On this basis, the specific implementation of obtaining the target document to be processed includes: creating a copy document of the original document, preprocessing the copy document according to a set preprocessing method, obtaining a preprocessed document, and determining the preprocessed document as the target document to be processed. As for how to preprocess the copy document of the original document according to a set preprocessing method to obtain a preprocessed document, and determine the preprocessed document as the target document to be processed, it will be explained through specific embodiments below, and will not be described in detail here.

[0108] Step 202: Identify redundant content from paragraphs in the target document, where the redundant content is paragraphs of determined element types.

[0109] From the above description, we can know that in real and valid text content, there are also some paragraphs whose element types can be directly determined by the server based on the paragraph format. For these paragraphs whose element types can be directly determined by the server, there is no need to perform tedious analysis and identification through the model.

[0110] The real and effective text content is the text content in the document that can be rendered as visible pixels after typesetting. For example, if the tab character is set not to be displayed, the rendered document will not display the text content of the tab character during display, that is, the visible pixels corresponding to the tab character will not be displayed, but will be displayed as blank.

[0111] Therefore, in step 202, paragraphs whose element types can be determined on the server side are identified from the paragraphs of the target document, and these paragraphs are regarded as redundant content. Redundant content here refers to information that no longer needs to be repeatedly processed for the model to perform the element type identification task.

[0112] In one embodiment, the specific implementation of identifying redundant content from paragraphs of the target document includes: identifying paragraphs that meet a specific element format from the paragraphs of the target document (for the convenience of description, the identified paragraph that meets the specific element format is referred to as the first paragraph), and determining the first paragraph as redundant content.

[0113] The specific element format may include one or more types, and different specific element formats correspond to different element types. The judgment basis of different element types is usually different.

[0114] Exemplarily, a specific element format indicates a specific paragraph length. In electronic documents, the main body often contains longer paragraphs, while elements such as titles and abstracts are relatively short. Therefore, if the content length of a paragraph far exceeds the scope of a regular title or brief description, then we can reasonably infer that this paragraph is likely to belong to the main body, rather than element types such as titles or abstracts. Accordingly, by setting a length threshold, extra-long paragraphs that may belong to the main body can be screened out from the target document based on the length threshold, and these extra-long paragraphs can be identified as the first paragraph, that is, redundant content, without the need to identify the element type through the model.

[0115] In addition, here is another point to add: in practical applications, the structure of electronic documents is diverse and complex, and the diversity is reflected in the different format requirements of electronic documents of different categories. Due to the diversity and complexity of the structure of electronic documents, there may be some problems in the method of determining redundant content based on paragraph length. For example, in a contract document, the title and specific content of a clause are usually integrated in one paragraph, which results in a large number of words in the paragraph and relatively complex content. If a paragraph is simply determined as a body paragraph according to the rule that the paragraph length exceeds the set length threshold, that is, the paragraph is regarded as redundant content of the document element recognition model, and it is no longer sent to the document element recognition model for further element type recognition, it is likely to miss some key information, resulting in errors in the recognition result. In order to solve this problem, as an embodiment, it is determined whether to determine the paragraph in the target document whose length exceeds the set length threshold as redundant content according to the category of the target document. This method can better meet the needs of practical applications when facing various document processing requirements.

[0116] Exemplarily, a specific element format indicates a specific content and structure. In an electronic document, the structure and content of objects such as tables, pictures, charts, text boxes, shapes, etc. are relatively fixed. For example, a table has a row and column structure, usually including a title row and a data row. Pictures are usually surrounded by borders. Charts are similar to pictures and usually include annotations such as titles, legends, and coordinate axes. Therefore, by identifying the characteristics of paragraph content and structure, paragraphs formed by these objects can be distinguished. For example, in the paragraph content, the object type of the object included in the paragraph can be determined according to the object label in the document data, such as any one of a table, an image, and a shape. These paragraphs can then be identified as the first paragraph, i.e., redundant content, without the need to perform element type identification through a model. It should be noted that the so-called paragraph formed by an object means that the paragraph content only includes objects, or it can be said that the object is a paragraph in itself. The reason for this setting is: if an object is inserted into a body paragraph as an illustration or data support, the paragraph should not be identified as the first paragraph. If a paragraph that contains both the above-mentioned objects and the main text is identified as the first paragraph, there may be recognition errors, which may lead to errors in subsequent processing (such as summary generation, information extraction, etc.). For example, the user will set an icon (picture) at the beginning of a paragraph whose element type is a title to better present the title. However, if it is simply assumed that there is a picture in the paragraph, then the element type of the paragraph is a main text paragraph, which will lead to an error in the identification of the element type of the paragraph. Therefore, for paragraph content that does not only include objects, its element type cannot be directly determined and needs to be further identified by the model.

[0117] Exemplarily, a specific element format indicates a specific format and layout. In an electronic document, some paragraphs may follow fixed format and layout requirements, such as headers, footers, date marks, etc. Therefore, by identifying the characteristics of paragraph format and layout, paragraphs of element types such as headers, footers, and date marks can be distinguished, and these paragraphs can be identified as the first paragraph, that is, redundant content, without the need to perform element type identification through the model.

[0118] In addition, the target document may also contain empty paragraphs, that is, paragraphs formed by space characters, which have no substantial content and therefore do not require the model to identify the element type. Therefore, empty paragraphs can also be identified from the target document and determined as redundant content.

[0119] Step 203: Use the document element recognition model to perform document recognition on the non-redundant content in the target document to obtain the model recognition results of the paragraphs in the non-redundant content. The non-redundant content refers to the other content in the target document except the redundant content. The model recognition results include the element type of the paragraph.

[0120] It can be seen from the description of step 203 that, in one embodiment, only the other content in the target document except the redundant content determined in the aforementioned step 202 (referred to as non-redundant content) will be sent to the model side, and the model side will call the document element recognition model to perform element type recognition on the paragraphs in this part of non-redundant content, thereby enabling the model to complete the element type recognition task more efficiently and accurately.

[0121] Step 204: Determine a final recognition result of the paragraph in the target document according to the model recognition result of the paragraph in the non-redundant content and the element type of the paragraph in the redundant content.

[0122] It can be known from the description of the above steps that the element type of the paragraph in the redundant content is determined on the server side. For example, the server side determines the element type of the first paragraph according to the specific element format satisfied by the paragraph in the redundant content (i.e., the first paragraph). The element type of the paragraph in the non-redundant content is determined by the model side. Then, according to the model recognition result of the paragraph in the non-redundant content and the element type determined by the server side for the paragraph in the redundant content, the final recognition result of the paragraph in the target document can be determined.

[0123] Exemplarily, in the process of identifying redundant content from the paragraphs of the target document, the identifier of the paragraph in the non-redundant content is determined, and then in the process of the document element recognition model performing document recognition on the non-redundant content in the target document, a mapping relationship between the paragraph identifier and the model recognition result can be established. In step 204, through the mapping relationship, the model recognition result can be mapped back to the corresponding paragraph in the target document, thereby determining the model recognition result of the paragraph in the non-redundant content in the target document.

[0124] In addition, the document element recognition model usually does not pay attention to the document's formatting attributes, such as font, font size, bold, italic, underline, alignment, etc. when identifying the element type of a paragraph. This is because the main goal of the document element recognition model is to understand and classify the information content in the document from a semantic level, rather than the visual presentation of the document content. Therefore, before sending the above-mentioned non-redundant content to the model end, the formatting attributes of the non-redundant content can also be ignored to reduce the burden on the model and avoid increasing the complexity of the model's processing tasks due to the model's failure to recognize complex formatting attributes, which also leads to a decrease in recognition accuracy.

[0125] The technical solution provided by the embodiment of the present application obtains the target document to be processed, identifies paragraphs of determined element types from the paragraphs of the target document as redundant content, uses the document element recognition model to perform document recognition on the non-redundant content in the target document, and obtains the model recognition result of the paragraph in the non-redundant content. The non-redundant content refers to the other content in the target document except the redundant content. According to the model recognition result of the paragraph in the non-redundant content and the redundant content, the final recognition result of the paragraph in the target document is determined, thereby realizing the collaborative work between the server and the model. This collaborative work mechanism can more effectively utilize the processing power of the model and avoid the model from processing unnecessary redundant content, which can not only improve the recognition efficiency and accuracy of the model, but also reduce the burden of the model.

[0126] It can be seen from the description of the above embodiments that for the non-redundant content in the target document, the server cannot determine the element type of the paragraph after preliminary identification, so the server calls the document element recognition model deployed on the model side to identify the paragraph element type of the non-redundant content. However, as far as non-redundant content is concerned, there may also be some information that is not required for the model to perform the element type recognition task. This information may include content that is invisible in the visual presentation effect, such as continuous space characters and separators in the paragraph. This information occupies the document space but is of no substantial use for the model to identify the element type. In addition, for objects such as tables and pictures embedded in paragraphs, the model only needs to identify their identity as embedded objects in the paragraph during processing, without paying attention to the specific content of the object itself.

[0127] Based on this, in order to further optimize the model processing flow, improve the recognition efficiency of the model and reduce its burden, the server can further streamline the non-redundant content before sending it to the model, aiming to remove information in the non-redundant content that is not helpful for the model to identify the element type. This processing method further enables the model to focus more on the core recognition task, significantly improves the overall processing efficiency of the model, and ensures the accuracy and timeliness of document element recognition.

[0128] For example, see Figure 3 , which is a flow chart of another document processing method provided in an embodiment of the present application. Figure 3 The process shown in Figure 2 Based on the process shown, the following steps are included:

[0129] Step 301: Obtain the target document to be processed.

[0130] Step 302: Identify redundant content from paragraphs in the target document, where the redundant content is paragraphs of determined element types.

[0131] For detailed description of steps 301 and 302, please refer to Figure 2 The relevant description in the illustrated embodiment will not be repeated here.

[0132] Step 303: Identify the content to be simplified from the paragraphs of the non-redundant content of the target document, perform a simplification operation on the content to be simplified, and obtain simplified non-redundant content.

[0133] From the above description, it can be seen that the content to be simplified in the non-redundant content refers to the information that is not necessary for the model to identify the element type. In other words, the content that is not necessary for the document element identification model to identify the element type is identified from the paragraphs of the non-redundant content of the target document, and the identified content is simplified to obtain the simplified non-redundant content. These contents usually include two categories. One is the content that is invisible in the visual presentation effect, such as continuous space characters and separators in a paragraph. The other is the content that the model only needs to identify as an embedded object in a paragraph during processing, without paying attention to its specific content itself, such as tables, pictures and other objects embedded in a paragraph. The following explains these two situations respectively:

[0134] In one embodiment, identifying the content to be simplified from the paragraphs of the non-redundant content of the target document, and performing a simplification operation on the content to be simplified to obtain the simplified non-redundant content specifically includes: identifying non-visual content from the paragraphs of the non-redundant content of the target document, determining the non-visual content as the content to be simplified, and deleting the non-visual content from the paragraphs of the non-redundant content of the target document to obtain the simplified non-redundant content.

[0135] The above non-visual content is the content that is invisible in the visual presentation effect, that is, the non-visual content that is not displayed in the displayed document. Non-visual content does not have semantics and is only used to adjust the document layout. The model does not need it when performing the feature type recognition task. Therefore, the non-visual content can be directly deleted from the paragraphs of non-redundant content in the target document to obtain the streamlined non-redundant content.

[0136] Exemplarily, the specific implementation of identifying non-visual content from paragraphs of non-redundant content in the target document and determining the non-visual content as content to be simplified includes: identifying space characters from paragraphs of non-redundant content in the target document and determining the space characters as content to be simplified; and / or, identifying separators from the target document and determining the separators as content to be simplified.

[0137] In another embodiment, content to be simplified is identified from paragraphs of non-redundant content in the target document, and a simplification operation is performed on the content to be simplified to obtain the simplified non-redundant content. The specific implementation includes: identifying elements of a preset type from paragraphs of non-redundant content in the target document, converting the elements of the preset type in the paragraphs into content indicator symbols, which are used to represent elements of the preset type.

[0138] The above-mentioned preset type of elements, that is, the model only needs to identify its identity as an embedded object in the paragraph when processing, without paying attention to its specific content. In other words, for the preset type of elements, the model only needs to identify its identity (such as the type and position of the content), without paying attention to the specific content therein. Therefore, the preset type of elements in the paragraph can be converted into corresponding content indicator symbols, and then the model can identify the element type according to the content indicator symbols.

[0139] Exemplarily, the specific implementation of identifying the content to be simplified from the paragraphs of the non-redundant content of the target document, simplifying the content to be simplified, and obtaining the simplified non-redundant content includes: identifying object elements from the paragraphs of the non-redundant content of the target document, determining the object elements as the content to be simplified, and converting the object elements into corresponding placeholders; and / or, identifying table elements from the paragraphs of the non-redundant content of the target document, determining the table elements as the content to be simplified, and converting the table elements into the position information of the table elements in the target document. The above-mentioned object elements include but are not limited to: objects such as pictures, charts, shapes, and text boxes. The placeholder can be a label indicating the type of object, such as the label of an image can be "img" and the label of a chart can be "tab".

[0140] In addition, in actual applications, there is a type of content to be simplified. This type of content to be simplified is interspersed in the document in an intermediate form. However, when the model performs feature type recognition tasks, it does not need to pay attention to these intermediate forms. The model is more concerned about the final form of these contents, that is, the final form displayed in the document.

[0141] In this regard, in one embodiment, the specific implementation of identifying the content to be simplified from the paragraphs of the non-redundant content of the target document, and performing a simplification operation on the content to be simplified to obtain the simplified non-redundant content includes: identifying the intermediate content to be processed as the final content from the paragraphs of the non-redundant content of the target document, and converting the intermediate content into the final content. By converting the intermediate content into the corresponding final content, it is possible to avoid the intermediate content from interfering with the model's document element recognition, ensuring that the model can focus on key information when performing the document element recognition task, thereby improving the accuracy of the model's document element recognition.

[0142] Exemplarily, domain elements are identified from paragraphs of non-redundant content in the target document, the domain elements are determined as content to be simplified, and the domain elements are converted into corresponding domain results.

[0143] Domain elements are used to insert dynamic content in documents, such as page numbers, dates, etc. In the final typesetting of the document, domain elements should be replaced with domain results. Therefore, domain elements are identified in paragraphs of non-redundant content and converted into corresponding domain results.

[0144] Exemplarily, revised content with revision marks is identified from paragraphs of non-redundant content in the target document, the revised content is determined as content to be streamlined, and the revised content is converted into corresponding revised content.

[0145] Revision marks indicate that the document has been modified, such as strikethrough, underline, text of different colors, etc. In the final typesetting of the document, revision marks should be replaced with revised content. Therefore, the revised content with revision marks is identified in the paragraphs of non-redundant content, and the revised content is converted into the corresponding revised content.

[0146] In one embodiment, it is further possible to determine whether the streamlined non-redundant content is redundant content. For example, after the domain element is converted into the corresponding domain result, the paragraph where the length of the paragraph where the domain is located exceeds the set length threshold is determined to be redundant content.

[0147] In addition, the document element recognition model usually does not pay attention to the document's formatting attributes, such as font, font size, bold, italic, underline, alignment and the like, when identifying the element type of a paragraph. This is because the main goal of the document element recognition model is to understand and classify the information content in the document from a semantic level, rather than the visual presentation of the document content. Therefore, after identifying the content to be streamlined from the paragraphs of non-redundant content in the target document and streamlining the content to be streamlined, the specific implementation of obtaining the streamlined non-redundant content also includes: cleaning the formatting attributes of the non-redundant content to reduce the burden on the model, avoiding the increase in the complexity of the model's processing tasks due to the model's failure to recognize complex formatting attributes, and also resulting in a decrease in recognition accuracy.

[0148] Step 304: Use the document element recognition model to perform document recognition on the streamlined non-redundant content to obtain model recognition results of paragraphs in the non-redundant content of the target document.

[0149] Step 305: Determine a final recognition result of the paragraph in the target document according to the model recognition result of the paragraph in the non-redundant content and the element type of the paragraph in the redundant content.

[0150] For detailed description of steps 304 and 305, please refer to Figure 2 The relevant descriptions in the process shown will not be repeated here.

[0151] Figure 3 In the illustrated embodiment, the server further simplifies the non-redundant content before sending it to the model, so as to remove unnecessary information in the non-redundant content for the model to identify the element type. This processing method further enables the model to focus more on the core recognition task, significantly improves the overall processing efficiency of the model, and ensures the accuracy and timeliness of document element recognition.

[0152] Here is another point to add: In practical applications, the structure of electronic documents is diverse and complex, and the diversity is reflected in the different format requirements of electronic documents of different categories. Due to the diversity and complexity of the structure of electronic documents, there may be some problems in the method of determining redundant content based on paragraph length. For example, in a contract document, the title and specific content of the clause are usually integrated in one paragraph, which results in a large number of words in the paragraph and relatively complex content. If a paragraph is simply determined as a body paragraph according to the rule that the paragraph length exceeds the set length threshold, that is, the paragraph is regarded as redundant content of the document element recognition model, and it is no longer sent to the document element recognition model for further element type recognition, it is likely to miss some key information, resulting in errors in the recognition result. In order to solve this problem, as an embodiment, part of the redundant content determined according to the rule that the paragraph length exceeds the set length threshold can be sent to the document element recognition model for further recognition, which can not only reduce the recognition task of the document element recognition model to a certain extent, but also ensure the accuracy of document element recognition. Among them, it is usually chosen to send the front content of the redundant content to the document element recognition model for recognition. The reason is that in many cases, important information such as the title is located at the beginning of the paragraph. In this way, it can be ensured as much as possible that key information such as the title can be sent to the document element recognition model for recognition, thereby effectively avoiding the problem of recognition errors caused by missing information. In addition, when segmenting the front content from the redundant content, it can be segmented based on punctuation marks, such as periods, to ensure that the semantics of the part of the content given to the model is as coherent and smooth as possible. Correspondingly, the element type of the paragraph in the redundant content is also the element type recognized by the document element recognition model.

[0153] Based on the above content, in one embodiment, when paragraphs in the target document whose length exceeds a set length threshold are determined to be redundant content according to the category of the target document, part of the redundant content can also be sent to a document element recognition model for further recognition. Such processing can also reduce the recognition task of the document element recognition model to a certain extent, and ensure the accuracy of document element recognition.

[0154] The following describes through a specific embodiment how to pre-process the copy document of the original document according to a set pre-processing method to obtain the target document processed in the embodiment of the present application.

[0155] In one embodiment, the copy document is preprocessed according to a set preprocessing method, and the specific implementation of obtaining the preprocessed document includes: identifying non-standard typesetting styles related to segmentation or paging from the copy document, converting the non-standard typesetting styles into a set standard typesetting style, and obtaining the preprocessed document.

[0156] Among them, non-standard typesetting styles refer to segmentation or paging operations that do not meet the model recognition requirements. These operations may be due to the document creator using specific tools or specific habits. For example, the segmentation operation required by the model recognition can be based on the hard return character in the document data to determine the segmentation. Non-standard typesetting styles usually cause the model to be unable to accurately recognize the paragraph structure of the document, thereby affecting subsequent feature type recognition tasks. For example, the document creator uses space characters to achieve a segmentation effect in a certain paragraph of the document. Although this can achieve the segmentation effect visually, the document feature recognition model deployed on the model side cannot understand this segmentation achieved by space characters. Therefore, this segmentation operation is regarded as a non-standard typesetting style.

[0157] The standard typesetting style refers to the segmentation operation that meets the model recognition requirements. Such segmentation operation is usually based on the general specifications or standards in the document processing field, such as specific paragraph markers.

[0158] It can be seen that by identifying non-standard typesetting styles related to paragraphing or paging from the copy document, converting the non-standard typesetting styles into the set standard typesetting styles, and obtaining the pre-processed document, it can ensure that the non-standard typesetting styles in the copy document are corrected and converted into standard typesetting styles that the model can accurately recognize. This helps the model accurately recognize the paragraph structure of the document, thereby supporting the subsequent feature type recognition task.

[0159] The standardization of the paragraphing style used in the document (the standardization here refers to the recognition ability of the model) will directly affect the model's recognition of paragraphs in the document. In addition, the separation style used in the document is also crucial to the coherence of the model's paragraph recognition. Therefore, the following discusses the specific implementation of identifying non-standard typesetting styles related to paragraphing or paging from the copy document from the two dimensions of paragraphing style and separation style, and converting non-standard typesetting styles into the set standard typesetting style.

[0160] Among them, from the perspective of segmentation style, as an optional implementation method, non-standard typesetting styles related to segmentation or paging are identified from the copy document, and the specific implementation of converting the non-standard typesetting style into a set standard typesetting style includes: identifying non-standard segmentation styles from the copy document, and converting the non-standard segmentation styles into a set standard segmentation style.

[0161] Exemplarily, the standard segmentation style is a paragraph marker (also known as a hard return), that is, the paragraph marker is used as a standard separation mark between document paragraphs, and the document element recognition model can accurately recognize the standard separation mark as the basis for paragraph recognition.

[0162] In this embodiment, based on the recognition rules corresponding to the preset non-standard typesetting styles, the non-standard typesetting styles can be identified from the copy document based on the recognition rules; in addition, the conversion rules for converting different non-standard typesetting styles into standard typesetting styles can also be determined, and standardized conversion can be performed based on the conversion rules.

[0163] The following examples describe the following non-standard segmentation styles that may exist in the copy document and their corresponding conversion rules:

[0164] The first type: soft return segmentation

[0165] In document editing scenarios, soft returns are often used to wrap lines within the same paragraph, rather than to create new paragraphs. However, due to the user's specific editing habits or insufficient understanding of document typesetting specifications, soft returns may be mistakenly used to try to visually create a segmentation effect.

[0166] In view of this situation, the non-standard segmentation style is identified from the copy document, and the specific implementation of converting the non-standard segmentation style into the set standard segmentation style includes: identifying the soft carriage return from the copy document, and converting the soft carriage return into a paragraph marker. By converting the soft carriage return into a paragraph marker, it is helpful to restore the correct paragraph structure of the document, thereby helping the subsequent model to accurately identify the paragraphs in the copy document and then identify the element type of the paragraph.

[0167] For example, see Figure 4 , which is a diagram comparing the effects of using soft carriage returns and hard carriage returns in text content.

[0168] The second type: the separator is glued to the content paragraph

[0169] In document editing scenarios, separators (such as page breaks and section breaks) are usually used to visually or logically separate document content, and can usually also achieve segmentation in visual effects. For example, Figure 5 The string "I am a free paragraph at the top of the page" on page 2 and the string "I am a glued paragraph at the top of the page, I am part of the paragraph on the previous page" on page 3 are visually displayed as two paragraphs due to the existence of page breaks. However, if the hard return character is used as the paragraph separation criterion, the string "I am a free paragraph at the top of the page" on page 2 and the string "I am a glued paragraph at the top of the page, I am part of the paragraph on the previous page" on page 3 will still be recognized as the same paragraph (see Figure 5 , there is no hard return after the string in page 2), resulting in a difference between the segmentation based on document data and the visual effect segmentation that conforms to user intent.

[0170] That is to say, due to the user's editing habits or specific needs, these separators are sometimes attached to the paragraph content, making it look like two paragraphs visually. However, if hard returns are used as the segmentation standard, the actual segmentation results will not be correctly separated in the document structure, that is, the two paragraphs separated by the separator will still be divided into the same paragraph.

[0171] In view of this situation, the non-standard paragraphing style is identified from the copy document, and the specific implementation of converting the non-standard paragraphing style to the set standard paragraphing style includes: identifying the separators attached to the paragraph content from the copy document, inserting paragraph markers between the separators and the attached paragraph content, so as to restore the correct paragraph structure of the document. Figure 5 , where page 1 is an example of correct paragraph structure. Figure 5 As shown on page 1 of the text, the string "I am a free paragraph at the top of the page" is followed by a page break, and there is also a hard return after the page break. Based on the hard return as the segmentation standard, the string "I am a free paragraph at the top of the page" in page 1 and the string "I am a free paragraph at the top of the page" in page 2 are identified as two paragraphs. In addition, there is another correct paragraph structure: the delimiter is at the beginning of the paragraph, that is, the delimiter is after the hard return. For example, suppose Figure 5 The content of page 1 is "I am a free paragraph hard return at the top of the page", and the content of page 2 is "Separator I am a free paragraph hard return at the top of the page", which is also the correct paragraph structure and can be used to Figure 4 The string in page 1 and the string in page 2 are correctly identified as two paragraphs.

[0172] Based on this, firstly, the separator mark (such as the specific code of the page break, the identifier of the section break, etc., which vary depending on the document format) is identified from the copy document, and then, for the identified separator, it is analyzed whether it is attached to the paragraph content. Here, the attachment of the separator mark to the paragraph content is manifested as the absence of a paragraph mark between the separator and the paragraph content. For example, see Figure 5 In the example on page 2 of the article, the delimiter tag is attached to the string "I am a free paragraph at the top of the page". Another case is that there is no paragraph tag after the delimiter tag. In other words, if there is no paragraph tag before or after the delimiter tag, it can be considered that the delimiter tag is attached to the paragraph content. In this case, insert a paragraph tag (such as a hard return) between the delimiter tag and the attached paragraph content to ensure that the paragraphs can be separated correctly. For example, Figure 5 Change page 2 in to "I am a free paragraph hard return section break at the top of the page (next page)".

[0173] The third type: space or tab segment

[0174] In a document editing context, users may use spaces or tabs to try to visually separate paragraphs, but these characters do not actually form true paragraph breaks in the document structure. For example, see Figure 6 , which is a schematic diagram of the segmentation effect of space or tab characters.

[0175] In view of this situation, the specific implementation of identifying the non-standard segmentation style from the copy document and converting the non-standard segmentation style into the set standard segmentation style includes: identifying the line ending with multiple consecutive spaces or tabs from the copy document, deleting the multiple consecutive spaces or tabs at the end of the line, and inserting a paragraph mark at the end of the line. Exemplarily, the copy document content is traversed line by line, and it is checked whether the end of each line contains multiple consecutive spaces or tabs. If continuous spaces or tabs are found, these characters are deleted and a paragraph mark is inserted at the end of the line. Specifically, the copy document content is traversed line by line, and for the current traversed line, it is traversed one by one from the first character (for example, in order from left to right), and when a space character or tab character is traversed, the traversal continues. If a valid character (for example, a non-space character and a non-tab character) is traversed subsequently, the traversal stops for the current line and continues to traverse the next line. If the last character position of the current line is still a space character or a tab character, it means that the current line ends with consecutive space characters or tab characters. Then, delete these consecutive space characters or tab characters and insert a paragraph mark at the end of the current line. In this way, the line between the current line and the previous paragraph mark will be separated into a paragraph, and the next line of the current line will become the starting line of a new paragraph.

[0176] The fourth case: the text content is not segmented

[0177] In document editing scenarios, users may fail to divide text content into multiple paragraphs correctly due to negligence or specific editing habits, resulting in multiple paragraphs being merged into one paragraph. This merging is particularly obvious when the document contains numbered lists, because each numbered item should usually be considered an independent paragraph. For example, see Figure 7 , which is a schematic diagram of the effect of not correctly segmenting text content.

[0178] In view of this situation, the specific implementation of identifying the non-standard paragraphing style from the copy document and converting the non-standard paragraphing style into the set standard paragraphing style includes: identifying a paragraph including at least two numbers from the copy document, and inserting a paragraph marker before the non-first number in the paragraph. Exemplarily, the document content is traversed line by line or paragraph by paragraph to find the paragraphs containing numbers, and for each paragraph containing at least two numbers, the position of the non-first number is found. A paragraph marker is inserted before each non-first number to ensure that after the paragraph marker is inserted, each numbered item is located in a new paragraph.

[0179] In addition, when the text content is not segmented and does not contain a numbered list, the model can also be used to segment the text content. For example, the model segments the text content using contextual semantic recognition technology. This is not limited here.

[0180] From the perspective of separation style, as an optional implementation method, non-standard typesetting styles related to paragraphing or paging are identified from the copy document, and the specific implementation of converting the non-standard typesetting styles into the set standard typesetting styles includes: identifying non-standard separation styles from the copy document, and converting the non-standard separation styles into the set standard separation styles.

[0181] For example, the following non-standard separation styles may exist in the copy document:

[0182] The first type: Section / page break paging

[0183] In a document editing scenario, users may use different paging or sectioning methods to organize content, such as directly inserting page breaks or inserting blank pages (which usually introduce section breaks) to achieve paging effects. However, these different paging or sectioning methods may lead to inconsistent document structures, thus affecting the readability of the document and the efficiency and accuracy of document analysis and recognition.

[0184] In response to this situation, a non-standard separation style is identified from the copy document. The specific implementation of converting the non-standard separation style to the set standard separation style includes: identifying a section break or page break at the end of a line from the copy document, and moving the section break or page break at the end of a line to the beginning of the next line of the line to ensure consistency of the paging or sectioning method in the copy document.

[0185] The second type: paragraph break and line break

[0186] In a document editing scenario, users may sometimes achieve page breaks by simply pressing Enter to add a blank line, but this is not recommended because it will become ineffective as the document layout is adjusted. For example, see Figure 7 , a schematic diagram of using consecutive blank lines to achieve paging effect

[0187] In response to this situation, a non-standard separation style is identified from the copy document. The specific implementation of converting the non-standard separation style to the set standard separation style includes: identifying a page ending with continuous blank lines from the copy document, inserting a page break at the beginning of the first line of the next page of the page, and deleting the continuous blank lines at the end of the page to ensure the consistency and stability of the paging method in the copy document.

[0188] Specifically, for each page of the copy document, traverse line by line. For the current traversed line, if it is found that there are no valid characters in the line, for example, all are space characters or tab characters, then continue to traverse the next line. If there are valid characters in the traversed line, stop traversing the current page and continue to traverse the next line by line. If it is traversed to the last line of the current page and it is found that there are still no valid characters in the last line, it means that the current page ends with continuous blank lines. Then, insert a page break at the beginning of the first line of the next page of the page, and delete the continuous blank lines at the end of the page. Through this processing method, page redundancy caused by continuous blank lines can be avoided, and the page division of the document can also be made clearer and more reasonable.

[0189] The third type: paragraph attribute paging

[0190] In a document editing scenario, users may use paragraph properties (such as "page break before" or "page break after") to achieve paging effects. This method is more common for professional users such as Office, but it may also be inadvertently introduced into the document due to operations such as copying and pasting, causing other users to be unaware or overwhelmed.

[0191] In view of this situation, the specific implementation of identifying the non-standard delimiting style from the copy document and converting the non-standard delimiting style into the set standard delimiting style includes: identifying the paragraphs with paragraph attributes set from the copy document, determining the first paragraph of the next page according to the paragraph attributes set for the paragraph, and inserting a page break at the beginning of the first paragraph of the next page to ensure the consistency and readability of the paging method of the copy document. Exemplarily, the copy document is traversed line by line or paragraph by paragraph to identify the paragraphs with specific paragraph attributes (such as page break before paragraph, page break after paragraph, etc.). After determining the first paragraph of the next page, a page break is inserted at the beginning of the paragraph, and the paragraph attributes originally used for paging are cleared.

[0192] By deeply analyzing the paragraph and separation styles in the copy documents, identifying the non-standard typesetting styles formed by the document creator's use of specific tools or habits, and converting these non-standard typesetting styles into the set standard typesetting styles, it can ensure that the model can accurately recognize and understand the paragraph structure of the copy documents.

[0193] The consistency and standardization of numbering are also crucial for the model to understand the logical structure of the document. In the document editing scenario, users may use a variety of numbering methods, including manual numbering, single-level automatic numbering, multi-level automatic numbering, etc., and these numbering styles may involve numbers, Chinese characters, special symbols, etc. In addition, the complex relationship between numbering and paragraphs, such as indentation and spacing, may also cause the typesetting to not meet user expectations. Therefore, unifying the numbering method and achieving controllability and consistency of typesetting is an important task in document processing.

[0194] Based on this, as an optional implementation, the specific implementation of identifying the non-standard typesetting style related to paragraphing or paging from the copy document and converting the non-standard typesetting style to the set standard typesetting style includes: identifying the non-standard numbering style from the copy document and converting the non-standard numbering style to the set standard numbering style. Exemplarily, the specific implementation of identifying the non-standard numbering style from the copy document and converting the non-standard numbering style to the set standard numbering style includes: identifying the number set with the automatic numbering style from the copy document and canceling the automatic numbering style setting for the number.

[0195] It can be seen that manual numbering is used in the copy documents to avoid the complexity brought by automatic numbering, thereby improving the reliability and consistency of the copy document layout and laying the foundation for the subsequent model to accurately understand the logical structure of the document.

[0196] In addition, there will be some non-standard content in the document. The non-standard content here refers to the content form in the document that does not conform to the set standard specifications. The set standard specifications are, for example, customized by document processing software, industry practices or users, and are intended to ensure the consistency and readability of the document. Exemplary, non-standard content includes: domain elements, revision marks, etc. Domain elements are used to insert dynamic content into the document, such as page numbers, dates, etc. In the final layout of the document, domain elements should be replaced with domain results. Modification marks indicate that the document has been modified, such as strikethroughs, underlines, texts of different colors, etc. In the final layout of the document, revision marks should be replaced with revised content. It can also be seen from this that non-standard content usually refers to the intermediate form of content generated during the document editing process, while standard content refers to the content presented in the final layout of the document.

[0197] Due to the limited computing power of the model, non-standard content may interfere with the model's paragraph recognition. In this regard, as an optional implementation method, non-standard typesetting styles related to segmentation or paging are identified from the copy document, and the specific implementation of converting the non-standard typesetting styles to the set standard typesetting styles includes: identifying non-standard content styles from the copy document, and converting the non-standard content styles to the set standard content styles. By normalizing the non-standard content styles in the copy document, it can be ensured that they will not interfere with the model's recognition.

[0198] Exemplarily, the specific implementation of identifying non-standard content styles from the copy document and converting the non-standard content styles into the set standard content styles includes: identifying domain elements from the copy document, determining domain results of the domain elements, and using the domain results to replace the domain elements in the copy document.

[0199] Exemplarily, the specific implementation of identifying non-standard content styles from a copy document and converting the non-standard content styles into a set standard content style includes: identifying revised content with revision marks from the copy document, determining the revised content of the revised content, and replacing the revised content with the revised content in the copy document.

[0200] In addition to domain elements and revision content, there may be other types of non-standard content, such as hidden text, footnotes / endnotes, comments, etc. For these contents, corresponding identification and conversion strategies can be formulated according to specific needs, and the embodiments of the present application do not limit this.

[0201] In the above embodiment, a copy document of the original document is created, the copy document is preprocessed according to a set preprocessing method, and a preprocessed document is obtained. The preprocessed document is determined as a target document to be processed, thereby ensuring that the content in the target document is correctly segmented, thereby improving the accuracy of identifying and streamlining redundant content in the document.

[0202] Furthermore, redundant content and non-redundant content are determined in the accurately segmented target document, a model recognition result of the streamlined non-redundant content is determined, and the element type of the redundant content is determined based on the paragraph format, content, structure, etc. of the redundant content. Based on the model recognition result of the paragraph in the non-redundant content and the element type of the paragraph in the redundant content, a final recognition result of the paragraph in the target document and / or the original document of the target document is determined.

[0203] Further, after obtaining the final recognition result of the paragraph in the target document, the final recognition result of the paragraph in the original document can also be determined based on the paragraph mapping relationship between the original document and the target document, and the final recognition result of the paragraph in the target document. Exemplary, in the process of pre-processing the copy document, the paragraph mapping relationship between the original document and the pre-processed document (i.e., the target document) can also be established. Specifically, in the original document, a unique identifier (such as paragraph number, paragraph ID, etc.) is assigned to each paragraph. In the process of pre-processing the copy document of the original document, the original position and the modified position of each paragraph are recorded to establish the mapping relationship between the paragraph in the original document and the corresponding paragraph in the pre-processed document. In the above embodiment, through the paragraph mapping relationship, the final recognition result of the paragraph in the target document can be corresponded back to the corresponding paragraph in the original document, thereby determining the final recognition result of the paragraph in the original document.

[0204] Fig. 9 This is a block diagram of an embodiment of a document processing device provided in an embodiment of the present application. Fig. 9 As shown, the device comprises:

[0205] The document acquisition module 91 is used to acquire the target document to be processed;

[0206] A redundancy identification module 92, configured to identify redundant content from paragraphs of the target document, wherein the redundant content is a paragraph of a determined element type;

[0207] A model recognition module 93 is used to perform document recognition on the non-redundant content in the target document using the document element recognition model to obtain a model recognition result of a paragraph in the non-redundant content, wherein the non-redundant content refers to other content in the target document except the redundant content, and the model recognition result includes an element type of the paragraph;

[0208] The result determination module 94 is used to determine the final recognition result of the paragraph in the target document according to the model recognition result of the paragraph in the non-redundant content and the element type of the paragraph in the redundant content.

[0209] In a possible implementation manner, the redundancy identification module 92 is specifically configured to:

[0210] A first paragraph that meets a specific element format is identified from paragraphs of the target document, and the first paragraph is determined to be redundant content.

[0211] In a possible implementation, the model identification module 93 includes:

[0212] A simplification element is used to identify the content to be simplified from the paragraphs of the non-redundant content of the target document, and perform a simplification operation on the content to be simplified to obtain the simplified non-redundant content;

[0213] The recognition unit is used to perform document recognition on the simplified non-redundant content using the document element recognition model to obtain a model recognition result of the paragraphs in the non-redundant content of the target document.

[0214] In a possible implementation, the simplified element includes:

[0215] A first identification subunit is used to identify non-visual content from the paragraphs of non-redundant content of the target document, and determine the non-visual content as content to be simplified;

[0216] The first simplification subunit is used to delete the non-visual content from the paragraph of the non-redundant content of the target document to obtain simplified non-redundant content.

[0217] In a possible implementation manner, the first identification subunit is specifically used to:

[0218] Identify space characters from paragraphs of non-redundant content in the target document, and determine the space characters as content to be simplified; and / or,

[0219] A separator is identified from the target document, and the separator is determined as content to be simplified.

[0220] In a possible implementation manner, the simplified element is specifically used for:

[0221] Identifying intermediate content to be processed into final content from paragraphs of non-redundant content of the target document, and converting the intermediate content into final content; and / or,

[0222] An element of a preset type is identified from a paragraph of non-redundant content of the target document, and the element of the preset type in the paragraph is converted into a content indicator symbol, where the content indicator symbol is used to represent the element of the preset type.

[0223] In a possible implementation manner, the simplified element is specifically used for:

[0224] Identifying object elements from paragraphs of non-redundant content in the target document, determining the object elements as content to be streamlined, and converting the object elements into corresponding placeholders; and / or,

[0225] Identifying a table element from a paragraph of non-redundant content of the target document, determining the table element as content to be streamlined, and converting the table element into position information of the table element in the target document; and / or,

[0226] Identifying domain elements from paragraphs of non-redundant content of the target document, determining the domain elements as content to be streamlined, and converting the domain elements into corresponding domain results; and / or,

[0227] Revision contents with revision marks are identified from paragraphs of non-redundant contents of the target document, the revision contents are determined as contents to be simplified, and the revision contents are converted into corresponding revised contents.

[0228] In a possible implementation, the result determination module 94 is specifically configured to:

[0229] A final recognition result of the paragraph in the target document is determined according to the model recognition result of the paragraph in the non-redundant content and the specific element format satisfied by the first paragraph.

[0230] In a possible implementation, the document acquisition module 91 includes:

[0231] A copy creation unit, used for creating a copy document of the original document;

[0232] The copy processing unit is used to pre-process the copy document according to a set pre-processing method to obtain a pre-processed document, and determine the pre-processed document as a target document to be processed.

[0233] In a possible implementation manner, the replica processing unit is specifically used to:

[0234] A non-standard typesetting style related to segmentation or paging is identified from the copy document, and the non-standard typesetting style is converted into a set standard typesetting style to obtain a pre-processed document.

[0235] In a possible implementation, the result determination module 94 is further configured to:

[0236] Based on the paragraph mapping relationship between the original document and the target document, and the final recognition result of the paragraph in the target document, the final recognition result of the paragraph in the original document is determined.

[0237] like Fig.10 As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0238] Memory 113, used for storing computer programs;

[0239] In one embodiment of the present application, the processor 111 is used to execute the program stored in the memory 113 to implement the document processing method provided by any of the above method embodiments, including:

[0240] Get the target document to be processed;

[0241] Identifying redundant content from paragraphs of the target document, wherein the redundant content is a paragraph of a determined element type;

[0242] Using the document element recognition model to perform document recognition on non-redundant content in the target document, obtaining a model recognition result of a paragraph in the non-redundant content, wherein the non-redundant content refers to other content in the target document except the redundant content, and the model recognition result includes an element type of the paragraph;

[0243] A final recognition result of the paragraph in the target document is determined according to the model recognition result of the paragraph in the non-redundant content and the redundant content.

[0244] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the document processing method provided in any of the aforementioned method embodiments are implemented.

[0245] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0246] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0247] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0248] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A document processing method, characterized in that: The method comprises: Get the target document to be processed; Identifying redundant content from paragraphs of the target document, wherein the redundant content is a paragraph of a determined element type; Using a document element recognition model to perform document recognition on non-redundant content in the target document, obtaining a model recognition result of a paragraph in the non-redundant content, wherein the non-redundant content refers to other content in the target document except the redundant content, and the model recognition result includes an element type of the paragraph; A final recognition result of the paragraph in the target document is determined according to the model recognition result of the paragraph in the non-redundant content and the element type of the paragraph in the redundant content.

2. The method according to claim 1, characterized in that The step of identifying redundant content from the paragraphs of the target document includes: A first paragraph that meets a specific element format is identified from paragraphs of the target document, and the first paragraph is determined to be redundant content.

3. The method according to claim 1 or 2, characterized in that: The step of using the document element recognition model to perform document recognition on the non-redundant content in the target document to obtain a model recognition result of a paragraph in the non-redundant content includes: Identifying content to be simplified from paragraphs of non-redundant content in the target document, and performing a simplification operation on the content to be simplified to obtain simplified non-redundant content; The document element recognition model is used to perform document recognition on the simplified non-redundant content to obtain a model recognition result of the paragraphs in the non-redundant content of the target document.

4. The method according to claim 3, characterized in that The step of identifying the content to be simplified from the paragraphs of the non-redundant content of the target document, and performing a simplification operation on the content to be simplified to obtain the simplified non-redundant content includes: Identifying non-visual content from paragraphs of non-redundant content in the target document, and determining the non-visual content as content to be simplified; The non-visual content is deleted from the paragraph of the non-redundant content of the target document to obtain simplified non-redundant content.

5. The method according to claim 4, characterized in that The step of identifying non-visual content from the paragraphs of non-redundant content of the target document and determining the non-visual content as content to be simplified includes: Identify space characters from paragraphs of non-redundant content in the target document, and determine the space characters as content to be simplified; and / or, A separator is identified from the target document, and the separator is determined as content to be simplified.

6. The method according to claim 3, characterized in that The step of identifying the content to be simplified from the paragraphs of the non-redundant content of the target document, and performing a simplification operation on the content to be simplified to obtain the simplified non-redundant content includes: Identifying intermediate content to be processed into final content from paragraphs of non-redundant content of the target document, and converting the intermediate content into final content; and / or, An element of a preset type is identified from a paragraph of non-redundant content of the target document, and the element of the preset type in the paragraph is converted into a content indicator symbol, where the content indicator symbol is used to represent the element of the preset type.

7. The method according to claim 3, characterized in that The step of identifying the content to be simplified from the paragraphs of the non-redundant content of the target document, and performing a simplification operation on the content to be simplified to obtain the simplified non-redundant content includes: Identifying object elements from paragraphs of non-redundant content in the target document, determining the object elements as content to be streamlined, and converting the object elements into corresponding placeholders; and / or, Identifying a table element from a paragraph of non-redundant content of the target document, determining the table element as content to be streamlined, and converting the table element into position information of the table element in the target document; and / or, Identifying domain elements from paragraphs of non-redundant content of the target document, determining the domain elements as content to be streamlined, and converting the domain elements into corresponding domain results; and / or, Revision contents with revision marks are identified from paragraphs of non-redundant contents of the target document, the revision contents are determined as contents to be simplified, and the revision contents are converted into corresponding revised contents.

8. The method according to claim 2, characterized in that: The determining, according to the model recognition result of the paragraph in the non-redundant content and the element type of the paragraph in the redundant content, a final recognition result of the paragraph in the target document comprises: A final recognition result of the paragraph in the target document is determined according to the model recognition result of the paragraph in the non-redundant content and the specific element format satisfied by the first paragraph.

9. The method according to any one of claims 1 to 7, characterized in that: The step of obtaining the target document to be processed includes: Create a copy of the original document; The copy document is preprocessed according to a set preprocessing method to obtain a preprocessed document, and the preprocessed document is determined as a target document to be processed.

10. The method according to claim 9, characterized in that The preprocessing of the copy document according to the set preprocessing method to obtain the preprocessed document includes: A non-standard typesetting style related to segmentation or paging is identified from the copy document, and the non-standard typesetting style is converted into a set standard typesetting style to obtain a pre-processed document.

11. The method according to claim 9, characterized in that The method further comprises: Based on the paragraph mapping relationship between the original document and the target document, and the final recognition result of the paragraph in the target document, the final recognition result of the paragraph in the original document is determined.

12. A document processing device, characterized in that: The device comprises: A document acquisition module is used to acquire the target document to be processed; A redundancy identification module, used to identify redundant content from paragraphs of the target document, wherein the redundant content is a paragraph of a determined element type; A model recognition module, used to perform document recognition on the non-redundant content in the target document using a document element recognition model to obtain a model recognition result of a paragraph in the non-redundant content, wherein the non-redundant content refers to other content in the target document except the redundant content, and the model recognition result includes an element type of the paragraph; The result determination module is used to determine the final recognition result of the paragraph in the target document according to the model recognition result of the paragraph in the non-redundant content and the element type of the paragraph in the redundant content.

13. An electronic device, characterized in that: include: A processor and a memory, wherein the processor is used to execute a document processing program stored in the memory to implement the document processing method according to any one of claims 1 to 11.

14. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the document processing method according to any one of claims 1 to 11.

Images