Document processing, content generation method and apparatus, and electronic device

Abstract

Embodiments of the present disclosure provide a document processing method and device, a content generation method and device, and an electronic device. The method comprises: in response to receiving a first instruction for a target document in a preset format, parsing non-text object partitions and text object partitions in the target document to obtain at least one non-text object partition and at least one text object partition; performing paragraph splicing on non-text objects and text objects in the non-text object partitions and the text object partitions based on semantic information and geometric information of the objects to obtain a plurality of paragraphs; and generating a semantic label of a paragraph according to position information and semantic information of the paragraph. The document objects in the target document can be accurately identified, and the layout of the document objects can be restored more accurately.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of information processing and the technical field of artificial intelligence, in particular to a document processing method and apparatus, a content generation method and apparatus, and an electronic device. BACKGROUND

[0002] In many application scenarios, a large amount of information needs to be collected. The collected information is structured for subsequent analysis and processing based on the collected information.

[0003] Information can be recorded in a document, such as a word document. In some documents, layout information in the document is recorded. The layout information can be used to structure the document. SUMMARY

[0004] Embodiments of the present disclosure provide a document processing method and apparatus, a content generation method and apparatus, and an electronic device.

[0005] In a first aspect, embodiments of the present disclosure provide a document processing method, which includes: in response to receiving a first instruction for a target document in a preset format, parsing non-text object partitions and text object partitions in the target document to obtain at least one non-text object partition and at least one text object partition; performing paragraph stitching on non-text objects and text objects in the non-text object partitions and the text object partitions based on semantic information and geometric information of the objects, respectively, to obtain a plurality of paragraphs; and generating semantic labels of the paragraphs according to position information and semantic information of the paragraphs.

[0006] In a second aspect, embodiments of the present disclosure provide a content generation method, which includes: receiving a third content generation instruction; and generating third content corresponding to the third content generation instruction based on the content generation instruction and a target paragraph from a plurality of paragraphs extracted from a target document in a preset format, wherein the target document includes text objects and non-text objects, and the plurality of paragraphs are obtained based on the following steps: parsing at least one non-text object partition and at least one text object partition from the target document; and performing paragraph stitching on non-text objects and text objects in the non-text object partitions and the text object partitions based on semantic information and geometric information of the objects, respectively, to obtain the plurality of paragraphs.

[0007] In a third aspect, the embodiments of the present disclosure provide a document processing apparatus, which comprises: an analyzing unit configured to analyze non-text object partitions and text object partitions in a target document in a preset format to obtain at least one non-text object partition and at least one text object partition in response to receiving a first instruction for the target document; a splicing unit configured to perform paragraph splicing on non-text objects and text objects in the non-text object partitions and the text object partitions respectively based on semantic information and geometric information of the objects to obtain a plurality of paragraphs; and a first generating unit configured to generate semantic labels of the paragraphs according to position information and semantic information of the paragraphs.

[0008] In a fourth aspect, the embodiments of the present disclosure provide a content generating apparatus, which comprises: a receiving unit configured to receive a third content generating instruction; and a second generating unit configured to generate third content corresponding to the third content generating instruction based on the content generating instruction and a target paragraph from a plurality of paragraphs extracted from a target document in a preset format, wherein the target document comprises text objects and non-text objects, and the plurality of paragraphs are obtained based on the following steps: analyzing at least one non-text object partition and at least one text object partition from the target document; and performing paragraph splicing on non-text objects and text objects in the non-text object partitions and the text object partitions respectively based on semantic information and geometric information of the objects to obtain the plurality of paragraphs.

[0009] In a fifth aspect, the embodiments of the present disclosure provide an electronic device, which comprises: a processor and a memory.

[0010] The memory stores computer-executable instructions.

[0011] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method described in the first aspect and various possible designs of the first aspect.

[0012] In a sixth aspect, the embodiments of the present disclosure provide a computer-readable storage medium, which stores computer-executable instructions, and when a processor executes the computer-executable instructions, the method described in the first aspect and various possible designs of the first aspect is implemented.

[0013] In a seventh aspect, the embodiments of the present disclosure provide a computer program product, which comprises a computer program, and when a processor executes the computer program, the method described in the first aspect and various possible designs of the first aspect is implemented.

[0014] The document processing and content generation method, device and electronic device provided by this embodiment, in response to receiving a first instruction for a target document in a preset format, parse the non-text object partitions and text object partitions in the target document to obtain at least one non-text object partition and at least one text object partition; based on the semantic information and geometric information of the objects, paragraph splicing is performed on the non-text object partitions and text objects in the text object partitions to obtain multiple paragraphs; semantic tags of paragraphs are generated according to the position information and semantic information of the paragraphs, so that document objects in the target document can be accurately identified and the layout of objects in the document can be restored more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0016] Figure 1 Schematic diagram of the document processing method provided by this disclosure Figure 1 ;

[0017] Figure 2A This is a schematic diagram of an application scenario;

[0018] Figure 2B This is a schematic diagram of an application scenario;

[0019] Figure 2C This is a schematic diagram of an application scenario;

[0020] Figure 3 A schematic flowchart of parsing a first image partition of a non-vector graphic object provided by the present disclosure;

[0021] Figure 4 A schematic flowchart of parsing a second image partition of a vector graphics object provided by the present disclosure;

[0022] Figure 5 A schematic flow chart of parsing a first table partition of a wireframe table provided by the present disclosure;

[0023] Figure 6 A schematic flow chart of parsing a second table partition of a semi-wireframe table provided by the present disclosure;

[0024] Figure 7 A schematic flow chart of parsing text object partitions provided by the present disclosure;

[0025] Figure 8A flowchart of a content generation method provided by the present disclosure is shown in FIG. 1.

[0026] Figure 9 An illustrative structural block diagram of a document processing apparatus is shown in FIG. 2.

[0027] Figure 10 An illustrative structural block diagram of a content generation apparatus is shown in FIG. 3.

[0028] Figure 11 An illustrative hardware structure diagram of an electronic device provided by an embodiment of the present disclosure is shown in FIG. 4. DETAILED DESCRIPTION

[0029] In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some but not all of the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present disclosure.

[0030] In some application scenarios, part of content can be extracted from a document according to layout information of the document, and the part of content and the layout information corresponding to the part of content in the document are processed, so as to use the content subsequently. For example, when a language model is used to generate content according to an instruction, part of content (for example, some paragraphs) is usually extracted from some documents according to layout information, and the extracted part of content is sent to the language model, so that the language model analyzes and processes the part of content to obtain a corresponding response content of the instruction.

[0031] The layout information includes information of an object. A document object includes a basic unit constituting a document, including: a title, a paragraph title, a natural paragraph, a header and footer, a footnote, a picture, a table, a formula, and the like.

[0032] Some formats of documents, for example, a Portable Document Format (PDF) document, only store information such as font, coordinate, size, and color of characters, and do not have document object information. It is necessary to recognize these document objects and restore the layout of the PDF.

[0033] In order to recognize the document objects of the above-mentioned formats of documents and restore the layout, in some implementation manners, parsing and structure restoration can be performed on a certain type of document object (for example, one type of object in a text, a picture, or a table), for example, parsing and restoration are performed on a table in a PDF document, and parsing and restoration are performed on a picture, but these implementation manners cannot completely parse and restore all of these document objects.

[0034] In some other implementations, the document in these formats is converted into a picture format, and then deep learning is used to identify the objects in the document and restore the layout of the document. These implementations can completely identify the various document objects and the layout of the various document objects in these documents, but the efficiency of this way is low, and the identification result is not accurate enough, so that the accuracy of the restoration result is poor.

[0035] The method provided by the present disclosure can accurately identify the document objects in the target document and restore the document objects and the layout in the target document by segmenting the non-text objects and the text objects in the non-text object partitions and the text object partitions based on the object semantic information and the geometric information.

[0036] Please refer to Figure 1 , which shows an illustrative flow of the document processing method provided by the present disclosure Figure 1 . As shown in Figure 1 , the method includes the following steps:

[0037] S101: In response to receiving a first instruction for a target document in a preset format, parse the non-text object partitions and the text object partitions in the target document to obtain at least one non-text object partition and at least one text object partition.

[0038] In this embodiment, the execution subject of the document processing method can be a client running in a terminal device, or a server providing services for the client.

[0039] The first instruction described above can be an instruction sent by a user, or an instruction sent by an electronic device. That is, the document processing method can be triggered by a user, or the document processing method can be triggered by an electronic device. The electronic device described above can send the first instruction described above to the execution subject according to program instructions.

[0040] The first instruction described above is used to instruct to parse the objects in the target document and restore the layout of the target document.

[0041] After receiving the first instruction described above, the execution subject described above can use various document parsing methods to parse the non-text object partitions and the text object partitions in the target document.

[0042] The text object partition here refers to a partition containing only characters, numbers, letters, etc. The non-text object partition described above includes a picture partition, a table partition, and a formula partition, etc.

[0043] That is, the target document is partitioned into text object partitions and non-text object partitions, and the non-text object partitions and the text object partitions are obtained.

[0044] Specifically, text objects and non-text objects in the target document are identified, and the target document is divided into at least one non-text object partition and at least one text object partition. The text object partitions and the non-text object partitions are independent of each other.

[0045] S102: Based on the semantic information and the geometric information of the objects, the non-text objects and the text objects in the non-text object partitions and the text object partitions are respectively subjected to paragraph stitching, and a plurality of paragraphs are obtained.

[0046] The geometric information includes position information, size information of the objects, and distance information between different objects.

[0047] After obtaining the text object partitions and the non-text object partitions, for each text object partition, the text blocks in the text object partition can be subjected to paragraph stitching according to the semantic information and the geometric information of the text blocks in the text object partition, and at least one text paragraph is obtained. The position information of the text blocks can include a page number and position information in a page corresponding to the page number. The position information of the text blocks in the page can be coordinate information of the text blocks in the page. The coordinate information of the text blocks in the page can include coordinate information of the text blocks in a preset coordinate system. In some embodiments, the preset coordinate system is, for example, a coordinate system constructed with a reference point in the page as an origin, a straight line passing through the origin and parallel to the short side of the page as an X axis, and a straight line passing through the origin and parallel to the long side of the page as a Y axis. The reference point can be any specified point in the page. The specified point can be, for example, a point corresponding to the lower left corner of the page, a point corresponding to the upper left corner of the page, or the like.

[0048] The semantic information of the text objects can include semantic information obtained by performing semantic recognition on the content of the text objects. The semantic recognition on the text objects can be performed using various semantic recognition methods.

[0049] For the non-text object partitions, the non-text objects in the non-text object partitions can be subjected to paragraph stitching according to the semantic information and the geometric information of the non-text objects, and at least one non-text object paragraph is obtained.

[0050] In some application scenarios, each non-text object partition can include a plurality of local partitions, and different local partitions can have semantic associations. For example, the size ratio relationship, the connection relationship, the adjacent relationship, the relative distance, and the like between different local partitions. The semantic associations between different local partitions can be regarded as semantic information of the non-text object partition.

[0051] In some application scenarios, a non-text object partition can include multiple local partitions, and different local partitions can include content such as text. The semantic information between different local partitions in the non-text object partition can be determined according to the text.

[0052] For example, the corresponding non-text object partition of a table object can include multiple cells. The multiple cells can be filled with text and the like. The semantic information corresponding to the cells can be determined according to the text in the multiple cells. For example, cell A recording "weather" has corresponding semantic information. Other cells B recording weather information can be located in the same column or row as cell A.

[0053] The semantic association relationship between different cells can also be determined according to the size ratio relationship, connection relationship, and adjacent relationship between different cells. For example, cell 1 is connected to cell 2, which means that cell 1 and cell 2 are connected together in space and belong to a table and the like. For another example, the size of cell 3 is determined according to the size of cell 4, which is twice the size of cell 4. Cell 4 and cell 3 belong to the same non-text object partition.

[0054] The position information can include coordinate information. The non-text objects in different non-text object partitions can be spliced according to the semantic information and geometric information.

[0055] The multiple paragraphs can be obtained through the above step S102.

[0056] S103: generating semantic tags of the paragraphs according to the position information and semantic information of the paragraphs.

[0057] The semantic tags of the paragraphs in each page can be generated in the order of the target document from front to back.

[0058] Please refer to Figures 2A-2C , Figure 2A for an application scenario diagram; Figure 2B for an application scenario diagram; Figure 2C for an application scenario diagram.

[0059] As shown in 2A, a page 201 of a target document can be subjected to text object recognition to obtain a result, which includes multiple text blocks 202.

[0060] The page can be subjected to text object partitioning and non-text object partitioning to obtain multiple partitions, such as Figure 2B multiple text object partitions 2031, 2032, 2033, 2034, and 2036, and a non-text object (table) partition 2035.

[0061] For each partition, perform paragraph concatenation to obtain multiple paragraphs. Figure 2C The obtained multiple paragraphs include text paragraphs 210, 211, 212, 213, 214, 215, 216, 217, 218, 220, 221, and a table paragraph 219.

[0062] Specifically, for each page, the page layout of the target document can be determined based on the geometric information of the multiple partitions in the page. The geometric information includes the position, size, and distance between different partitions.

[0063] The above page layout forms include single-column layout and double-column layout. The above double-column layout means that the page is divided into two columns in the horizontal direction.

[0064] If two partitions are detected to have the same row, the widths of the two partitions with the same row are checked to see if they are similar. If so, the multiple partitions on the page are divided into partitions on the left side of the page and partitions on the right side of the page based on their position information. The cumulative area of ​​the partitions on the left side of the page is further calculated to see if it is approximately the same as the cumulative area of ​​the partitions on the right side of the page. If so, the page is determined to be in a double-column layout.

[0065] If no sections with the same row are detected on the page, the page layout is considered to be single-column. If two sections are detected to have the same row, but other sections do not, and the widths of the two sections are significantly different, the page layout is considered to be single-column.

[0066] For different partitions in a page, the order of the partitions in the page can be determined from top to bottom and from left to right.

[0067] For a page with a single column layout, the order of different sections in the page can be determined from top to bottom. Furthermore, for the target document, the order of different sections in each page can be determined.

[0068] For a double-column page, the order of different sections in the page can be determined from left to right and from top to bottom.

[0069] The paragraphs in each partition are analyzed in order, and semantic labels for different paragraphs in each partition are generated.

[0070] The semantic tags may include the order in which the paragraphs appear in the document and / or the document objects corresponding to the paragraphs.

[0071] For a single-column page or a double-column page, the paragraphs in each partition in the page are analyzed in the order of the partitions in the page. Specifically, for each partition, the order of the different paragraphs in the partition is determined according to the position information of the different paragraphs in the partition. The semantic labels of the paragraphs in the partition are generated according to the order.

[0072] Specifically, for a first partition containing the first line of the first page of the document, if the partition is a text object partition and the partition includes one paragraph, the semantic information of the paragraph is determined. If the semantic of the paragraph is determined to express a complete semantic, the paragraph can be regarded as a document title of the document. The semantic label added to the paragraph can include a paragraph number indicating that the paragraph is the first paragraph, and a title identifier indicating that the paragraph is a document title.

[0073] For a second partition appearing after the first partition, if the partition includes at least one paragraph, the semantic of the paragraph is further analyzed. If the semantic of the at least one paragraph includes at least a person name and / or a unit name, it can be determined that the paragraph is a paragraph recording author information, and an author information semantic label is added to the paragraph.

[0074] Illustratively, for a third partition after the second partition, if the partition is a text object partition and the partition includes one paragraph, the semantic of the paragraph is analyzed. If the paragraph includes "abstract" and the semantic of the paragraph is a summary description of the document, the paragraph can be regarded as an abstract, and an abstract semantic label is added to the paragraph.

[0075] For a fourth partition after the third partition, if the partition is a text object partition and the partition includes at least one paragraph, the semantic information of each paragraph can be further analyzed. According to the semantic recognition result, a first paragraph of the text, a second paragraph of the text, and the like are set for each paragraph.

[0076] If an image partition appears after the fourth partition, the position of the image partition in the document is determined, and it is determined whether the image partition is the first image partition. If it is the first image partition and the image partition includes only one image, the semantic label of the image is image 1. If the image partition includes multiple images, the order of the images can be determined according to the order from left to right and from top to bottom, and the corresponding semantic labels are set for the images.

[0077] For a subsequent image partition, the order of the images in the subsequent image partition can be determined. Then the order of the images in the subsequent image partition is continued with the order of the images in the previous image partition.

[0078] For the last partition of the page, if the partition only includes a number, the number can be regarded as a page number. The paragraph corresponding to the number can be regarded as a footer.

[0079] By analogy, semantic labels can be generated for paragraphs in different partitions of the document.

[0080] In this embodiment, in response to receiving a first instruction for a target document in a preset format, the non-text object partitions and the text object partitions in the target document are parsed to obtain at least one non-text object partition and at least one text object partition; the non-text objects and the text objects in the non-text object partitions and the text object partitions are respectively subjected to paragraph stitching based on semantic information and geometric information of the objects to obtain a plurality of paragraphs; and semantic labels of the paragraphs are generated according to position information and semantic information of the paragraphs. By first dividing the target document into text object partitions and non-text object partitions, then respectively subjecting the partitions to paragraph stitching, and finally generating semantic labels of the paragraphs, the document objects of the target document can be accurately identified, so that the layout information of the objects in the target document can be accurately restored.

[0081] In some embodiments, the non-text objects include non-vector graphics objects, please refer to Figure 3 The step S101 includes the following steps of parsing a first picture partition corresponding to a non-vector picture object in the target document:

[0082] S301: For a non-vector picture, first position information corresponding to the non-vector picture is obtained based on an underlying protocol of the target document, where the first position information is position information of the first picture partition.

[0083] S302: Based on the first position information, the non-vector picture is intercepted in the target document to obtain a non-vector picture corresponding to the first picture partition.

[0084] The non-vector picture includes pictures in PNG format, JPG format, etc. For the non-vector picture, a preset interface can be used to detect the underlying protocol of the target document, and information of the non-vector picture inserted in the target document can be extracted from the underlying protocol. The information of the non-vector picture includes position information. Then, a rectangular frame surrounding the non-vector picture can be determined according to the position information. The rectangular frame can be regarded as the first picture partition. The position information of the rectangular frame can be regarded as the position information of the first picture partition. The position information of the first picture can include coordinates of the top-left vertex and the bottom-right vertex of the rectangular frame in a preset coordinate system.

[0085] After the position information of the first picture partition is determined, the picture contained in the partition can be obtained. Specifically, a screenshot can be taken in the target document according to the position information of the first picture partition, so as to obtain the picture included in the first picture partition.

[0086] According to the above manner, all first picture partitions of the target document and pictures inserted in each first picture partition can be obtained.

[0087] In these embodiments, the identification and restoration of the non-vector picture document object is achieved by partitioning the first picture partition of the non-vector picture and obtaining the picture in the first picture partition.

[0088] In some embodiments, the non-text object includes a vector graph object, please refer to Figure 4 The step S101 includes the following steps of analyzing the first picture partition corresponding to the non-vector picture object in the target document:

[0089] S401: hide the text content in the target document.

[0090] A vector graph is a scalable and distortion-free image format. A vector graph is generated by combining multiple objects, and the record of each object is achieved by mathematical functions. Unlike a bitmap, a vector graph records the algorithm of shape and color instead of the information of each point on the screen. When opening a vector graph, the document application program performs operations on the functions corresponding to the objects in the vector graph, and displays the operation results (shape and color of the graph). Therefore, it is impossible to obtain the position information of the picture from the underlying protocol.

[0091] The underlying protocol of the target document can be detected using a preset interface, and the display information of the text content in each page can be extracted from the underlying protocol by page. The display information of the text content of each page is modified to hide the text content of each page.

[0092] It can be understood that after hiding the text content of each page, the text content in the original vector graph will also be hidden. After the text content in the page is hidden, the vector graph in the page only includes points, line frames, etc.

[0093] S402: in the target document with hidden text content, identify the second position information corresponding to the vector picture and the first vector picture without text content; the second position information is used as the position information of the second picture partition.

[0094] For each page, after hiding the text content, the page is converted into a picture through format conversion. Then the picture corresponding to the page is converted into a grayscale picture, and the grayscale picture is binarized. For the binarized page picture, the first vector picture in the page binarized picture is identified using a projection segmentation algorithm (XY-cut algorithm), a rectangular frame surrounding the first vector picture is determined, and the second position information corresponding to the vector picture is determined by the projection segmentation algorithm. The second position information can include the coordinates of the rectangular frame in a preset coordinate system corresponding to the page. The rectangular frame can be regarded as the second picture partition corresponding to the first vector picture.

[0095] S403: determining the text content in the first vector picture according to the second position information.

[0096] S404: fusing the first vector picture and the text content as a vector picture corresponding to the second picture partition.

[0097] After the second position information of the second picture partition is determined, the text content corresponding to the first vector picture can be obtained from the target document according to the second position information. Specifically, a screenshot can be taken in the target document according to the second position information of the second picture partition, and the text content can be obtained from the screenshot.

[0098] The text content can be fused into the first vector picture to obtain a vector picture corresponding to the second picture partition.

[0099] By analogy, all the second picture partitions in the page and the vector pictures corresponding to each second partition can be obtained.

[0100] In these embodiments, the page where the vector picture is located is converted into a page picture, the position information of the vector picture without text content and the corresponding second picture partition is recognized in the page picture, the text content in the vector picture is obtained by taking a screenshot in the original page, and the text content is fused with the second picture without text content. Further, the second picture partition and the vector picture corresponding to the second picture are obtained, and the recognition and restoration of the vector picture are realized.

[0101] In some embodiments, the non-text object includes a wired frame table object, please refer to Figure 5 The step S101 includes analyzing the first table partition corresponding to the wired frame table object in the target document based on the following steps:

[0102] S501: obtaining third position information of a rectangular frame surrounding the wired frame table and information of each first cell in the wired frame table, wherein the third position information is the position information of the first table partition.

[0103] S502: merging a plurality of first cells according to the information of the adjacent cells of each first cell to obtain a plurality of second cells after merging.

[0104] S503: determining the size of the smallest cell in the plurality of second cells and determining the proportional relationship between the other second cells and the smallest cell.

[0105] S504: generating a blank wired frame table corresponding to the first table partition according to the proportional relationship.

[0106] S505: Fuse the text content rectangular frame corresponding to the third position information in the target document with the blank lined frame table to obtain a lined frame table corresponding to the first table partition.

[0107] The interface provided by the lined frame table extraction tool can be used to extract the position of the lined frame table in each page of the target document and determine the minimum rectangular frame including the table position. At the same time, the information (position and content) of each first cell in the lined frame table is extracted.

[0108] The first table partition corresponding to the lined frame table in each page can be parsed page by page.

[0109] For each page, the minimum rectangular frame surrounding a lined frame in the page can be regarded as a first table partition. The position information of the above-mentioned minimum rectangular frame (including the vertex of the minimum rectangular frame in the preset coordinate system of the page) can be regarded as the third position information of the first table partition.

[0110] For each first cell in the extracted lined frame, the adjacent cells in the upper, lower, left and right directions of the first cell can be found according to the position information of the first cell and the position information of other cells, and the position information of the adjacent cells is stored to obtain the adjacency matrix of the first cell.

[0111] For the above-mentioned adjacency matrix, it is detected whether the upper and left cells of each first cell have blank first cells that can be merged, and if so, they are merged. After completing the merging of multiple first cells, multiple second cells are obtained. The position and content information of each second cell is obtained. The width and height of each second cell are calculated to determine the minimum cell. The minimum cell is taken as the reference cell.

[0112] For other second cells, the width and height of each other second cell are divided by the width and minimum height of the minimum cell respectively to obtain the number of columns and rows occupied by each other second cell in the lined frame.

[0113] According to the number of rows and columns of each second cell, the coordinates (including the starting row, ending row, starting column and ending column) of the second cell in the lined frame are calculated. Thus, the restoration information of each cell (without text content filling) in the lined frame is obtained.

[0114] According to the third position information, a text content rectangular frame is extracted from the target document. The text content corresponding to each cell of the blank lined frame table is extracted from the position information of each cell in the text content rectangular frame. Each text content is filled into the corresponding blank second cell. Thus, the first table partition corresponding to the lined frame and the restoration of the table content in the first table partition are obtained.

[0115] In this way, each wireframe table in the target document can be recognized and restored.

[0116] In some embodiments, the non-text object includes a semi-wireframe table object, please refer to Figure 6 The step S101 includes analyzing the second table partition corresponding to the semi-wireframe table object in the target document based on the following steps:

[0117] S601: Based on the projection segmentation algorithm and the image recognition algorithm, the frame line used to define the semi-wireframe table is recognized.

[0118] The semi-wireframe table can be a table defined by two parallel frame lines.

[0119] For the frame line of the non-vector graph format, the image recognition algorithm can be used to recognize the frame line used to define the semi-wireframe. The frame line can be a line segment.

[0120] S602: According to the position and size of the recognized frame line, the frame line pair used to define the semi-wireframe is determined.

[0121] For example, according to the distance between the two endpoints of different frame lines, the two frame lines closest to each other are determined, and the two frame lines are approximately the same size, which can be used as a frame line pair.

[0122] S603: The fourth position information determined by the frame line pair is used as the position information of the second table partition.

[0123] The smallest rectangular frame surrounding the semi-wireframe is used as the second table partition. The position information of the smallest rectangular frame is used as the position information (fourth position information) of the second table partition. The fourth position information includes the coordinates of the vertices of the smallest rectangular frame in the page preset coordinate system.

[0124] S604: The semi-wireframe table content corresponding to the fourth position information is obtained from the target document.

[0125] S605: According to the semi-wireframe table content and the frame line pair, the semi-wireframe table corresponding to the second table partition is generated.

[0126] According to the position and size of the frame line pair, a blank semi-wireframe table is generated. The text content in the semi-wireframe table can be extracted line by line from the screenshot according to the fourth position information in the target document.

[0127] The text content is fused with the blank semi-wireframe to obtain the semi-wireframe table corresponding to the second table partition.

[0128] Through the above process, the semi-wireframe table document object can be recognized and restored from the target document.

[0129] In some embodiments, the non-text object of the target document includes a formula object, and step S101 includes resolving a formula partition corresponding to the formula object in the target document based on the following steps:

[0130] The formula content is identified using a regular expression and a preset formula font, and the formula content is spliced to obtain the formula in the target document.

[0131] In these embodiments, for a formula partition, special symbols in the formula can be identified in the partition by combining a regular expression with a formula special font, and after identifying the formula symbols, the formula content is spliced to obtain the formula in the target document. By the above-mentioned manner, all formulas in the target document can be identified. The identification and restoration of the formulas in the target document are achieved.

[0132] In some embodiments, please refer to Figure 7 The above step S101 includes the following steps of resolving the text object partition:

[0133] S701: Obtain a plurality of text blocks corresponding to the text content of the target document outside the non-text object partition.

[0134] For the text content in the target document, a plurality of text blocks can be obtained by text object resolution. Specifically, a text object resolution tool corresponding to a preset format can be used to resolve the text content of the target document, thereby obtaining a plurality of text blocks. Each text block can correspond to position information and text content of the text block.

[0135] Each text block text content can include one or more characters. Please refer to Figure 2A The target document is subjected to text object resolution to obtain a plurality of text blocks.

[0136] S702: Based on a projection segmentation algorithm, the plurality of text blocks are partitioned to obtain at least one text object partition.

[0137] Each page can be subjected to text object partition page by page. For each page, each text block is subjected to horizontal projection and vertical projection on the page. The position information of each text block in the coordinate system of the page is determined, as well as the rectangular frame surrounding the text block. Then the distance between the rectangular frames corresponding to each text block is calculated.

[0138] The first distance threshold can be preset to determine whether the distance between the rectangular frames corresponding to the upper and lower adjacent text blocks is greater than the first distance threshold. If the distance is greater than the first distance threshold, a line is used to divide the upper and lower text blocks. The second distance threshold can be preset to determine whether the distance between the left and right adjacent text blocks is greater than the second distance threshold. If the distance is greater than the second distance threshold, a line is used to divide the left and right adjacent text blocks. The vertical and horizontal directions are repeatedly determined and divided, so that at least one text object partition is obtained.

[0139] It can be understood that the first distance threshold and the second distance threshold can be set according to specific application scenarios, which are not limited here.

[0140] In some embodiments, the method further comprises:

[0141] According to the geometric information of the non-text object partition and / or the text object partition determined in the target document page, the layout form of the target document is determined, and the order of each partition is determined according to the layout form;

[0142] For each text object partition, the plurality of text blocks are sorted according to the position information of the text blocks.

[0143] After obtaining the text object partition, the order of the plurality of text objects is determined according to the position information of the text object partition.

[0144] In some embodiments, the step S102 comprises:

[0145] For each text object partition, the plurality of text blocks in the text object partition are paragraph spliced according to the position information and / or semantic information of the text blocks, to obtain a plurality of text paragraphs.

[0146] In some application scenarios, for each text object partition, the text object partition can include a plurality of text blocks. Whether the plurality of text blocks are in the same row can be determined according to the position information of the rectangular frames of the plurality of text blocks. If the plurality of text blocks are in the same row, the plurality of text blocks are spliced to obtain a row of text blocks. If there are adjacent rows of text blocks below, the distance between the adjacent rows and the left alignment information of the corresponding rectangular frames of the adjacent rows of text blocks are determined according to the position information of the corresponding rectangular frames of the adjacent rows of text blocks. If the row spacing is less than a certain distance threshold, the coordinates of the leftmost text block of the adjacent row and whether the leftmost text block is on the left side of the corresponding leftmost coordinates of the row or aligned with the corresponding leftmost coordinates of the row are determined.

[0147] If the leftmost text block of the adjacent row is on the left side of the corresponding leftmost coordinates of the row or aligned with the corresponding leftmost coordinates of the row, the adjacent row is divided into the same text paragraph as the row. Otherwise, a new paragraph is started for the adjacent row of text blocks. By analogy, the text blocks in the text object partition can be spliced into a plurality of text paragraphs.

[0148] In some application scenarios, semantic information of different text block combinations is analyzed, and text blocks representing a complete semantic content are spliced to obtain a text paragraph.

[0149] In some embodiments, the above-mentioned paragraph splicing of the plurality of text blocks in each text object partition according to the position information and / or semantic information of the text blocks obtains a plurality of text paragraphs, including:

[0150] For adjacent paragraphs in different two text object partitions, it is determined whether to splice the adjacent paragraphs according to at least two paragraphs corresponding to the two text object partitions, respectively, and the at least two paragraphs corresponding to the two text object partitions, respectively, include the adjacent paragraphs.

[0151] The above-mentioned different two text object partitions can be two text object partitions adjacent across columns or two text object partitions adjacent across pages.

[0152] In the adjacent two text object partitions, the last paragraph of the former text object partition and the first paragraph of the latter text object partition are adjacent paragraphs. Whether the adjacent two paragraphs need to be spliced needs to be determined according to the last at least two paragraphs in the former text object partition and the first at least two paragraphs in the latter text object partition, whether to splice the adjacent two paragraphs in different partitions.

[0153] For example, for the last paragraph S K in the former text object partition Q1, the second last paragraph S K-1 and the third last paragraph S K-2 in the text object partition Q1 are obtained. The leftmost coordinates and the rightmost coordinates of each paragraph S K-1 , S K-2 in the text object partition Q1 are determined. For the first paragraph W1 in the latter text object partition Q2, the second paragraph W2 and the third paragraph W3 in the text object partition Q2 are obtained. The leftmost coordinates and the rightmost coordinates of each paragraph W2, W3 in the latter text object partition Q2 are determined. If the leftmost coordinate of the last line of the paragraph S K is aligned with the leftmost coordinates of the paragraphs S K-1 , S K-2 , and the leftmost coordinates of the first paragraph W1 and the second paragraph W2 and the third paragraph W3 in the text object partition Q2 are aligned, it can be preliminarily judged that the paragraph S k needs to be spliced with the paragraph W1, and then the semantics of the paragraph S k and the paragraph W1 are determined. If the semantics of the two paragraphs are coherent, the two paragraphs are spliced into one paragraph.

[0154] In some embodiments, the step S101 comprises:

[0155] The non-text object partitions and the text object partitions of the target document are parsed in the order of first parsing the non-text object partitions and then parsing the text object partitions.

[0156] In these embodiments, the non-text objects in the target document can be identified first, and then the non-text object partitions corresponding to the non-text objects are determined.

[0157] After the non-text object partitions are parsed, the text object partitions including the text objects are parsed.

[0158] Since the non-text objects can include text content, the non-text object partitions are parsed first and then the text object partitions are parsed, which can avoid incorrectly dividing the text content in the non-text objects into the text object partitions, thereby obtaining more accurate results of the text object partitions and the non-text object partitions.

[0159] In some embodiments, the non-text object partitions and the text object partitions of the target document are parsed in the order of first parsing the non-text object partitions and then parsing the text object partitions, comprising:

[0160] The picture partitions corresponding to the pictures, the table partitions corresponding to the tables, and / or the formula partitions corresponding to the formulas in the target document are parsed in sequence.

[0161] In these embodiments, if the target document includes pictures, tables, and formulas, the non-text object partitions in the target document can be parsed in the order of first parsing the picture partitions, then parsing the table partitions, and finally parsing the formula partitions.

[0162] In these embodiments, the pictures are identified first, which can avoid misidentification of the subsequent tables and formulas due to the content in the pictures. The pictures and the tables are identified first, and then the formulas are identified, which can avoid misidentification of the formulas due to the content in the pictures and the content in the tables. The accuracy of the identification of the non-text objects is further improved, thereby improving the accuracy of the non-text object partitions.

[0163] In some embodiments, the method further comprises the following steps:

[0164] First, the target paragraph corresponding to the content generation instruction is determined from the plurality of paragraphs according to the received first content generation instruction.

[0165] Second, the task information is generated according to the first content generation instruction and the target paragraph.

[0166] Finally, the first model is called to generate the first content corresponding to the first content generation instruction according to the task information by the first model.

[0167] After obtaining the semantic labels of the paragraphs in the target document, the semantic labels of the paragraphs and the corresponding text content can be stored in association and a vector index can be established.

[0168] The first content generation instruction can be an instruction for generating various content, such as an instruction for generating reply information to a question in the first content generation instruction. For example, a content generation instruction for generating summary content of a document. Or a content generation instruction for rewriting the content of a document.

[0169] After receiving the first content generation instruction, the first content generation instruction can be matched with the multiple paragraphs of the target document, and the target paragraph can be determined according to the matching result.

[0170] The first content generation instruction and the content generation task information of the target paragraph can be written into the prompt template (prompt template) generation task information. The target content can include non-text objects, such as tables, pictures, or formulas.

[0171] The first model can be called through a preset interface, and the target paragraph can be processed by the first model according to the task information to obtain the first content matched with the first content generation instruction.

[0172] The first model can be a language model. The language model can be a model that has been pre-trained using a large amount of training corpus and has preliminary reasoning ability.

[0173] In these embodiments, the first content is generated by the target paragraph in the target document that matches the first content generation instruction. Since the target paragraph is a paragraph that is accurately identified and restored from the multiple paragraphs in the target document, the accuracy of the determined target paragraph is relatively high and does not contain redundant information. Therefore, the first content matched with the first content generation instruction can be obtained relatively quickly and accurately.

[0174] In some embodiments, the method further includes the following steps:

[0175] First, the received second content generation instruction;

[0176] Second, call the second model to generate the second content by the second model according to the second content instruction, wherein the second content is generated by the second model based on the second target paragraph determined by the second model from the multiple paragraphs that matches the second content generation instruction;

[0177] The second content generation instruction can be an instruction for generating various contents, such as an instruction for generating reply information to a question in the second content generation instruction. For example, an instruction for generating summary content of a document. Or an instruction for rewriting the content of a document.

[0178] The second content generation instruction can be sent to a second model. The second model can be a language model. The language model can be a model with preliminary reasoning ability pre-trained using a large amount of training corpus.

[0179] The second model can obtain a plurality of paragraphs corresponding to the target document according to the second content generation instruction. And determine the target paragraph from the plurality of paragraphs according to the matching result with the second content instruction. Then generate the second content according to the second content instruction according to the target paragraph.

[0180] Illustratively, if the target document records a picture A of an animal A, a table B recording the food eaten, and a paragraph C describing the daily life of the animal A, etc. The second content generation instruction can be to summarize the appearance characteristics of the animal A. Then the second model can determine the picture A as the corresponding target paragraph according to the second content generation instruction, and generate the appearance characteristics description of the animal A according to the picture A.

[0181] In these embodiments, by sending the second content generation instruction to the second model, the second content can be obtained by the second model according to the second content generation instruction to obtain the target paragraph in the target document to generate the second content.

[0182] Reference Figure 8 , which shows a schematic flowchart of the content generation method provided by the present disclosure. As Figure 8 shown, the method comprises the following steps:

[0183] S801: receiving a third content generation instruction.

[0184] The execution subject of the above content generation method can be a client running in a terminal device, or a server providing services for the client.

[0185] The third content generation instruction can be an instruction for generating various contents, such as an instruction for generating reply information to a question in the third content generation instruction. For example, an instruction for generating summary content of a document. Or an instruction for rewriting the content of a document.

[0186] The user can send the third content instruction to the above execution subject through the instruction input window in the client.

[0187] In some application scenarios, the content of the target document can be displayed in the client interface. The target document can be a document in the preset format in the embodiments shown in Figure 1 、 Figure 3 , and details are not repeated here.

[0188] S802: generating third content corresponding to the third content generation instruction based on the third content generation instruction and a target paragraph from a plurality of paragraphs extracted from the target document in the preset format; wherein the target document includes text objects and non-text objects, and the plurality of paragraphs are obtained based on the following steps: parsing at least one non-text object partition and at least one text object partition from the target document; performing paragraph stitching on non-text objects and text objects in the non-text object partition and the text object partition based on semantic information and geometric information of the objects, respectively, to obtain the plurality of paragraphs.

[0189] After receiving the third content generation instruction, the target paragraph can be determined from the plurality of paragraphs corresponding to the target document. For example, the target paragraph is determined from the plurality of paragraphs according to the matching degree with the third content generation instruction.

[0190] In some embodiments, the plurality of paragraphs in the target document can be parsed in advance according to the method of the embodiments shown in Figure 1 、 Figures 3-7 .

[0191] In some other embodiments, after receiving the third content instruction, the target document indicated by the third content instruction can be parsed according to the method of the embodiments shown in Figure 1 、 Figures 3-7 , to obtain a plurality of paragraphs with semantic labels.

[0192] The execution subject can generate third content matching the third content instruction according to the target paragraph.

[0193] In some embodiments, step S802 includes sending the third content generation instruction to a third model to determine a target paragraph matching the third content from the plurality of paragraphs of the target document according to the third instruction by the third model, and generating the third content according to the target paragraph.

[0194] In these embodiments, the third content generation instruction can be sent to the third model. The third model can be a language model.

[0195] The third model can obtain the plurality of paragraphs corresponding to the target document according to the third content generation instruction. For example, the target document is parsed according to Figure 1 、 Figures 3-7The method of the embodiment shown parses a plurality of paragraphs with semantic labels. Then, a target paragraph is determined from the plurality of paragraphs according to a matching degree with the third content instruction. Then, the third content is generated according to the target paragraph.

[0196] In some embodiments, step S802 includes the following steps:

[0197] First, a target paragraph matching the third content generation instruction is determined from a plurality of paragraphs extracted from the target document in the preset format.

[0198] Second, a content generation request including the third content generation instruction and the target paragraph is sent to the third model, and the third content is generated by the third model according to the target paragraph.

[0199] The target document can be parsed according to the preset format. Figure 1 、 Figures 3-7 The method of the embodiment shown parses a plurality of paragraphs with semantic labels.

[0200] After obtaining the semantic labels of each paragraph of the target document, the semantic labels of each paragraph and the corresponding text content can be associated and stored, and a vector index can be established.

[0201] After receiving the third content generation instruction, the first content generation instruction can be matched with a plurality of paragraphs from the target document, and a target paragraph is determined according to the matching result.

[0202] The content generation request can be generated according to the third content generation instruction and the content of the target paragraph. The content generation request is sent to the third model, and the third content is generated by the third model according to the content of the target paragraph.

[0203] In some embodiments, the method further includes:

[0204] The third content is displayed.

[0205] In these embodiments, after obtaining the third content, the third content can be displayed for the user to read.

[0206] In this embodiment, after receiving the third content generation instruction, the third content is generated according to the third content generation instruction and the target paragraph in the target document matching the first content generation instruction. Since the target paragraph is a paragraph in the plurality of paragraphs accurately identified and restored from the target document, the accuracy of the determined target paragraph is relatively high, and no redundant information is included. Therefore, the third content matching the third content generation instruction can be obtained relatively quickly and accurately.

[0207] Corresponding to the above Figure 1 The document processing method of the embodiment shown, Figure 9A schematic structural diagram of a document processing apparatus is provided for the embodiments of the present disclosure. For ease of illustration, only parts related to the embodiments of the present disclosure are shown. Refer to Figure 9 The apparatus 90 comprises an analyzing unit 901, a first generating unit 902, and a first generating unit 903. Wherein,

[0208] The analyzing unit 901 is configured to, in response to receiving a first instruction for a target document in a preset format, analyze non-text object partitions and text object partitions in the target document to obtain at least one non-text object partition and at least one text object partition.

[0209] The splicing unit 902 is configured to perform paragraph splicing on non-text objects and text objects in the non-text object partitions and the text object partitions respectively based on semantic information and geometric information of the objects to obtain a plurality of paragraphs.

[0210] The first generating unit 903 is configured to generate semantic labels of the paragraphs according to position information and semantic information of the paragraphs.

[0211] In some embodiments, the non-text objects comprise non-vector graphics objects, and the analyzing unit 901 is further configured to analyze a first picture partition corresponding to a non-vector picture object in the target document based on the following steps:

[0212] For the non-vector picture, first position information corresponding to the non-vector picture is obtained based on an underlying protocol of the target document, wherein the first position information is position information of the first picture partition.

[0213] The non-vector picture corresponding to the first picture partition is obtained by intercepting the non-vector picture in the target document based on the first position information.

[0214] In some embodiments, the non-text objects comprise vector graphics objects, and the analyzing unit 901 is further configured to analyze a second picture partition corresponding to a vector picture object in the target document based on the following steps:

[0215] Hiding text content in the target document;

[0216] In the target document with the hidden text content, second position information corresponding to the vector picture and a first vector picture without text content are identified, and the second position information is used as position information of the second picture partition.

[0217] Text content in the first vector picture is determined according to the second position information.

[0218] The first vector picture and the text content are fused as a vector picture corresponding to the second picture partition.

[0219] In some embodiments, the non-text object comprises a wired frame table object, and the parsing unit 901 is further configured to parse the wired frame table object in the target document based on the following steps:

[0220] obtain third position information of a rectangular frame surrounding the wired frame table and information of each first cell in the wired frame table, wherein the third position information is position information of the first table partition;

[0221] merge the plurality of first cells according to the information of the adjacent cells of each first cell to obtain a plurality of second cells after merging;

[0222] determine the size of the smallest cell in the plurality of second cells and determine the proportional relationship between other cells and the smallest cell;

[0223] generate a blank wired frame table corresponding to the first table partition according to the proportional relationship;

[0224] fuse the text content rectangular frame corresponding to the third position information in the target document with the blank wired frame table to obtain the wired frame table corresponding to the first table partition.

[0225] In some embodiments, the non-text object comprises a semi-wired frame table object, and the parsing unit 901 is further configured to parse the semi-wired frame table object in the target document based on the following steps:

[0226] identify the frame lines used to define the semi-wired frame table based on a projection segmentation algorithm and an image recognition algorithm;

[0227] determine a pair of frame lines used to define the semi-wired frame according to the position and size of the identified frame lines;

[0228] determine fourth position information of the pair of frame lines as position information of the second table partition;

[0229] obtain semi-wired frame table content corresponding to the fourth position information from the target document;

[0230] write the semi-wired frame table content into the semi-wired frame table to obtain the semi-wired frame table corresponding to the second table partition.

[0231] In some embodiments, the non-text object comprises a formula object, and the parsing unit 901 is further configured to parse the formula object in the target document based on the following steps:

[0232] identify formula content using a regular expression and a preset formula font, and splice the formula content to obtain the formula in the target document.

[0233] In some embodiments, the parsing unit 901 is further configured to:

[0234] obtaining a plurality of text blocks corresponding to the text content of the target document except for the non-text object partition;

[0235] partitioning the plurality of text blocks based on a projection segmentation algorithm to obtain at least one text object partition.

[0236] In some embodiments, the parsing unit 901 is further configured to:

[0237] determining a layout form of the target document according to the geometric information of the plurality of partitions determined in the target document page, and determining an order of the partitions according to the layout form.

[0238] In some embodiments, the splicing unit 902 is further configured to:

[0239] performing paragraph splicing on the plurality of text blocks in each text object partition according to the position information and / or semantic information of the text blocks to obtain a plurality of text paragraphs.

[0240] In some embodiments, the splicing unit 902 is further configured to:

[0241] for adjacent paragraphs in different two text object partitions, determining whether to splice the adjacent paragraphs according to at least two paragraphs corresponding to the two text object partitions respectively, the at least two paragraphs corresponding to the two text object partitions respectively including the adjacent paragraphs.

[0242] In some embodiments, the parsing unit 901 is further configured to:

[0243] parsing the non-text object partition and the text object partition of the target document in the order of the non-text object partition first and the text object partition second.

[0244] In some embodiments, the parsing unit 901 is further configured to:

[0245] sequentially parsing a picture partition corresponding to a picture, a table partition corresponding to a table, and a formula partition corresponding to a formula in the target document.

[0246] In some embodiments, the apparatus 90 further comprises a first content generation unit (not shown in the figure). The first content generation unit is configured to:

[0247] generating task information according to the first content generation instruction and the target paragraph;

[0248] calling the first model to generate first content corresponding to the first content generation instruction according to the task information by the first model.

[0249] In some embodiments, the apparatus 90 further comprises a second content generation unit (not shown in the figure). The second content generation unit is configured to:

[0250] receive the second content generation instruction;

[0251] invoke the second model to generate the second content according to the second content instruction by the second model, wherein the second content is generated by the second model from a second target paragraph determined by the second model from the plurality of paragraphs according to the second content generation instruction.

[0252] corresponding to the above Figure 7 content generation method of the embodiments shown above, Figure 10 a schematic structural diagram of a content generation apparatus provided by the embodiments of the present disclosure. For ease of illustration, only parts related to the embodiments of the present disclosure are shown. Referring to Figure 10 , the apparatus 100 comprises a receiving unit 1001 and a second generation unit 1002.

[0253] wherein,

[0254] the receiving unit 1001 is configured to receive a third content generation instruction;

[0255] the second generation unit 1002 is configured to generate a third content corresponding to the third content generation instruction based on the content generation instruction and a target paragraph from a plurality of paragraphs extracted from a target document of a preset format; wherein the target document comprises a text object and a non-text object, and the plurality of paragraphs are obtained based on the following steps: parsing at least one non-text object partition and at least one text object partition from the target document; performing paragraph splicing on non-text objects and text objects in the non-text object partition and the text object partition based on semantic information and geometric information of the objects, respectively, to obtain the plurality of paragraphs.

[0256] In some embodiments, the second generation unit 1002 is further configured to:

[0257] send the third content generation instruction to the third model to determine a target paragraph matching the third content from the plurality of paragraphs of the target document by the third model according to the third instruction, and generate the third content according to the target paragraph.

[0258] In some embodiments, the second generation unit 1002 is further configured to:

[0259] determine a target paragraph matching the third content generation instruction from the plurality of paragraphs extracted from the target document of the preset format;

[0260] send a content generation request comprising the third content generation instruction and the target paragraph to the third model, and generate the third content by the third model according to the target paragraph.

[0261] In some embodiments, the apparatus 100 described above further includes a display unit (not shown in the figure). The display unit is configured to:

[0262] present the third content.

[0263] To implement the above-described embodiments, the present embodiments also provide an electronic device.

[0264] Reference Figure 11 which shows a structural schematic diagram of an electronic device 1100 suitable for implementing the present embodiments, which can be a terminal device or a server. Among them, the terminal device can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDA), tablet computers (PAD), portable multimedia players (PMP), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 11 The electronic device shown is only an example and should not impose any limitation on the functions and use range of the present embodiments.

[0265] As shown in Figure 11 , the electronic device 1100 can include a processing device (such as a central processor, a graphics processor, etc.) 1101, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1102 or loaded into a random access memory (RAM) 1103 from a storage device 1108. In the RAM 1103, various programs and data required for the operation of the electronic device 1100 are also stored. The processing device 1101, the ROM 1102, and the RAM 1103 are connected to each other through a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0266] Generally, the following devices can be connected to the I / O interface 1105: input devices 1106 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 1107 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage devices 1108 including, for example, a magnetic tape, a hard disk, and the like; and communication devices 1109. The communication devices 1109 can allow the electronic device 1100 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 11The electronic device 1100 is illustrated with various means for performing various functions, but it is understood that not all of the means illustrated are required to perform the functions. More or less means can be implemented.

[0267] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from the network via the communication device 1109, or installed from the storage device 1108, or installed from the ROM 1102. When the computer program is executed by the processing device 1101, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0268] It should be noted that the computer readable medium described above in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any suitable combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used or used in conjunction with an instruction execution system, apparatus or device. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium that can send, propagate or transmit a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, a RF (radio frequency) or the like, or any suitable combination thereof.

[0269] The above-mentioned computer readable medium can be contained in the above-mentioned electronic device; or can exist separately without being assembled into the electronic device.

[0270] The computer readable medium described above can carry one or more programs, when the one or more programs (computer-executable instructions) are executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0271] Computer program code for carrying out operations of the present disclosure can be written in any one or more programming languages, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0272] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that noted in the figures. For example, two blocks noted in succession can in fact be executed substantially concurrently or in the opposite order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flow diagrams, and combinations of blocks in the block diagrams and / or flow diagrams, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware-based systems and computer instructions.

[0273] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself.

[0274] The functionality described above in this document can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, non- transitory machine-readable media can include RAM, ROM, programmable ROM (EPROM, EEPROM or flash memory), or any other storage device(s) through which program instructions can be stored and executed by a processing unit. The above described functions can be implemented as software modules or software functions using object-oriented programming techniques (e.g., C++). The software modules or functions can be stored on one or more of the respective storage devices, in the RAM, or elsewhere by a processor executing at a device as instructions stored in non-transitory machine-readable media.

[0275] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of a processor, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0276] The above description is only preferred embodiments of the present disclosure and the explanation of the technical principles of the application. It should be understood by those skilled in the art that the disclosure range involved in the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and also covers other technical solutions formed by any combination of the above technical features or equivalent features without departing from the above disclosed concept. For example, the technical solutions formed by replacing the above features with the technical features disclosed in the present disclosure (but not limited to) having similar functions.

[0277] Further, while operations are depicted in a particular order, this should not be understood as requiring the operations to be performed in the particular order shown or in sequential order, and that certain operations can be performed in parallel or in any suitably ordered order. Similarly, while several specific implementation details are discussed in the above discussion, these should not be construed as limiting the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0278] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A document processing method characterized by, The method comprises the following steps: In response to receiving a first instruction for a target document in a preset format, parsing non-text object partitions and text object partitions in the target document to obtain at least one non-text object partition and at least one text object partition; Based on the semantic information and geometric information of the objects, the non-text objects and the text objects in the non-text object partitions and the text object partitions are respectively subjected to paragraph splicing to obtain a plurality of paragraphs; the geometric information comprises position, size and distance between different partitions; According to the position information and the semantic information of the paragraphs, a semantic label of the paragraphs is generated; If the non-text objects comprise non-vector graphics objects, the parsing of the non-text object partitions and the text object partitions in the target document to obtain at least one non-text object partition and at least one text object partition comprises the following steps of parsing a first picture partition corresponding to a non-vector picture object in the target document, wherein the first picture partition belongs to the non-text object partition: For the non-vector picture, first position information corresponding to the non-vector picture is obtained based on an underlying protocol of the target document, wherein the first position information is position information of the first picture partition; Based on the first position information, the non-vector picture corresponding to the first picture partition is obtained by intercepting the non-vector picture in the target document.

2. The method of claim 1, wherein, If the non-text objects comprise vector graphics objects, the parsing of the non-text object partitions and the text object partitions in the target document to obtain at least one non-text object partition and at least one text object partition comprises the following steps of parsing a second picture partition corresponding to a vector picture object in the target document: Hiding the text content in the target document; In the target document in which the text content is hidden, a second position information corresponding to a vector picture and a first vector picture without text content are identified; the second position information is used as position information of the second picture partition; According to the second position information, the text content in the first vector picture is determined; The first vector picture and the text content are fused as a vector picture corresponding to the second picture partition.

3. The method of claim 1, wherein, If the non-text objects comprise wired frame table objects, the parsing of the non-text object partitions and the text object partitions in the target document to obtain at least one non-text object partition and at least one text object partition comprises the following steps of parsing a first table partition corresponding to a wired frame table object in the target document: Third position information of a rectangular frame surrounding the wired frame table and information of each first cell in the wired frame table are obtained, wherein the third position information is position information of the first table partition; According to the information of the adjacent cells of each first cell, a plurality of second cells after merging are obtained by merging the plurality of first cells; The size of the smallest cell in the plurality of second cells is determined, and a proportional relationship between other cells and the smallest cell is determined; According to the proportional relationship, a blank wired frame table corresponding to the first table partition is generated. Fusing the text content rectangular frame corresponding to the third position information in the target document with the blank lined frame table to obtain a wired frame table corresponding to the first table partition.

4. The method of claim 1, wherein, If the non-text object includes a half-lined frame object, the parsing of the non-text object partition and the text object partition in the target document to obtain at least one non-text object partition and at least one text object partition includes parsing a second table partition corresponding to the half-lined frame object in the target document based on the following steps: Based on the projection segmentation algorithm and the image recognition algorithm, the frame lines used to define the half-lined frame are recognized. According to the position and size of the recognized frame lines, a pair of frame lines used to define the half-lined frame is determined. The fourth position information determined by the pair of frame lines is taken as the position information of the second table partition. The half-lined frame content corresponding to the fourth position information is obtained from the target document. The half-lined frame content is written into the half-lined frame to obtain a half-lined frame corresponding to the second table partition.

5. The method of claim 1, wherein, If the non-text object includes a formula object, the parsing of the non-text object partition and the text object partition in the target document to obtain at least one non-text object partition and at least one text object partition includes parsing a formula partition corresponding to the formula object in the target document based on the following steps: The formula content is recognized using regular expressions and a preset formula font, and the formula content is spliced to obtain the formula in the target document.

6. The method of claim 1, wherein, The parsing of the non-text object partition and the text object partition in the target document to obtain at least one non-text object partition and at least one text object partition includes: Obtaining a plurality of text blocks corresponding to the text content of the target document other than the non-text object partition; Based on the projection segmentation algorithm, the plurality of text blocks are partitioned to obtain at least one text object partition.

7. The method according to claim 6, characterized in that The method further includes: According to the geometric information of the plurality of partitions determined in the target document page, the layout form of the target document is determined, and the order of each partition is determined according to the layout form.

8. The method of claim 1, wherein, The paragraph splicing of non-text objects and text objects in the non-text object partition and the text object partition based on the semantic information and the geometric information of the objects respectively to obtain a plurality of paragraphs includes: For each text object partition, a plurality of text blocks in the text object partition are spliced into a plurality of text paragraphs according to the position information and / or semantic information of the text blocks.

9. The method of claim 8, wherein, The paragraph splicing of a plurality of text blocks in each text object partition according to the position information and / or semantic information of the text blocks to obtain a plurality of text paragraphs further includes: For adjacent paragraphs in different two text object partitions, it is determined whether to splice the adjacent paragraphs according to at least two paragraphs corresponding to the two text object partitions respectively, the at least two paragraphs corresponding to the two text object partitions respectively including the adjacent paragraphs.

10. The method of claim 1, wherein, The parsing of the at least one non-text object partition and the at least one text object partition in the target document includes: The non-text object partition and the text object partition of the target document are parsed in the order of the non-text object partition first and the text object partition second.

11. The method of claim 10, wherein, The non-text object partition and the text object partition of the target document are parsed in the order of first parsing the non-text object partition and then parsing the text object partition, and the parsing of the non-text object partition comprises parsing the following non-text object partition: The picture partition corresponding to the picture, the table partition corresponding to the table, and / or the formula partition corresponding to the formula in the target document are parsed in sequence.

12. The method of claim 1, wherein, The method further comprises: determining a target paragraph corresponding to the content generation instruction from the plurality of paragraphs according to the received first content generation instruction; generating task information according to the first content generation instruction and the target paragraph; calling a first model to generate first content corresponding to the first content generation instruction according to the task information by the first model.

13. The method of claim 1, wherein, The method further comprises: receiving a second content generation instruction; calling a second model to generate second content according to the second content generation instruction by the second model, wherein the second content is generated by the second model based on a second target paragraph determined by the second model from the plurality of paragraphs that matches the second content generation instruction.

14. A content generation method, comprising: receiving a third content generation instruction; generating third content corresponding to the third content generation instruction based on the third content generation instruction and a target paragraph from a plurality of paragraphs extracted from a target document in a preset format; wherein the target document comprises text objects and non-text objects, and the plurality of paragraphs are obtained based on the following steps: parsing at least one non-text object partition and at least one text object partition from the target document; and performing paragraph splicing on non-text objects and text objects in the non-text object partition and the text object partition based on semantic information and geometric information of the objects, respectively, to obtain the plurality of paragraphs; the geometric information comprises position, size, and distance between different partitions; if the non-text object comprises a non-vector graphics object, the parsing of the at least one non-text object partition and the at least one text object partition from the target document comprises parsing a first picture partition corresponding to a non-vector picture object in the target document based on the following steps, and the first picture partition belongs to the non-text object partition: for a non-vector picture, obtaining first position information corresponding to the non-vector picture based on the underlying protocol of the target document, wherein the first position information is position information of the first picture partition; obtaining a non-vector picture corresponding to the first picture partition by intercepting the non-vector picture in the target document based on the first position information.

15. The method of claim 14, wherein, The generation of the third content corresponding to the third content generation instruction based on the content generation instruction and the target paragraph from the plurality of paragraphs extracted from the target document in the preset format comprises: sending the third content generation instruction to a third model to determine a target paragraph matching the third content from the plurality of paragraphs of the target document according to the third content generation instruction by the third model, and generating the third content according to the target paragraph.

16. The method of claim 14, wherein, The generating, based on the content generation instruction and a target paragraph from a plurality of paragraphs extracted from a target document in a preset format, of third content corresponding to the third content generation instruction comprises: determining a target paragraph matching the third content generation instruction from a plurality of paragraphs extracted from a target document in a preset format; sending a content generation request comprising the third content generation instruction and the target paragraph to a third model, and generating the third content by the third model according to the target paragraph.

17. The method according to any one of claims 14 to 16, characterized in that, The method further comprises: displaying the third content.

18. A document processing apparatus characterized by comprising: Comprise: The parsing unit is configured to, in response to receiving a first instruction for a target document in a preset format, parse non-text object partitions and text object partitions in the target document to obtain at least one non-text object partition and at least one text object partition; The splicing unit is configured to perform paragraph splicing on non-text objects and text objects in the non-text object partitions and the text object partitions respectively based on semantic information and geometric information of the objects to obtain a plurality of paragraphs; the geometric information comprises position, size, and distance between different partitions; The first generation unit is configured to generate semantic tags of the paragraphs according to position information and semantic information of the paragraphs; If the non-text object comprises a non-vector graphics object, the parsing unit is specifically configured to parse a first picture partition corresponding to the non-vector picture object in the target document based on the following steps, and the first picture partition belongs to the non-text object partition: For a non-vector picture, first position information corresponding to the non-vector picture is obtained based on an underlying protocol of the target document, wherein the first position information is position information of the first picture partition; The non-vector picture corresponding to the first picture partition is obtained by intercepting the non-vector picture in the target document based on the first position information.

19. A content generation apparatus, comprising: a receiving unit configured to receive a third content generation instruction; a second generation unit configured to generate third content corresponding to the third content generation instruction based on the content generation instruction and a target paragraph from a plurality of paragraphs extracted from a target document in a preset format; wherein the target document comprises text objects and non-text objects, and the plurality of paragraphs are obtained based on the following steps: parsing at least one non-text object partition and at least one text object partition from the target document; performing paragraph splicing on non-text objects and text objects in the non-text object partitions and the text object partitions respectively based on semantic information and geometric information of the objects to obtain a plurality of paragraphs; the geometric information comprises position, size, and distance between different partitions; If the non-text object comprises a non-vector graphics object, the parsing at least one non-text object partition and at least one text object partition from the target document comprises parsing a first picture partition corresponding to the non-vector picture object in the target document based on the following steps, and the first picture partition belongs to the non-text object partition: For a non-vector picture, first position information corresponding to the non-vector picture is obtained based on an underlying protocol of the target document, wherein the first position information is position information of the first picture partition; Intercept the non-vector picture in the target document based on the first position information to obtain a non-vector picture corresponding to the first picture partition.

20. An electronic device, comprising: Comprise: a processor and a memory; the memory stores computer-executed instructions; the processor executes the computer-executed instructions stored in the memory, so that the processor executes the method as claimed in any one of claims 1 to 17.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executed instructions, and when the processor executes the computer-executed instructions, the method as claimed in any one of claims 1 to 17 is realized.

Images