Document alignment method and device, computer equipment and storage medium

Abstract

The invention relates to a document alignment method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a first format document and a second format document corresponding to the first format document; the first format document comprises at least one layout block, and the second format document comprises at least one reference text line; generating a dictionary tree corresponding to the second format document based on the reference text line in the second format document; aiming at each layout block, determining an aligned text line corresponding to the layout block based on the layout block and the dictionary tree; and obtaining text alignment information of the layout blocks based on the layout block information corresponding to the layout blocks and the text line information of the aligned text lines. By adopting the method, the accuracy of the document alignment information can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a document alignment method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of computer technology, artificial intelligence products have developed rapidly. Before using documents in the training of artificial intelligence products such as document intelligence, document structured analysis, and retrieval-augmented generation, it is necessary to determine the alignment information of the documents in order to accurately extract the information required for training from the documents. In addition, the alignment information of the documents can also be used to visualize the difference information between documents, and can also be used in the RAG (Retrieval-Augmented Generation) system to improve the accuracy of document information positioning, facilitate tracking and locating the original location of the document, etc.

[0003] In traditional technology, a DOC (Document) document is converted into a PDF (Portable Document Format) document. Based on the PDF document, a layout analysis model is used to determine the layout structure information of the document, a reading order sorting model is used to determine the reading order information of the document, and a table restoration model is used to determine the table structure information of the document. Then, based on the layout structure information, reading order information and table structure information, the alignment information of the document is determined, resulting in low accuracy of the document alignment information. Summary of the invention

[0004] Based on this, it is necessary to provide a document alignment method, apparatus, computer device, computer-readable storage medium and computer program product that can improve the accuracy of document alignment information in response to the above technical problems.

[0005] In a first aspect, the present application provides a document alignment method. The method comprises:

[0006] Acquire a first format document and a second format document corresponding to the first format document; the first format document includes at least one layout block, and the second format document includes at least one reference text line;

[0007] generating a dictionary tree corresponding to the second format document based on the reference text line in the second format document;

[0008] For each of the layout blocks, based on the layout block and the dictionary tree, determining the aligned text line corresponding to the layout block;

[0009] Based on the layout block information corresponding to the layout block and the text line information of the aligned text line, the text alignment information of the layout block is obtained.

[0010] In a second aspect, the present application also provides a document alignment device. The device comprises:

[0011] An acquisition module, used for acquiring a first format document and a second format document corresponding to the first format document; the first format document includes at least one layout block, and the second format document includes at least one reference text line;

[0012] A generating module, configured to generate a dictionary tree corresponding to the second format document based on the reference text lines in the second format document;

[0013] A determination module, configured to determine, for each layout block, an aligned text line corresponding to the layout block based on the layout block and the dictionary tree;

[0014] The alignment module is used to obtain the text alignment information of the layout block based on the layout block information corresponding to the layout block and the text line information of the aligned text line.

[0015] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0016] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0017] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0018] The document alignment method, device, computer equipment, storage medium and computer program product described above obtain a first format document and a second format document corresponding to the first format document. Since the first format document is a document with a flow layout, the first format document includes at least one layout block, and the layout block information of the layout block includes information such as the layout block reading order and the layout block type, but the position coordinates of the layout block cannot be determined based on the first format document. Since the second format document is a document with a fixed layout, the second format document includes at least one reference text line, and the text line information of the reference text line includes text line page and text line Position information, etc., but the reading order of the reference text lines cannot be determined according to the second format document; based on the reference text lines in the second format document, a dictionary tree corresponding to the second format document is generated, and for each layout block, based on the layout block and the dictionary tree, the aligned text line corresponding to the layout block is determined, that is, by traversing the dictionary tree, the aligned text line corresponding to the layout block is determined, and then based on the layout block information corresponding to the layout block and the text line information of the aligned text line, the target position coordinates of the layout block and the target reading order of the text lines are determined, so as to obtain the text alignment information of the layout block, thereby improving the accuracy of the document alignment information. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] FIG1 is a diagram of an application environment of a document alignment method in one embodiment;

[0020] FIG2 is a schematic diagram of a flow chart of a document alignment method in one embodiment;

[0021] FIG3 is a schematic diagram of a dictionary tree in one embodiment;

[0022] FIG4 is a schematic diagram of a process flow of a dictionary tree construction step in one embodiment;

[0023] FIG5 is a schematic flow chart of a step of determining an aligned text line in one embodiment;

[0024] FIG6 is a schematic flow chart of a step of determining text alignment information in one embodiment;

[0025] FIG7 is a schematic diagram of document alignment in one embodiment;

[0026] FIG8 is a structural block diagram of a document alignment device in one embodiment;

[0027] FIG. 9 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0029] The document alignment method provided in the embodiment of the present application can be applied in the application environment shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Both the terminal and the server can be used alone to execute the document alignment method provided in the embodiment of the present application. The terminal and the server can also be used together to execute the document alignment method provided in the embodiment of the present application. For example, a first format document and a second format document corresponding to the first format document are obtained; the first format document includes at least one layout block, and the second format document includes at least one reference text line; based on the reference text line in the second format document, a dictionary tree corresponding to the second format document is generated; for each layout block, based on the layout block and the dictionary tree, the alignment text line corresponding to the layout block is determined; based on the layout block information corresponding to the layout block and the text line information of the alignment text line, the text alignment information of the layout block is obtained. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0030] In one embodiment, as shown in FIG. 2 , a document alignment method is provided. This embodiment is described by taking the method applied to a computer device as an example, and includes steps 202 to 208 .

[0031] Step 202, obtaining a first format document and a second format document corresponding to the first format document; the first format document includes at least one layout block, and the second format document includes at least one reference text line.

[0032] Among them, the first format document is a document with a flow layout. A document with a flow layout means that the structure of the content in the document is organized based on logical units such as paragraphs, chapters, sentences, lists, tables, etc., rather than being directly bound to a fixed page position. It can be understood that the text and images in the first format document do not have fixed page numbers and position information. The layout of the text and images will automatically adjust according to the editor's window size, zoom ratio or font size. The first format document can be a DOC (Document) format document and a DOCX (Document XML) format document. The second format document refers to a document with a fixed layout. A document with a fixed layout means that the position of the content in the document is fixed. It can be understood that the text and images in the second format document are bound to the position coordinates. The layout of the text and images will not automatically adjust according to the editor's window size, zoom ratio or font size. The second format document can be a PDF document. The layout block refers to the independent rectangular areas or logical blocks in the document that divide the document content according to the physical layout and structure of the page. A layout block can contain one or more different types of content elements, such as one or more of a text paragraph, image, table, and title. A layout block is considered a content unit that can be processed separately during document parsing. A text line refers to a line of characters, and a reference text line refers to a line of characters in the second format document.

[0033] Exemplarily, a computer device acquires a document in a first format and converts the document in the first format into a document in a second format; the document in the first format includes at least one layout block, and the document in the second format includes at least one reference text line.

[0034] In one embodiment, a computer device obtains a document in a first format, converts the document in the first format into a document in a second format through an Apache (Apache Software Foundation) open source parsing package POI (Apache Poor Obfuscation Implementation, library) of java (Java programming language), obtains at least one layout block from the document in the first format, and obtains at least one reference text line from the document in the second format.

[0035] In one embodiment, the computer device obtains a document in a first format, converts the document in the first format into an HTML (Hyper Text Markup Language) format document, parses the HTML format document based on a preset tag pair to obtain document layout information corresponding to the document in the first format, and determines at least one layout block corresponding to the document in the first format based on the document layout information; converts the document in the first format into a document in a second format, and obtains at least one reference text line from the document in the second format. The document layout information includes information such as images, tables, headers, footers, titles, paragraphs, and tables of contents of the document in the first format.

[0036] Step 204: Generate a trie corresponding to the document in the second format based on the reference text lines in the document in the second format.

[0037] The trie is a data structure for storing and retrieving strings. The trie corresponds to the document in the second format one by one. The trie stores information of all reference text lines in the document in the second format, including information of the reference character sequence of the reference text line, the text line length of the reference text line, the page number of the reference text line in the document in the second format, and the initial reading order of the reference text line. The trie is constructed by all reference text lines in the document in the second format. The trie includes multiple nodes. The node information of each node includes a character, a text line length, a page number, and a reading order. There is a connection relationship between nodes, and some nodes have corresponding failure pointers. The connection relationship between nodes represents the parent-child relationship between nodes, and the failure pointer corresponding to a node represents the jump relationship of the node. For example, as shown in Figure 3, the trie includes 12 nodes. The solid line between nodes represents the parent-child relationship between nodes, and the dotted line between nodes represents the jump relationship of nodes. The node information corresponding to the node "Jing" is {Jing, 2, [(1,1)]}, where "Jing" is the character corresponding to the node, 2 is the text line length of the reference text line "Nanjing" where "Jing" is located, the first dimension 1 in (1,1) represents that the reference text line "Nanjing" where "Jing" is located is on the first page of the document in the second format, and the second dimension 1 in (1,1) represents that the initial reading order of the reference text line "Nanjing" where "Jing" is located in the document in the second format is 1.

[0038] Exemplarily, for each reference text line in the document in the second format, determine the reference character sequence corresponding to the reference text line, the target page number in the document in the second format, and the initial reading order; based on the reference character sequence corresponding to each reference text line, the target page number in the document in the second format, and the initial reading order, determine the trie corresponding to the document in the second format.

[0039] Step 206: For each layout block, determine the aligned text line corresponding to the layout block based on the layout block and the trie.

[0040] The aligned text line refers to the reference text line corresponding to the layout block, which can be understood as the reference text line included in the layout block. For example, the aligned text lines of the layout block "Nanjing Yangtze River Bridge" are "Nanjing", "Yangtze River" and "Bridge".

[0041] Exemplarily, when the layout block is text to be aligned, determine the character sequence to be aligned corresponding to the layout block, and based on the arrangement order of the characters to be aligned in the character sequence to be aligned, traverse the dictionary tree corresponding to the second format document to obtain the aligned text line corresponding to the layout block.

[0042] Step 208, obtaining text alignment information of the layout block based on the layout block information corresponding to the layout block and the text line information of the aligned text line.

[0043] The layout block information refers to the information of the layout block in the first format document, including but not limited to the layout block code, layout block type, layout block string, table information contained in the table type layout block and layout position coordinates. The layout block information can be represented by a layout block vector. For example, the layout block information is ζ, ζ={(ζ r k ,ζ t k ,ζ s' k ,ζ f k ,ζ p k ) k=1,…,D}, where k represents the identifier of the layout block, and the identifier of the layout block represents the reading order of the layout block; ζ r k Indicates the layout block code, r k = k; ζ s' k The information representing the layout block, for example, the layout block is the text information of a paragraph, then s' k Contains all the characters in the paragraph; s' k ={c' h k} t=1,…,L ',ζ s' k (h)=c' h k , c' h k is the hth character of the kth layout block, L' is the string length of the layout block; s' kIt can also be represented as a byte stream of images in a layout block; t k Indicates the type information of the layout block, for example, table, title, directory, paragraph, etc.; D is the total number of layout blocks in the first format document; f k The table information contained in the layout block represented as a table type includes the row and column information of the table cells and the text information in the cells; p k is the initial position coordinate of the layout block; r k , t k and s' k The target position coordinates of the layout block are obtained by parsing the first format document and are determined based on the text line information of the aligned text lines.

[0044] The text line information refers to the information of the aligned text line in the second format document, and the text line information includes but is not limited to the text line page, the initial reading order of the text line, the text line string and the text line position information. The text line information can be represented by a text line vector. For example, the text line information is ω, ω={ω i =(ω n i ,ω r i , ω s i ,ω p i ) i=1,…,M}, wherein i represents the arrangement order of the aligned text lines in the page where the second format document is located, for example, the arrangement order of the aligned text lines is obtained after the reference text lines in the page where the aligned text lines in the second format document are sorted in a manner of first up and down and then left and right; M represents the number of reference text lines in the page where the aligned text lines are located in the second format document; ω n i Indicates the page number of the page where the aligned text line is located in the second format document; r i Indicates the initial reading order of aligned text lines. Before document alignment is performed, the default is ω r i =i, the target reading order of aligned text lines is determined based on the corresponding layout blocks; ω s i A string representing the alignment of text lines, where ω s i ={c j i} j=1,…,L ,ω si (j)=c j i , L is the length of the string of aligned text lines, c j i Indicates the jth character of the aligned text line; ω p i Indicates the position information of the aligned text line in the second format document. The position information of the aligned text line can be represented by the position coordinates of the upper left corner of the rectangular box surrounding the first character of the aligned text line and the position of the lower right corner of the rectangular box surrounding the last character of the aligned text line. For example, ω p i =(x 0 i ,y 0 i ,x 1 i ,y 1 i ), (x 0 i ,y 0 i ) is the position coordinate of the upper left corner of the rectangular box surrounding the first character of the aligned text line, (x 1 i ,y 1 i ) is the position of the lower right corner of the rectangular box surrounding the last character of the aligned text line.

[0045] Text alignment information refers to information that characterizes the correspondence between layout blocks and aligned text lines. Text alignment information includes updated layout block information corresponding to layout blocks and updated text line information of all aligned text lines. Updated layout block information refers to information obtained by updating the initial position coordinates in the layout block information to the target position coordinates. Updated text line information refers to information obtained by updating the initial reading order in the text line information to the target reading order. For example, if the version block only includes text, the text alignment information is ζ r k ,ζ t k ,ζ s' k ,ζ f k ,ζ p k ,(ω n i ,ω r i ,ω s i ,ω p i ) i,…,T ,r k ,ζ t k ,ζ s' k ,ζ f k ,ζ p k To update the page block information, p k is the target position coordinate of the layout block, (ω n i ,ω r i ,ω s i ,ω p i ) i,…,T Update text line information corresponding to multiple aligned text lines, ω r i The target reading order for justifying lines of text.

[0046] Exemplarily, the computer device determines the target reading order corresponding to the aligned text lines and the target position coordinates corresponding to the layout block based on the layout block information corresponding to the layout block and the text line information of the aligned text lines, and obtains the text alignment information of the layout block based on the layout block information, the target position coordinates of the layout block, the text line information corresponding to the aligned text lines, and the target reading order.

[0047] In one embodiment, step 208 further includes: combining text alignment information corresponding to multiple layout blocks based on the layout block codes of the layout blocks to obtain document alignment information corresponding to the first format document.

[0048] In the above document alignment method, a first format document and a second format document corresponding to the first format document are obtained. Since the first format document is a document with a streaming layout, the first format document includes at least one layout block, and the layout block information of the layout block includes information such as the layout block reading order and the layout block type, but the position coordinates of the layout block cannot be determined based on the first format document. Since the second format document is a document with a fixed layout, the second format document includes at least one reference text line, and the text line information of the reference text line includes text line page and text line position information, etc., but the reading order of the reference text line cannot be determined based on the second format document; based on the reference text line in the second format document, a dictionary tree corresponding to the second format document is generated, and for each layout block, based on the layout block and the dictionary tree, the aligned text line corresponding to the layout block is determined, that is, by traversing the dictionary tree, the aligned text line corresponding to the layout block is determined, and then based on the layout block information corresponding to the layout block and the text line information of the aligned text line, the target position coordinates of the layout block and the target reading order of the text line are determined, and the text alignment information of the layout block is obtained, thereby improving the accuracy of the document alignment information.

[0049] In one embodiment, as shown in FIG4 , generating a dictionary tree corresponding to the second format document based on the reference text line in the second format document includes:

[0050] Step 402: for each reference text line in the second format document, determine the target page number and initial reading order of the reference text line in the second format document.

[0051] The target page number refers to the identifier of the page where the reference text line is located in the second format document. The initial reading order refers to the code of the reference text line in the page where the reference text line is located in the second format document, which can be understood as the arrangement order obtained by arranging all reference text lines in the page where the reference text line is located in the second format document according to the preset coding order.

[0052] Exemplarily, for each reference text line in the second format document, the computer device determines the target page number and initial reading order of the reference text line in the second format document.

[0053] Step 404 , obtaining a reference character sequence corresponding to the reference text line, and traversing from a preset root node based on the arrangement order of characters in the reference character sequence.

[0054] The reference character sequence refers to a sequence determined according to the characters and the order of characters in the reference text line, and the reference character sequence includes all characters in the reference text line. The preset root node refers to a pre-set root node, the character corresponding to the preset root node is " / ", the text line length corresponding to the preset root node is "-1", and the page number and initial reading order corresponding to the preset root node are both empty. For example, the root node of the dictionary tree shown in FIG3 is the preset root node, and the node information of the preset root node is { / ,-1,[]}.

[0055] Exemplarily, the computer device obtains a reference character sequence corresponding to a reference text line, and traverses from a preset root node based on the arrangement order of characters in the reference character sequence.

[0056] Step 406, for the non-last character in the reference character sequence, if there is no node corresponding to the non-last character, a node corresponding to the non-last character is added, and the node information of the node corresponding to the non-last character is determined based on a preset length, a preset page number and a preset reading order.

[0057] Among them, non-last characters refer to other characters in the reference character sequence except the last character. Preset length refers to the preset text line length, which can be set to -1. Preset page refers to the preset page number, which can be empty. Preset reading order refers to the preset reading order, which can be empty.

[0058] Exemplarily, for a non-end character in a reference character sequence, if there is no node corresponding to the non-end character, the computer device adds a node corresponding to the non-end character, and determines the node information of the node corresponding to the non-end character based on a preset length, a preset page number, and a preset reading order.

[0059] Step 408 , for the last character in the reference character sequence, the node information of the node corresponding to the last character is determined based on the text line length corresponding to the reference text line, the target page and the initial reading order.

[0060] The last character is the last character in the reference character sequence.

[0061] Exemplarily, for the last character in the reference character sequence, the computer device determines the node information of the node corresponding to the last character based on the text line length corresponding to the reference text line, the target page, and the initial reading order.

[0062] In one embodiment, the root node contains no characters, the text line length L of the root node is -1, the page number and the encoding sequence number of the text line are empty, and each node except the root node contains only one character; for each text line ω i Add character c to the dictionary tree starting from the root node ij , if the character c i j The next node P->next is not in the current node P (the same character c does not exist in the next node P->next i j ), then add the character c to the next node P->next of the current node P in the dictionary tree. i j If the character c i j Yes i The last character of the text line needs to be updated to the length of node P->next, which is L, and the page number of the text line is ω n i and the text line encoding number ω r i When the node P->next information is updated, the node P is updated to the next node P=P->next, and the next character is added. When the next node P->next of the node P exists with the character c i j Same characters, if character c i j Yes i The last character of the text line needs to be updated to the length of node P->next, which is L, and the page number of the text line is ω n i and the text line encoding number ω r i Information, if the character c i j If it is not the last character, there is no need to update the node P->next information. Finally, update the node P to the next node of P = P->next, and continue to add the next character. The length of the text line from the root node to a child node is greater than 0. The string passed is the string corresponding to the node, that is, the text line string ω i , for a table, it is a single-line string in a single cell.

[0063] Step 410: Generate a dictionary tree corresponding to the second format document based on the preset root node and the node information corresponding to each node.

[0064] Exemplarily, the computer device generates a dictionary tree corresponding to the second format document according to a preset root node and node information corresponding to each node.

[0065] In one embodiment, the computer device generates an initial dictionary tree according to the preset root node and the node information corresponding to each node, determines the failure pointer of the node in the initial dictionary tree, and obtains the dictionary tree corresponding to the second format document. Determining the failure pointer of the node in the initial dictionary tree includes: traversing the child nodes of the root node, and the failure pointers of the child nodes of the root node all point to the root node; using the depth-first algorithm to traverse and set the failure pointers of other nodes on the dictionary tree (except the root node and the child nodes of the root node), and setting the failure pointer of its P->next child node based on the failure pointer of the current node P, that is, the current node P has a fail pointer, and attempts to set the failure pointer of its child node P->next. If its child node P->next is in the child node P->fail->next of the failure pointer of the current node P, assign the node with the same character value as the child node character of P in the child node of the failure pointer of the current node P to the failure pointer of the child node P->next, otherwise update the failure pointer of the current node P to the failure pointer of the failure pointer of the current node P until a node with the same character as the child node of P is found or the failure pointer of the current node P does not exist. When the failure pointer of the current node P does not exist, let the failure of the child node P->next point to the root node. For example, construct a dictionary tree for the text lines Nanjing City, Nanjing, Mayor, Yangtze River, Bridge and Changgan Bridge, and obtain the dictionary tree shown in Figure 3.

[0066] In this embodiment, based on the arrangement order of characters in the reference character sequence, traversal is performed from a preset root node. For non-end characters in the reference character sequence, if there is no node corresponding to the non-end character, a node corresponding to the non-end character is added, and the node information of the node corresponding to the non-end character is determined based on the preset length, preset page number and preset reading order. For the end character in the reference character sequence, the node information of the node corresponding to the end character is determined based on the text line length, target page and initial reading order corresponding to the reference text line; based on the preset root node and the node information corresponding to each node, a dictionary tree corresponding to the second format document is generated, that is, the generated dictionary tree includes the information of the characters in each reference text line in the second format document, as well as the target page number and initial reading order of the reference text line, providing accurate basic data for the subsequent determination of aligned text lines.

[0067] In one embodiment, as shown in FIG5 , determining the aligned text line corresponding to the layout block based on the layout block and the dictionary tree includes:

[0068] Step 502, obtaining the sequence of characters to be aligned corresponding to the layout block.

[0069] The character sequence to be aligned refers to a sequence determined by the characters in the layout block and the order of the characters, and the character sequence to be aligned includes all the characters in the layout block.

[0070] Exemplarily, the computer device obtains the character sequence to be aligned corresponding to the layout block.

[0071] Step 504: Determine the character sequence to be aligned as the current sequence. Based on the arrangement order of the characters to be aligned in the current sequence, start matching from the child nodes of the preset root node of the trie until the character corresponding to the next child node of the matching branch of the trie does not include the next character to be aligned. Based on the matched nodes in the matching branch, determine the candidate text line; the candidate text line is composed of the characters corresponding to at least one matched node.

[0072] Wherein, the current sequence refers to the character sequence currently being matched. The matching branch refers to the branch composed of the nodes that have completed matching with the characters in the current sequence. For example, when the current sequence is {Nan, Jing, Shi, Chang, Jiang, Da, Qiao} and traversing in the trie shown in Figure 3, if the current character to be aligned is "Shi", then the matching branch is { / , -1, []}, {Nan, -1, []}, {Jing, 2, [(1, 1)]}, {Shi, 3, [(1, 2)]}. The next child node refers to the child node of the current matching node, and the current matching node refers to the node that matches the current character to be aligned. For example, if the current matching node is {Chang, -1, [()]} then the next child nodes are {Jiang, 2, [(1, 4)]} and {Gan, -1, [()]}. The next character to be aligned refers to the next character to be aligned, which can be understood as the next character to be aligned after the current character to be aligned. For example, when the current sequence is {Nan, Jing, Shi, Chang, Jiang, Da, Qiao} and the current character to be aligned is "Shi", the next character to be aligned is "Chang". The matched node refers to the node in the matching branch.

[0073] Exemplarily, the computer device determines the character sequence to be aligned as the current sequence. Based on the arrangement order of the characters to be aligned in the current sequence, start matching from the child nodes of the preset root node of the trie until the character corresponding to the next child node of the matching branch of the trie does not include the next character to be aligned, obtaining the matching branch. Determine the target matching nodes as the matched nodes in the matching branch whose text line length is not equal to the preset length. For each target matching node, based on the characters corresponding to the matched nodes before the target matching node and the target matching node in the matching branch, obtain the candidate text line corresponding to the target matching node. For example, if the matching branch is { / , -1, []}, {Nan, -1, []}, {Jing, 2, [(1, 1)]}, {Shi, 3, [(1, 2)]}, the obtained candidate text lines are "Nan Jing" and "Nan Jing Shi".

[0074] Step 506: Determine the next character to be aligned in the character sequence to be aligned and the characters to be aligned after the next character to be aligned as the updated current sequence.

[0075] Exemplarily, the computer device determines the next character to be aligned and the character to be aligned after the next character to be aligned in the sequence of characters to be aligned as the updated current sequence.

[0076] Step 508, repeatedly execute the step of matching from the child nodes of the preset root node of the dictionary tree based on the arrangement order of the characters to be aligned in the current sequence, until the characters corresponding to the next child node of the matching branch of the dictionary tree do not include the next character to be aligned, and determine the candidate text lines based on the matched nodes in the matching branch.

[0077] Exemplarily, the computer device executes matching from the child nodes of the preset root node of the dictionary tree based on the arrangement order of the characters to be aligned in the current sequence, until the characters corresponding to the next child node of the matching branch of the dictionary tree do not include the next character to be aligned, and determines the candidate text line based on the matched nodes in the matching branch.

[0078] In one embodiment, traversing the layout block z s' k The character c' h k , select the root node in the dictionary tree as the starting node, and determine the character c' h k Is it in the next node of the current node P? If the character c' h k In the next node of P, the P node is updated to the next node of P node that contains the character, and the character length of the node is checked to see if it is greater than 0. If it is greater than 0, it means that an aligned text line has been matched. Then, according to the failure pointer of P node, backtrack (that is, repeatedly update the P node to the failure pointer of P node) to check whether there is a node with a character length greater than 0 until the failure pointer of the node points to the root node, indicating that there is also a matching text line in the previous link. If the next node of the current P node does not contain the character c' h k When the character is found, update the P node to the failure pointer of the P node, and continue to see whether the P node contains the character. If it does, continue the above steps. Otherwise, continue to update the P node to the failure pointer of the P node until the P node is equal to the root node, and then match the next character.

[0079] Step 510, until the next character to be aligned is empty, determine the aligned text line corresponding to the layout block based on the candidate text lines.

[0080] Exemplarily, until the next character to be aligned is empty, the computer device determines the aligned text line corresponding to the layout block based on the candidate text lines.

[0081] In this embodiment, a dictionary tree is used to traverse the characters in the character sequence to be aligned of the layout block to obtain candidate text lines corresponding to the layout block, and the alignment text lines are determined based on the candidate text lines, providing accurate basic data for document alignment of the candidates.

[0082] In one embodiment, determining the aligned text line corresponding to the layout block based on the candidate text line includes:

[0083] Determine the starting position information of the candidate text lines in the layout block; combine the candidate text lines based on the starting position information of the candidate text lines in the layout block to obtain multiple candidate blocks; determine the candidate text lines corresponding to the candidate blocks that are the same as the layout block as the aligned text lines corresponding to the layout block.

[0084] Among them, the starting position information refers to the information composed of the position of the first character in the candidate text line in the layout block and the position of the last character in the candidate text line in the layout block. The starting position information can be composed of the arrangement order of the first character in the candidate text line in the layout block and the arrangement order of the last character in the candidate text line in the layout block. For example, the candidate text line is "Nanjing", the layout block is "Nanjing Yangtze River Bridge", "South" is the first character of the layout block, and "Beijing" is the second character of the layout block, then the starting position information of "Nanjing" is (1,2). A candidate block refers to a text block composed of candidate text lines. For example, the layout block is "Nanjing Yangtze River Bridge". According to the dictionary tree shown in Figure 3, the candidate text lines are determined to be "Nanjing", "Nanjing City", "Mayor", "Yangtze River", and "Bridge". The starting position information of "Nanjing" is (1,2), the starting position information of "Nanjing City" is (1,3), the starting position information of "Mayor" is (3,4), the starting position information of "Yangtze River" is (4,5), and the starting position information of "Bridge" is (6,7). According to the starting position information of the candidate text lines in the layout block, the candidate text lines are combined to obtain multiple candidate blocks "Nanjing Mayor" and "Nanjing Yangtze River Bridge".

[0085] Exemplarily, a computer device determines the starting position information of a candidate text line in a layout block, and based on the starting position information of the candidate text line in the layout block, combines the candidate text lines so that the first positions of the starting position information of the candidate text lines are connected to obtain multiple candidate blocks, and determines the candidate text lines corresponding to the candidate blocks that are the same as the layout block as the aligned text lines corresponding to the layout block.

[0086] In this embodiment, candidate text lines are combined according to their starting position information in the layout block to obtain multiple candidate blocks, and the candidate text lines corresponding to the candidate blocks that are the same as the layout block are determined as the aligned text lines corresponding to the layout block, thereby improving the accuracy of aligning the lines.

[0087] In one embodiment, as shown in FIG6 , based on the layout block information corresponding to the layout block and the text line information of the aligned text line, the text alignment information of the layout block is obtained, including:

[0088] Step 602, based on the first format document, determine the layout block information corresponding to the layout block; the layout block information includes the layout block code.

[0089] The layout block code refers to an identifier that represents the reading order of the layout block.

[0090] Exemplarily, the computer device parses the first format document to obtain layout block information corresponding to the layout block.

[0091] Step 604: determine text line information of the aligned text lines based on the second format document; the text line information includes text line position information.

[0092] The text line position information refers to the position of the aligned text line in the second format document. The text line position information can be represented by the position coordinates of the upper left corner of the rectangular box surrounding the first character of the text line and the position of the lower right corner of the rectangular box surrounding the last character of the text line. For example, the text line position information ω p i =(x 0 i ,y 0 i ,x 1 i ,y 1 i ), (x 0 i ,y 0 i ) is the position coordinate of the upper left corner of the rectangular box surrounding the first character of the text line, (x 1 i ,y 1 i ) is the position of the lower right corner of the rectangular box surrounding the last character of the text line.

[0093] Exemplarily, the computer device determines text line information of the aligned text lines according to the second format document.

[0094] Step 606, for each aligned text line, based on the starting position information of the aligned text line in the layout block and the layout block code corresponding to the layout block, determine the target reading order corresponding to the aligned text line.

[0095] Exemplarily, for each aligned text line, based on the starting position information of the aligned text line in the layout block, the arrangement order of the aligned text line in the layout block is determined, based on the layout block code corresponding to the layout block, the target reading order of the last aligned text line of the previous layout block is obtained, and based on the target reading order of the last aligned text line of the previous layout block and the arrangement order of the aligned text line in the layout block, the target reading order corresponding to the aligned text line is determined. For example, if the layout block is "Nanjing Yangtze River Bridge", the aligned text lines are "Nanjing", "Yangtze River" and "Bridge", then the arrangement order of "Nanjing" is 1, the arrangement order of "Yangtze River" is 2, and the arrangement order of "Bridge" is 3, and the target reading order of the last aligned text line of the previous layout block is 102, then the target reading order of "Nanjing" is 103, the target reading order of "Yangtze River" is 104, and the target reading order of "Bridge" is 105.

[0096] Step 608, based on the text line position information of the multiple aligned text lines corresponding to the layout block, determine the target position coordinates corresponding to the layout block.

[0097] Exemplarily, the computer device determines the minimum bounding rectangle corresponding to each aligned text line, determines the vertex coordinates of the minimum bounding rectangle in the second format document, compares the vertex coordinates corresponding to all aligned text lines, obtains the maximum value of the first direction coordinate, the minimum value of the first direction coordinate, the maximum value of the second direction coordinate, and the minimum value of the second direction coordinate, and obtains the target position coordinates corresponding to the layout block based on the maximum value of the first direction coordinate, the minimum value of the first direction coordinate, the maximum value of the second direction coordinate, and the minimum value of the second direction coordinate. max , the minimum value of the first direction coordinate x min , the maximum value of the second direction coordinate y max and the minimum value y of the second direction coordinate min , then the target position coordinates of the layout block are ((x min ,y min ),(x max ,y min ), (x max ,y max ),(x min ,y max )).

[0098] Step 610, based on the layout block information, the target position coordinates of the layout block, the text line information corresponding to the aligned text line and the target reading order, obtain the text alignment information of the layout block.

[0099] Exemplarily, the computer device updates the layout block information corresponding to the layout block based on the target position coordinates of the layout block to obtain updated layout block information, updates the text line information of the aligned text line based on the target reading order corresponding to the aligned text line to obtain updated text line information, and combines the updated layout block information and the updated line information to form text alignment information of the layout block.

[0100] In this embodiment, text alignment information of the layout block is obtained based on the layout block information, the target position coordinates of the layout block, the text line information corresponding to the aligned text line, and the target reading order. The text alignment information includes the information of the layout block in the first format document and the information of the aligned text line in the second format document, and also includes the target position coordinates of the layout block and the target reading order corresponding to the aligned text line.

[0101] In one embodiment, when the layout block is an image to be aligned, the method further includes:

[0102] Acquire candidate images from a second format document; calculate the similarity between the image to be aligned and the candidate image to obtain the similarity corresponding to the candidate image; compare the aspect ratio of the image to be aligned with the aspect ratio of the candidate image, and determine the candidate image with the same aspect ratio as the image to be aligned as the reference image; determine the reference image with the smallest similarity as the alignment image of the image to be aligned; and obtain image alignment information of the layout block based on the layout block information corresponding to the layout block and the image information of the alignment image in the second format document.

[0103] The candidate image refers to an image in the second format document. Similarity refers to a numerical value that characterizes the similarity between the image to be aligned and the candidate image. The similarity can be LPIPS (Learned Perceptual Image PatchSimilarity). Aspect ratio refers to the ratio of height to width. Image information characterizes the information of the aligned image in the second format document. The image information includes the page of the aligned image in the second format document, the initial reading order, the aligned image byte stream information, and the position information of the aligned image in the second format document. The image information can be represented by an image vector. For example, the image vector is ρ, ρ={ρ i =(ρ n i ,ρ r i ,ρ s i ,ρ p i ) i=1,…,N}, where i represents the order of the aligned images in the page where the second format document is located. For example, the images of the aligned images in the page where the second format document is located are sorted first up and down and then left and right, and the resulting arrangement order of the aligned images is i; N is the number of images of the aligned images in the page where the second format document is located; ρ n Indicates the page number of the page where the aligned image is located in the second format document; r Indicates the reading order of the aligned image. Before alignment, the default value is ρ r i =i;ρ s Indicates the alignment image byte stream information; ρ p i represents the position information of the aligned image in the second format document, ρ p i =(x 0 i ,y 0 i ,x 1 i ,y 1 i ), (x 0 i ,y 0 i ) represents the position coordinates of the upper left corner of the aligned image in the second format document, (x 1 i ,y 1 i ) represents the position coordinates of the lower right corner of the aligned image in the second format document.

[0104] Image alignment information refers to information that characterizes the correspondence between layout blocks and aligned images. Image alignment information includes updated layout block information corresponding to the layout blocks and updated image information of the aligned images. Updated layout block information refers to information obtained by updating the initial position coordinates in the layout block information to the target position coordinates. Updated image information refers to information obtained by updating the initial reading order in the image information to the target reading order. For example, if the version block only includes images, the image alignment information is ζ r k ,ζ t k ,ζ s' k ,ζ f k ,ζ p k ,ρ n i ,ρ r i ,ρ s i ,ρp i , ρ n i ,ρ r i ,ρ s i ,ρ p i To update the image information, ρ r i is the target reading order, r k ,ζ t k ,ζ s' k ,ζ f k ,ζ p k To update the page block information, p k The target position coordinates of the layout block.

[0105] Exemplarily, a computer device obtains a candidate image from a second format document, calculates the similarity between the image to be aligned and the candidate image, obtains the similarity corresponding to the candidate image, compares the aspect ratio of the image to be aligned with the aspect ratio of the candidate image, determines the candidate image having the same aspect ratio as the image to be aligned as the reference image, determines the reference image having the smallest similarity as the alignment image of the images to be aligned, determines the target position coordinates of the layout block and the target reading order of the alignment image based on the layout block information corresponding to the layout block and the image information of the alignment image in the second format document, updates the layout block information corresponding to the layout block based on the target position coordinates of the layout block to obtain updated layout block information, updates the text line information of the alignment text line based on the target reading order corresponding to the alignment text line to obtain updated text line information, and aligns the updated layout block information and the image information of the updated line information that constitutes the layout block.

[0106] In this embodiment, in the case where the content in the layout block is the image to be aligned, the alignment image of the image to be aligned is determined based on the similarity between the image to be aligned and the candidate image, and the size relationship between the aspect ratio of the image to be aligned and the aspect ratio of the candidate image, thereby improving the accuracy of the aligned image, and then based on the layout block information corresponding to the layout block and the image information of the aligned image in the second format document, the image alignment information of the layout block is obtained, thereby improving the accuracy of the image alignment information.

[0107] In an exemplary embodiment, the document alignment system includes a DOC document parser, a PDF document parser, and a document alignment module. The DOC document parser is used to parse the DOC document to obtain the layout information of the DOC document, including the diagrams, tables, headers, footers, titles, paragraphs, directories and other information of the DOC document, and determine at least one layout block of the DOC document based on the layout information; the PDF document parser is used to convert the DOC document into a PDF document, and use the PDF parsing tool to obtain the reference text line corresponding to each page in the DF document. The document alignment module is used to align the layout blocks and text lines, including:

[0108] Construct a dictionary tree corresponding to a PDF document. For each reference text line in the PDF document, the computer device determines the target page number and initial reading order of the reference text line in the PDF document, obtains a reference character sequence corresponding to the reference text line, and traverses from a preset root node based on the arrangement order of characters in the reference character sequence. For a non-last character in the reference character sequence, if there is no node corresponding to the non-last character, a node corresponding to the non-last character is added, and node information of the node corresponding to the non-last character is determined based on a preset length, a preset page number, and a preset reading order; for a last character in the reference character sequence, node information of the node corresponding to the last character is determined based on the text line length, the target page, and the initial reading order corresponding to the reference text line; an initial dictionary tree is generated based on the preset root node and the node information corresponding to each node, and a failure pointer of a node in the initial dictionary tree is determined to obtain a dictionary tree corresponding to the PDF document. Determine the failure pointer of the node in the initial dictionary tree, including: traversing the child nodes of the root node, the failure pointers of the child nodes of the root node all point to the root node; using the depth-first algorithm to traverse and set the failure pointers of other nodes on the dictionary tree (except the root node and the child nodes of the root node), and setting the failure pointer of its P->next child node based on the failure pointer of the current node P, that is, the current node P has a fail pointer, and try to set the failure pointer of its child node P->next. If its child node P->next is in the child node P->fail->next of the failure pointer of the current node P, assign the node with the same character value as the child node character of P in the child node of the failure pointer of the current node P to the failure pointer of the child node P->next, otherwise, update the failure pointer of the current node P to the failure pointer of the failure pointer of the current node P until a node with the same character as the child node of P is found or the failure pointer of the current node P does not exist. When the failure pointer of the current node P does not exist, let the failure of the child node P->next point to the root node.

[0109] For a layout block whose content is text to be aligned, the computer device obtains a sequence of characters to be aligned corresponding to the layout block, determines the sequence of characters to be aligned as the current sequence, matches from the child nodes of the preset root node of the dictionary tree based on the arrangement order of the characters to be aligned in the current sequence, until the characters corresponding to the next child node of the matching branch of the dictionary tree do not include the next character to be aligned, and obtains a matching branch, determines the matched node in the matching branch whose text line length is not equal to the preset length as the target matching node, and for each target matching node, obtains the candidate text line corresponding to the target matching node based on the characters corresponding to the matched nodes before the target matching node and the target matching node in the matching branch; determines the next character to be aligned in the sequence of characters to be aligned and the character to be aligned after the next character to be aligned as the updated current sequence, repeats the step of matching from the child nodes of the preset root node of the dictionary tree based on the arrangement order of the characters to be aligned in the current sequence, until the characters corresponding to the next child node of the matching branch of the dictionary tree do not include the next character to be aligned, and determines the candidate text line based on the matched nodes in the matching branch, until the next character to be aligned is empty, and obtains multiple candidate text lines. Determine the starting position information of the candidate text lines in the layout block, and based on the starting position information of the candidate text lines in the layout block, combine the candidate text lines so that the first positions of the starting position information of the candidate text lines are connected to obtain multiple candidate blocks, and determine the candidate text lines corresponding to the candidate blocks that are the same as the layout block as the aligned text lines corresponding to the layout block.

[0110] Parse the DOC document to obtain the layout block information corresponding to the layout block, and determine the text line information of the aligned text line according to the PDF document; for each aligned text line, determine the arrangement order of the aligned text line in the layout block based on the starting position information of the aligned text line in the layout block, obtain the target reading order of the last aligned text line of the previous layout block based on the layout block code corresponding to the layout block, and determine the target reading order corresponding to the aligned text line based on the target reading order of the last aligned text line of the previous layout block and the arrangement order of the aligned text line in the layout block; determine the minimum bounding rectangle corresponding to each aligned text line respectively, and determine the top position of the minimum bounding rectangle in the PDF document. Point coordinates, compare the vertex coordinates corresponding to all aligned text lines to obtain the maximum value of the first direction coordinate, the minimum value of the first direction coordinate, the maximum value of the second direction coordinate and the minimum value of the second direction coordinate, based on the maximum value of the first direction coordinate, the minimum value of the first direction coordinate, the maximum value of the second direction coordinate and the minimum value of the second direction coordinate, obtain the target position coordinates corresponding to the layout block; based on the target position coordinates of the layout block, update the layout block information corresponding to the layout block to obtain the updated layout block information, based on the target reading order corresponding to the aligned text line, update the text line information of the aligned text line to obtain the updated text line information, and combine the updated layout block information and the updated line information to form the text alignment information of the layout block.

[0111] For a layout block whose content is an image to be aligned, a candidate image is obtained from the PDF document, the similarity between the image to be aligned and the candidate image is calculated to obtain the similarity corresponding to the candidate image, the aspect ratio of the image to be aligned is compared with the aspect ratio of the candidate image, the candidate image having the same aspect ratio as the image to be aligned is determined as the reference image, the reference image having the smallest LIPIS similarity is determined as the alignment image of the image to be aligned, the target position coordinates of the layout block and the target reading order of the alignment image are determined based on the layout block information corresponding to the layout block and the image information of the alignment image in the PDF document, the layout block information corresponding to the layout block is updated based on the target position coordinates of the layout block to obtain the updated layout block information, the text line information of the alignment text line is updated based on the target reading order corresponding to the alignment text line to obtain the updated text line information, and the updated layout block information and the updated line information are combined to form the image alignment information of the layout block.

[0112] For example, in the document alignment diagram shown in FIG7 , the DOC document includes a version block whose content is the text to be aligned and a version block whose content is the image to be aligned. First, the text alignment information corresponding to the version block whose content is the text to be aligned is determined, and then the image alignment information corresponding to the version block whose content is the image to be aligned is determined. In order to use a unified format for marking the text alignment information and the image alignment information, ζ is used. r k ,ζt k ,ζ s' k ,ζ f k ,ζ p k ,ρ s i ,(ω n i ,ω r i ,ω s i ,ω p i ) i,…,T To indicate that the text alignment information is s i Is empty, image alignment information (ω n i ,ω r i ,ω s i ,ω p i ) i,…,T Is empty.

[0113] In the above document alignment method, a DOC document and a PDF document corresponding to the DOC document are obtained. Since the DOC document is a document with a streaming layout, the DOC document includes at least one layout block. The layout block information of the layout block includes information such as a layout block reading order and a layout block type, but the position coordinates of the layout block cannot be determined according to the DOC document. Since the PDF document is a document with a fixed layout, the PDF document includes at least one reference text line. The text line information of the reference text line includes text line page and text line position information, etc., but the reading order of the reference text line cannot be determined according to the PDF document. Based on the reference text lines in the PDF document, a dictionary tree corresponding to the PDF document is generated. For each layout block, based on the layout block and the dictionary tree, the aligned text line corresponding to the layout block is determined, that is, by traversing the dictionary tree, the aligned text line corresponding to the layout block is determined, and then based on the layout block information corresponding to the layout block and the text line information of the aligned text line, the target position coordinates of the layout block and the target reading order of the text line are determined, and the text alignment information of the layout block is obtained, thereby improving the accuracy of document alignment.

[0114] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0115] Based on the same inventive concept, the embodiment of the present application also provides a document alignment device for implementing the document alignment method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more document alignment device embodiments provided below can refer to the limitations on the document alignment method above, and will not be repeated here.

[0116] In one embodiment, as shown in FIG. 8 , a document alignment device is provided, including: an acquisition module 802 , a generation module 804 , a determination module 806 and an alignment module 808 , wherein:

[0117] An acquisition module 802 is used to acquire a first format document and a second format document corresponding to the first format document; the first format document includes at least one layout block, and the second format document includes at least one reference text line;

[0118] A generating module 804, configured to generate a dictionary tree corresponding to the second format document based on the reference text lines in the second format document;

[0119] A determination module 806 is used to determine, for each layout block, the aligned text line corresponding to the layout block based on the layout block and the dictionary tree;

[0120] The alignment module 808 is used to obtain the text alignment information of the layout block based on the layout block information corresponding to the layout block and the text line information of the aligned text line.

[0121] In one embodiment, the generation module 804 is also used to: determine, for each reference text line in the second format document, the target page number and initial reading order of the reference text line in the second format document; obtain a reference character sequence corresponding to the reference text line, and traverse from a preset root node based on the arrangement order of the characters in the reference character sequence; for a non-end character in the reference character sequence, if there is no node corresponding to the non-end character, add a node corresponding to the non-end character, and determine the node information of the node corresponding to the non-end character based on the preset length, preset page number and preset reading order; for the end character in the reference character sequence, determine the node information of the node corresponding to the end character based on the text line length, target page and initial reading order corresponding to the reference text line; and generate a dictionary tree corresponding to the second format document based on the preset root node and the node information corresponding to each node.

[0122] In one embodiment, the determination module 806 is also used to: obtain the sequence of characters to be aligned corresponding to the layout block; determine the sequence of characters to be aligned as the current sequence, match from the child nodes of the preset root node of the dictionary tree based on the arrangement order of the characters to be aligned in the current sequence, until the characters corresponding to the next child node of the matching branch of the dictionary tree do not include the next character to be aligned, and determine the candidate text line based on the matched nodes in the matching branch; the candidate text line consists of characters corresponding to at least one matched node; determine the next character to be aligned in the sequence of characters to be aligned and the character to be aligned after the next character to be aligned as the updated current sequence; repeat the steps of matching from the child nodes of the preset root node of the dictionary tree based on the arrangement order of the characters to be aligned in the current sequence, until the characters corresponding to the next child node of the matching branch of the dictionary tree do not include the next character to be aligned, and determine the candidate text line based on the matched nodes in the matching branch; until the next character to be aligned is empty, determine the aligned text line corresponding to the layout block based on the candidate text line.

[0123] In one embodiment, the determination module 806 is also used to: determine the starting position information of the candidate text line in the layout block; combine the candidate text lines based on the starting position information of the candidate text line in the layout block to obtain multiple candidate blocks; and determine the candidate text line corresponding to the candidate block that is the same as the layout block as the aligned text line corresponding to the layout block.

[0124] In one embodiment, the alignment module 808 is also used to: determine the layout block information corresponding to the layout block based on the first format document; the layout block information includes the layout block code; determine the text line information of the aligned text line based on the second format document; the text line information includes the text line position information; for each aligned text line, determine the target reading order corresponding to the aligned text line based on the starting position information of the aligned text line in the layout block and the layout block code corresponding to the layout block; determine the target position coordinates corresponding to the layout block based on the text line position information of multiple aligned text lines corresponding to the layout block; obtain the text alignment information of the layout block based on the layout block information, the target position coordinates of the layout block, the text line information corresponding to the aligned text line and the target reading order.

[0125] In one embodiment, the alignment module 808 is also used to: obtain candidate images from a second format document; calculate the similarity between the image to be aligned and the candidate image to obtain the similarity corresponding to the candidate image; compare the aspect ratio of the image to be aligned with the aspect ratio of the candidate image, and determine the candidate image with the same aspect ratio as the image to be aligned as the reference image; determine the reference image with the smallest similarity as the alignment image of the image to be aligned; and obtain the image alignment information of the layout block based on the layout block information corresponding to the layout block and the image information of the alignment image in the second format document.

[0126] Each module in the above-mentioned document alignment device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0127] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG9. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a document alignment method is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0128] Those skilled in the art will understand that the structure shown in FIG. 9 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0129] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0130] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0131] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0132] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0133] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0134] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A document alignment method, characterized in that: The method comprises: Acquire a first format document and a second format document corresponding to the first format document; the first format document includes at least one layout block, and the second format document includes at least one reference text line; generating a dictionary tree corresponding to the second format document based on the reference text line in the second format document; For each of the layout blocks, based on the layout block and the dictionary tree, determining the aligned text line corresponding to the layout block; Based on the layout block information corresponding to the layout block and the text line information of the aligned text line, the text alignment information of the layout block is obtained.

2. The method according to claim 1, characterized in that The step of generating a dictionary tree corresponding to the second format document based on the reference text line in the second format document includes: For each reference text line in the second format document, determining a target page number and an initial reading order of the reference text line in the second format document; Obtaining a reference character sequence corresponding to the reference text line, and traversing from a preset root node based on the arrangement order of characters in the reference character sequence; For a non-last character in the reference character sequence, if there is no node corresponding to the non-last character, a node corresponding to the non-last character is added, and node information of the node corresponding to the non-last character is determined based on a preset length, a preset page number, and a preset reading order; For the last character in the reference character sequence, determining the node information of the node corresponding to the last character based on the text line length corresponding to the reference text line, the target page and the initial reading order; Based on the preset root node and the node information corresponding to each of the nodes, a dictionary tree corresponding to the second format document is generated.

3. The method according to claim 1, characterized in that The step of determining the aligned text line corresponding to the layout block based on the layout block and the dictionary tree includes: Obtaining a sequence of characters to be aligned corresponding to the layout block; Determine the sequence of characters to be aligned as the current sequence, match from the child nodes of the preset root node of the dictionary tree based on the arrangement order of the characters to be aligned in the current sequence, until the characters corresponding to the next child node of the matching branch of the dictionary tree do not include the next character to be aligned, and determine a candidate text line based on the matched nodes in the matching branch; the candidate text line is composed of characters corresponding to at least one of the matched nodes; Determine the next character to be aligned and the character to be aligned after the next character to be aligned in the character sequence to be aligned as an updated current sequence; Repeat the step of matching from the child nodes of the preset root node of the dictionary tree based on the arrangement order of the characters to be aligned in the current sequence until the characters corresponding to the next child node of the matching branch of the dictionary tree do not include the next character to be aligned, and determining the candidate text line based on the matched nodes in the matching branch; Until the next character to be aligned is empty, the aligned text line corresponding to the layout block is determined based on the candidate text lines.

4. The method according to claim 3, characterized in that: The step of determining the aligned text line corresponding to the layout block based on the candidate text line includes: Determine the starting position information of the candidate text line in the layout block; Based on the starting position information of the candidate text lines in the layout block, the candidate text lines are combined to obtain a plurality of candidate blocks; The candidate text line corresponding to the candidate block that is the same as the layout block is determined as the aligned text line corresponding to the layout block.

5. The method according to claim 1, characterized in that The obtaining of the text alignment information of the layout block based on the layout block information corresponding to the layout block and the text line information of the aligned text line includes: Based on the first format document, determine the layout block information corresponding to the layout block; the layout block information includes a layout block code; Based on the second format document, determining text line information of the aligned text line; the text line information includes text line position information; For each of the aligned text lines, based on the starting position information of the aligned text line in the layout block and the layout block code corresponding to the layout block, determine the target reading order corresponding to the aligned text line; Determining the target position coordinates corresponding to the layout block based on the text line position information of the multiple aligned text lines corresponding to the layout block; Based on the layout block information, the target position coordinates of the layout block, the text line information corresponding to the aligned text line and the target reading order, the text alignment information of the layout block is obtained.

6. The method according to claim 1, characterized in that In the case where the layout block is an image to be aligned, the method further includes: Acquire a candidate image from the second format document; Calculating the similarity between the image to be aligned and the candidate image to obtain the similarity corresponding to the candidate image; Comparing the aspect ratio of the image to be aligned with the aspect ratio of the candidate image, and determining the candidate image with the same aspect ratio as the image to be aligned as the reference image; Determine the reference image with the smallest similarity as the alignment image of the image to be aligned; Based on the layout block information corresponding to the layout block and the image information of the aligned image in the second format document, the image alignment information of the layout block is obtained.

7. A document alignment device, characterized in that: The device comprises: An acquisition module, used for acquiring a first format document and a second format document corresponding to the first format document; the first format document includes at least one layout block, and the second format document includes at least one reference text line; A generating module, configured to generate a dictionary tree corresponding to the second format document based on the reference text lines in the second format document; A determination module, configured to determine, for each layout block, an aligned text line corresponding to the layout block based on the layout block and the dictionary tree; The alignment module is used to obtain the text alignment information of the layout block based on the layout block information corresponding to the layout block and the text line information of the aligned text line.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images