Information processing method, apparatus, device, and ticket processing method

Abstract

The application discloses an information processing method, comprising: obtaining a target text containing layout information, wherein the layout information comprises one or more cells; obtaining vector feature information corresponding to coordinate information and vector feature information corresponding to text information in the cells according to the coordinate information and the text information; encoding the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information to obtain a vector representation of the encoded cells; determining a key vector representation and a value vector representation of the cells according to the vector representation of the cells; and determining a modification relationship between a first cell in the layout information and a second cell in the layout information according to the key vector representation and the value vector representation of the cells. The above method can solve the problem that the prior art cannot be migrated in different fields when extracting information from a text containing layout information.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an information processing method, apparatus, electronic device, and storage device. This application also relates to a bill processing method. Background Technology

[0002] In recent years, more and more works have explored how to extract information such as consumption items and amounts from text containing graph / table structures, for example, from value-added tax invoices. Previous works have mostly used sequence labeling to extract key information from graph / table structured text. This method relies on predefined label types, and its transferability is poor when transferring to different domains due to differences in labeling systems.

[0003] In current technologies, the use of biaffine mechanisms to calculate dependency arc fractions between words in a sentence has achieved significant improvements in dependency parsing tasks. However, dependency parsing tasks are based on plain text expansion and cannot be applied to texts containing graph / table structures.

[0004] How to extract information from text containing graph / table structures and better transfer it to different fields is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This application provides an information processing method, apparatus, electronic device, and storage device to solve the problem that existing methods for extracting information from text containing layout information cannot be transferred to different fields.

[0006] This application provides an information processing method, including:

[0007] Obtain target text containing layout information, wherein the layout information includes one or more cells;

[0008] Based on the coordinate information and text information in the cell, obtain the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information;

[0009] The vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information are encoded to obtain the vector representation of the encoded cell;

[0010] The key vector representation and value vector representation of the cell are determined based on the vector representation of the cell;

[0011] Based on the key vector representation and value vector representation of the cell, the modification relationship between the first cell and the second cell in the layout information is determined.

[0012] Optionally, the layout information includes a graph structure and / or a table structure.

[0013] Optionally, it also includes: obtaining the label vector representation of the cell;

[0014] Accordingly, determining the key vector representation and value vector representation of the cell based on the vector representation of the cell includes:

[0015] The key vector representation and value vector representation of the cell are determined based on the vector representation of the cell and the label representation of the cell.

[0016] Optionally, obtaining the label vector representation of the cell includes:

[0017] The relationship between the table header and the table body is marked for the cells, as well as the modification relationship of the key-value pairs corresponding to the cells;

[0018] Based on the pointing relationship information and the modification relationship, the label vector representation of the cell is obtained.

[0019] Optionally, determining the modification relationship between the first cell and the second cell in the layout information based on the key vector representation and value vector representation of the cell includes:

[0020] Calculate the first score between the first cell and the second cell based on the key vector representation and value vector representation of the cell;

[0021] Calculate the second score based on the coordinate information to determine the closeness of the relationship between the first cell and the second cell;

[0022] Based on the first score and the second score, the modification relationship between the first cell and the second cell is obtained.

[0023] Optionally, obtaining the modification relationship between the first cell and the second cell based on the first score and the second score includes:

[0024] Add the first score and the second score to obtain the modification relationship score.

[0025] Optionally, the step of calculating the second score of the relationship between the first cell and the second cell based on the coordinate information includes:

[0026] Calculate the difference in horizontal and vertical coordinate positions between the first and second cells based on the coordinate information;

[0027] Based on the difference in the horizontal coordinate position and the difference in the vertical coordinate position, a second score is determined to indicate the closeness of the relationship between the first cell and the second cell.

[0028] This application also provides an information processing apparatus, including:

[0029] A target text acquisition unit is used to acquire target text containing layout information, wherein the layout information includes one or more cells;

[0030] The vector feature information acquisition unit is used to obtain vector feature information corresponding to the coordinate information and vector feature information corresponding to the text information based on the coordinate information and text information in the cell.

[0031] The vector representation of the cell is used to encode the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information to obtain the vector representation of the encoded cell;

[0032] A key vector representation and value vector representation determination unit is used to determine the key vector representation and value vector representation of the cell based on the vector representation of the cell;

[0033] The modification relationship determination unit is used to determine the modification relationship between the first cell and the second cell in the layout information based on the key vector representation and value vector representation of the cell.

[0034] This application also provides an electronic device, including:

[0035] Processor; and

[0036] The memory stores a program for an information processing method. After the device is powered on and the program for the information processing method is run by the processor, the following steps are performed:

[0037] Obtain target text containing layout information, wherein the layout information includes one or more cells;

[0038] Based on the coordinate information and text information in the cell, obtain the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information;

[0039] The vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information are encoded to obtain the vector representation of the encoded cell;

[0040] The key vector representation and value vector representation of the cell are determined based on the vector representation of the cell;

[0041] Based on the key vector representation and value vector representation of the cell, the modification relationship between the first cell and the second cell in the layout information is determined.

[0042] This application also provides a storage device storing a program for an information processing method, which is executed by a processor to perform the following steps:

[0043] Obtain target text containing layout information, wherein the layout information includes one or more cells;

[0044] Based on the coordinate information and text information in the cell, obtain the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information;

[0045] The vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information are encoded to obtain the vector representation of the encoded cell;

[0046] The key vector representation and value vector representation of the cell are determined based on the vector representation of the cell;

[0047] Based on the key vector representation and value vector representation of the cell, the modification relationship between the first cell and the second cell in the layout information is determined.

[0048] This application also provides a bill processing method, including:

[0049] Obtain a ticket; the ticket comprises one or more cells;

[0050] Based on the coordinate information and text information in the cell, obtain the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information;

[0051] The vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information are encoded to obtain the vector representation of the encoded cell;

[0052] The key vector representation and value vector representation of the cell are determined based on the vector representation of the cell;

[0053] Based on the key vector representation and value vector representation of the cell, determine the modification relationship between the first cell and the second cell in the layout information;

[0054] Based on the aforementioned modification relationship, the amount of the bills is statistically analyzed to obtain the total amount of the bills.

[0055] Compared with the prior art, this application has the following advantages:

[0056] This application provides an information processing method, comprising: obtaining target text containing layout information, the layout information including one or more cells; obtaining vector feature information corresponding to the coordinate information and vector feature information corresponding to the text information based on the coordinate information and text information in the cells; encoding the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information to obtain a vector representation of the encoded cell; determining the key vector representation and value vector representation of the cell based on the vector representation of the cell; and determining the modification relationship between a first cell and a second cell in the layout information based on the key vector representation and value vector representation of the cell. Compared with sequence labeling, this application, when extracting information from target text containing a graph / table structure, does not rely on predefined label types, thus solving the technical problem that information cannot be transferred between different domains while extracting information from text containing a graph / table structure. Attached Figure Description

[0057] Figure 1A This is an application scenario diagram of an information processing method provided in the first embodiment of this application.

[0058] Figure 1 This is a flowchart of an information processing method provided in the first embodiment of this application.

[0059] Figure 2 This is a schematic diagram of the input and output of an information processing method provided in the first embodiment of this application.

[0060] Figure 3 This is a schematic diagram of an information processing device provided in the second embodiment of this application.

[0061] Figure 4 This is a schematic diagram of an electronic device provided in the third embodiment of this application. Detailed Implementation

[0062] Numerous specific details are set forth in the following description to provide a full understanding of the invention. However, the invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0063] To illustrate this application more clearly, we will first briefly introduce the application scenarios of the information processing method provided in the first embodiment of this application.

[0064] The information processing method provided in the first embodiment of this application can be applied to scenarios involving interaction between a client and a server, such as... Figure 1AWhen the total amount of a VAT invoice needs to be obtained, the client typically establishes a connection with the server first. After the connection is established, the client sends the target text of the VAT invoice to the server. The server obtains the target text of the VAT invoice in the target text acquisition unit 101. The layout information of the VAT invoice includes one or more cells. Then, in the vector feature information acquisition unit 102, the server obtains the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information based on the coordinate information and text information in the cells. Next, in the cell vector representation acquisition unit 103, the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information are... The vector feature information is encoded to obtain the vector representation of the encoded cell; then, in the key vector representation and value vector representation determination unit 104, the key vector representation and value vector representation of the cell are determined according to the vector representation of the cell; then, in the modification relationship obtaining unit 105, the modification relationship between the first cell and the second cell in the layout information is determined according to the key vector representation and value vector representation of the cell; finally, according to the modification relationship, the amount of the VAT invoice is counted to obtain the total amount of the VAT invoice, and the total amount of the VAT invoice is provided to the client, and the client receives the total amount of the VAT invoice.

[0065] The first embodiment of this application provides an information processing method, which will be described below in conjunction with... Figure 1 , Figure 2 Let me introduce it.

[0066] like Figure 1 As shown, in step S101, target text containing layout information is obtained, wherein the layout information includes one or more cells.

[0067] The layout information includes a graph structure and / or a table structure.

[0068] The target text refers to electronic text containing layout information. Examples include value-added tax invoices and customs documents. The target text can be a text image obtained by scanning the original text (e.g., the original value-added tax invoice) or a text image obtained by photographing the original text.

[0069] The information processing method can run on the server side. The target text can be obtained from the client. When a user needs to obtain the textual relationship between two cells in the target text, the target text can be uploaded to the server via the client, and the server receives the target text sent by the client. The information processing method can also run on the client side.

[0070] like Figure 1As shown, in step S102, vector feature information corresponding to the coordinate information and vector feature information corresponding to the text information are obtained based on the coordinate information and text information in the cell.

[0071] In practical implementation, the general OCR (Optical Character Recognition) technology can be used to obtain cell information containing coordinate information and text information from the target text. OCR technology refers to the technology of locating text coordinate information and recognizing text information from target text.

[0072] The cell refers to a rectangle containing text information.

[0073] The coordinate information of the cell refers to the coordinate information of the four vertices of the rectangle in the target text.

[0074] For example, if the text information of a cell is "Beijing XXX", the coordinates of the four vertices of the rectangle containing the text content are: (86,162), (337,162), (337,182), (86,182).

[0075] From a target text, multiple cells containing coordinate and text information can be obtained. For example, from a value-added tax invoice, 20 cells can be obtained, represented as cell 0, cell 1, up to cell 20, with each cell containing coordinate and text information.

[0076] Based on the coordinate information and text information in the cell, the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information are obtained, which provides the prerequisite for encoding the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information to obtain the vector representation of the encoded cell.

[0077] The step of obtaining vector feature information corresponding to the coordinate information and vector feature information corresponding to the text information based on the coordinate information and text information in the cell can refer to mapping the coordinate information in the cell information to vector feature information corresponding to the coordinate information (position), and mapping the text information in the cell information to vector feature information corresponding to the text information (word).

[0078] In practical implementation, when mapping text information in cell information to vector feature information of text information (word), sentence embedding can be used to transform the text information in cell information into vector feature information corresponding to the text information. Sentence embedding is a sentence vector technique, which is a method to represent a piece of text information as vector feature information.

[0079] For example, Figure 2 In the diagram, 2-1 represents the vector feature information corresponding to the text information (word), and 2-2 represents the vector feature information corresponding to the coordinate information.

[0080] like Figure 1 As shown, in step S103, the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information are encoded to obtain the vector representation of the encoded cell.

[0081] In practice, the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information can be input into the pre-trained model for encoding to obtain the vector representation of the encoded cell.

[0082] The pre-trained model can include layout information, such as the LayoutLM pre-trained model. LayoutLM is a Microsoft general-purpose document pre-trained model that performs well in form understanding and invoice understanding. The pre-trained model can also be one that does not include style information, such as the BERT (Bidirectional Encoder Representation from Transformers) pre-trained model, whose main innovation lies in its pre-training method.

[0083] like Figure 2 As shown, the vector feature information corresponding to the text information and the vector feature information corresponding to the coordinate information in the cell are input into LayoutLM for encoding processing to obtain the vector representation of the encoded cell.

[0084] like Figure 1 As shown, in step S104, the key vector representation and value vector representation of the cell are determined based on the vector representation of the cell.

[0085] After obtaining the vector representation of the encoded cell in step S103, the key vector representation and value vector representation of the cell are determined based on the vector representation of the cell.

[0086] In practice, the vector representation of a cell can be input into the encoding layer, and the key vector representation and value vector representation of the cell can be output.

[0087] The key refers to the key in a key-value pair, which is a keyword. Keywords are usually located at the beginning of a row or column in a table in the target text.

[0088] The value refers to the value in the key-value pair. The value is the specific value, which is usually located in the body of the table in the target text.

[0089] like Figure 2 The vector representation of a cell is input into the encoding layer (GCN / Transformer / LSTM), and the output is the key vector representation and value vector representation of the cell, where MLP... head MLP is a key vector representation. dep This is represented by a value vector. Among them, GCN (Graph Convolutional Networks) is a graph structure model. Transformer is a classic NLP model. LSTM (Long Short-Term Memory) is a time-recurrent neural network specifically designed to address the long-term dependency problem inherent in general RNNs (Recurrent Neural Networks).

[0090] The first embodiment of this application may further include: obtaining the label vector representation of the cell.

[0091] Accordingly, determining the key vector representation and value vector representation of the cell based on the vector representation of the cell includes:

[0092] The key vector representation and value vector representation of the cell are determined based on the vector representation of the cell and the label representation of the cell.

[0093] In practice, the cell vector and the cell label vector can be concatenated together and input into the encoding layer, and the cell key vector and value vector representation can be output.

[0094] The obtained label vector representation of the cell includes:

[0095] The relationship between the table header and the table body is marked for the cells, as well as the modification relationship of the key-value pairs corresponding to the cells;

[0096] Based on the pointing relationship information and the modification relationship, the label vector representation of the cell is obtained.

[0097] The relationship information can include question-answer relationships.

[0098] For example, the cell that registers the number is the question cell, and "333" is the answer cell corresponding to the cell that registers the number.

[0099] like Figure 2 2-3 are the label vector representations.

[0100] By concatenating the vector representation of a cell with its label vector representation and inputting it into the encoding layer, the output is the key vector representation and value vector representation of the cell. Compared with only inputting the vector representation of the cell, the introduction of the cell label vector can eliminate some pairs to a certain extent, making the cell dependencies obtained by the model more accurate. This is because two answer cells are unlikely to form a pair, two question cells are unlikely to form a pair, while question-answer cells are more likely to form a pair.

[0101] like Figure 1 As shown, in step S105, the modification relationship between the first cell and the second cell in the layout information is determined based on the key vector representation and value vector representation of the cell.

[0102] Specifically, the key vector representation and value vector of the cell can be input into a dual affine mechanism classifier for processing to obtain the modification relationship between the first cell in the layout information and the second cell in the layout information.

[0103] The modification relationship between the first cell and the second cell in the layout information refers to the relationship score of the key-value pair between the first cell and the second cell.

[0104] Determining the modification relationship between the first cell and the second cell in the layout information based on the key vector representation and value vector representation of the cell includes:

[0105] Calculate the first score between the first cell and the second cell based on the key vector representation and value vector representation of the cell;

[0106] Calculate the second score based on the coordinate information to determine the closeness of the relationship between the first cell and the second cell;

[0107] Based on the first score and the second score, the modification relationship between the first cell and the second cell is obtained.

[0108] The step of obtaining the modification relationship between the first cell and the second cell based on the first score and the second score includes:

[0109] Add the first score and the second score to obtain the modification relationship score.

[0110] In practice, weighting coefficients can be set for the first and second scores. Based on these weighting coefficients and the first and second scores, the modification relationship between the first and second cells can be determined. For example, if the weighting coefficient for the first score is set to w1, and the weighting coefficient for the second score is set to w2, with the first score being score1 and the second score being score2, then the modification relationship score between the first and second cells is:

[0111] w1*score1+w2*score2.

[0112] The second score for calculating the closeness of the relationship between the first cell and the second cell based on coordinate information includes:

[0113] Calculate the difference in horizontal and vertical coordinate positions between the first and second cells based on the coordinate information;

[0114] Based on the difference in the horizontal coordinate position and the difference in the vertical coordinate position, a second score is determined to indicate the closeness of the relationship between the first cell and the second cell.

[0115] Specifically, to calculate the second score of the relationship between the first cell and the second cell based on the coordinate information, the key vector representation and value vector of the cell can be input into the feature scoring model for processing to obtain the second score of the relationship between the first cell and the second cell.

[0116] For example, Figure 2 The key vector representation and value vector of the cell are input into the Biaffine classifier and the Feature Score model for processing to obtain the first score and the second score. The two scores are then added together to obtain the modification relationship score between the first cell and the second cell.

[0117] By adding the first score to the second score, the modification relationship score between the first cell and the second cell is obtained. Since the closeness of the relationship between the coordinates of two cells is related to the difference in position of the cells on the horizontal or vertical axis, the model tends to think that cells that are close in interval and on the same horizontal or vertical axis are more likely to have a modification relationship. Therefore, the modification relationship score between the coordinates of two cells output by the model is more accurate.

[0118] This concludes the introduction of the first embodiment of this application. The first embodiment applies a dependency parsing model to the task of extracting information from target text with a graph / table structure. Compared to sequence labeling, it does not rely on predefined label types, solving the technical problem of not being able to transfer information extracted from text containing graph / table structures across different domains. In a preferred embodiment, the key vector representation and value vector representation of a cell are determined based on the cell's vector representation and label representation. Compared to determining the key vector representation and value vector representation of a cell based solely on its vector representation, the introduction of cell label vectors makes the obtained dependencies between cells more accurate. Furthermore, when obtaining the modification relationship between two cells, the introduction of a second score calculating the closeness of the relationship between the first and second cells based on coordinate information also makes the score of the modification relationship between the two cells output by the model more accurate.

[0119] Corresponding to the information processing method provided in the first embodiment of this application, the second embodiment of this application also provides an information processing device.

[0120] like Figure 3 As shown, the information processing device includes:

[0121] The target text acquisition unit 301 is used to acquire target text containing layout information, wherein the layout information includes one or more cells;

[0122] The vector feature information acquisition unit 302 is used to obtain vector feature information corresponding to the coordinate information and vector feature information corresponding to the text information based on the coordinate information and text information in the cell.

[0123] The vector representation of the cell is obtained by unit 303, which is used to encode the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information to obtain the vector representation of the encoded cell;

[0124] The key vector representation and value vector representation determination unit 304 is used to determine the key vector representation and value vector representation of the cell based on the vector representation of the cell;

[0125] The modification relationship determination unit 305 is used to determine the modification relationship between the first cell in the layout information and the second cell in the layout information based on the key vector representation and value vector representation of the cell.

[0126] As one implementation, the layout information includes a graph structure and / or a table structure.

[0127] In one embodiment, the apparatus further includes: a label vector representation obtaining unit, used to obtain the label vector representation of a cell;

[0128] Accordingly, the key vector representation and value vector representation determination unit is specifically used for:

[0129] The key vector representation and value vector representation of the cell are determined based on the vector representation of the cell and the label representation of the cell.

[0130] In one implementation, the label vector representation acquisition unit is specifically used for:

[0131] The relationship between the table header and the table body is marked for the cells, as well as the modification relationship of the key-value pairs corresponding to the cells;

[0132] Based on the pointing relationship information and the modification relationship, the label vector representation of the cell is obtained.

[0133] In one implementation, the modification relationship determination unit is specifically used for:

[0134] Calculate the first score between the first cell and the second cell based on the key vector representation and value vector representation of the cell;

[0135] Calculate the second score based on the coordinate information to determine the closeness of the relationship between the first cell and the second cell;

[0136] Based on the first score and the second score, the modification relationship between the first cell and the second cell is obtained.

[0137] In one implementation, the modification relationship determination unit is specifically used for:

[0138] Add the first score and the second score to obtain the modification relationship score.

[0139] In one implementation, the modification relationship determination unit is specifically used for:

[0140] Calculate the difference in horizontal and vertical coordinate positions between the first and second cells based on the coordinate information;

[0141] Based on the difference in the horizontal coordinate position and the difference in the vertical coordinate position, a second score is determined to indicate the closeness of the relationship between the first cell and the second cell.

[0142] It should be noted that for a detailed description of the apparatus provided in the second embodiment of this application, please refer to the relevant description of the first embodiment of this application, which will not be repeated here.

[0143] Corresponding to the information processing method provided in the first embodiment of this application, the third embodiment of this application also provides an electronic device.

[0144] like Figure 4 As shown, the electronic device includes:

[0145] Processor 401; and

[0146] The memory 402 stores a program for the information processing method. After the device is powered on and the program for the information processing method is run by the processor, the following steps are performed:

[0147] Obtain target text containing layout information, wherein the layout information includes one or more cells;

[0148] Based on the coordinate information and text information in the cell, obtain the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information;

[0149] The vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information are encoded to obtain the vector representation of the encoded cell;

[0150] The key vector representation and value vector representation of the cell are determined based on the vector representation of the cell;

[0151] Based on the key vector representation and value vector representation of the cell, the modification relationship between the first cell and the second cell in the layout information is determined.

[0152] As one implementation, the layout information includes a graph structure and / or a table structure.

[0153] In one implementation, the electronic device further performs the following steps: obtaining the label vector representation of the cell;

[0154] Accordingly, determining the key vector representation and value vector representation of the cell based on the vector representation of the cell includes:

[0155] The key vector representation and value vector representation of the cell are determined based on the vector representation of the cell and the label representation of the cell.

[0156] As one implementation method, obtaining the label vector representation of the cell includes:

[0157] The relationship between the table header and the table body is marked for the cells, as well as the modification relationship of the key-value pairs corresponding to the cells;

[0158] Based on the pointing relationship information and the modification relationship, the label vector representation of the cell is obtained.

[0159] As one implementation, determining the modification relationship between the first cell and the second cell in the layout information based on the key vector representation and value vector representation of the cell includes:

[0160] Calculate the first score between the first cell and the second cell based on the key vector representation and value vector representation of the cell;

[0161] Calculate the second score based on the coordinate information to determine the closeness of the relationship between the first cell and the second cell;

[0162] Based on the first score and the second score, the modification relationship between the first cell and the second cell is obtained.

[0163] As one implementation method, obtaining the modification relationship between the first cell and the second cell based on the first score and the second score includes:

[0164] Add the first score and the second score to obtain the modification relationship score.

[0165] As one implementation method, the step of calculating the second score of the relationship between the first cell and the second cell based on coordinate information includes:

[0166] Calculate the difference in horizontal and vertical coordinate positions between the first and second cells based on the coordinate information;

[0167] Based on the difference in the horizontal coordinate position and the difference in the vertical coordinate position, a second score is determined to indicate the closeness of the relationship between the first cell and the second cell.

[0168] It should be noted that for a detailed description of the electronic device provided in the third embodiment of this application, please refer to the relevant description of the first embodiment of this application, which will not be repeated here.

[0169] Corresponding to the information processing method provided in the first embodiment of this application, the fourth embodiment of this application also provides a storage device storing a program for the information processing method, which is executed by a processor to perform the following steps:

[0170] Obtain target text containing layout information, wherein the layout information includes one or more cells;

[0171] Based on the coordinate information and text information in the cell, obtain the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information;

[0172] The vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information are encoded to obtain the vector representation of the encoded cell;

[0173] The key vector representation and value vector representation of the cell are determined based on the vector representation of the cell;

[0174] Based on the key vector representation and value vector representation of the cell, the modification relationship between the first cell and the second cell in the layout information is determined.

[0175] It should be noted that for a detailed description of the storage device provided in the fourth embodiment of this application, please refer to the relevant description of the first embodiment of this application, which will not be repeated here.

[0176] The fifth embodiment of this application provides a bill processing method, including:

[0177] Obtain the target text of the ticket; the layout information of the ticket includes one or more cells;

[0178] Based on the coordinate information and text information in the cell, obtain the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information;

[0179] The vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information are encoded to obtain the vector representation of the encoded cell;

[0180] The key vector representation and value vector representation of the cell are determined based on the vector representation of the cell;

[0181] Based on the key vector representation and value vector representation of the cell, determine the modification relationship between the first cell and the second cell in the layout information;

[0182] Based on the aforementioned modification relationship, the amount of the bills is statistically analyzed to obtain the total amount of the bills.

[0183] The documents include: value-added tax invoices, general invoices, customs declarations, etc.

[0184] It should be noted that for a detailed description of the bill processing method provided in the fifth embodiment of this application, please refer to the relevant description of the first embodiment of this application, which will not be repeated here.

[0185] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

[0186] In a typical configuration, a computing device includes one or more processors (CPUs), memory-mapped input / output interfaces, network interfaces, and memory.

[0187] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0188] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0189] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. An information processing method characterized by comprising: The method comprises: obtaining target text containing layout information, the layout information comprising one or more cells; obtaining vector feature information corresponding to the coordinate information and vector feature information corresponding to the text information according to the coordinate information and the text information in the cell; encoding the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information by using a pre-trained model containing the layout information to obtain a vector representation of the encoded cell; determining a key vector representation and a value vector representation of the cell according to the vector representation of the cell; determining a modification relationship between a first cell in the layout information and a second cell in the layout information according to the key vector representation and the value vector representation of the cell and the coordinate information, wherein the coordinate information is used to calculate a relationship closeness between the first cell and the second cell, the position of the first cell is different from the position of the second cell, the modification relationship is obtained by weighting a first score and a second score, the first score is determined by the key vector representation and the value vector representation of the cell, and the second score is determined by the coordinate information.

2. The method of claim 1, wherein, The layout information comprises a graph structure and / or a table structure.

3. The method of claim 1, wherein, The method further comprises: obtaining a label vector representation of the cell; correspondingly, the determining of the key vector representation and the value vector representation of the cell according to the vector representation of the cell comprises: determining the key vector representation and the value vector representation of the cell according to the vector representation of the cell and the label representation of the cell.

4. The method of claim 3, wherein, The obtaining of the label vector representation of the cell comprises: annotating a pointing relationship between a table header and a table body and a modification relationship of a key-value pair corresponding to the cell for the cell; obtaining the label vector representation of the cell according to the pointing relationship information and the modification relationship.

5. The method of claim 1, wherein, The method further comprises: adding the first score and the second score to obtain a modification relationship score.

6. The method of claim 1, wherein, The method further comprises: calculating a horizontal coordinate position difference and a vertical coordinate position difference between the first cell and the second cell according to the coordinate information; determining the second score of the relationship closeness according to the horizontal coordinate position difference and the vertical coordinate position difference.

7. An information processing apparatus, characterized by comprising: The method comprises: a target text obtaining unit configured to obtain target text containing layout information, the layout information comprising one or more cells; a vector feature information obtaining unit configured to obtain vector feature information corresponding to the coordinate information and vector feature information corresponding to the text information according to the coordinate information and the text information in the cell; a cell vector representation obtaining unit configured to encode the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information by using a pre-trained model containing the layout information to obtain a vector representation of the encoded cell; a key vector representation and value vector representation determining unit configured to determine a key vector representation and a value vector representation of the cell according to the vector representation of the cell; The modification relationship determining unit is configured to determine a modification relationship between a first cell in the layout information and a second cell in the layout information according to the key vector representation and the value vector representation of the cells and the coordinate information, wherein the coordinate information is used to calculate a relationship closeness between the first cell and the second cell, the first cell is different from the second cell in position, the modification relationship is weighted by a first score and a second score, the first score is determined by the key vector representation and the value vector representation of the cells, and the second score is determined by the coordinate information.

8. An electronic device, comprising: The information processing method comprises the following steps: a processor; and a memory configured to store a program of an information processing method, and the device is powered on and executes the program of the information processing method by the processor to perform the following steps: obtain a target text containing layout information, wherein the layout information comprises one or more cells; obtain vector feature information corresponding to the coordinate information and vector feature information corresponding to the text information according to the coordinate information and the text information in the cells; encode the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information by using a pre-trained model containing the layout information to obtain vector representations of the encoded cells; determine key vector representations and value vector representations of the cells according to the vector representations of the cells; determine a modification relationship between a first cell in the layout information and a second cell in the layout information according to the key vector representation and the value vector representation of the cells and the coordinate information, wherein the coordinate information is used to calculate a relationship closeness between the first cell and the second cell, the first cell is different from the second cell in position, the modification relationship is weighted by a first score and a second score, the first score is determined by the key vector representation and the value vector representation of the cells, and the second score is determined by the coordinate information.

9. A storage device, comprising: The information processing method comprises the following steps: a processor; and a memory configured to store a program of an information processing method, and the device is powered on and executes the program of the information processing method by the processor to perform the following steps: obtain a target text containing layout information, wherein the layout information comprises one or more cells; obtain vector feature information corresponding to the coordinate information and vector feature information corresponding to the text information according to the coordinate information and the text information in the cells; encode the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information by using a pre-trained model containing the layout information to obtain vector representations of the encoded cells; determine key vector representations and value vector representations of the cells according to the vector representations of the cells; determine a modification relationship between a first cell in the layout information and a second cell in the layout information according to the key vector representation and the value vector representation of the cells and the coordinate information, wherein the coordinate information is used to calculate a relationship closeness between the first cell and the second cell, the first cell is different from the second cell in position, the modification relationship is weighted by a first score and a second score, the first score is determined by the key vector representation and the value vector representation of the cells, and the second score is determined by the coordinate information. According to the key vector representation and the value vector representation of the cell, and the coordinate information, a modification relationship between a first cell in the layout information and a second cell in the layout information is determined, wherein the coordinate information is used to calculate a relationship closeness between the first cell and the second cell, the position of the first cell is different from the position of the second cell, the modification relationship is obtained by weighting a first score and a second score, the first score is determined by the key vector representation and the value vector representation of the cell, and the second score is determined by the coordinate information.

10. A method of processing a document, characterized by, Comprise: Obtaining the target text of the bill; the layout information of the bill comprises one or more cells; According to the coordinate information and the text information in the cell, vector feature information corresponding to the coordinate information and vector feature information corresponding to the text information are obtained; Using a pre-trained model containing the layout information to encode the vector feature information corresponding to the coordinate information and the vector feature information corresponding to the text information, to obtain the vector representation of the encoded cell; According to the vector representation of the cell, the key vector representation and the value vector representation of the cell are determined; According to the key vector representation and the value vector representation of the cell, and the coordinate information, a modification relationship between a first cell in the layout information and a second cell in the layout information is determined, wherein the coordinate information is used to calculate a relationship closeness between the first cell and the second cell, the position of the first cell is different from the position of the second cell, the modification relationship is obtained by weighting a first score and a second score, the first score is determined by the key vector representation and the value vector representation of the cell, and the second score is determined by the coordinate information; According to the modification relationship, the total amount of the bill is obtained by counting the amount of the bill.

Images