Invoice information extraction method and device, electronic equipment and storage medium

Abstract

Embodiments of the invention disclose an invoice information extraction method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a target invoice and a target to-be-extracted information category; wherein the target to-be-extracted information category is an information category of invoice information needing to be acquired in the target invoice; performing character recognition on the target invoice to obtain m candidate words; m is a positive integer; determining a feature matrix of each candidate word in the m candidate words to obtain m feature matrixes; performing named entity recognition and word classification on the m candidate words based on the m feature matrixes to obtain m target tags of the m candidate words; each target label corresponds to the information category of one invoice information; and determining invoice information based on the m target tags and the target to-be-extracted information category. According to the embodiment of the invention, when the invoice information is extracted, the invoice information extraction cost can be reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of information extraction, and in particular to an invoice information extraction method, device, electronic device and storage medium. Background Art

[0002] Administrative documents include invoices, forms, payrolls, etc., which are very common in daily life and are required in many administrative procedures. Among them, invoices have attracted much attention because they are related to financial activities. A large number of invoices are received every day and usually need to be paid as soon as possible. Any mistakes are unacceptable.

[0003] Currently, most of these procedures have been digitized and automated, and information extraction systems extract key fields from documents, such as identifiers, types, amounts, dates, etc. However, current information extraction methods have many problems. For example, early sequence labeling methods were rule-based, with rules defined manually, relying on trigger words, regular expressions, and language descriptors. However, designing such models is time-consuming and language- and template-dependent, requiring manual annotation of a large number of documents, and is very cumbersome when integrating new languages ​​and templates. Deep learning methods require a large amount of data for training, especially for the extraction of information from semi-structured documents such as invoices. There are limitations, such as processing information in tables, dealing with different templates and languages, which results in high costs for invoice information extraction.

[0004] Therefore, there is an urgent need for an invoice information extraction method to reduce the cost of extracting invoice information when extracting invoice information. Summary of the invention

[0005] In order to solve the above problems, the embodiments of the present invention provide an invoice information extraction method, device, electronic device and storage medium, which can reduce the cost of extracting invoice information when extracting invoice information.

[0006] In a first aspect, an embodiment of the present invention provides a method for extracting invoice information, comprising:

[0007] Obtaining a target invoice and a target information category to be extracted; the target information category to be extracted is the information category of the invoice information to be obtained in the target invoice;

[0008] Perform character recognition on the target invoice to obtain m candidate words, where m is a positive integer;

[0009] Determine a feature matrix of each of the m candidate words to obtain m feature matrices;

[0010] Based on the m feature matrices, named entity recognition and word classification are performed on the m candidate words to obtain m target labels of the m candidate words; each target label corresponds to an information category of invoice information;

[0011] The invoice information is determined based on the m target tags and the target information category to be extracted.

[0012] In a second aspect, an embodiment of the present invention provides an invoice information extraction device, including an acquisition unit and a processing unit;

[0013] The acquisition unit is used to acquire a target invoice and a target information category to be extracted; the target information category to be extracted is the information category of the invoice information to be acquired in the target invoice;

[0014] The processing unit is used to perform character recognition on the target invoice to obtain m candidate words; m is a positive integer;

[0015] Determine a feature matrix of each of the m candidate words to obtain m feature matrices;

[0016] Based on the m feature matrices, named entity recognition and word classification are performed on the m candidate words to obtain m target labels of the m candidate words; each target label corresponds to an information category of invoice information;

[0017] The invoice information is determined based on the m target tags and the target information category to be extracted.

[0018] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor and a memory, wherein the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the method described in the first aspect.

[0019] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method as described in the first aspect.

[0020] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer is operable to cause the computer to execute the method described in the first aspect.

[0021] Implementing the embodiments of the present application has the following beneficial effects:

[0022] In the implementation mode of the present application, the target invoice and the target information category to be extracted are first obtained, wherein the target information category to be extracted is the information category of the invoice information to be obtained in the target invoice, then, character recognition is performed on the target invoice to obtain m candidate words, wherein m is a positive integer, and then, the feature matrix of each candidate word in the m candidate words is determined to obtain m feature matrices, and then, named entity recognition and word classification are performed on the m candidate words based on the m feature matrices to obtain m target labels of the m candidate words, each target label corresponds to an information category of the invoice information, and finally, the invoice information is determined based on the m target labels and the target information category to be extracted. Thus, by performing named entity recognition and word classification on the m candidate words, m target labels of the m candidate words are obtained, and then the invoice information is determined according to the m target labels and the target information category to be extracted, when the invoice information is extracted, the cost of extracting the invoice information can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the background technology, the drawings required for use in the embodiments of the present invention or the background technology will be described below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 It is a flow chart of a method for extracting invoice information provided in an embodiment of the present application;

[0025] Figure 2 It is a structural schematic diagram of an invoice information extraction device provided in an embodiment of the present application;

[0026] Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other steps or modules inherent to these processes, methods, products or devices.

[0029] Reference to "embodiments" herein means that a particular feature, result, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0030] See also Figure 1 , Figure 1 : is a flow chart of a method for extracting invoice information provided by an embodiment of the present application. Figure 1 As shown, the invoice information extraction method provided by the embodiment of the present invention includes but is not limited to the following steps:

[0031] Step S101: Obtain the target invoice and the target information category to be extracted;

[0032] The target information category to be extracted is the information category of the invoice information that needs to be obtained in the target invoice;

[0033] Step S102: Perform character recognition on the target invoice to obtain m candidate words;

[0034] Wherein, m is a positive integer;

[0035] Step S103: determining a feature matrix of each of the m candidate words to obtain m feature matrices;

[0036] Step S104: performing named entity recognition and word classification on the m candidate words based on the m feature matrices to obtain m target labels for the m candidate words;

[0037] Wherein, each target tag corresponds to an information category of invoice information;

[0038] Step S105: Determine the invoice information based on the m target tags and the target information category to be extracted.

[0039] In a possible embodiment, a target invoice and a target information category to be extracted are obtained, wherein the target information category to be extracted is the information category of the invoice information that needs to be obtained in the target invoice, the target invoice can be a paper invoice or an electronic invoice, and the target information category to be extracted can be the invoice number, invoice date, purchaser's name, seller's name, amount, goods or service details, etc.

[0040] In a possible embodiment, character recognition is performed on the target invoice to obtain m candidate words, where m is a positive integer. When character recognition is performed on the target invoice, an optical character recognition (OCR) method is used to obtain the characters of the target invoice through a scanner, determine their shapes by detecting dark and light patterns, and then use a character recognition method to translate the shapes into computer text. For printed characters, an optical method is used to convert the text in the target invoice paper document into a black and white dot matrix image file, and the text in the image is converted into a text format through recognition software, thereby obtaining m candidate words.

[0041] In a possible embodiment, a feature matrix of each of the m candidate words is determined to obtain m feature matrices, wherein the feature matrix of each candidate word is used to reflect the text feature, layout feature, pattern feature and logical feature of the candidate word, wherein the text feature refers to all candidate words that can define the context of the candidate word in the target invoice, including the candidate words closest to the left, right, top and bottom, and the top and bottom lines; the layout feature refers to the position of the candidate word in the document, block or line and its coordinates, including page descriptors such as page width, height and page margins; the pattern feature refers to the normalized pattern representation of the candidate word, for example, uppercase letters are represented by C, lowercase letters are represented by c, numbers are represented by d, and email addresses and Uniform Resource Locators (URLs) also correspond to normalized forms; the logical feature refers to whether the candidate word is a title or a result of a mathematical operation through a Boolean value; if the font of the candidate word is larger than that of other candidate words, then the candidate word may be a title, and whether the candidate word is a result of a mathematical operation can be determined through a vertically or horizontally aligned numeric triple.

[0042] In a possible embodiment, named entity recognition and word classification are performed on m candidate words based on m feature matrices to obtain m target labels of the m candidate words, wherein each target label corresponds to an information category of invoice information. For a target candidate word among the m candidate words, the target candidate word is any candidate word among the m candidate words. When named entity recognition is performed on the target candidate word, the named entity in the target candidate word is identified, and the target candidate word is classified into predefined categories, such as names of people, places, names of organizations, dates, times, currencies, etc. When word classification is performed on the target candidate word, it is performed according to a preset classification system, and the classification system can be based on multiple factors such as semantics, part of speech, and theme. By performing named entity recognition on m candidate words, the related information of named entities can be accurately identified based on pre-defined entity categories. However, the information extraction scope of named entity recognition is relatively narrow, and it mainly focuses on identifying specific entity names and their categories. By combining the word classification method, the understanding and information extraction capabilities of the target invoice can be improved. The information extraction scope of the word classification method is wider, and all words of the target invoice can be classified. It is not limited to named entities. In addition to entity names, it can also process common vocabulary. By analyzing the combination of words of different categories, various information such as themes and semantic roles can be determined.

[0043] In a possible embodiment, named entity recognition can be performed on m candidate words based on m feature matrices to obtain m target labels for the m candidate words, or word classification can be performed on the m candidate words based on the m feature matrices to obtain m target labels for the m candidate words.

[0044] In a possible embodiment, the invoice information is determined based on m target tags and the target information category to be extracted. When the target tag is consistent with the target information category to be extracted, the candidate word corresponding to the target tag is used as the invoice information corresponding to the target information category to be extracted.

[0045] In the embodiments of the present application, compared with the early rule-based methods, the two deep learning methods of the present invention reduce the reliance on manually designed rules, can automatically learn features, are more adaptable to different languages ​​and templates, and improve processing efficiency and flexibility. For example, the word classification-based method does not need to build resources like traditional methods, such as town dictionaries, for pre-marking, avoiding cumbersome manual operations. Compared with other deep learning methods, the present invention can achieve better results with a small amount of data. For example, cloud security and compliance monitoring tools require more than 300k invoices for training, while the performance of the present invention is very impressive with only 20k training samples, and the performance is further improved at 100k samples, greatly reducing the cost of data construction, while achieving comparable or even better precision and recall. The system of the present invention can effectively handle the special organizational forms of information in invoices, such as challenges brought by label-value pairs, different reading directions, etc., can better capture contextual data, and improve the accuracy of information extraction. For example, by reasonably selecting features and improving the model architecture, it has stronger extraction capabilities for key fields in different locations in the document, including tables.

[0046] Optionally, step S102, performing character recognition on the target invoice to obtain m candidate words, may include the following steps:

[0047] Step S201: Determine the reading direction and information structure of the target invoice;

[0048] Step S202: determining a recognition order based on a reading direction and information structure;

[0049] Step S203: performing character recognition on the target invoice based on the recognition order to obtain n characters; n is an integer greater than or equal to m;

[0050] Step S204: determining the initial position of each character in the n characters to obtain n initial positions;

[0051] Step S205: determining the centroid coordinates of each character in the n characters according to the n initial positions, and obtaining n centroid coordinates;

[0052] Step S206: Arrange the n characters based on the n centroid coordinates to obtain text information;

[0053] Step S207: Screen the text information according to the target information category to be extracted to obtain m candidate words.

[0054] In a possible embodiment, the reading direction and information structure of the target invoice are determined. The reading direction is determined according to the layout and text features of the target invoice. If the target invoice has horizontal or vertical dividing lines and borders, and the text is mostly arranged neatly along these lines, then the direction of the lines is the reading direction. The information structure is determined according to the regional division and hierarchical relationship of the target invoice. According to the common sections of the invoice, the target invoice is divided into large areas such as the header area, the text area, and the signature area. Among them, the header area usually contains the invoice name, code, number, invoice date, and basic information of the buyer and seller. The text area includes a detailed description of the goods or services, involving key data such as name, specification, quantity, unit price, amount, etc. The signature area includes the total amount, tax amount, price and tax total, and signatures of the issuer, payee, and reviewer.

[0055] In a possible embodiment, the recognition order is determined based on the reading direction and the information structure, and the target invoice is subjected to character recognition based on the recognition order to obtain n characters, where n is an integer greater than or equal to m. The initial position of each character in the n characters is determined to obtain n initial positions. When performing character recognition, the position information of each character in the original pixel coordinate system of the invoice image is synchronously obtained. If the invoice has multiple pages or is partially enlarged and scanned, it is necessary to unify the coordinate systems of different pages and different zoom ratios. Relative scale positioning or absolute coordinate conversion methods based on key reference points of the page, such as page corners and fixed pattern centers, can be used to achieve a consistent description of the character positions of the entire invoice.

[0056] In a possible embodiment, the centroid coordinates of each of the n characters are determined according to the n initial positions to obtain n centroid coordinates, the n characters are arranged based on the n centroid coordinates to obtain text information, and the text information is screened according to the target information category to be extracted to obtain m candidate words. The text extracted by optical character recognition is processed, only the words are retained and the lines are reconstructed according to the centroid coordinates of the words, and a sequence containing the entire document text is defined to ensure that the context and the field to be extracted are in the same sequence. The method for calculating the centroid of a word is based on the assumption that the character graphics are evenly distributed, and the character is regarded as a rectangular area with uniform mass to determine the center of gravity position. If the character has an irregular shape, such as deformation of artistic fonts, handwriting and connected strokes, resulting in a large deviation between the circumscribed rectangle and the actual character center of gravity, a centroid algorithm based on pixel grayscale distribution is used to traverse the character pixels, and the centroid is weighted according to the position and grayscale value of each pixel. The text pixel is the foreground color and has a higher grayscale value. For invoice pages that may be tilted or distorted, such as when the paper is not laid flat during scanning, the calculated centroid coordinates are corrected. The tilt detection and correction algorithm in the image geometric transformation is used to first detect the overall tilt angle of the invoice, and then reversely rotate all centroid coordinates to ensure that the centroid position reflects the true relative position relationship of the characters. Then, the centroid coordinates are standardized and converted to a specific range to facilitate comparisons between different invoice sizes and subsequent text arrangement operations based on relative positions, eliminating the impact of absolute coordinate scale differences.

[0057] Optionally, step S103, determining a feature matrix of each of the m candidate words to obtain m feature matrices, may include the following steps:

[0058] Step S301: determining an associated word set of each candidate word among m candidate words, and obtaining m associated word sets; each associated word set includes at least one associated word;

[0059] Step S302: constructing a text feature matrix of each of the m candidate words based on the m associated word sets to obtain m text feature matrices;

[0060] Step S303: determining the page descriptor of the target invoice, and determining m position coordinates corresponding to the m candidate words based on the n centroid coordinates;

[0061] Step S304: determining a layout feature matrix of each of the m candidate words based on the page descriptor and the m position coordinates, to obtain m layout feature matrices;

[0062] Step S305: determining a pattern feature matrix of each of the m candidate words based on a preset pattern standardization method, and obtaining m pattern feature matrices;

[0063] Step S306: determining m logical feature matrices according to the fonts and numeric tuples of the m candidate words;

[0064] Step S307: Integrate m text feature matrices, m layout feature matrices, m pattern feature matrices and m logic feature matrices to obtain m feature matrices.

[0065] In a possible embodiment, the associated word set is words associated with the candidate word, including all candidate words that can define the context of the candidate word in the target invoice, including the candidate words closest to the left and right, top and bottom, and up and down, and the associated word set of each candidate word in the m candidate words is determined to obtain m associated word sets, each associated word set includes at least one associated word, and a text feature matrix of each candidate word in the m candidate words is constructed based on the m associated word sets to obtain m text feature matrices.

[0066] In a possible embodiment, the page descriptor of the target invoice is determined, and the m position coordinates corresponding to the m candidate words are determined based on the n centroid coordinates. The layout feature matrix of each candidate word in the m candidate words is determined based on the page descriptor and the m position coordinates to obtain m layout feature matrices. The m position coordinates corresponding to the m candidate words obtained based on the invoice page pixel coordinate system are normalized, and the length and width of the invoice page are scaled to the [0, 1] interval respectively. The coordinates of each candidate word are also converted to normalized relative coordinates accordingly. The relative position ratio of each candidate word relative to the key area of ​​the page, such as the title bar, the table area, etc., is calculated and determined as auxiliary coordinate information to enrich the position feature expression. When constructing the layout feature matrix according to the position of the candidate words, first, the distance between adjacent candidate words is calculated, including the horizontal distance and the vertical distance. These distance values ​​are used as matrix elements to reflect the density of vocabulary distribution. If the two candidate words are very close and in the same row, then they belong to the same item description. If the distance is far, the association is weak. The distance relationship between the candidate word and the page boundary and the area dividing line is analyzed to form a feature value.

[0067] In a possible embodiment, the pattern feature matrix of each candidate word among the m candidate words is determined based on a preset pattern specification method to obtain m pattern feature matrices. According to the common format specifications of invoices, multiple pattern templates are defined. For example, for the amount area, the template is set to require the number to be followed by the monetary unit, and the number format must comply with the financial counting specifications. For the date area, the template specifies the arrangement order and separator form of the year, month, and day. The template is refined according to the characteristics of invoices in different industries, such as the product specification description mode of the manufacturing invoice, the service time and billing mode of the service industry invoice, etc., covering font format, character combination rules, symbol usage specifications and other aspects, forming a comprehensive pattern specification system. Each candidate word is compared with the preset pattern template one by one, and technical means such as regular expression matching and string similarity calculation are adopted. For candidate words that match a specific pattern, the corresponding position in the pattern feature matrix is ​​marked as 1, and unmatched is marked as 0. By using the pattern feature matrix and the inherent format rules of invoices, we can quickly screen out candidate words with specific format meanings. In massive invoice texts, many key information follow fixed patterns. This method can quickly focus on the key points and reduce the complexity of subsequent processing, such as accurately locating core data such as amount, date, and number, reducing interference from irrelevant vocabulary, and improving the accuracy and speed of overall information extraction.

[0068] In a possible embodiment, m logical feature matrices are determined based on the fonts and digital tuples of m candidate words, and the font attributes of the m candidate words are analyzed to extract style features such as font name, font size, boldness, and italicity. Different fonts often have different functions in invoices. The title is usually large and bold, and the text is normal. Vectors are formed by quantifying these features. For digital candidate words, digital logic such as numerical range, parity, and digit features are analyzed. For example, the amount number is generally positive and has a specific numerical interval, and the invoice number is mostly a continuous integer or a specific rule encoding. These digital logics are converted into feature values. The font features and digital features are combined into a tuple, and the association logic between them is further determined. For example, if the font size is large and bold and the number is an integer, it may be the serial number of an important project. If the number in the small font size is regular font with two decimal places, it is identified as a subdivision of the amount. A logical feature matrix is ​​generated based on such association rules. In addition, combined with the invoice context information, if the context of a digital candidate word is a commodity name, then the candidate word may be a unit price or quantity. If it is in the total area, then the candidate word may be the total price.

[0069] In a possible embodiment, m text feature matrices, m layout feature matrices, m pattern feature matrices and m logic feature matrices are integrated to obtain m feature matrices, and the m text feature matrices, m layout feature matrices, m pattern feature matrices and m logic feature matrices are sequentially spliced ​​together by column to form a comprehensive feature matrix with higher dimensions and richer information. In the splicing process, different weights can be assigned according to the importance of each feature. If it is found through experiments that the text feature has a great influence on the specific invoice classification task, the weight proportion of the text feature matrix can be appropriately increased so that the final feature matrix focuses more on key information and optimizes the feature expression effect. The spliced ​​feature matrix is ​​standardized, and the feature values ​​of each dimension are converted into a distribution with a mean of 0 and a standard deviation of 1, eliminating the difference in the magnitude of different features, ensuring the stability of subsequent training, and avoiding the dominant learning process due to the excessive value of individual features. If the dimension of the feature matrix is ​​too high, the dimensionality reduction technology of principal component analysis can be used to reduce the dimension while retaining most of the key information, reduce the amount of calculation and training complexity, avoid the risk of overfitting, improve the efficiency of training and prediction, and focus on the core features for learning and decision-making.

[0070] Optionally, step S104, performing named entity recognition and word classification on the m candidate words based on the m feature matrices to obtain m target labels for the m candidate words, may include the following steps:

[0071] Step S401: determining a named entity recognition dataset corresponding to a target invoice;

[0072] Step S402: determining the named entity ratios of the m candidate words based on the named entity recognition dataset;

[0073] Step S403: performing named entity recognition on the m candidate words based on the m feature matrices to obtain m first labels;

[0074] Step S404: classifying the m candidate words based on the m feature matrices to obtain m second labels;

[0075] Step S405: Integrate the m first labels and the m second labels based on the named entity ratio to obtain m target labels.

[0076] In a possible embodiment, a named entity recognition dataset corresponding to a target invoice is determined, the named entity recognition dataset includes multiple named entities, the named entity ratio of m candidate words is determined based on the named entity recognition dataset, the multiple named entities are compared with the m candidate words one by one to obtain the named entity ratio, then, the first label and the second label of each of the m candidate words are determined by named entity recognition and word classification, named entity recognition is performed on the m candidate words based on m feature matrices to obtain m first labels, word classification is performed on the m candidate words based on the m feature matrices to obtain m second labels, and finally the m first labels and the m second labels are integrated based on the named entity ratio to obtain m target labels.

[0077] In a possible embodiment, when integrating the first label and the second label, the precision in the field of information retrieval, i.e., the ratio of predicted key fields that are correctly extracted and classified, and the recall rate, i.e., the ratio of fields that exist in the reference and are correctly found and classified, are used for evaluation according to the ratio of named entities of the target invoice. Since word classification is superior to named entity recognition in most fields, named entity recognition has better precision in fields other than the amount, and word classification performs better in precision and recall rate, named entity recognition has limitations in processing the target invoice due to the large ratio of undefined words to named entities, especially the amount field often appears in the summary table containing irrelevant words. Therefore, when integrating, the first credibility of the first label and the second credibility of the second label are determined according to the ratio of named entities. If the ratio of named entities is high, then the first credibility of the first label is high, and the first label can be used as the target label at this time. If the ratio of named entities is low, then the second credibility of the second label is high, and the second label can be used as the target label at this time. When the first credibility and the second credibility are both within the preset intermediate credibility range, the target label is determined by combining the first label and the second label.

[0078] Optionally, step S403, performing named entity recognition on the m candidate words based on the m feature matrices to obtain m first labels, may include the following steps:

[0079] Step S501: Determine the word matrix of each candidate word among the m candidate words to obtain m word matrices;

[0080] Step S502: Connect the m word matrices with the corresponding feature matrices in the m feature matrices to obtain m combination matrices;

[0081] Step S503: Determine the number of encoder layers and feature weights of the attention layer;

[0082] Step S504: construct a target encoder based on the number of layers and feature weights;

[0083] Step S505: encoding the m combination matrices through a target encoder to obtain m first encoding results;

[0084] Step S506: Decoding the m first encoding results by using the conditional random field to obtain m first decoding results;

[0085] Step S507: Determine m first labels according to the m first decoding results.

[0086] In a possible embodiment, the word matrix can be represented in the form of a word vector, and the candidate words are represented as numerical vectors through natural language processing. For the target candidate word, the mapping dimension of the vector space is first determined, and then the specific value in the word vector is determined according to the target candidate word and the mapping dimension. The determination of the mapping dimension makes the semantically similar candidate words in the target invoice close in the vector space. The word vector can be represented by one-hot encoding, distributed representation, or dynamic word vector based on deep learning. For two candidate words with the same text, after being represented by dynamic word vectors, the word vectors of the two may be consistent or inconsistent, which will change according to the position of the candidate word in the entire sentence and the semantic structure of the sentence.

[0087] In a possible embodiment, m word matrices are connected with corresponding feature matrices in m feature matrices to obtain m combination matrices. When connecting, the text feature matrix is ​​connected first, and then the layout feature matrix, the pattern feature matrix and the logic feature matrix are connected. The combination matrix can also be represented in the form of a vector, and the vector size corresponding to the combination matrix can be 1155. For the m candidate words, a pre-trained bidirectional encoder representation (Bidirectional Encoder Representations from Transformers, BERT) model is used for word embedding, and the built-in word segmenter of the model is used for word segmentation. Specifically, the combination matrix is ​​input into the BERT model to obtain a first encoding result. For the first encoding result, the best label sequence is jointly decoded through the top layer of the conditional random field (Conditional Random Field, CRF), wherein the conditional random field can predict the probability distribution of the state sequence of the first encoding result.

[0088] In a possible embodiment, the number of layers of the encoder layer and the feature weights of the attention layer are determined, and the number of layers of the added Transformer block is determined according to the number of layers to alleviate errors in optical character recognition. The Transformer block includes a multi-head attention mechanism and a feedforward neural network, and also includes a normalization operation and a residual connection structure between the multi-head attention mechanism and the feedforward neural network. The multi-head attention mechanism consists of multiple parallel attention heads, each of which can be regarded as an independent attention layer. According to the basic calculation logic of the attention mechanism, the context information of the candidate word is obtained by calculating the correlation score between the relevant factors and then performing weighted summation, determining the characteristics of the input sequence and the correlation between the elements from different angles, splicing the results of multiple attention heads and transforming and integrating them, and outputting feature representations of multi-faceted correlations. After being processed by the multi-head attention mechanism, the combined matrix will enter the feedforward neural network for further nonlinear transformation to enhance the model's ability to express features. The feedforward neural network consists of two fully connected layers and an activation function in the middle. Normalization can normalize the input data of each layer, which helps to speed up model training and improve model stability. The residual connection adds the input of each layer directly to the output of that layer, which allows the gradient to propagate back more smoothly, avoids problems such as gradient disappearance in deep networks, and makes the model easier to train.

[0089] Optionally, step S503, determining the number of encoder layers and the feature weights of the attention layer, may include the following steps:

[0090] Step S601: Compare the target invoice and text information to obtain the misrecognition rate;

[0091] Step S602: Determine the semantic level of the text information; the semantic level is used to reflect the complexity of the text information;

[0092] Step S603: determining the number of layers based on the misrecognition rate and the semantic level;

[0093] Step S604: determining the recognition speed of each of the m candidate words, and obtaining m recognition speeds;

[0094] Step S605: Determine feature weights based on the m recognition speeds.

[0095] In a possible embodiment, when the recognition speed of a target candidate word is slow, the candidate word may have a recognition error during optical character recognition, and the feature weight of the target candidate word can be increased to correct the error through a multi-head attention mechanism.

[0096] In a possible embodiment, the target invoice and text information are compared to obtain the misrecognition rate, determine the semantic level of the text information, and determine the number of layers based on the misrecognition rate and the semantic level. If the number of layers is determined to be two, add two layers of Transformer blocks to alleviate optical character recognition errors. The text of optical character recognition often has contextual semantic associations. For example, when recognizing a passage, there are grammatical and semantic connections between words in a sentence. The multi-head attention mechanism in the Transformer block can determine this contextual relationship. When a certain recognized character or word appears to be inaccurate, the semantic association between the context and the background is used to correct this error through the aggregation and feature extraction of context information by the multi-head attention layer. The feedforward neural network can further perform deep processing on the features processed by the attention mechanism, and adjust the feature space through nonlinear transformation, so that more complex text patterns and rules can be learned, thereby improving the ability to distinguish and correct misrecognition situations. Therefore, through the synergy of a series of components such as the multi-head attention mechanism contained in the Transformer block, the ability to handle optical character recognition errors is improved from multiple dimensions.

[0097] Optionally, step S404, classifying the m candidate words based on the m feature matrices to obtain m second labels, may include the following steps:

[0098] Step S701: Determine the word sequence length of each candidate word among m candidate words to obtain m word sequence lengths;

[0099] Step S702: Encode the m text feature matrices through a convolutional neural network to obtain m second encoding results;

[0100] Step S703: encoding the m layout feature matrices, the m pattern feature matrices, and the m logic feature matrices through a bidirectional long short-term memory network to obtain m third encoding results;

[0101] Step S704: combining the m second encoding results and the m third encoding results to obtain m combined encoding results;

[0102] Step S705: Decoding the m combined encoding results according to the lengths of the m word sequences to obtain m second decoding results;

[0103] Step S706: Determine m second labels according to the m second decoding results.

[0104] In a possible embodiment, the second label of the candidate word is determined based on the text feature matrix, the layout feature matrix, the pattern feature matrix and the logic feature matrix in the feature vector, thereby determining the information category of the candidate word in the target invoice.

[0105] In a possible embodiment, when classifying m candidate words, the word sequence length of each candidate word is first determined. The word sequence length is determined based on the real field, for example, based on related words such as date, amount, title, mathematical assertion, etc., thereby reducing undefined elements and reducing information redundancy during word classification.

[0106] In a possible embodiment, the input is a list of feature matrices of m candidate words of the target invoice. For text features, the text feature matrix is ​​encoded by a convolutional neural network. For spatial ordered features, the layout feature matrix, pattern feature matrix and logical feature matrix are encoded by a bidirectional LSTM encoder of a long short-term memory network (LSTM). Then, the encoding results of the convolutional neural network and the bidirectional LSTM encoder are combined by a combiner, and the combined results are input into a preset decoder for decoding, and the second label is determined according to the decoding result.

[0107] In a possible embodiment, before using the invoice information extraction method of the embodiment of the present application, it is necessary to conduct training and evaluate the impact of the data volume on the training situation. The training is conducted on 100k database documents. The results show that the recall rate and precision are improved when the data volume increases. The present invention significantly reduces the required data volume by selecting only reliable data for training and development. It can cope with unprecedented templates and multi-language situations, does not require parameter settings and prior knowledge, and obtains considerable performance with a small amount of data. Reducing the demand for training data is of great significance to commercial document processing.

[0108] In a possible embodiment, the improvement direction related to the interactive incremental method and two-dimensional information is adopted. For example, a more intuitive interactive interface, real-time feedback and response, and multimodal interactive support are adopted to enhance user interactivity, and a concise, easy-to-use interface with visual elements is designed to enable users to more intuitively understand and operate the interactive incremental process. Adaptive incremental strategies, data screening and preprocessing, and model fusion and updating are adopted to optimize the incremental learning process, and data management and storage efficiency are improved through incremental data storage optimization, data version management, and distributed storage and computing. High-dimensional representation, compression coding, and semantic enhancement representation are adopted to increase information representation capabilities, and data structures such as tensors are used to represent two-dimensional information, which can more comprehensively capture multiple features and interrelationships of information, reduce data storage space and transmission bandwidth without losing important information, and integrate semantic information into the representation of two-dimensional information, so that the information not only has numerical features, but also contains semantic meanings. For example, by adding tags, annotations, or using knowledge graphs, more clear semantics are given to two-dimensional information for better understanding and processing.

[0109] In summary, in the implementation of the present application, the target invoice and the target information category to be extracted are first obtained, wherein the target information category to be extracted is the information category of the invoice information that needs to be obtained in the target invoice, then, character recognition is performed on the target invoice to obtain m candidate words, wherein m is a positive integer, and then, the feature matrix of each candidate word in the m candidate words is determined to obtain m feature matrices, and then, based on the m feature matrices, named entity recognition and word classification are performed on the m candidate words to obtain m target labels of the m candidate words, each target label corresponds to an information category of the invoice information, and finally, the invoice information is determined based on the m target labels and the target information category to be extracted. Thus, by performing named entity recognition and word classification on the m candidate words, m target labels of the m candidate words are obtained, and then the invoice information is determined according to the m target labels and the target information category to be extracted, when the invoice information is extracted, the cost of extracting the invoice information can be reduced.

[0110] The method of the embodiment of the present invention is described in detail above, and the device of the embodiment of the present invention is provided below.

[0111] See also Figure 2 , Figure 2 800 is a schematic diagram of the structure of an invoice information extraction device provided in an embodiment of the present application. The invoice information extraction device 800 includes an acquisition unit 801 and a processing unit 802;

[0112] The acquisition unit 801 is used to acquire the target invoice and the target information category to be extracted; the target information category to be extracted is the information category of the invoice information to be acquired in the target invoice;

[0113] Processing unit 802 is used to perform character recognition on the target invoice to obtain m candidate words, where m is a positive integer;

[0114] Determine the feature matrix of each candidate word among the m candidate words to obtain m feature matrices;

[0115] Based on the m feature matrices, named entity recognition and word classification are performed on the m candidate words to obtain m target labels of the m candidate words; each target label corresponds to an information category of invoice information;

[0116] The invoice information is determined based on the m target tags and the target information category to be extracted.

[0117] In some possible embodiments, in performing character recognition on a target invoice to obtain m candidate words, the processing unit 802 is specifically configured to:

[0118] Determine the reading direction and information structure of the target invoice;

[0119] Determine recognition order based on reading direction and information structure;

[0120] Perform character recognition on the target invoice based on the recognition order to obtain n characters; n is an integer greater than or equal to m;

[0121] Determine the initial position of each character in the n characters, and obtain n initial positions;

[0122] Determine the centroid coordinates of each character in the n characters according to the n initial positions, and obtain n centroid coordinates;

[0123] Arrange n characters based on n centroid coordinates to obtain text information;

[0124] The text information is screened according to the target information category to be extracted to obtain m candidate words.

[0125] In some possible embodiments, in determining the feature matrix of each candidate word among the m candidate words to obtain the m feature matrices, the processing unit 802 is specifically configured to:

[0126] Determine the associated word set of each candidate word among the m candidate words, and obtain m associated word sets; each associated word set includes at least one associated word;

[0127] Construct a text feature matrix of each of the m candidate words based on the m associated word sets to obtain m text feature matrices;

[0128] Determine the page descriptor of the target invoice, and determine m position coordinates corresponding to the m candidate words based on the n centroid coordinates;

[0129] Determine a layout feature matrix of each of the m candidate words based on the page descriptor and the m position coordinates, and obtain m layout feature matrices;

[0130] Determine a pattern feature matrix of each of the m candidate words based on a preset pattern standardization method to obtain m pattern feature matrices;

[0131] Determine m logical feature matrices according to the fonts and digital tuples of the m candidate words;

[0132] Integrate m text feature matrices, m layout feature matrices, m pattern feature matrices and m logic feature matrices to obtain m feature matrices.

[0133] In some possible embodiments, in performing named entity recognition and word classification on m candidate words based on m feature matrices to obtain m target labels for the m candidate words, the processing unit 802 is specifically configured to:

[0134] Determine the named entity recognition dataset corresponding to the target invoice;

[0135] Determine the named entity ratio of m candidate words based on the named entity recognition dataset;

[0136] Perform named entity recognition on the m candidate words based on the m feature matrices to obtain m first labels;

[0137] Based on the m feature matrices, m candidate words are classified to obtain m second labels;

[0138] Based on the named entity ratio, m first labels and m second labels are integrated to obtain m target labels.

[0139] In some possible embodiments, in performing named entity recognition on m candidate words based on m feature matrices to obtain m first labels, the processing unit 802 is specifically configured to:

[0140] Determine the word matrix of each candidate word among the m candidate words, and obtain m word matrices;

[0141] Connect the m word matrices with the corresponding feature matrices in the m feature matrices to obtain m combination matrices;

[0142] Determine the number of encoder layers and feature weights of the attention layer;

[0143] Build the target encoder based on the number of layers and feature weights;

[0144] Encoding the m combination matrices through a target encoder to obtain m first encoding results;

[0145] Decoding the m first encoding results by using a conditional random field to obtain m first decoding results;

[0146] Determine m first labels according to the m first decoding results.

[0147] In some possible embodiments, in determining the number of encoder layers and the feature weights of the attention layer, the processing unit 802 is specifically configured to:

[0148] Compare the target invoice and text information to obtain the misrecognition rate;

[0149] Determine the semantic level of text information; the semantic level is used to reflect the complexity of text information;

[0150] Determine the number of layers based on the misrecognition rate and semantic level;

[0151] Determine the recognition speed of each of the m candidate words, and obtain m recognition speeds;

[0152] The feature weights are determined based on the m recognition speeds.

[0153] In some possible embodiments, in terms of performing word classification on m candidate words based on m feature matrices to obtain m second labels, the processing unit 802 is specifically configured to:

[0154] Determine the word sequence length of each candidate word in the m candidate words, and obtain m word sequence lengths;

[0155] Encode the m text feature matrices through a convolutional neural network to obtain m second encoding results;

[0156] Encoding the m layout feature matrices, the m pattern feature matrices, and the m logic feature matrices through a bidirectional long short-term memory network to obtain m third encoding results;

[0157] Combining the m second encoding results and the m third encoding results to obtain m combined encoding results;

[0158] Decoding the m combined encoding results according to the lengths of the m word sequences to obtain m second decoding results;

[0159] Determine m second labels according to the m second decoding results.

[0160] See also Figure 3 , Figure 3 Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 3 As shown, the electronic device 900 includes a transceiver 901, a processor 902 and a memory 903, which are connected via a bus 904. The memory 903 is used to store computer programs and data, and can transmit the data stored in the memory 903 to the processor 902. The electronic device can be the above-mentioned invoice information extraction device, and the processor 902 can be the above-mentioned acquisition unit 801 and processing unit 802.

[0161] The processor 902 is used to read the computer program in the memory 903 and perform the following operations:

[0162] Obtain the target invoice and the target information category to be extracted; the target information category to be extracted is the information category of the invoice information to be obtained in the target invoice;

[0163] Perform character recognition on the target invoice to obtain m candidate words; m is a positive integer;

[0164] Determine the feature matrix of each candidate word among the m candidate words to obtain m feature matrices;

[0165] Based on the m feature matrices, named entity recognition and word classification are performed on the m candidate words to obtain m target labels of the m candidate words; each target label corresponds to an information category of invoice information;

[0166] The invoice information is determined based on the m target tags and the target information category to be extracted.

[0167] In some possible embodiments, in performing character recognition on the target invoice to obtain m candidate words, the processor 902 is specifically configured to perform the following operations:

[0168] Determine the reading direction and information structure of the target invoice;

[0169] Determine the recognition order based on reading direction and information structure;

[0170] Perform character recognition on the target invoice based on the recognition order to obtain n characters; n is an integer greater than or equal to m;

[0171] Determine the initial position of each character in the n characters, and obtain n initial positions;

[0172] Determine the centroid coordinates of each character in the n characters according to the n initial positions, and obtain n centroid coordinates;

[0173] Arrange n characters based on n centroid coordinates to obtain text information;

[0174] The text information is screened according to the target information category to be extracted to obtain m candidate words.

[0175] In some possible embodiments, in determining the feature matrix of each candidate word among the m candidate words to obtain the m feature matrices, the processor 902 is specifically configured to perform the following operations:

[0176] Determine the associated word set of each candidate word among the m candidate words, and obtain m associated word sets; each associated word set includes at least one associated word;

[0177] Based on the m associated word sets, a text feature matrix of each of the m candidate words is constructed to obtain m text feature matrices;

[0178] Determine the page descriptor of the target invoice, and determine m position coordinates corresponding to the m candidate words based on the n centroid coordinates;

[0179] Determine a layout feature matrix of each of the m candidate words based on the page descriptor and the m position coordinates, and obtain m layout feature matrices;

[0180] Determine a pattern feature matrix of each of the m candidate words based on a preset pattern standardization method to obtain m pattern feature matrices;

[0181] Determine m logical feature matrices according to the fonts and digital tuples of the m candidate words;

[0182] Integrate m text feature matrices, m layout feature matrices, m pattern feature matrices and m logic feature matrices to obtain m feature matrices.

[0183] In some possible embodiments, in performing named entity recognition and word classification on m candidate words based on m feature matrices to obtain m target labels of the m candidate words, the processor 902 is specifically configured to perform the following operations:

[0184] Determine the named entity recognition dataset corresponding to the target invoice;

[0185] Determine the named entity ratio of m candidate words based on the named entity recognition dataset;

[0186] Perform named entity recognition on the m candidate words based on the m feature matrices to obtain m first labels;

[0187] Based on the m feature matrices, m candidate words are classified to obtain m second labels;

[0188] Based on the named entity ratio, m first labels and m second labels are integrated to obtain m target labels.

[0189] In some possible embodiments, in performing named entity recognition on m candidate words based on m feature matrices to obtain m first labels, the processor 902 is specifically configured to perform the following operations:

[0190] Determine the word matrix of each candidate word among the m candidate words, and obtain m word matrices;

[0191] Connect the m word matrices with the corresponding feature matrices in the m feature matrices to obtain m combination matrices;

[0192] Determine the number of encoder layers and feature weights of the attention layer;

[0193] Build the target encoder based on the number of layers and feature weights;

[0194] Encoding the m combination matrices through a target encoder to obtain m first encoding results;

[0195] Decoding the m first encoding results by using a conditional random field to obtain m first decoding results;

[0196] Determine m first labels according to the m first decoding results.

[0197] In some possible embodiments, in determining the number of encoder layers and the feature weights of the attention layer, the processor 902 is specifically configured to perform the following operations:

[0198] Compare the target invoice and text information to obtain the misrecognition rate;

[0199] Determine the semantic level of text information; the semantic level is used to reflect the complexity of text information;

[0200] Determine the number of layers based on the misrecognition rate and semantic level;

[0201] Determine the recognition speed of each of the m candidate words, and obtain m recognition speeds;

[0202] The feature weights are determined based on the m recognition speeds.

[0203] In some possible embodiments, in terms of performing word classification on m candidate words based on m feature matrices to obtain m second labels, the processor 902 is specifically configured to perform the following operations:

[0204] Determine the word sequence length of each candidate word in the m candidate words, and obtain m word sequence lengths;

[0205] Encode the m text feature matrices through a convolutional neural network to obtain m second encoding results;

[0206] Encoding the m layout feature matrices, the m pattern feature matrices, and the m logic feature matrices through a bidirectional long short-term memory network to obtain m third encoding results;

[0207] Combining the m second encoding results and the m third encoding results to obtain m combined encoding results;

[0208] Decoding the m combined encoding results according to the lengths of the m word sequences to obtain m second decoding results;

[0209] Determine m second labels according to the m second decoding results.

[0210] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement part or all of the steps of any invoice information extraction method recorded in the above method embodiments.

[0211] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any invoice information extraction method recorded in the above method embodiments.

[0212] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0213] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0214] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the modules, which is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or module can be electrical or other forms.

[0215] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0216] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software program modules.

[0217] If the integrated module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0218] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for extracting invoice information, characterized in that: include: Get the target invoice and the target information category to be extracted; The target information category to be extracted is the information category of the invoice information that needs to be obtained in the target invoice; Performing character recognition on the target invoice to obtain m candidate words; m is a positive integer; Determine a feature matrix of each of the m candidate words to obtain m feature matrices; Perform named entity recognition and word classification on the m candidate words based on the m feature matrices to obtain m target labels for the m candidate words; Each target tag corresponds to an information category of invoice information; The invoice information is determined based on the m target tags and the target information category to be extracted.

2. The method according to claim 1, characterized in that The target invoice is subjected to character recognition to obtain m candidate words, including: Determining the reading direction and information structure of the target invoice; determining a recognition order based on the reading direction and the information structure; Performing character recognition on the target invoice based on the recognition sequence to obtain n characters, where n is an integer greater than or equal to m; Determine the initial position of each character in the n characters to obtain n initial positions; Determine the centroid coordinates of each of the n characters according to the n initial positions to obtain n centroid coordinates; Arranging the n characters based on the n centroid coordinates to obtain text information; The text information is screened according to the target information category to be extracted to obtain the m candidate words.

3. The method according to claim 2, characterized in that The step of determining a feature matrix of each of the m candidate words to obtain m feature matrices includes: Determine an associated word set of each of the m candidate words to obtain m associated word sets; each associated word set includes at least one associated word; Constructing a text feature matrix of each of the m candidate words based on the m associated word sets to obtain m text feature matrices; Determine a page descriptor of the target invoice, and determine m position coordinates corresponding to the m candidate words based on the n centroid coordinates; Determine a layout feature matrix of each of the m candidate words based on the page descriptor and the m position coordinates to obtain m layout feature matrices; Determine a pattern feature matrix of each of the m candidate words based on a preset pattern standardization method to obtain m pattern feature matrices; Determine m logical feature matrices according to the fonts and numeric tuples of the m candidate words; The m text feature matrices, the m layout feature matrices, the m pattern feature matrices and the m logic feature matrices are integrated to obtain the m feature matrices.

4. The method according to claim 3, characterized in that The performing named entity recognition and word classification on the m candidate words based on the m feature matrices to obtain m target labels of the m candidate words includes: Determine a named entity recognition dataset corresponding to the target invoice; Determine the named entity ratios of the m candidate words based on the named entity recognition dataset; Performing named entity recognition on the m candidate words based on the m feature matrices to obtain m first labels; Perform word classification on the m candidate words based on the m feature matrices to obtain m second labels; The m first labels and the m second labels are integrated based on the named entity ratio to obtain the m target labels.

5. The method according to claim 4, characterized in that The performing named entity recognition on the m candidate words based on the m feature matrices to obtain m first labels includes: Determine a word matrix for each of the m candidate words to obtain m word matrices; Connecting the m word matrices with corresponding feature matrices in the m feature matrices to obtain m combination matrices; Determine the number of encoder layers and feature weights of the attention layer; Constructing a target encoder based on the number of layers and the feature weights; Encoding the m combination matrices by the target encoder to obtain m first encoding results; Decoding the m first encoding results by using a conditional random field to obtain m first decoding results; The m first labels are determined according to the m first decoding results.

6. The method according to claim 5, characterized in that The determining the number of encoder layers and the feature weights of the attention layer includes: Comparing the target invoice with the text information to obtain a false recognition rate; Determining the semantic level of the text information; the semantic level is used to reflect the complexity of the text information; Determining the number of layers based on the misrecognition rate and the semantic level; Determine the recognition speed of each of the m candidate words to obtain m recognition speeds; The feature weight is determined based on the m recognition speeds.

7. The method according to any one of claims 4 to 6, characterized in that: The performing word classification on the m candidate words based on the m feature matrices to obtain m second labels includes: Determine the word sequence length of each candidate word in the m candidate words to obtain m word sequence lengths; Encoding the m text feature matrices through a convolutional neural network to obtain m second encoding results; Encoding the m layout feature matrices, the m pattern feature matrices, and the m logic feature matrices through a bidirectional long short-term memory network to obtain m third encoding results; Combining the m second encoding results and the m third encoding results to obtain m combined encoding results; Decoding the m combined encoding results according to the m word sequence lengths to obtain m second decoding results; The m second tags are determined according to the m second decoding results.

8. An invoice information extraction device, characterized in that: The device comprises an acquisition unit and a processing unit; The acquisition unit is used to acquire a target invoice and a target information category to be extracted; the target information category to be extracted is the information category of the invoice information to be acquired in the target invoice; The processing unit is used to perform character recognition on the target invoice to obtain m candidate words; m is a positive integer; Determine a feature matrix of each of the m candidate words to obtain m feature matrices; Perform named entity recognition and word classification on the m candidate words based on the m feature matrices to obtain m target labels for the m candidate words; Each target tag corresponds to an information category of invoice information; The invoice information is determined based on the m target tags and the target information category to be extracted.

9. An electronic device, characterized in that: include: A processor and a memory, the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.

Images