An accounting document classification method and system based on artificial intelligence

Abstract

The application provides an accounting document classification method and system based on artificial intelligence. The label dependency relationship between the text information and the label data set is determined according to the part of speech of the effective word units in each text keyword. The text label features in the target accounting document are determined through the label dependency relationship and the semantic loss of each text keyword. The semantic features of the document image and each data label in the label data set are semantically associated and matched to obtain the semantic inline relationship between the document image and the label data set. The image label features of the semantic word units in the target accounting document are determined based on the semantic inline relationship. The image label features and the text label features are modality fused, and then the trusted label of the target accounting document is generated based on the modality fused label features. The target accounting document is classified by using the trusted label. Based on the above scheme, the multi-label classification of the accounting document can be realized by combining the feature fusion of the text mode and the image mode.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pattern recognition, more specifically, the present application relates to an accounting document classification method and system based on artificial intelligence. BACKGROUND

[0002] With the development of information technology, computers and electronic data processing gradually penetrate into the field of accounting, and automated and digitized accounting documents emerge as the times require. Electronic accounting documents not only improve data processing speed, but also ensure the accuracy and consistency of information. In recent years, with the rapid development of big data, cloud computing and artificial intelligence technologies, intelligent accounting documents have begun to appear, which can realize automatic data input, intelligent auditing and real-time processing, providing more efficient, secure and flexible solutions for enterprise financial management.

[0003] In accounting documents, single modal may have incomplete or ambiguous information, which affects the accuracy of classification. For example, the text in the document image may become blurred or unclear due to image quality problems, resulting in incomplete or unrecognizable information extracted from the image. Traditional image classification methods may miss key information, thereby reducing classification performance. At the same time, text information may also contain some difficult-to-explain words, abbreviations or specific accounting terms, which makes it easy to cause understanding bias by relying solely on text for classification. Therefore, how to combine the feature fusion of text modal and image modal to realize the multi-label classification of accounting documents has become a difficult problem in the industry. SUMMARY

[0004] The present application provides an accounting document classification method and system based on artificial intelligence, which can realize multi-label classification of accounting documents by combining feature fusion of text modal and image modal.

[0005] In a first aspect, the present application provides an accounting document classification method based on artificial intelligence, comprising:

[0006] Obtaining a label data set of an accounting document and a target accounting document to be classified, and then extracting text information and document images in the target accounting document;

[0007] Performing semantic word segmentation on the text information to obtain a plurality of text keywords of the target accounting document, and then determining a label dependency relationship between the text information and the label data set according to the part of speech of the effective word units in each text keyword, and determining the text label features of the semantic word units in the target accounting document through the label dependency relationship and the semantic loss of each text keyword in semantic word segmentation;

[0008] The semantic features of the voucher image are semantically associated and matched with each data label in the label data set, to obtain a semantic in-line relationship between the voucher image and the label data set, and an image label feature of a semantic word element in the target accounting voucher is determined based on the semantic in-line relationship and a self-attention mechanism;

[0009] The image label feature and the text label feature are modality fused, and then a trusted label of the target accounting voucher is generated based on the modality fused label feature, and the target accounting voucher is classified using the trusted label.

[0010] In some embodiments, the text information is semantically segmented to obtain a plurality of text keywords of the target accounting voucher, which specifically includes:

[0011] A plurality of segmented word groups in the text information are extracted;

[0012] The semantic association degree between each segmented word group and the text information is determined;

[0013] The plurality of text keywords of the target accounting voucher are filtered from all segmented word groups through each semantic association degree.

[0014] In some embodiments, the label dependency relationship between the text information and the label data set is determined according to the part of speech of the effective word element in each text keyword, which specifically includes:

[0015] All associated labels are filtered from the label data set based on the semantic features of the text information of the target accounting voucher;

[0016] For each text keyword, a correlation coefficient between the text keyword and each associated label is determined according to the part of speech feature of the text keyword;

[0017] The label dependency value between the text keyword and the label data set is determined through all correlation coefficients, and then the label dependency value between each text keyword and the label data set is obtained;

[0018] The label dependency relationship between the text information and the label data set is determined according to all label dependency values.

[0019] In some embodiments, the text label feature of the semantic word element in the target accounting voucher is determined through the label dependency relationship and the semantic loss of each text keyword in semantic segmentation, which specifically includes:

[0020] For each text keyword, the semantic loss of the text keyword in semantic segmentation is extracted;

[0021] The label dependency value of the text keyword is obtained from the label dependency relationship;

[0022] determine semantic wordpiece labels of the text keywords according to the label dependent value and the semantic loss, and further obtain semantic wordpiece labels of each text keyword;

[0023] determine text label features of semantic wordpieces in the target accounting document according to all semantic wordpiece labels.

[0024] In some embodiments, the semantic correlation matching of the semantic features of the document image and each data label in the label data set obtains a semantic in-line relationship between the document image and the label data set, and specifically includes:

[0025] For each data label in the label data set, extract the label semantics of the data label and the semantic features of the document image;

[0026] match the label semantics and the semantic features to obtain a semantic correlation value of the data label, and further obtain semantic correlation values of each data label in the label data set;

[0027] determine the semantic in-line relationship between the document image and the label data set through all semantic correlation values.

[0028] In some embodiments, determining image label features of semantic wordpieces in the target accounting document based on the semantic in-line relationship combined with a self-attention mechanism specifically includes:

[0029] construct an association degree matrix of semantic wordpieces in the document image through the semantic in-line relationship;

[0030] perform feature weighting on all semantic correlation values in the association degree matrix based on a self-attention mechanism to obtain image label features of semantic wordpieces in the target accounting document.

[0031] In some embodiments, a tensor fusion algorithm is used to perform modal fusion on the image label features and the text label features.

[0032] In a second aspect, the present application provides an accounting document classification system based on artificial intelligence, comprising:

[0033] an acquisition module configured to acquire a label data set of an accounting document and a target accounting document to be classified, and further extract text information and a document image in the target accounting document;

[0034] The processing module is configured to perform semantic word segmentation on the text information to obtain a plurality of text keywords of the target accounting voucher, and then determine a label dependency relationship between the text information and the label data set according to the part-of-speech of an effective word element in each text keyword, and determine a text label feature of a semantic word element in the target accounting voucher by using the label dependency relationship and semantic loss of each text keyword in the semantic word segmentation;

[0035] The processing module is further configured to perform semantic association matching between the semantic features of the voucher image and each data label in the label data set to obtain a semantic inline relationship between the voucher image and the label data set, and determine an image label feature of a semantic word element in the target accounting voucher based on the semantic inline relationship and a self-attention mechanism;

[0036] The execution module is configured to perform modality fusion on the image label feature and the text label feature, and then generate a trusted label of the target accounting voucher based on the label feature after the modality fusion, and perform multi-label classification on the target accounting voucher by using the trusted label.

[0037] In a third aspect, a computer device is provided. The computer device includes a memory and a processor. The memory is configured to store a computer program. The processor is configured to call and run the computer program from the memory, so that the computer device performs the accounting voucher classification method based on artificial intelligence.

[0038] In a fourth aspect, a computer readable storage medium is provided. The computer readable storage medium stores instructions or codes. When the instructions or codes are run on a computer, the computer is caused to perform the accounting voucher classification method based on artificial intelligence.

[0039] The technical scheme provided by the embodiments of the present application has the following beneficial effects:

[0040] The application provides an accounting document classification method and system based on artificial intelligence. Label data set of an accounting document and target accounting document to be classified are obtained, and then text information and document image in the target accounting document are extracted. Semantic word segmentation is performed on the text information to obtain a plurality of text keywords of the target accounting document, and then a label dependency relationship between the text information and the label data set is determined according to the part-of-speech of effective word units in each text keyword. The text label feature of a semantic word unit in the target accounting document is determined through the label dependency relationship and semantic loss of each text keyword in semantic word segmentation. The semantic features of the document image and each data label in the label data set are semantically associated and matched to obtain a semantic inline relationship between the document image and the label data set. The image label feature of a semantic word unit in the target accounting document is determined based on the semantic inline relationship and a self-attention mechanism. The image label feature and the text label feature are modality fused, and then a trusted label of the target accounting document is generated based on the modality fused label feature. The target accounting document is classified by using the trusted label.

[0041] Therefore, in the application, the image label feature and the text label feature are modality fused, and then a trusted label of the target accounting document is generated based on the modality fused label feature, and the target accounting document is classified by using the trusted label. First, the text label feature not only effectively extracts semantic information in the text, but also ensures the accuracy and effectiveness of the information through the calculation of the semantic loss, so as to provide accurate text label information for subsequent modality fusion, so that the image feature and the text feature can form a complement when being fused, thereby improving the classification accuracy and the depth of semantic understanding. Through the dimension reduction process, the keywords extracted from a large amount of text information can highly summarize the key information in the text, thereby reducing the burden of the model when processing high-dimensional data. At the same time, by analyzing the relationship between the effective word units and the parts-of-speech of the text keywords, the dependency modeling between the text and the label is further strengthened. Then, the determination of the image label feature provides a more complete image feature representation for modality fusion, enhances the performance of multi-label classification, ensures the contribution of image information to the final classification, and facilitates the subsequent fusion of the text modality and the image modality to eliminate the information blind area in the traditional single modality classification method. Especially when the image quality is poor, the text information can make up for the possible missing part in the image, and the image label feature can be accurately weighted by combining the self-attention mechanism, thereby further improving the effect of feature fusion. In summary, based on the above scheme, the multi-label classification of the accounting document can be realized by combining the feature fusion of the text modality and the image modality. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the accompanying drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0043] Figure 1 is an exemplary flowchart of an accounting document classification method based on artificial intelligence according to some embodiments of the present application;

[0044] Figure 2 is a logic diagram of matching images and text labels according to some embodiments of the present application;

[0045] Figure 3 is a flowchart of determining image label features according to some embodiments of the present application;

[0046] Figure 4 is a structural schematic diagram of an accounting document classification system based on artificial intelligence according to some embodiments of the present application;

[0047] Figure 5 is a structural schematic diagram of a computer device for implementing an accounting document classification method based on artificial intelligence according to some embodiments of the present application. DETAILED DESCRIPTION

[0048] In order to better understand the technical solutions of the present application, the technical solutions of the present application will be described in detail below in combination with the accompanying drawings of the specification and specific embodiments.

[0049] Reference Figure 1 The figure is an exemplary flowchart of an accounting document classification method based on artificial intelligence according to some embodiments of the present application, which mainly includes the following steps:

[0050] In step 101, the label data set of the accounting document and the target accounting document to be classified are obtained, and then the text information and the document image in the target accounting document are extracted.

[0051] It should be noted that in the present application, the label data set represents a set of labels of a group of labeled accounting documents, wherein each data instance is associated with one or more data labels; the target accounting document represents a specific accounting record document; the text information represents the textual description part in the target accounting document; and the document image represents the image file of the target accounting document, including the scanned image or the photographed image of the accounting document.

[0052] In a specific implementation, data tags of accounting vouchers submitted in a specified time period are collected from a voucher database, so that a set of all data tags is taken as a label data set of the accounting vouchers, and a current accounting voucher to be classified is taken as a target accounting voucher, text category information in the target accounting voucher is identified by using a scanner, so that the text category information is taken as text information in the target accounting voucher.

[0053] In some embodiments, with reference to Figure 2 The figure is a matching logic diagram of image and text labels according to the image and text labels shown in some embodiments of the present application. First, the image part generates image encoding through a visual encoder (for example: CLIP), and then generates image semantics through a text decoder (BERT-Generation). After data cleaning, m word pieces are retained to form word piece encoding of a text encoder (for example: CLIP). At the same time, similarity calculation is performed on the image encoding and the word piece encoding, and the first k keywords are extracted through sorting and screening. Second, the text label matching part, the label encoding is generated through a word embedding model (GloVe) on the data set label. Then, similarity calculation is performed to match the keyword encoding, and k correlation scores are obtained. These scores are processed through a Softmax function to obtain k fusion weights. Finally, the similarity scores of the image encoding and the fusion weights of the text labels are combined to generate the final loss value L through the Softmax function. The whole process realizes the deep fusion and matching of image and text through the visual encoder and the text encoder and the word embedding model (GloVe).

[0054] In step 102, the text information is semantically segmented to obtain a plurality of text keywords of the target accounting voucher, and then a label dependency relationship between the text information and the label data set is determined according to the part-of-speech of the valid word piece in each text keyword, and a text label feature of a semantic word piece in the target accounting voucher is determined through the label dependency relationship and the semantic loss of each text keyword in the semantic segmentation.

[0055] In some embodiments, the text information is semantically segmented to obtain a plurality of text keywords of the target accounting voucher, which can be implemented by the following steps:

[0056] Extracting a plurality of segmented word groups in the text information;

[0057] Determining a semantic correlation degree between each segmented word group and the text information;

[0058] Filtering a plurality of text keywords of the target accounting voucher from all segmented word groups through each semantic correlation degree.

[0059] In a specific implementation, first, the plurality of segmented word groups in the text information can be extracted in the following manner: a natural language processing tool (for example, Jieba segmentation) can be used to divide the text information into a plurality of segmented word groups according to the grammatical structure in the text information; then, the semantic correlation degree between each segmented word group and the text information can be determined in the following manner: for each segmented word group, a word vector model (for example, Word2Vec) can be used to calculate the vector representation of the segmented word group in the semantic space as a segmented semantic vector, and a word embedding algorithm can be used to vectorize the text information in the semantic space as a text semantic vector, so that the cosine similarity between the segmented semantic vector and the text semantic vector can be used as the semantic correlation degree between the segmented word group and the text information, and the semantic correlation degree between each segmented word group and the text information can be obtained through the above manner; finally, the plurality of text keywords of the target accounting voucher can be selected from all segmented word groups through each semantic correlation degree in the following manner: a semantic correlation threshold of the keyword can be preset through historical experience, so that the segmented word group corresponding to the semantic correlation degree greater than the semantic correlation threshold can be used as a text keyword, and thus the plurality of text keywords of the target accounting voucher can be obtained.

[0060] It should be noted that in the present application, the text keyword represents a word in the text with important semantic information; the segmented word group is a basic unit in the text; and the semantic correlation degree is a correlation degree of quantifying the semantic information transmitted between the segmented word group and the text information.

[0061] In some embodiments, determining the label dependency relationship between the text information and the label data set according to the part of speech of each valid word unit in the text keyword can be implemented in the following steps:

[0062] Filtering all associated labels from the label data set based on the semantic features of the text information of the target accounting voucher;

[0063] For each text keyword, determining the correlation coefficient between the text keyword and each associated label according to the part of speech feature of the text keyword;

[0064] Determining the label dependency value between the text keyword and the label data set through all correlation coefficients, and thus obtaining the label dependency value between each text keyword and the label data set;

[0065] Determining the label dependency relationship between the text information and the label data set according to all label dependency values.

[0066] In a specific implementation, first, the following method can be used to filter all associated labels from the label data set based on the semantic features of the text information of the target accounting voucher, that is, the text semantic vector corresponding to the text information is taken as the semantic features of the text information of the target accounting voucher, and for each data label in the label data set, the vector representation of the data label in the semantic space is calculated as the label semantic vector using a word vector model (for example, Word2Vec), so that the cosine similarity between the label semantic vector and the semantic features of the text information is taken as the association value between the data label and the text information, and the association value between each data label and the text information can be obtained by the above method. The association threshold can be preset by historical experience, so that the data label corresponding to the association value greater than the association threshold is taken as the associated label, and all associated labels can be obtained; second, for each text keyword, the following method can be used to determine the correlation coefficient between the text keyword and each associated label according to the part-of-speech features of the text keyword, that is, for each text keyword, the part-of-speech semantic vector of the part-of-speech group corresponding to the text keyword is taken as the part-of-speech feature of the text keyword, so that the cosine similarity between the part-of-speech feature and the label semantic vector of each associated label is taken as the correlation coefficient between the text keyword and the corresponding associated label, and the correlation coefficient between the text keyword and each associated label can be obtained. The part-of-speech feature represents the grammatical function and role of the word in natural language processing.

[0067] Then, in a specific implementation, the following method can be used to determine the label dependency value between the text keyword and the label data set through all correlation coefficients, and then the label dependency value between each text keyword and the label data set can be obtained, that is, the set of all correlation coefficients is taken as the value range of the label dependency value, and the label dependency value between the text keyword and the label data set can be obtained, so that the label dependency value between each text keyword and the label data set can be obtained by the above method; finally, the following method can be used to determine the label dependency relationship between the text information and the label data set according to all label dependency values, that is, the set of all label dependency values is taken as the label dependency relationship between the text information and the label data set.

[0068] It should be noted that in this application, the label dependency relationship represents the mutual influence relationship between the features in the text and the labels in the label data set; the associated label refers to the label in the label data set that has similar semantics with the text keyword; the correlation coefficient represents the strength and direction of the linear relationship between the text keyword and the associated label; and the label dependency value refers to the quantitative value of the dependency degree between the associated label and the specific text feature.

[0069] In some embodiments, the determination of the text label features of the semantic word units in the target accounting document by the label dependency relationship and the semantic loss of each text keyword in semantic segmentation can be implemented by the following steps:

[0070] For each text keyword, the semantic loss of the text keyword in semantic segmentation is extracted;

[0071] The label dependency value of the text keyword is obtained from the label dependency relationship;

[0072] The semantic word unit label of the text keyword is determined by the label dependency value and the semantic loss, and the semantic word unit label of each text keyword is obtained;

[0073] The text label features of the semantic word units in the target accounting document are determined according to all the semantic word unit labels.

[0074] It should be noted that in the present application, the text label features are features describing the semantic relationship between the text content and the label; the semantic loss represents the loss degree of the original semantic information in semantic segmentation, and a smaller semantic loss means better information retention and more accurate semantic segmentation performance; the label dependency value represents the correlation or dependency degree between the text features and the label data set; the semantic word unit label is a unit label used to capture the key theme in the text.

[0075] In a specific implementation, firstly, for each text keyword, the semantic loss of the text keyword in semantic segmentation can be realized by the following method, that is, for each text keyword, the absolute value of the difference between the correlation coefficient of the text keyword and 1 is taken as the semantic loss of the text keyword in semantic segmentation; secondly, the label dependency value of the text keyword is obtained from the label dependency relationship; then, the semantic word unit label of the text keyword is determined through the label dependency value and the semantic loss, and the semantic word unit label of each text keyword can be realized by the following method, that is, for each associated label, the correlation coefficient between the associated label and the text keyword is filtered from the value range of the label dependency value, so that the ratio of the correlation coefficient to the semantic loss is taken as the semantic dependency value between the associated label and the text keyword, and the semantic dependency value between each associated label and the text keyword can be obtained by the above method, so that the associated label with the maximum semantic dependency value is taken as the semantic word unit label of the text keyword, if the absolute value of the difference between the semantic dependency value of other associated labels and the semantic dependency value of the semantic word unit label is less than 0.01, the above all associated labels can be combined as the semantic word unit label of the text keyword by using a graph sorting algorithm (for example: TextRank), and the semantic word unit label of each text keyword can be obtained by the above method; finally, the text label feature of the semantic word unit in the target accounting document can be realized by the following method, that is, the set of all semantic word unit labels is taken as the text label feature of the semantic word unit in the target accounting document.

[0076] In step 103, the semantic features of the voucher image and each data label in the label data set are semantically associated and matched to obtain the semantic inline relationship between the voucher image and the label data set, and the image label feature of the semantic word unit in the target accounting document is determined based on the semantic inline relationship combined with the self-attention mechanism.

[0077] In some embodiments, the semantic features of the voucher image and each data label in the label data set are semantically associated and matched to obtain the semantic inline relationship between the voucher image and the label data set can be realized by the following steps:

[0078] For each data label in the label data set, the label semantics of the data label and the semantic features of the voucher image are extracted;

[0079] The label semantics and the semantic features are matched to obtain the semantic association value of the data label, and then the semantic association value of each data label in the label data set is obtained;

[0080] The semantic inline relationship between the voucher image and the label data set is determined through all the semantic association values.

[0081] In a specific implementation, firstly, for each data label in the label data set, the label semantics of the data label and the semantic features of the credential image can be implemented in the following manner: for each data label in the label data set, taking the label semantic vector corresponding to the data label as the label semantics of the data label, using a convolutional neural network to extract the content features of the credential image, mapping the content features to a semantic space through an embedding layer, and taking all semantic token vectors output by the semantic space as the semantic features of the credential image; then, matching the label semantics and the semantic features to obtain the semantic correlation value of the data label, and further obtaining the semantic correlation value of each data label in the label data set can be implemented in the following manner: the label semantics and the semantic features can be matched using a similarity evaluation algorithm, that is, taking the cosine similarity between each semantic token vector in the label semantics and the semantic features as the matching result, so that the matching result is taken as the value range of the semantic correlation value, and thus the semantic correlation value of the data label is obtained, and the semantic correlation value of each data label in the label data set is obtained through the above manner; finally, the semantic in-link relationship between the credential image and the label data set can be determined through all the semantic correlation values in the following manner: taking the set of all semantic correlation values as the semantic in-link relationship between the credential image and the label data set.

[0082] It should be noted that in the present application, the semantic in-link relationship represents the inherent correlation between the credential image and the label data set at the semantic level; the label semantics of the data label represents the actual meaning conveyed by each label in the label data set; the semantic features of the credential image are features describing the semantic information in the credential image; and the semantic correlation value represents the semantic correlation degree between the data label and the credential image.

[0083] In some embodiments, based on the semantic in-link relationship, the image label features of the semantic tokens in the target accounting credential are determined in combination with a self-attention mechanism, and the image label features of the semantic tokens in the target accounting credential are determined in combination with the self-attention mechanism. Figure 3 The graph is a flowchart for determining the image label features in some embodiments of the present application, and the determination of the image label features in the present embodiment can be implemented in the following steps:

[0084] In step 1031, an association degree matrix of the semantic tokens in the credential image is constructed through the semantic in-link relationship.

[0085] In step 1032, the feature of all semantic correlation values in the association degree matrix is weighted based on the self-attention mechanism, and the image label features of the semantic tokens in the target accounting credential are obtained.

[0086] It should be noted that in the present application, the image label features are features that can reflect the semantic relationship between the image content and the label; and the association degree matrix is a matrix quantifying the correlation between the image and the label features.

[0087] In a specific implementation, first, the construction of the correlation degree matrix of the semantic word units in the voucher image through the semantic inline relationship can be implemented in the following manner: initializing an empty relationship matrix, each row of the matrix representing a semantic word unit vector, and each column of the matrix representing a data label; filling the semantic correlation values in the semantic inline relationship into the relationship matrix, so that the filled relationship matrix can be used as the correlation degree matrix of the semantic word units in the voucher image; then, the feature weighting of all semantic correlation values in the correlation degree matrix based on the self-attention mechanism can be implemented in the following manner: using the self-attention mechanism to weight each semantic word unit vector and the label semantics, the self-attention mechanism can help the model determine which labels are more important and which semantic word units are more critical among multiple labels and image semantic word units; for each semantic correlation value in the correlation degree matrix, the product of the weights of the semantic correlation value corresponding semantic word unit vector and the label semantics is used as the weight of the semantic correlation value, so that the weight of each semantic correlation value in the correlation degree matrix can be obtained through the above manner, and the weighted sum of all semantic correlation values in the correlation degree matrix is calculated as the image label feature of the semantic word units in the target accounting voucher.

[0088] In step 104, the image label feature and the text label feature are modality fused, and then a trusted label of the target accounting voucher is generated based on the modality fused label feature. The trusted label is used for multi-label classification of the target accounting voucher.

[0089] In some embodiments, the modality fusion of the image label feature and the text label feature using the tensor fusion algorithm can be implemented in the following steps:

[0090] Initializing an attention mechanism based tensor fusion model;

[0091] Using the attention mechanism to weight the image and text in the target accounting voucher, and using the image label feature and the image label feature as the fusion objects of the tensor fusion model;

[0092] Using the tensor fusion model to perform modality fusion of the labels, and using the set of modality fused labels as the modality fused label feature.

[0093] It should be noted that in the present application, the tensor fusion model is a model for realizing multi-modal data fusion by combining information of multiple modalities (such as images and texts). The tensor fusion model learns the importance and correlation between different modalities by representing image label features and text label features as high-dimensional tensors respectively and weighting the image label features and the text label features by using an attention mechanism. The attention mechanism can automatically adjust the weight of each modality, so that the tensor fusion model can focus on more important parts when performing fusion. Specifically, the label features of images and texts are mapped to tensor form and are merged by weighting through the attention mechanism. Finally, the modalities are fused by tensor multiplication, addition or other operations. The result of the fusion is the set of all labels as the final label features, which represents the comprehensive semantic information of the target accounting document under multiple modalities. The tensor fusion model can effectively improve the label expression ability after modal fusion and provide more rich and accurate feature representation for subsequent label prediction and classification.

[0094] In some embodiments, generating the trusted label of the target accounting document based on the label features after modal fusion can be realized in the following manner: after semantic deduplication of all labels after modal fusion, the labels are used as the trusted label of the target accounting document. It should be noted that in the present application, the trusted label refers to a label list that can effectively reflect the real information in the target accounting document.

[0095] In some embodiments, using the trusted label to perform multi-label classification of the target accounting document can be realized in the following manner: the content in the trusted label is used as the classification label of the target accounting document. A mapping table of classification labels and document categories is obtained from the current accounting document classification system. The document category corresponding to the classification label in the trusted label is obtained from the mapping table, thereby completing the multi-label classification of the target accounting document.

[0096] In addition, another aspect of the present application provides an accounting document classification system based on artificial intelligence in some embodiments, referring to Figure 4 The figure is a structural schematic diagram of an accounting document classification system based on artificial intelligence according to some embodiments of the present application. The accounting document classification system based on artificial intelligence includes an acquisition module 201, a processing module 202 and an execution module 203, which are described as follows:

[0097] The acquisition module 201 is mainly used to acquire the label data set of the accounting document and the target accounting document to be classified, and then extract the text information and the document image in the target accounting document in the present application.

[0098] The processing module 202 is configured to perform semantic segmentation on the text information to obtain a plurality of text keywords of the target accounting voucher, and then determine a label dependency relationship between the text information and the label data set according to the part-of-speech of an effective word element in each text keyword, and determine a text label feature of a semantic word element in the target accounting voucher through the label dependency relationship and semantic loss of each text keyword in semantic segmentation;

[0099] It should be noted that the processing module 202 is further configured to perform semantic association matching between the semantic features of the voucher image and each data label in the label data set to obtain a semantic inline relationship between the voucher image and the label data set, and determine an image label feature of a semantic word element in the target accounting voucher based on the semantic inline relationship and a self-attention mechanism;

[0100] The execution module 203 is mainly configured to perform modality fusion on the image label feature and the text label feature, and then generate a trusted label of the target accounting voucher based on the fused label feature, and perform multi-label classification on the target accounting voucher using the trusted label.

[0101] The above describes an example of the accounting voucher classification method and system based on artificial intelligence provided by the embodiments of the present application in detail. It can be understood that the corresponding device contains the hardware structure and / or software module corresponding to the execution of each function in order to achieve the above functions. Those skilled in the art should easily realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0102] In some embodiments, the present application also provides a computer device, which comprises a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the above-mentioned accounting voucher classification method based on artificial intelligence.

[0103] In some embodiments, with reference to Figure 5 The dashed line in the figure indicates that the unit or module is optional, and the figure is a structural schematic diagram of a computer device for implementing the accounting voucher classification method based on artificial intelligence according to the embodiments of the present application. The accounting voucher classification method based on artificial intelligence described in the above embodiments can be implemented by Figure 5The computer device shown is implemented by a computer device including at least one processor 301, a memory 302, and at least one communication unit 305, and can be a terminal device or a server or a chip.

[0104] The processor 301 can be a general-purpose processor or a special-purpose processor. For example, the processor 301 can be a central processing unit (CPU), which can be used to control the computer device, execute a software program, and process data of the software program. The computer device can further include a communication unit 305 to implement input (reception) and output (transmission) of signals.

[0105] For example, the computer device can be a chip, and the communication unit 305 can be an input and / or output circuit of the chip, or the communication unit 305 can be a communication interface of the chip. The chip can be a component of a terminal device or a network device or other device.

[0106] For another example, the computer device can be a terminal device or a server, and the communication unit 305 can be a transceiver of the terminal device or the server, or the communication unit 305 can be a transceiver circuit of the terminal device or the server.

[0107] One or more memories 302 can be included in the computer device, and programs 304 can be stored on the memories 302. The programs 304 can be run by the processor 301 to generate instructions 303, so that the processor 301 executes the methods described in the above method embodiments according to the instructions 303. Alternatively, data (such as a target review model) can also be stored in the memory 302. Alternatively, the processor 301 can also read data stored in the memory 302. The data can be stored in the same storage address as the programs 304, or the data can be stored in different storage addresses from the programs 304.

[0108] The processor 301 and the memory 302 can be separately arranged or integrated together, for example, integrated on a system on chip (SOC) of the terminal device.

[0109] It should be understood that each step of the above method embodiments can be accomplished by logic circuitry in the form of hardware within the processor 301 or by instructions in the form of software, and the processor 301 can be a CPU, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof.

[0110] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0111] For example, in some embodiments, the present application also provides a computer readable storage medium, wherein instructions or codes are stored in the computer readable storage medium, and when the instructions or codes are run on a computer, the computer is caused to perform the above-mentioned artificial intelligence-based accounting voucher classification method.

[0112] Although preferred embodiments of the application have been described, those skilled in the art will appreciate that additional modifications and variations to the preferred embodiments can be made without departing from the spirit and scope of the application. Accordingly, it is intended that the appended claims be interpreted as including all such alterations and modifications as fall within the scope of the application.

[0113] Obviously, various modifications and changes are possible in the present application without departing from the scope and spirit of the application. Accordingly, it is intended that the application embrace all modifications and changes that fall within the scope of the claims and their equivalents.

Claims

1. An artificial intelligence-based accounting document classification method, characterized by, The method comprises the following steps: obtaining a label data set of an accounting voucher and a target accounting voucher to be classified, and then extracting text information and a voucher image in the target accounting voucher, wherein the label data set represents a label set of a group of labeled accounting vouchers, and each data instance is associated with one or more data labels; performing semantic segmentation on the text information to obtain a plurality of text keywords of the target accounting voucher, and then determining a label dependency relationship between the text information and the label data set according to the part-of-speech of the valid word units in each text keyword, and determining a text label feature of a semantic word unit in the target accounting voucher through the label dependency relationship and the semantic loss of each text keyword in semantic segmentation; performing semantic association matching between the semantic features of the voucher image and each data label in the label data set to obtain a semantic inline relationship between the voucher image and the label data set, and determining an image label feature of a semantic word unit in the target accounting voucher based on the semantic inline relationship combined with a self-attention mechanism, wherein the semantic inline relationship represents the internal association between the voucher image and the label data set at the semantic level; performing modal fusion on the image label feature and the text label feature, and then generating a trusted label of the target accounting voucher based on the label feature after modal fusion, and using the trusted label to perform multi-label classification on the target accounting voucher; wherein determining the label dependency relationship between the text information and the label data set according to the part-of-speech of the valid word units in each text keyword specifically comprises: filtering all associated labels from the label data set based on the semantic features of the text information of the target accounting voucher; for each text keyword, determining the correlation coefficient between the text keyword and each associated label according to the part-of-speech feature of the text keyword, wherein the part-of-speech feature represents the grammatical function and role of the word in natural language processing; determining the label dependency value between the text keyword and the label data set through all correlation coefficients, and then obtaining the label dependency value between each text keyword and the label data set; determining the label dependency relationship between the text information and the label data set according to all label dependency values; wherein the modal fusion of the image label feature and the text label feature is realized by the following steps: initializing a tensor fusion model based on an attention mechanism; using the attention mechanism to weight the image and text in the target accounting voucher, and taking the image label feature and the image label feature as the fusion objects of the tensor fusion model; using the tensor fusion model to perform modal fusion of the labels, and taking the set of all labels after modal fusion as the label feature after modal fusion.

2. The method of claim 1, wherein, performing semantic segmentation on the text information to obtain a plurality of text keywords of the target accounting voucher specifically comprises: extracting a plurality of segmented word groups from the text information; determining the semantic association degree between each segmented word group and the text information; filtering the plurality of text keywords of the target accounting voucher from all segmented word groups through each semantic association degree.

3. The method of claim 1, wherein, The text label features of the semantic word units in the target accounting document are determined according to the semantic loss of each text keyword in semantic segmentation and the label dependency relationship, and specifically include: For each text keyword, the semantic loss of the text keyword in semantic segmentation is extracted; The label dependency value of the text keyword is obtained from the label dependency relationship; The semantic word unit label of the text keyword is determined through the label dependency value and the semantic loss, and then the semantic word unit label of each text keyword is obtained; The text label features of the semantic word units in the target accounting document are determined according to all semantic word unit labels.

4. The method of claim 1, wherein, The semantic features of the document image and each data label in the label data set are semantically associated and matched to obtain the semantic in-line relationship between the document image and the label data set, and specifically include: For each data label in the label data set, the label semantics of the data label and the semantic features of the document image are extracted; The semantic association value of the data label is obtained by matching the label semantics and the semantic features, and then the semantic association value of each data label in the label data set is obtained; The semantic in-line relationship between the document image and the label data set is determined through all semantic association values.

5. The method of claim 1, wherein, Based on the semantic in-line relationship, the image label features of the semantic word units in the target accounting document are determined in combination with the self-attention mechanism, and specifically include: An association degree matrix of the semantic word units in the document image is constructed through the semantic in-line relationship; Based on the self-attention mechanism, all semantic association values in the association degree matrix are weighted to obtain the image label features of the semantic word units in the target accounting document.

6. The method of claim 1, wherein, The image label features and the text label features are fused using a tensor fusion algorithm.

7. An artificial intelligence-based accounting document classification system that classifies accounting documents using the method of any one of claims 1 to 6, characterized by, The system includes: An acquisition module is configured to acquire a label data set of an accounting document and a target accounting document to be classified, and then extract text information and a document image in the target accounting document; A processing module is configured to perform semantic segmentation on the text information to obtain a plurality of text keywords of the target accounting document, and then determine a label dependency relationship between the text information and the label data set according to the part-of-speech of the effective word units in each text keyword, and determine text label features of semantic word units in the target accounting document through the label dependency relationship and the semantic loss of each text keyword in semantic segmentation; The processing module is further configured to perform semantic association matching between the semantic features of the document image and each data label in the label data set to obtain the semantic in-line relationship between the document image and the label data set, and determine the image label features of the semantic word units in the target accounting document in combination with the self-attention mechanism based on the semantic in-line relationship; An execution module is configured to fuse the image label features and the text label features, and then generate a trusted label of the target accounting document based on the fused label features, and use the trusted label to perform multi-label classification on the target accounting document.

8. A computer device, comprising: The computer device comprises a memory for storing a computer program and a processor for calling and running the computer program from the memory, so that the computer device executes the artificial intelligence-based accounting voucher classification method in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions or codes, which, when running on a computer, enable the computer to execute the artificial intelligence-based accounting voucher classification method in any one of claims 1 to 6.

Images