Automatic Bill Recognition Method, Device, Computer Equipment and Storage Medium

Abstract

The embodiments of this application belong to the fields of artificial intelligence and digital medicine, and are applied in the field of intelligent medicine. It involves a method for automatic bill recognition, including inputting a training data set into a constructed BERT network model to obtain surface features, syntactic features, and semantic features, splicing them to obtain spliced features, outputting the prediction result of the BERT network model based on the spliced features, calculating a first loss function according to the prediction result, iteratively updating the BERT network model according to the first loss function, outputting the BERT network model as a bill recognition model, performing optical character recognition on the bill to be recognized to obtain text data, inputting the text data into the bill recognition model, outputting a classification result, and determining the field type of the text data. This application also provides a device for automatic bill recognition, a computer device, and a storage medium. In addition, this application also involves blockchain technology, and the bill to be recognized can be stored in the blockchain. This application can automatically recognize the field category corresponding to the text data.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence technology and digital medical technology, and in particular, to a method, device, computer device and storage medium for automatically identifying bills. Background Art

[0002] With the country's strong promotion of the construction of the national basic medical network and the off-site medical settlement system, the proportion of manual reimbursement has been decreasing year by year, but local and off-site sporadic reimbursements have long existed. Using manual reimbursement has high costs, is error-prone, and there are also problems such as high reimbursement pressure during the reimbursement peak period and difficulty in matching the three medical insurance catalogs.

[0003] In response to this, intelligent bill recognition technology is introduced to identify paper invoices. Currently, intelligent bill recognition technology mainly uses OCR technology. Although OCR technology can already detect the text of medical invoices and medical hospitalization lists, and accurately recognize the text content inside the high-precision detection frame, for medical bills of different formats, manual post-processing is still required to connect to the medical insurance bureau system for operations such as medical insurance reimbursement, and there are still problems such as cumbersome processes, large workloads, and low efficiency. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to propose a method, device, computer device and storage medium for automatically identifying bills to solve the technical problems in the related art that the applicability of bill recognition is low, resulting in cumbersome and inefficient subsequent operation processes.

[0005] To solve the above technical problems, the embodiments of the present application provide a method for automatically identifying bills, which adopts the following technical solutions:

[0006] Obtain an original bill data set, and preprocess the original bill data set to obtain a training data set;

[0007] Input the training data set into the constructed BERT network model to obtain surface features, syntactic features and semantic features;

[0008] Concatenate the surface features, the syntactic features and the semantic features to obtain a concatenated feature;

[0009] Output the prediction result of the BERT network model based on the concatenated feature;

[0010] Calculate a first loss function according to the prediction result, iteratively update the BERT network model according to the first loss function, and determine whether the iteratively updated BERT network model converges. When the iteratively updated BERT network model converges, output the BERT network model as a bill recognition model;

[0011] Obtain the bill to be recognized, perform optical character recognition on the bill to be recognized, and obtain the text data corresponding to the bill to be recognized;

[0012] Input the text data into the bill recognition model, output the classification result, and obtain the field type of the text data according to the classification result.

[0013] Further, the obtaining the surface feature, syntactic feature and semantic feature by inputting the training data set into the constructed BERT network model includes:

[0014] Input the training data set into the input layer of the BERT network model for vector conversion to obtain the feature vector data corresponding to the training data set;

[0015] Extract the surface feature, syntactic feature and semantic feature from the feature vector data through the feature extraction layer of the BERT network model.

[0016] Further, the feature extraction layer includes a bottom network, an intermediate network and a high-level network. The extracting the surface feature, syntactic feature and semantic feature from the feature vector data through the feature extraction layer of the BERT network model includes:

[0017] Extract the surface feature from the text feature vector through the bottom network, and input the surface feature into the intermediate network and the high-level network respectively;

[0018] Extract the syntactic feature from the surface feature through the intermediate network, and input the syntactic feature into the high-level network;

[0019] Extract the semantic feature from the surface feature and the syntactic feature through the high-level network.

[0020] Further, the outputting the prediction result of the BERT network model based on the concatenated feature includes:

[0021] Use the softmax layer of the BERT network model to predict the concatenated feature to obtain the prediction probability of each label corresponding to the training data set;

[0022] Obtain the prediction result according to the prediction probability.

[0023] Further, the iteratively updating the BERT network model according to the first loss function and determining whether the iteratively updated BERT network model converges includes:

[0024] Adjust the model parameters of the BERT network model according to the first loss function to obtain an intermediate BERT network model;

[0025] Calculate the second loss function of the intermediate BERT network model, and determine whether the intermediate BERT network model converges according to the first loss function and the second loss function.

[0026] Further, the determining whether the intermediate BERT network model converges according to the first loss function and the second loss function includes:

[0027] Calculate the difference between the first loss function and the second loss function. If the difference is within a preset range, the intermediate BERT network model converges.

[0028] Further, the preprocessing the original bill dataset to obtain a training dataset includes:

[0029] Perform field conversion on the original bill data in the original bill dataset to obtain an original bill dataset containing field names and corresponding field values, where the field names are the training labels corresponding to the field values;

[0030] Extract the preprocessed original bill dataset according to a preset ratio to obtain a training dataset.

[0031] To solve the above technical problems, an embodiment of the present application further provides a bill automatic recognition device, which adopts the following technical solutions:

[0032] An acquisition module, configured to acquire an original bill dataset, and preprocess the original bill dataset to obtain a training dataset;

[0033] A feature module, configured to input the training dataset into a constructed BERT network model to obtain surface features, syntactic features, and semantic features;

[0034] A splicing module, configured to splice the surface features, the syntactic features, and the semantic features to obtain spliced features;

[0035] A prediction module, configured to output a prediction result of the BERT network model based on the spliced features;

[0036] An iterative update module, configured to calculate a first loss function according to the prediction result, perform iterative update on the BERT network model according to the first loss function, and determine whether the iteratively updated BERT network model converges. When the iteratively updated BERT network model converges, output the BERT network model as a bill recognition model;

[0037] An identification module, configured to obtain a bill to be identified, perform optical character recognition on the bill to be identified, and obtain text data corresponding to the bill to be identified;

[0038] A classification module, configured to input the text data into a bill recognition model, output a classification result, and obtain the field type of the text data according to the classification result.

[0039] To solve the above technical problems, an embodiment of the present application further provides a computer device, which adopts the following technical solutions:

[0040] The computer device includes a memory and a processor. Computer-readable instructions are stored in the memory. When the processor executes the computer-readable instructions, the steps of the above-mentioned automatic bill recognition method are implemented.

[0041] To solve the above technical problems, an embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solutions:

[0042] Computer-readable instructions are stored on the computer-readable storage medium. When the computer-readable instructions are executed by a processor, the steps of the above-mentioned automatic bill recognition method are implemented.

[0043] Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects:

[0044] In the present application, by obtaining an original bill data set, preprocessing the original bill data set to obtain a training data set, inputting the training data set into a constructed BERT network model, obtaining surface features, syntactic features, and semantic features, splicing the surface features, syntactic features, and semantic features to obtain a spliced feature, outputting a prediction result of the BERT network model based on the spliced feature, calculating a first loss function according to the prediction result, iteratively updating the BERT network model according to the first loss function, and determining whether the iteratively updated BERT network model converges. When the iteratively updated BERT network model converges, outputting the BERT network model as a bill recognition model, obtaining a bill to be identified, performing optical character recognition on the bill to be identified to obtain text data corresponding to the bill to be identified, inputting the text data into the bill recognition model, outputting a classification result, and obtaining the field type of the text data according to the classification result; the present application can automatically process bills that have undergone optical character recognition through the bill recognition model. For the text data in each text box of the bill, the field category corresponding to the text data can be automatically recognized; at the same time, using the trained bill recognition model to replace the manual post-processing code increases the readability of the code, is easy to maintain, and can also reduce the deployment pressure and reduce unforeseen situations such as online operation failures. Description of the Drawings

[0045] To more clearly illustrate the solutions in this application, the following will briefly introduce the accompanying drawings required for the description of the embodiments of this application. Obviously, the accompanying drawings in the following description are some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0046] Figure 1 is an exemplary system architecture diagram to which this application can be applied;

[0047] Figure 2 A flowchart of an embodiment of the automatic bill recognition method according to this application;

[0048] Figure 3 is a schematic structural diagram of an embodiment of the automatic bill recognition device according to this application;

[0049] Figure 4 is a schematic structural diagram of an embodiment of the computer device according to this application. Detailed implementation manners

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs; the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above accompanying drawing descriptions are intended to cover non-exclusive inclusion. The terms "first", "second", etc. in the specification and claims of this application or the above accompanying drawings are used to distinguish different objects and not to describe a specific order.

[0051] Referring to "embodiments" herein means that specific features, structures, or characteristics described in connection with the embodiments can be included in at least one embodiment of this application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0052] To enable those skilled in the technical field to better understand the solutions of this application, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings.

[0053] This application provides an automatic bill recognition method that can be applied to, for example, Figure 1In the system architecture 100 shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0054] Users can use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0055] The terminal devices 101, 102, 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, and desktop computers, etc.

[0056] The server 105 can be a server that provides various services, such as a background server that supports the pages displayed on the terminal devices 101, 102, 103.

[0057] It should be noted that the automatic bill recognition method provided by the embodiments of the present application is generally executed by the server. Correspondingly, the automatic bill recognition device is generally set in the server.

[0058] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results.

[0059] Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technologies mainly include several major directions such as computer vision technology, robotics, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0060] It should be understood that Figure 1 the numbers of the terminal devices, networks, and servers in

[0061] Continuing to refer to Figure 2 , a flowchart of an embodiment of a method for automatic bill recognition according to the present application is shown. The automatic bill recognition method includes the following steps:

[0062] Step S201, obtaining an original bill data set and preprocessing the original bill data set to obtain a training data set.

[0063] In this embodiment, the original bill data set is a bill picture of a medical bill, which can be obtained by connecting various insurance companies and manually inputting. When transmitting the medical bill picture, the bill source, bill user, bill type, bill belonging region, and bill personal information can be transmitted together.

[0064] The steps of preprocessing the original bill data set to obtain a training data set include:

[0065] Using the field assignment method to perform field conversion on the original bill data in the original bill data set to obtain an original bill data set including field names and corresponding field values, where the field names are the training labels corresponding to the field values;

[0066] Extracting the preprocessed original bill data set according to a preset ratio to obtain a training data set.

[0067] Preprocessing the obtained original bill picture into "field names" and corresponding "field value formats", which can be expressed as {field name: field value}, to obtain an original bill data set including field names and corresponding field values. The field names are the training labels corresponding to the field values; for example: {Name: Wang Er}, {Invoice Number: 0443911990}, {Amount: 10.00}, {Specification: ml}, {Drug Name: Cefdinir Dispersible Tablets}, {Date: 20190305}, {Hospital Name: Nanjing Children's Hospital}, {Medical Insurance Type: Self-Paid}, {Gender: Male}, etc.

[0068] The field names are used as the field types corresponding to the field values. Different types of data should be as balanced and comprehensive as possible. The original bill data set is divided into a training data set and a test data set according to a certain ratio. For example, the training data set and the test data set are extracted according to a ratio of 9:1.

[0069] Step S202, inputting the training data set into the constructed BERT network model to obtain surface features, syntactic features, and semantic features.

[0070] The BERT (Bidirectional Encoder Representations from Transformers) network model aims to train and learn the rich semantic information expressed in the text using a large-scale unannotated corpus, and then fine-tune the semantic expression of the text in a specific NLP (Natural Language Processing) task, and finally apply it to this NLP task.

[0071] In this embodiment, the BERT network model includes at least an input layer, a feature extraction layer, and an output layer. Specifically, the training data set is input into the input layer of the BERT network model for vector conversion to obtain feature vector data corresponding to the training data set, and the feature extraction layer of the BERT network model extracts features from the feature vector data to obtain surface features, syntactic features, and semantic features.

[0072] In this embodiment, before inputting the training data into the BERT network model for feature extraction, it is necessary to perform vector conversion on the training data to obtain a text sequence. Specifically, a predefined start symbol is added before the first training data, and a number of semantic unit symbols corresponding to each training data are determined to obtain a text sequence. It should be noted that the predefined start symbol can be set manually, such as [start] or [CLS] or [C]. For the determination of the number of semantic unit symbols, the several characters included in each training data can be determined as the above-mentioned several semantic unit symbols, or each training data can be tokenized, and the obtained several tokens can be determined as the above-mentioned several semantic unit symbols. This means that the above-mentioned semantic unit symbols (tokens) can be characters or words in the training data.

[0073] A predefined separator symbol can also be added between every two adjacent training data, where the predefined separator symbol is used to separate two adjacent training data. The predefined separator symbol can be set manually, for example, it can be set as [SEP] or [S], etc. For example, the determined semantic symbol sequence is:

[0074] [CLS]Tok1_1Tok1_2[SEP]Tok2_1Tok2_2[SEP]Tok3_1...

[0075] Among them, [CLS] represents the predefined start symbol, Tokx_x represents the semantic unit symbol, and [SEP] represents the predefined separator symbol, where [SEP] can be more than two, so as to expand the number of input training data from two to more than two.

[0076] Convert the training data into vectors to obtain semantic unit (Token) vectors, text (Segment) vectors, and position (Position) vectors, which represent word information, sentence information, and position information respectively. Use the Token vectors, Segment vectors, and Position vectors as feature vector data.

[0077] In some alternative implementation manners of this embodiment, the above-mentioned feature extraction layer of the BERT network model extracts features from the feature vector data to obtain surface features, syntactic features, and semantic features, including:

[0078] Extract surface features from the text feature vectors through the underlying network, and input the surface features into the intermediate network and the high-level network respectively;

[0079] Extract syntactic features from the surface features through the intermediate network, and input the syntactic features into the high-level network;

[0080] Extract semantic features from the surface features and syntactic features through the high-level network.

[0081] The feature extraction layer includes a 12-layer network structure stacked together. Each layer of the network structure adopts the decoder (encoder) part of the Transformer. Each layer of the encoder includes a layer of multi-head attention mechanism (Muti-Head-Attention) and a layer of feed-forward neural network (Position-Wise-Feed-Forward-Networks). Among them, the 12-layer network structure contains 1 to 4 layers of underlying networks, 5 to 8 layers of intermediate networks, and 9 to 12 layers of high-level networks. The underlying networks are used to learn surface features at the phrase level, the intermediate networks are used to learn syntactic features at the syntactic level, and the high-level networks are used to learn semantic features at the semantic level.

[0082] In this embodiment, it is necessary to transform the feature extraction layer of the BERT network model, connect the 4th layer network, the 8th layer network, and the 12th layer network. That is, the input of the 8th layer network structure is the output of the 7th layer network structure and the output of the 4th layer network structure, and the input of the 12th layer network structure is the output of the 11th layer network structure and the output of the 8th layer network structure. The purpose is to perform feature enhancement to make the result of classification prediction more accurate.

[0083] Step S203: Concatenate the surface features, syntactic features, and semantic features to obtain concatenated features.

[0084] In this embodiment, when concatenating the surface feature X1, the syntactic feature X2, and the semantic feature X3, the specific concatenation formula that can be used is as follows:

[0085]

[0086] Among them, X m is a splicing feature, and is a vector splicing operation.

[0087] Step S204: Output the prediction result of the BERT network model based on the splicing feature.

[0088] Specifically, use the softmax layer of the BERT network model to predict the splicing feature, obtain the prediction probability of each label corresponding to the training data set, and obtain the prediction result according to the prediction probability.

[0089] In this embodiment, the splicing feature can be input into the softmax layer through a fully connected layer for prediction, and the prediction probability of each label corresponding to the training data is obtained. The calculation formula of the prediction probability is as follows:

[0090] Pr = softmax(W·X m +b)

[0091] Among them, X m is the splicing feature, W and b are the trainable weight matrix and bias of softmax respectively, and Pr is the prediction probability.

[0092] The splicing feature is input into the softmax layer to predict the prediction probabilities corresponding to different labels of the text data, and the probability values are between 0 and 1. Specifically, the predicted probability vector P = {p1, p2,..., pη} is output, where pη represents the probability that the output feature belongs to the ηth category, and η is the predicted category.

[0093] It should be noted that the fully connected layer can be constructed separately and connected between the 12th layer network structure and the softmax layer, or can be implemented by the feedforward neural network of the 12th layer network structure.

[0094] After calculating the prediction probability, compare the prediction probabilities of each label, and the label with the largest prediction probability is the category corresponding to the training data.

[0095] Step S205: Calculate the first loss function according to the prediction result, iteratively update the BERT network model according to the first loss function, and determine whether the iteratively updated BERT network model converges. When the iteratively updated BERT network model converges, output the BERT network model as the bill recognition model.

[0096] In this embodiment, the calculation formula of the loss function is as follows:

[0097]

[0098] Where N is the number of training data, and y i represents the output result of the i-th training data, and Pr(y i ) represents the predicted probability corresponding to y i .

[0099] In some alternative implementation manners of this embodiment, the above-mentioned iterative update of the BERT network model according to the first loss function and determining whether the iteratively updated BERT network model converges includes:

[0100] Adjust the model parameters of the BERT network model according to the first loss function to obtain an intermediate BERT network model;

[0101] Calculate the second loss function of the intermediate BERT network model, and determine whether the intermediate BERT network model converges according to the first loss function and the second loss function.

[0102] Adjust the model parameters according to the loss function and continue iterative training. When the model is trained to a certain extent, at this time, the performance of the model reaches the optimal state and the loss function cannot continue to decrease, that is, it converges. The specific way to determine convergence is to calculate the difference between the first loss function and the second loss function. If the difference is within the preset range, the intermediate BERT network model converges.

[0103] Specifically, the way to determine convergence is to only calculate the loss function in the previous and next two rounds of iteration. If the loss function is still changing, the training data set can continue to be selected and input into the BERT network model to continue iterative training of the BERT network model; if the loss function does not change significantly, it can be considered that the model converges. At this time, it is determined that the training of the BERT network model is completed, and the training is stopped, and the final BERT network model is output as the bill recognition model.

[0104] Step S206: Obtain the bill to be recognized, perform optical character recognition on the bill to be recognized, and obtain the text data corresponding to the bill to be recognized.

[0105] The bill to be recognized may include value-added tax bills, bank receipt bills, special invoices, toll bridge bills, shopping receipt bills, medical bills, bank acceptance bills, etc. The bill to be recognized in this embodiment is specifically a medical bill. The obtained bill to be recognized is analyzed and recognized through optical character recognition (OCR) technology, and the text data on the bill is returned. The text data may include the bill text information and the corresponding positions, table layouts, and regions, etc.

[0106] OCR technology can use Baidu's OCR text recognition interface or the Google open-source OCR engine Tesseract. Specifically, the bill to be recognized is scanned through OCR technology to obtain the original bill image of the bill; then the bill information of the original bill image is collected; this information collection process includes preprocessing processes such as image grayscale conversion, noise reduction, binarization, and character segmentation operations.

[0107] For preprocessing, it is mainly to reduce the useless information of the image, obtain the effective image, and facilitate subsequent processing. Specifically, when preprocessing the original bill image, first, the original bill image is divided into regions (the main way of region division is to identify the line information in the bill table), and the effective region including the effective bill information is divided to obtain the layout feature. Among them, the image of this effective region can be called the effective image, and then the subsequent character feature extraction and recognition are performed on this effective image to obtain the text recognition feature. For example, for a certain bill, first, the line rule feature of the table is recognized. Based on this line rule feature, the bill can be divided to obtain the layout feature, then the effective image is obtained, and then the effective image of this part is processed specifically to obtain the text recognition feature, thereby reducing the data calculation and processing volume and improving the recognition efficiency.

[0108] Of course, other steps can also be included in image preprocessing, such as operations like image binarization, noise reduction, and character segmentation. Among them, binarization means converting the original bill image into a binary image containing only two black and white colors. This is because the amount of information contained in a color image is huge, and the amount of calculation during subsequent processing will also increase accordingly, and the time spent will also be longer. Therefore, to improve the efficiency of processing and recognition, the color image can be pre-binarized first; noise reduction is to solve the problem of the decline in image quality caused by noise interference in the actual image through a filter or other means; and character segmentation means dividing a series of characters into individual characters (or words, numbers) with definite meanings and then recognizing them.

[0109] When the effective image is obtained, character feature extraction can be performed on the effective image. Character features can be said to be the main basis for recognition. Briefly, they can be divided into two categories: one is statistical features, such as the ratio of black / white dots in the text area. When the text is divided into several regions, the combination of the black / white dot ratios of these individual regions becomes a numerical vector in space. In comparison, basic mathematical theories are sufficient to handle it; the other type of feature is structural features, such as after the text image is thinned, obtaining the number and position of the stroke endpoints and intersection points of the character, or using stroke segments as features.

[0110] After obtaining the character features of the bill information, the corresponding comparison method can be used to match the character features with the alternative features in the character library, so as to find the closest feature, and determine the meaning of the bill information according to the character meaning of the closest feature, and then obtain the text data corresponding to the bill.

[0111] It should be emphasized that to further ensure the privacy and security of the bill to be recognized, the above bill to be recognized can also be stored in a node of a blockchain.

[0112] The blockchain referred to in this application is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. Blockchain, in essence, is a decentralized database, a series of data blocks generated by using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity of the information (anti-counterfeiting) and generate the next block. The blockchain can include the blockchain underlying platform, the platform product service layer, and the application service layer, etc.

[0113] Step S207: Input the text data into the bill recognition model, output the classification result, and obtain the field type of the text data according to the classification result.

[0114] In this embodiment, the text data is vectorized to obtain Token vectors, Segment vectors, and Position vectors, which represent word information, sentence information, and position information respectively. The Token vectors, Segment vectors, and Position vectors are used as text feature vectors to input into the bill recognition model for feature extraction. The concatenated features obtained are predicted through the softmax layer to obtain the prediction probability of each label corresponding to the text data, and the classification result is output.

[0115] It should be noted that the label is the field name corresponding to the text data. For example, if the text data is Wang Er, the corresponding field name is Name. The field name is used as the field type, and the label with the highest prediction probability is the field type corresponding to the text data.

[0116] This application can automatically process the bills that have undergone optical character recognition through the bill recognition model. For the text data in each text box of the bill, the field category corresponding to the text data can be automatically recognized. At the same time, using the trained bill recognition model to replace the manual post-processing code can increase the readability of the code, make it easy to maintain, and can also reduce the deployment pressure and reduce unforeseen situations such as online operation failures.

[0117] This application can be used in numerous general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This application can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0118] This application can be applied in the field of intelligent healthcare, thus promoting the construction of smart cities.

[0119] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, read-only memory (ROM), or a random access memory (RAM), etc.

[0120] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit and can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings can include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment but can be executed at different moments, and their execution order is not necessarily sequential but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0121] Further reference Figure 3 to Figure 2 As an implementation of the method shown above, this application provides an embodiment of an automatic bill recognition device. This device embodiment corresponds to the method embodiment shown in Figure 2 and can be specifically applied to various electronic devices.

[0122] As Figure 3 shown, the automatic bill recognition device 300 described in this embodiment includes: an acquisition module 301, a feature module 302, a splicing module 303, a prediction module 304, an iterative update module 305, an identification module 306, and a classification module 307. Among them:

[0123] The acquisition module 301 is used to acquire the original bill data set and preprocess the original bill data set to obtain the training data set;

[0124] The feature module 302 is used to input the training data set into the constructed BERT network model to obtain surface features, syntactic features, and semantic features;

[0125] The splicing module 303 is used to splice the surface features, the syntactic features, and the semantic features to obtain the spliced features;

[0126] The prediction module 304 is used to output the prediction result of the BERT network model based on the spliced features;

[0127] The iterative update module 305 is used to calculate the first loss function according to the prediction result, iteratively update the BERT network model according to the first loss function, and determine whether the iteratively updated BERT network model converges. When the iteratively updated BERT network model converges, output the BERT network model as the bill recognition model;

[0128] The identification module 306 is used to acquire the bill to be identified, perform optical character recognition on the bill to be identified, and obtain the text data corresponding to the bill to be identified;

[0129] The classification module 307 is used to input the text data into the bill recognition model, output the classification result, and obtain the field type of the text data according to the classification result.

[0130] It should be emphasized that, in order to further ensure the privacy and security of the bill to be identified, the above bill to be identified can also be stored in a node of a blockchain.

[0131] The above automatic bill recognition device automatically processes the bill after optical character recognition through the bill recognition model. For the text data in each text box of the bill, the field category corresponding to the text data can be automatically recognized; at the same time, using the trained bill recognition model to replace the manual post-processing code can increase the readability of the code, make it easy to maintain, and can also reduce the deployment pressure and reduce unpredictable situations such as online operation failures.

[0132] In this embodiment, the feature module 302 includes a vector conversion sub-module and a feature extraction sub-module. The vector conversion sub-module is configured to input the training data set into the input layer of the BERT network model for vector conversion, so as to obtain feature vector data corresponding to the training data set; the feature extraction sub-module is configured to perform feature extraction on the feature vector data through the feature extraction layer of the BERT network model to obtain the surface feature, the syntactic feature, and the semantic feature.

[0133] In this embodiment, the feature extraction sub-module is further configured to:

[0134] Extract surface features from the text feature vectors through the underlying network, and input the surface features into the intermediate network and the high-level network respectively;

[0135] Perform feature extraction on the surface features through the intermediate network to obtain the syntactic features, and input the syntactic features into the high-level network;

[0136] Perform feature extraction on the surface features and the syntactic features through the high-level network to obtain the semantic features.

[0137] In some alternative implementation manners of this embodiment, the prediction module 304 is further configured to:

[0138] Use the softmax layer of the BERT network model to predict the concatenated features to obtain the prediction probabilities of each label corresponding to the training data set;

[0139] Obtain a prediction result according to the prediction probabilities.

[0140] In this embodiment, the iterative update module 305 includes an adjustment sub-module and a calculation sub-module. The adjustment sub-module is configured to adjust the model parameters of the BERT network model according to the first loss function to obtain an intermediate BERT network model; the calculation sub-module is configured to calculate the second loss function of the intermediate BERT network model, and determine whether the intermediate BERT network model converges according to the first loss function and the second loss function.

[0141] In this embodiment, the calculation sub-module is further configured to calculate the difference between the first loss function and the second loss function. If the difference is within a preset range, the intermediate BERT network model converges.

[0142] In some alternative implementation manners of this embodiment, the acquisition module 301 is further configured to:

[0143] The original bill data in the original bill data set is subjected to field conversion by using the field assignment method to obtain an original bill data set containing field names and corresponding field values, where the field names are the training labels corresponding to the field values;

[0144] The preprocessed original bill data set is extracted according to a preset ratio to obtain a training data set.

[0145] To solve the above technical problems, an embodiment of the present application further provides a computer device. For details, please refer to Figure 4 , Figure 4 which is the basic structural block diagram of the computer device in this embodiment.

[0146] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are communicatively connected to each other through a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Among them, those skilled in the art of the present technology can understand that a computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0147] The computer device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device can perform human-computer interaction with the user through a keyboard, a mouse, a remote control, a touchpad, a voice control device, and other means.

[0148] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disc, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, FlashCard, etc. equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and the external storage device of the computer device 4. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the automatic bill recognition method. In addition, the memory 41 may also be used to temporarily store various data that have been output or will be output.

[0149] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the computer-readable instructions stored in the memory 41 or process data, such as running the computer-readable instructions of the automatic bill recognition method.

[0150] The network interface 43 may include a wireless network interface or a wired network interface, and the network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0151] In this embodiment, when the processor executes the computer-readable instructions stored in the memory, the steps of the automatic bill recognition method in the above embodiment are implemented. The bill is automatically processed by the bill recognition model after optical character recognition. For the text data in each text box of the bill, the field category corresponding to the text data can be automatically recognized. At the same time, the trained bill recognition model is used to replace the manual post-processing code, which increases the readability of the code, is easy to maintain, and can also reduce the deployment pressure and reduce unpredictable situations such as online operation failures.

[0152] The present application also provides another implementation manner, that is, to provide a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor, so that the at least one processor executes the steps of the above-mentioned automatic bill recognition method, automatically processes the optically character-recognized bill through a bill recognition model, and for the text data of each text box in the bill, the field category corresponding to the text data can be automatically recognized; at the same time, using the trained bill recognition model to replace the manual post-processing code can increase the readability of the code, make it easy to maintain, and can also reduce the deployment pressure and reduce unpredictable situations such as online operation failures.

[0153] Through the description of the above implementation manners, those skilled in the art can clearly understand that the above-mentioned method of the embodiment can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions to enable a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0154] Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The drawings show the preferred embodiments of the present application, but do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosed content of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing specific implementation manners, or perform equivalent replacements for some of the technical features. Any equivalent structure directly or indirectly using the content of the specification and drawings of the present application in other related technical fields shall be within the scope of the patent protection of the present application by the same token.

Claims

1. A method for automatic bill recognition, characterized in that, Including the following steps: Obtain the original bill dataset, and preprocess the original bill dataset to obtain a training dataset; Input the training dataset into the constructed BERT network model to obtain surface features, syntactic features, and semantic features; Concatenate the surface features, the syntactic features, and the semantic features to obtain concatenated features; Output the prediction result of the BERT network model based on the concatenated features; Calculate a first loss function according to the prediction result, iteratively update the BERT network model according to the first loss function, and determine whether the iteratively updated BERT network model converges. When the iteratively updated BERT network model converges, output the BERT network model as a bill recognition model; Obtain the bill to be recognized, perform optical character recognition on the bill to be recognized to obtain text data corresponding to the bill to be recognized; Input the text data into the bill recognition model, output a classification result, and obtain the field type of the text data according to the classification result; Among them, the inputting the training dataset into the constructed BERT network model to obtain surface features, syntactic features, and semantic features includes: Input the training dataset into the input layer of the BERT network model for vector conversion to obtain feature vector data corresponding to the training dataset; Extract features from the feature vector data through the feature extraction layer of the BERT network model to obtain the surface features, the syntactic features, and the semantic features; Among them, the feature extraction layer includes a bottom layer network, an intermediate layer network, and a high layer network. The extracting features from the feature vector data through the feature extraction layer of the BERT network model to obtain the surface features, the syntactic features, and the semantic features includes: Extract surface features from the text feature vector through the bottom layer network, and input the surface features into the intermediate layer network and the high layer network respectively; Extract features from the surface features through the intermediate layer network to obtain the syntactic features, and input the syntactic features into the high layer network; Extract features from the surface features and the syntactic features through the high layer network to obtain the semantic features; Among them, the feature extraction layer includes a 12-layer network structure stacked. Each layer of the network structure uses the decoder part of the Transformer. Each layer of the encoder includes a multi-head attention mechanism and a position-wise feed-forward network; among them, the 12-layer network structure includes 1 to 4 bottom-layer networks, 5 to 8 middle-layer networks, and 9 to 12 top-layer networks. The bottom-layer networks are used to learn surface features at the phrase level, the middle-layer networks are used to learn syntactic features at the syntactic level, and the top-layer networks are used to learn semantic features at the semantic level; among them, the feature extraction layer of the BERT network model is modified by connecting the 4th layer network, the 8th layer network, and the 12th layer network. That is, the input of the 8th layer network structure is the output of the 7th layer network structure and the output of the 4th layer network structure, and the input of the 12th layer network structure is the output of the 11th layer network structure and the output of the 8th layer network structure.

2. The automatic bill recognition method according to claim 1, characterized in that The output of the prediction result of the BERT network model based on the concatenated features includes: Using the softmax layer of the BERT network model to predict the concatenated features to obtain the prediction probability of each label corresponding to the training data set; Obtaining the prediction result according to the prediction probability.

3. The automatic bill recognition method according to claim 1, characterized in that The iterative update of the BERT network model according to the first loss function and determining whether the iteratively updated BERT network model converges includes: Adjusting the model parameters of the BERT network model according to the first loss function to obtain an intermediate BERT network model; Calculating the second loss function of the intermediate BERT network model, and judging whether the intermediate BERT network model converges according to the first loss function and the second loss function.

4. The automatic bill recognition method according to claim 3, characterized in that, The judgment of whether the intermediate BERT network model converges according to the first loss function and the second loss function includes: Calculating the difference between the first loss function and the second loss function. If the difference is within the preset range, the intermediate BERT network model converges.

5. The automatic bill recognition method according to claim 1, characterized in that The preprocessing of the original bill data set to obtain the training data set includes: Using the field assignment method to perform field conversion on the original bill data in the original bill data set to obtain an original bill data set containing field names and corresponding field values, where the field names are the training labels corresponding to the field values; Extracting the preprocessed original bill data set according to a preset ratio to obtain the training data set.

6. An automatic bill recognition device, characterized in that, Includes: An acquisition module for acquiring an original bill data set and preprocessing the original bill data set to obtain a training data set; A feature module for inputting the training data set into the constructed BERT network model to obtain surface features, syntactic features, and semantic features; A splicing module for splicing the surface features, the syntactic features, and the semantic features to obtain spliced features; A prediction module, configured to output a prediction result of the BERT network model based on the splicing features; An iterative update module, configured to calculate a first loss function according to the prediction result, iteratively update the BERT network model according to the first loss function, and determine whether the iteratively updated BERT network model converges. When the iteratively updated BERT network model converges, output the BERT network model as a bill recognition model; An identification module, configured to obtain a bill to be identified, perform optical character recognition on the bill to be identified, and obtain text data corresponding to the bill to be identified; A classification module, configured to input the text data into the bill recognition model, output a classification result, and obtain a field type of the text data according to the classification result; Wherein, the feature module includes a vector conversion sub-module and a feature extraction sub-module. The vector conversion sub-module is configured to input the training data set into the input layer of the BERT network model for vector conversion to obtain feature vector data corresponding to the training data set; the feature extraction sub-module is configured to perform feature extraction on the feature vector data through the feature extraction layer of the BERT network model to obtain the surface feature, the syntactic feature, and the semantic feature; Wherein, the feature extraction layer includes a bottom layer network, an intermediate layer network, and a high layer network. The feature extraction sub-module is further configured to: extract surface features from the text feature vector through the bottom layer network, and input the surface features into the intermediate layer network and the high layer network respectively; perform feature extraction on the surface features through the intermediate layer network to obtain the syntactic features, and input the syntactic features into the high layer network; perform feature extraction on the surface features and the syntactic features through the high layer network to obtain the semantic features; Wherein, the feature extraction layer includes a 12-layer network structure stacked. Each layer of the network structure adopts the decoder (encoder) part of the Transformer. Each layer of the encoder includes a layer of multi-head attention mechanism (Muti-Head-Attention) and a layer of position-wise feed-forward neural network (Position-Wise-Feed-Forward-Networks); wherein, the 12-layer network structure includes 1 to 4 layers of bottom layer networks, 5 to 8 layers of intermediate layer networks, and 9 to 12 layers of high layer networks. The bottom layer network is used to learn surface features at the phrase level, the intermediate layer network is used to learn syntactic features at the syntactic level, and the high layer network is used to learn semantic features at the semantic level; wherein, the feature extraction layer of the BERT network model is transformed, and the 4th layer network, the 8th layer network, and the 12th layer network are connected. That is, the input of the 8th layer network structure is the output of the 7th layer network structure and the output of the 4th layer network structure, and the input of the 12th layer network structure is the output of the 11th layer network structure and the output of the 8th layer network structure.

7. A computer device, comprising a memory and a processor, wherein computer-readable instructions are stored in the memory, and when the processor executes the computer-readable instructions, the automatic bill recognition method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that, Computer-readable instructions are stored on the computer-readable storage medium, and when the computer-readable instructions are executed by the processor, the automatic bill recognition method according to any one of claims 1 to 5 is implemented.

Images