Multi-bill data processing method and device based on time logic, equipment and medium

Abstract

The invention relates to the field of computer vision, and discloses a multi-bill data processing method and device based on time logic, equipment and a medium, and the method comprises the steps: collecting a plurality of to-be-processed bill images; inputting the images of the to-be-processed bills into a character detection and recognition model for parallel text detection and recognition to obtain a plurality of bill texts; sequentially connecting the multiple bill texts into a single text; inputting the single text into a named entity recognition model, and recognizing and outputting date entity information and amount entity information with a time sequence in the single text; and according to the identified date entity information and amount entity information, accumulating amount data corresponding to all dates to obtain an accumulated amount of the plurality of bills to be processed. Text merging is carried out on the basis of extracting the multi-bill texts, dates and amounts with time logic information in the merged texts are recognized and extracted through named entities, accurate accumulation calculation is carried out, and efficient and reliable data recognition and statistics in a multi-bill information scene are achieved.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision technology and financial technology, and in particular to a method, device, equipment and medium for processing multiple bill data based on time logic. Background Art

[0002] Invoices are the most effective information voucher in the financial industry. They can be used as a favorable voucher to prove that a transaction has occurred. Financial calculations of invoices often involve the problem of calculating multiple invoices together. In order to achieve full automation of information extraction, existing technical solutions use OCR technology to extract single invoice information, and then make associations and statistics based on location, keywords, QR codes and other information.

[0003] Although this method can solve problems in some scenarios, there is a chain reaction in the post-processing process. Once the amount recognized in one of the invoices is deviated, the final amount will be incorrect, resulting in insufficient accuracy in the data processing of multiple invoices, making it difficult to achieve accurate and reliable multi-invoice data extraction and processing. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a multi-bill data processing method, device, equipment and medium based on time logic that can be applied to financial technology or other related fields. Its main purpose is to improve the accuracy of multi-bill data processing and realize efficient and reliable data identification and statistics in multi-bill information scenarios.

[0005] The technical solution of the present invention is as follows:

[0006] The first aspect of the present invention provides a method for processing multiple bill data based on time logic, comprising:

[0007] Capturing images of multiple bills to be processed;

[0008] Inputting the images of the plurality of bills to be processed into a pre-built and trained text detection and recognition model, performing parallel text detection and recognition on the images of the plurality of bills to be processed, and obtaining the texts of the plurality of bills;

[0009] Connecting the multiple bill texts into a single text in sequence;

[0010] Input the single text into a pre-built and trained named entity recognition model, and recognize and output date entity information and amount entity information in time sequence in the single text;

[0011] According to the identified date entity information and amount entity information, the amount data corresponding to all dates are accumulated to obtain the cumulative amount of the multiple bills to be processed.

[0012] A second aspect of the present invention provides a multi-ticket data processing device based on time logic, comprising:

[0013] A collection module, used for collecting images of multiple bills to be processed;

[0014] A text recognition module, used for inputting the images of the multiple bills to be processed into a pre-built and trained text detection and recognition model, performing parallel text detection and recognition on the images of the multiple bills to be processed, and obtaining multiple bill texts;

[0015] A text connection module, used for connecting the multiple bill texts into a single text in sequence;

[0016] A named entity recognition module, used for inputting the single text into a pre-built and trained named entity recognition model, and identifying and outputting date entity information and amount entity information in a time sequence in the single text;

[0017] The accumulation module is used to accumulate the amount data corresponding to all dates according to the identified date entity information and amount entity information to obtain the accumulated amount of the multiple bills to be processed.

[0018] A third aspect of the present invention provides a computer device, comprising at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned multi-ticket data processing method based on time logic.

[0021] A fourth aspect of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors can execute the above-mentioned multi-ticket data processing method based on time logic.

[0022] Beneficial effects: The present invention discloses a method, device, equipment and medium for processing multiple bill data based on time logic. Compared with the prior art, the embodiment of the present invention collects images of multiple bills to be processed; inputs the images of the multiple bills to be processed into a pre-built and trained text detection and recognition model, performs parallel text detection and recognition on the images of the multiple bills to be processed, and obtains multiple bill texts; connects the multiple bill texts into a single text in sequence; inputs the single text into a pre-built and trained named entity recognition model, identifies and outputs the date entity information and amount entity information with time sequence in the single text; according to the identified date entity information and amount entity information, accumulates the amount data corresponding to all dates to obtain the cumulative amount of the multiple bills to be processed. By extracting multiple bill texts, performing named entity recognition after connecting the texts of multiple bills in time sequence, extracting the dates and amounts with time logic information in the multiple bill data, and performing accurate accumulation calculation, the accuracy of multi-bill data processing is improved, and efficient and reliable data recognition and statistics in the multi-bill information scenario are achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the scheme in the present invention, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 A schematic diagram of an application environment of a method for processing multiple bill data based on time logic provided by an embodiment of the present invention;

[0025] Figure 2 A flow chart of a method for processing multiple bill data based on time logic provided by an embodiment of the present invention;

[0026] Figure 3 A flow chart of step S202 in the method for processing multiple bill data based on time logic provided in an embodiment of the present invention;

[0027] Figure 4 A flow chart of step S204 in the method for processing multiple bill data based on time logic provided in an embodiment of the present invention;

[0028] Figure 5 A flow chart of step S205 in the method for processing multiple bill data based on time logic provided in an embodiment of the present invention;

[0029] Figure 6 A schematic diagram of the functional modules of a multi-ticket data processing device based on time logic provided by an embodiment of the present invention;

[0030] Figure 7 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The embodiments of the present invention are described below in conjunction with the accompanying drawings.

[0032] The multi-ticket data processing method based on time logic provided by the embodiment of the present invention can be applied to Figure 1 In the application environment of FIG. 1 , it includes a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0033] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only as examples).

[0034] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0035] The server 105 may be a server that provides various services, such as a background server that provides support for the content browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (for example only). The background server may analyze and process the received data such as the user request, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device. The server 105 may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server 105 may also be a server for a distributed system, or a server combined with a blockchain.

[0036] It should be noted that the multi-ticket data processing method based on time logic provided in the embodiment of the present application can generally be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103. Accordingly, the multi-ticket data processing device based on time logic provided in the embodiment of the present invention can also be set in the first terminal device 101, the second terminal device 102 or the third terminal device 103. Alternatively, the multi-ticket data processing method based on time logic provided in the embodiment of the present invention can also generally be executed by the server 105. Accordingly, the multi-ticket data processing device based on time logic provided in the embodiment of the present invention can generally be set in the server 105.

[0037] It should be understood that the numbers of the above terminal devices, networks and servers are only illustrative. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0038] like Figure 2 As shown, the multi-ticket data processing method based on time logic provided by the embodiment of the present invention specifically includes the following steps:

[0039] S201, collecting images of multiple bills to be processed.

[0040] In this embodiment, multiple bills to be processed are scanned or photographed by a high-speed scanner or smart device (such as a smart phone) to obtain digital image files. These image files can capture the details on the bills and provide clear image input for subsequent recognition. The specific bills to be processed can be bills that need to be counted for amount, such as bank checks, deposit slips, value-added tax invoices and other financial documents. These bills usually contain key financial information such as date, amount, payee information, etc., and the images of these bills need to be automatically recognized and data statistically processed to improve the efficiency of bill processing.

[0041] Preferably, during the image acquisition process, image processing techniques can also be used to pre-process the acquired images, such as grayscale, binarization, denoising and tilt correction, to improve image quality and ensure the accuracy of subsequent recognition.

[0042] S202: Input the images of the multiple bills to be processed into a pre-built and trained text detection and recognition model, perform text detection and recognition on the images of the multiple bills to be processed in parallel, and obtain multiple bill texts.

[0043] In this embodiment, the collected bill image is input into a pre-built and trained text detection and recognition model. The text detection and recognition model can be an OCR model based on deep learning, such as a model based on CNN and RNN structure, which can accurately capture and recognize the text on the bill, and perform parallel processing on multiple bill images to obtain multiple bill texts, thereby improving the processing speed.

[0044] Specifically, the text detection and recognition model includes a text detection sub-model and a text recognition sub-model connected in series, and the text detection sub-model includes a backbone network, a bottleneck layer and an output layer connected in sequence; the text recognition sub-model includes an image feature extraction network and a timing network.

[0045] That is, before processing the bill to be processed, a text detection sub-model and a text recognition sub-model are first constructed. The text detection model structure is constructed based on DBnet and is divided into a backbone network, a bottleneck layer and an output layer. The backbone network is composed of a convolutional layer, an activation layer, a BN layer and a short-circuit layer connected by a preset structure and quantity. The bottleneck layer mainly performs downsampling from top to bottom, and connects the sampled layer with the corresponding backbone network layer to form a feature fusion. Finally, the features of different resolutions in the bottleneck layer are fused (concat) and input into the output layer. The output layer adds a deconvolution layer to achieve upsampling, and has two channels that correspond to the generation of an adaptive threshold map T and a segmentation map P, and converts P into a final binary map B through differentiable binarization, thereby completing the construction of the text detection sub-model. The text recognition sub-model includes an image feature extraction network and a time series network. The image feature extraction network is mainly composed of 5 layers of blocks, each of which is composed of several layers. The layers include a funnel-shaped convolution layer structure (3x3 convolution->1x1 convolution->3x3 convolution), an activation layer, and a shortcut layer. The time series network is mainly LSTM, which is mainly connected by self-circulation layers. The self-circulation layer is composed of memory gates, forget gates, and output gates, and these gates are composed of parameterized matrices. After matrix product operations are performed on each parameterized matrix with the input implicit vector and the text vector, the output value and the new implicit vector can be obtained, so as to train the model and adjust the model parameters. The trained text detection sub-model and text recognition sub-model are connected in series to obtain a text detection and recognition model. The text detection and recognition model can detect the text area position of the input image and recognize the specific text content, providing a reliable data basis for bill data processing.

[0046] S203, connecting the multiple bill texts into a single text in sequence.

[0047] In this embodiment, after the images of multiple bills to be processed are subjected to text detection and recognition, the text content of each bill will be sorted in the order in which they are recognized and processed, and the recognized texts of multiple bills will be connected in sequence to form a continuous single text string. During the connection process, specific separators or marks can also be inserted to distinguish the content boundaries of different bills. For example, a unique bill identifier or a line break can be added after the text content of each bill to ensure that the start and end positions of each bill can be identified in the connected single text. By integrating the text content of multiple bills into a single text stream, it is convenient to use the temporal logical relationship between all bills to perform more accurate statistical processing during subsequent data processing, thereby improving the accuracy of data processing.

[0048] S204: input the single text into a pre-built and trained named entity recognition model, and recognize and output date entity information and amount entity information in time sequence in the single text.

[0049] In this embodiment, the connected single text is input into a pre-built and trained named entity recognition model, which can be a named entity recognition model based on FinBERT-CRF or other models with the same entity recognition function. The model uses deep learning technology, such as BERT architecture, combined with the CRF layer to identify and extract entity information in the text, and can more accurately identify entities in financial texts. In this embodiment, the model is pre-trained with the time logical relationship of the bill text in the financial field, so that the model can recognize the date entity information and amount entity information with time sequence in the single text. It improves the effective identification and extraction of date entity information and amount entity information with time sequence from continuous text, and provides structured data input for subsequent data processing.

[0050] S205. According to the identified date entity information and amount entity information, the amount data corresponding to all dates are accumulated to obtain the accumulated amounts of the multiple bills to be processed.

[0051] In this embodiment, based on the date entity information and amount entity information in the single text that have been identified by the named entity recognition model, the date entity details and amount entity information can be stored in the form of structured data. For example, in a database or data frame, each record contains a date and a corresponding amount, etc. According to the identified date entity information and amount entity information, the corresponding amount data under the corresponding date can be extracted, and the amount data corresponding to all dates can be accumulated. For example, by looping through each record, the amount obtained by each traversal is added to the cumulative total, and finally the cumulative amount of multiple bills to be processed is obtained. The text detection and recognition of multiple bills to be processed are performed in parallel, and the amount is accumulated after the named entity recognition, which realizes the combination of information extraction processes, unifies the scattered processing processes, and improves the processing efficiency of bill data. In addition, the text is merged on the basis of extracting multiple bill texts, and the time logic information in the multiple bill texts is effectively used, which can enhance the model's association recognition of time and amount entities and improve more accurate data recognition and statistics in multiple bill scenarios.

[0052] In one embodiment, Figure 3 As shown, step S202 includes:

[0053] S301, using the backbone network to extract features from the images of the plurality of bills to be processed, and outputting corresponding first feature maps;

[0054] S302, downsampling and feature fusion are performed on the first feature map through the bottleneck layer, and a second feature map with different resolutions is output;

[0055] S303, performing dual-channel upsampling processing on the second feature map through the output layer to generate an adaptive threshold map and a segmentation map, and converting the adaptive threshold map and the segmentation map into a binary map of the text area;

[0056] S304, inputting the binary image into the image feature extraction network, extracting the text features of the binary image, and predicting and outputting the corresponding text sequence after performing time series analysis on the text features through the time series network to obtain the corresponding multi-bill text.

[0057] In this embodiment, based on the structure of the text detection and recognition model, after inputting the images of multiple bills to be processed into the text detection and recognition model, in the text detection stage, the input bill image is first subjected to feature extraction processing through the convolution layer, activation layer, BN layer and short-circuit layer in the backbone network to obtain a first feature map, which captures the text features in the image. Then, the first feature map is downsampled through the bottleneck layer to reduce the size of the feature map and increase the receptive field, and the downsampled feature map is fused with the feature map of the corresponding layer in the backbone network to output second feature maps of different resolutions. These second feature maps contain rich text context information. Finally, the second feature map is subjected to dual-channel upsampling processing through the output layer to generate an adaptive threshold map (T) and a segmentation map (P) respectively, and the following differentiable binarization processing formula is used:

[0058] k is a trainable parameter

[0059] The segmentation map (P) is converted into a binary map (B) to complete the detection of the text area.

[0060] In the text recognition stage, the binary image output from the text detection stage is input into the image feature extraction network, and the text features are extracted through the image feature extraction network, such as capturing the shape, size and structural information of the text, and passed to the timing network for timing analysis. The timing network, such as the long short-term memory network (LSTM), can process sequence data and capture the contextual relationship between texts, thereby identifying the timing dependencies of the text sequence, such as the order of letters and characters. Therefore, after the timing network performs timing analysis on the features of the binary image, the corresponding text sequence is predicted and output, thereby automatically identifying the readable text information from the binary image obtained by text detection, and obtaining the corresponding multi-bill text, thereby achieving efficient and reliable bill text detection and recognition.

[0061] In one embodiment, before step S204, the method further includes:

[0062] Collecting relevant bill data, and constructing a pre-training data set and a fine-tuning training set according to the relevant bill data;

[0063] The pre-built named entity recognition model is pre-trained using the pre-training data set, and the pre-trained named entity recognition model is fine-tuned based on temporal logical relationships using the fine-tuning training set until a final named entity recognition model is obtained after a preset convergence condition is met.

[0064] In this embodiment, before processing multiple bill data, a named entity recognition model is pre-built and trained to realize entity extraction of bill text. Specifically, in the model construction stage, a named entity recognition model is constructed based on the Bert structure, which includes a word embedding layer, a position encoding layer and an attention layer, wherein the word embedding layer is mainly composed of a dictionary, which is an n x512 matrix, where n is the number of words. According to the input text id, the corresponding word vector is output to complete the word embedding work. The position encoding layer mainly completes the encoding of the position of the word in the sentence, and the response value can be obtained by the following calculation formula:

[0065] PE(pos,2i)=sin(pos / 10000^(2i / d_model))

[0066] PE(pos,2i+1)=cos(pos / 10000^(2i / d_model))

[0067] Among them, PE(pos,2i) represents the value at position pos and dimension 2i, PE(pos,2i+1) represents the value at position pos and dimension 2i+1, pos represents the position in the sequence, i represents the dimension index in the embedding vector, and d_model represents the dimension of the embedding vector. The attention layer includes a self-attention module, a forward propagation module, and a multi-head attention module. The attention layers are stacked in a preset number and constructed together with the word embedding layer and the position encoding layer to obtain the initial named entity recognition model.

[0068] In the model training phase, a large amount of related bill data is first collected. This data can come from the daily operations of banks, insurance companies or other financial institutions. Bill data can cover a variety of types, such as checks, invoices, insurance policies, etc., to ensure the generalization ability of the model. According to the collected related bill data, two parts of the data set are constructed respectively to obtain a pre-training data set for preliminary model training and a fine-tuning training set for model fine-tuning, thereby meeting the data requirements of the model training phase.

[0069] In one embodiment, the collecting of the associated bill data and constructing a pre-training data set and a fine-tuning training set according to the associated bill data include:

[0070] Collecting related bill images within a historical time period from a database, inputting the related bill images into the text detection and recognition model, and obtaining related bill texts;

[0071] Performing entity labeling on the associated bill text, adding corresponding entity annotations to the date entity and the amount entity in the associated bill text, and generating the pre-training data set;

[0072] The associated bill texts are connected in chronological order and date labels and amount labels are added to generate a continuous text sequence as the fine-tuning training set.

[0073] In this embodiment, the associated bill data specifically refers to the associated bill images within the historical time period collected from the database. For the text content in the bill, the associated bill images are firstly subjected to text detection and recognition using the above-mentioned trained text detection and recognition model to obtain the corresponding associated bill text. Preferably, the recognized text is further manually corrected to ensure the accuracy of the data.

[0074] When constructing the pre-training dataset, the associated bill text is entity labeled according to the BIO rule, mainly for the date and amount, so as to add corresponding entity annotations to the date entity and amount entity in the associated bill text, and generate the corresponding pre-training dataset. In the BIO rule, the B-label indicates the beginning of the entity (such as "B-DATE" indicates the beginning of the date entity), the I-label indicates the internal part of the entity (such as "I-DATE" indicates the internal part of the date entity), and the O-label indicates the part that does not belong to any entity. Through the pre-training dataset with date entity annotations and amount entities, the pre-built named entity recognition model is pre-trained according to the named entity recognition strategy. During pre-training, the loss function is CELoss, the optimizer is adam, the learning rate is 1e-3, the batch_size is 128, and the model loss is trained until convergence, so that the pre-trained model can recognize the date and amount entities in the text.

[0075] In addition, based on the collected and identified associated bill text, the text is connected in date order and date labels and amount labels are added. For example, the date label is 'DATE_1'...'DATA_N', and the corresponding amount label is 'AMOUNT_1'...'AMOUNT_N', thereby generating a continuous text sequence as a fine-tuning training set. Based on the fine-tuning training set, the pre-trained named entity recognition model is fine-tuned based on the temporal logical relationship. During the fine-tuning training, the text sequence connected in date order when fine-tuning the training data can simulate the temporal logical relationship, which helps the model learn the dependency on the time series and the correlation between different invoices. Especially in scenarios where operations need to be performed according to the time sequence (such as the accumulation of amounts), fine-tuning by connecting the data in time order can also help the model identify and process the changes of the same event or transaction at different time points, so as to improve the reliability of amount extraction in multi-bill scenarios. During fine-tuning training, the loss function is CELoss, the optimizer is adam, the learning rate is 1e-5, the batch_size is 128, and the model loss is trained until convergence to obtain the final named entity recognition model.

[0076] In one embodiment, Figure 4 As shown, step S204 includes:

[0077] S401, inputting the single text into a pre-built and trained named entity recognition model to identify and extract all date entities and amount entities in the single text;

[0078] S402: After adding corresponding date tags and amount tags to all date entities and amount entities in chronological order, the corresponding date entity sequence and amount entity sequence are output.

[0079] In this embodiment, a single text obtained by sequentially connecting multiple bill texts is input into a named entity recognition model. The model can extract features including word embeddings, contextual information, etc. from the single text, and then perform entity prediction on each single time to determine whether it is part of an entity and which category it belongs to (such as date or amount), thereby identifying all date entities and amount entities in the single text. For each identified date entity and amount entity, the model will add labels respectively, add corresponding date labels such as 'DATE_1'...'DATA_N', and corresponding amount labels such as 'AMOUNT_1'...'AMOUNT_N' in chronological order, and output the entity sequence with date and amount labels added in sequence to form a structured date and amount information list.

[0080] For example, after recognizing three bill images, the following three bill texts are obtained: "On January 15, 2024, the meeting venue rental fee of 300 yuan was paid."

[0081] "On February 20, 2024, pay the lecturer a fee of 500 yuan."

[0082] "On March 10, 2024, pay RMB 200 for equipment purchase."

[0083] Concatenate these three bill texts into a single text in sequence for the input of the named entity recognition model: "On January 15, 2024, pay the conference venue rental fee of 300 yuan. On February 20, 2024, pay the lecturer's remuneration of 500 yuan. On March 10, 2024, pay the equipment purchase fee of 200 yuan."

[0084] The model recognizes and extracts all date entities and amount entities in a single text, and uses the BIO annotation system to indicate the boundaries of the entities. For example:

[0085] "January 15, 2024" is labeled as "B-DATE I-DATE I-DATE", "300 yuan" is labeled as "B-AMOUNT I-AMOUNT", "February 20, 2024" is labeled as "B-DATE I-DATE I-DATE", "500 yuan" is labeled as "B-AMOUNT I-AMOUNT", "March 10, 2024" is labeled as "B-DATE I-DATE I-DATE", and "200 yuan" is labeled as "B-AMOUNT I-AMOUNT". Finally, the model converts these BIO tags into more concise entity tags, such as 'DATE_1', 'AMOUNT_1', etc., and outputs the recognized entities and their labels to obtain date entity sequences and amount entity sequences. For example:

[0086] 'DATE_1': "January 15, 2024",

[0087] 'AMOUNT_1': "300 yuan",

[0088] 'DATE_2': "February 20, 2024",

[0089] 'AMOUNT_2': "500 yuan",

[0090] 'DATE_3': "March 10, 2024",

[0091] 'AMOUNT_3': "200 yuan".

[0092] In this way, the named entity recognition model can identify and extract all date entities and amount entities in a single text, and mark them in the order in which they appear in the original bill text, converting unstructured text information into a structured entity list, which not only helps to improve the accuracy of entity recognition, but also provides structured entity information for subsequent data processing and analysis.

[0093] In one embodiment, Figure 5 As shown, step S205 includes:

[0094] S501, extracting the amount data corresponding to each date data according to the date tag and the amount tag;

[0095] S502: Convert the extracted amount data into floating point numbers and then add them up to obtain the cumulative amount of multiple bills to be processed.

[0096] In this embodiment, based on the date label and amount label in the date entity sequence and the amount entity sequence, the amount data corresponding to each date data can be extracted, that is, the data labeled 'AMOUNT_X', and the extracted amount data is converted from the string format to a floating point number for mathematical operations, and the converted floating point amount is accumulated. Specifically, each record can be traversed in a loop, and the amount obtained by each traversal conversion is added to the cumulative total, thereby obtaining the cumulative amount of multiple bills to be processed. By extracting key date and amount information from the text of multiple bills and structuring it, the efficiency and accuracy of financial bill processing are greatly improved, and efficient and reliable bill data statistical processing is achieved.

[0097] In the above embodiment, the present invention discloses a method for processing multiple bill data based on time logic, which collects images of multiple bills to be processed; inputs the images of the multiple bills to be processed into a pre-built and trained text detection and recognition model, performs parallel text detection and recognition on the images of the multiple bills to be processed, and obtains multiple bill texts; connects the multiple bill texts into a single text in sequence; inputs the single text into a pre-built and trained named entity recognition model, identifies and outputs the date entity information and amount entity information with time sequence in the single text; according to the identified date entity information and amount entity information, accumulates the amount data corresponding to all dates to obtain the cumulative amount of the multiple bills to be processed. By extracting multiple bill texts, performing named entity recognition on the texts of multiple bills after connecting them in time sequence, extracting the dates and amounts with time logic information in the multiple bill data, and performing accurate accumulation calculation, the accuracy of multi-bill data processing is improved, and efficient and reliable data recognition and statistics in the multi-bill information scenario are achieved.

[0098] It should be noted that there is not necessarily a certain order between the above-mentioned steps. A person skilled in the art can understand, based on the description of the embodiments of the present invention, that in different embodiments, the above-mentioned steps may have different execution orders, that is, they may be executed in parallel, may be executed interchangeably, and so on.

[0099] Further references Figure 6 , as a response to the above Figure 2 The present invention provides an embodiment of a multi-ticket data processing device based on time logic, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0100] like Figure 6 As shown, the multi-ticket data processing device 60 based on time logic described in this embodiment includes:

[0101] The acquisition module 601 is used to acquire images of multiple bills to be processed;

[0102] The text recognition module 602 is used to input the images of the multiple bills to be processed into a pre-built and trained text detection and recognition model, perform text detection and recognition on the images of the multiple bills to be processed in parallel, and obtain multiple bill texts;

[0103] A text connection module 603, used to connect the multiple bill texts into a single text in sequence;

[0104] A named entity recognition module 604 is used to input the single text into a pre-built and trained named entity recognition model, and recognize and output date entity information and amount entity information in a time sequence in the single text;

[0105] The accumulation module 605 is used to accumulate the amount data corresponding to all dates according to the identified date entity information and amount entity information to obtain the accumulated amount of the multiple bills to be processed. The module referred to in the present invention refers to a series of computer program instruction segments that can complete specific functions, which is more suitable for describing the execution process of multi-bill data processing based on time logic than a program. For the specific implementation of each module, please refer to the corresponding method embodiment above, which will not be repeated here.

[0106] In one embodiment, the text detection and recognition model includes a text detection sub-model and a text recognition sub-model connected in series, the text detection sub-model includes a backbone network, a bottleneck layer and an output layer connected in sequence; the text recognition sub-model includes an image feature extraction network and a timing network.

[0107] In one embodiment, the text recognition module 602 includes:

[0108] A feature extraction unit, configured to extract features from the images of the plurality of bills to be processed using the backbone network, and output a corresponding first feature map;

[0109] A feature fusion unit, used to downsample and fuse the first feature map through the bottleneck layer, and output a second feature map with different resolutions;

[0110] A detection output unit, configured to perform dual-channel upsampling processing on the second feature map through the output layer to generate an adaptive threshold map and a segmentation map, and convert the adaptive threshold map and the segmentation map into a binary map of the text area;

[0111] The text detection unit is used to input the binary image into the image feature extraction network to extract the text features of the binary image, and predict and output the corresponding text sequence after performing time series analysis on the text features through the time series network to obtain the corresponding multi-bill text.

[0112] In one embodiment, the device 60 further includes:

[0113] A data collection module, used to collect relevant bill data, and construct a pre-training data set and a fine-tuning training set according to the relevant bill data;

[0114] The pre-training and fine-tuning module is used to pre-train the pre-built named entity recognition model through the pre-training data set, and to fine-tune the pre-trained named entity recognition model based on temporal logical relationships through the fine-tuning training set until the final named entity recognition model is obtained after the preset convergence conditions are met.

[0115] In one embodiment, the data acquisition module includes:

[0116] A collection unit, used for collecting related bill images within a historical time period from a database, inputting the related bill images into the text detection and recognition model, and obtaining related bill texts;

[0117] A first data set generating unit is used to perform entity tagging on the associated bill text, add corresponding entity annotations to the date entity and the amount entity in the associated bill text, and generate the pre-training data set;

[0118] The second data set generating unit is used to connect the associated bill texts in chronological order and add date labels and amount labels to generate a continuous text sequence as the fine-tuning training set.

[0119] In one embodiment, the named entity recognition module 604 includes:

[0120] An entity extraction unit, used for inputting the single text into a pre-built and trained named entity recognition model, and identifying and extracting all date entities and amount entities in the single text;

[0121] The sequence output unit is used to add corresponding date tags and amount tags to all date entities and amount entities in chronological order, and then output the corresponding date entity sequence and amount entity sequence.

[0122] In one embodiment, the accumulation module 605 includes:

[0123] An amount data extraction unit, used to extract the amount data corresponding to each date data according to the date tag and the amount tag;

[0124] The accumulation unit is used to convert the extracted amount data into floating point numbers and then accumulate them to obtain the cumulative amount of multiple bills to be processed.

[0125] In the above embodiment, the present invention discloses a multi-bill data processing device based on time logic, which collects images of multiple bills to be processed; inputs the images of the multiple bills to be processed into a pre-built and trained text detection and recognition model, performs parallel text detection and recognition on the images of the multiple bills to be processed, and obtains multiple bill texts; connects the multiple bill texts into a single text in sequence; inputs the single text into a pre-built and trained named entity recognition model, identifies and outputs the date entity information and amount entity information with time sequence in the single text; according to the identified date entity information and amount entity information, accumulates the amount data corresponding to all dates to obtain the cumulative amount of the multiple bills to be processed. By extracting multiple bill texts, performing named entity recognition on the texts of multiple bills after connecting them in time sequence, extracting the dates and amounts with time logic information in the multiple bill data, and performing accurate accumulation calculation, the accuracy of multi-bill data processing is improved, and efficient and reliable data recognition and statistics in the multi-bill information scenario are achieved.

[0126] Another embodiment of the present invention provides a computer device, such as Figure 7 As shown, the computer device 70 includes:

[0127] One or more processors 701 and memory 702, Figure 7 A processor 701 is used as an example for description. The processor 701 and the memory 702 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0128] Processor 701 is used to complete various control logics of computer device 70, and it can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component or any combination of these components. In addition, processor 701 can also be any traditional processor, microprocessor or state machine. Processor 701 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0129] The memory 702 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions corresponding to the multi-ticket data processing method based on time logic in the embodiment of the present invention. The processor 701 executes various functional applications and data processing of the computer device 70 by running the non-volatile software programs, instructions and units stored in the memory 702, that is, implementing the multi-ticket data processing method based on time logic in the above method embodiment.

[0130] The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device 70, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 702 may optionally include a memory remotely arranged relative to the processor 701, and these remote memories may be connected to the computer device 70 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. One or more units are stored in the memory 702, and when executed by one or more processors 701, the steps of the multi-ticket data processing method based on time logic in any of the above-mentioned method embodiments are executed.

[0131] In the above embodiment, the present invention discloses a computer device, which collects images of multiple bills to be processed; inputs the images of the multiple bills to be processed into a pre-built and trained text detection and recognition model, performs parallel text detection and recognition on the images of the multiple bills to be processed, and obtains multiple bill texts; connects the multiple bill texts into a single text in sequence; inputs the single text into a pre-built and trained named entity recognition model, identifies and outputs the date entity information and amount entity information with time sequence in the single text; according to the identified date entity information and amount entity information, accumulates the amount data corresponding to all dates to obtain the cumulative amount of the multiple bills to be processed. By extracting multiple bill texts, performing named entity recognition on the texts of multiple bills after connecting them in time sequence, extracting the dates and amounts with time logic information in the multiple bill data, and performing accurate accumulation calculation, the accuracy of multiple bill data processing is improved, and efficient and reliable data recognition and statistics in multiple bill information scenarios are achieved.

[0132] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the steps of the multi-ticket data processing method based on time logic in any of the above method embodiments are executed.

[0133] In the above embodiment, the present invention discloses a non-volatile computer-readable storage medium, which collects images of multiple bills to be processed; inputs the images of the multiple bills to be processed into a pre-built and trained text detection and recognition model, performs parallel text detection and recognition on the images of the multiple bills to be processed, and obtains multiple bill texts; connects the multiple bill texts into a single text in sequence; inputs the single text into a pre-built and trained named entity recognition model, identifies and outputs the date entity information and amount entity information with time sequence in the single text; according to the identified date entity information and amount entity information, accumulates the amount data corresponding to all dates to obtain the cumulative amount of the multiple bills to be processed. By extracting multiple bill texts, performing named entity recognition on the texts of multiple bills after connecting them in time sequence, extracting the dates and amounts with time logic information in the multiple bill data, and performing accurate accumulation calculation, the accuracy of multiple bill data processing is improved, and efficient and reliable data recognition and statistics in multiple bill information scenarios are achieved.

[0134] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0135] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0136] In summary, the method, device, equipment and medium for processing multiple bill data based on time logic disclosed in the present invention include: collecting images of multiple bills to be processed; inputting the images of the multiple bills to be processed into a pre-built and trained text detection and recognition model, performing parallel text detection and recognition on the images of the multiple bills to be processed, and obtaining multiple bill texts; connecting the multiple bill texts into a single text in sequence; inputting the single text into a pre-built and trained named entity recognition model, identifying and outputting the date entity information and amount entity information with time sequence in the single text; according to the identified date entity information and amount entity information, accumulating the amount data corresponding to all dates to obtain the cumulative amount of the multiple bills to be processed. By extracting multiple bill texts, performing named entity recognition after connecting the texts of multiple bills in time sequence, extracting the dates and amounts with time logic information in the multiple bill data, and performing accurate accumulation calculation, the accuracy of multi-bill data processing is improved, and efficient and reliable data recognition and statistics in the multi-bill information scenario are achieved.

[0137] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, and the computer program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, a floppy disk, a flash memory, an optical storage device, etc.

[0138] It should be noted that if any software tools or components other than those of the Company appear in the embodiments of the present application, they are merely used for illustration and do not represent actual use. It should be understood that the application of the present invention is not limited to the above examples, and that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes shall fall within the scope of protection of the claims attached to the present invention.

Claims

1. A multi-ticket data processing method based on time logic, characterized in that: include: Capturing images of multiple bills to be processed; Inputting the images of the plurality of bills to be processed into a pre-built and trained text detection and recognition model, performing parallel text detection and recognition on the images of the plurality of bills to be processed, and obtaining the texts of the plurality of bills; Connecting the multiple bill texts into a single text in sequence; Input the single text into a pre-built and trained named entity recognition model, and recognize and output date entity information and amount entity information in time sequence in the single text; According to the identified date entity information and amount entity information, the amount data corresponding to all dates are accumulated to obtain the cumulative amount of the multiple bills to be processed.

2. The method for processing multiple bill data based on time logic according to claim 1 is characterized in that: The text detection and recognition model includes a text detection sub-model and a text recognition sub-model connected in series, wherein the text detection sub-model includes a backbone network, a bottleneck layer and an output layer connected in sequence; the text recognition sub-model includes an image feature extraction network and a temporal network.

3. The method for processing multiple bill data based on time logic according to claim 2 is characterized in that: The step of inputting the images of the plurality of bills to be processed into a pre-built and trained text detection and recognition model, performing parallel text detection and recognition on the images of the plurality of bills to be processed, and obtaining the text of the plurality of bills includes: Using the backbone network to extract features from the images of the plurality of bills to be processed, and outputting corresponding first feature maps; Downsampling and feature fusion are performed on the first feature map through the bottleneck layer to output a second feature map with different resolutions; Performing dual-channel upsampling processing on the second feature map through the output layer to generate an adaptive threshold map and a segmentation map, and converting the adaptive threshold map and the segmentation map into a binary map of the text area; The binary image is input into the image feature extraction network to extract the text features of the binary image, and the text features are analyzed in time series by the time series network to predict and output the corresponding text sequence to obtain the corresponding multi-bill text.

4. The method for processing multiple bill data based on time logic according to claim 1 is characterized in that: Before inputting the single text into a pre-built and trained named entity recognition model and identifying and outputting the date entity information and the amount entity information in the single text in time sequence, the method further includes: Collecting relevant bill data, and constructing a pre-training data set and a fine-tuning training set according to the relevant bill data; The pre-built named entity recognition model is pre-trained using the pre-training data set, and the pre-trained named entity recognition model is fine-tuned based on temporal logical relationships using the fine-tuning training set until a final named entity recognition model is obtained after a preset convergence condition is met.

5. The method for processing multiple bill data based on time logic according to claim 4 is characterized in that: The collecting of the associated bill data and constructing a pre-training data set and a fine-tuning training set according to the associated bill data respectively include: Collecting related bill images within a historical time period from a database, inputting the related bill images into the text detection and recognition model, and obtaining related bill texts; Performing entity labeling on the associated bill text, adding corresponding entity annotations to the date entity and the amount entity in the associated bill text, and generating the pre-training data set; The associated bill texts are connected in chronological order and date labels and amount labels are added to generate a continuous text sequence as the fine-tuning training set.

6. The method for processing multiple bill data based on time logic according to claim 1 is characterized in that: The step of inputting the single text into a pre-built and trained named entity recognition model, and identifying and outputting date entity information and amount entity information in a time sequence in the single text, includes: Input the single text into a pre-built and trained named entity recognition model to identify and extract all date entities and amount entities in the single text; After adding corresponding date labels and amount labels to all date entities and amount entities in chronological order, the corresponding date entity sequence and amount entity sequence are output.

7. The method for processing multiple bill data based on time logic according to claim 6 is characterized in that: According to the identified date entity information and amount entity information, the amount data corresponding to all dates are accumulated to obtain the cumulative amount of the multiple bills to be processed, including: Extracting the amount data corresponding to each date data according to the date label and the amount label; The extracted amount data is converted into floating point numbers and then accumulated to obtain the cumulative amount of multiple bills to be processed.

8. A multi-ticket data processing device based on time logic, characterized in that: include: A collection module, used for collecting images of multiple bills to be processed; A text recognition module, used for inputting the images of the multiple bills to be processed into a pre-built and trained text detection and recognition model, performing parallel text detection and recognition on the images of the multiple bills to be processed, and obtaining multiple bill texts; A text connection module, used for connecting the multiple bill texts into a single text in sequence; A named entity recognition module, used for inputting the single text into a pre-built and trained named entity recognition model, and identifying and outputting date entity information and amount entity information in a time sequence in the single text; The accumulation module is used to accumulate the amount data corresponding to all dates according to the identified date entity information and amount entity information to obtain the accumulated amount of the multiple bills to be processed.

9. A computer device, characterized in that: comprising at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the multi-ticket data processing method based on time logic as described in any one of claims 1-7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the multi-ticket data processing method based on time logic as described in any one of claims 1-7.

Images