Program, information processing device and method

Abstract

To provide a technique capable of reading a bill by OCR processing and performing payment processing from the read text data.SOLUTION: A server of a bill reading system causes a processor to perform the steps of: storing image data (2021) of a bill related to a user and generating text data by reading the image data of the bill by OCR processing as the function (S121); generating classification text data by classifying the generated text data per bill description item (S122); determining whether items necessary to perform payment processing for the bill exist on the basis of the classified classification text data (S123); and when the determination results indicate that all necessary items exist, generating payment data for performing payment processing for the bill (S126).SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to a program, an information processing device, and a method. [Background technology]

[0002] Optical Character Recognition (OCR) is a well-known technology that uses optical reading devices such as cameras and scanners to recognize documents printed on paper forms. There is also a well-known technology that uses image analysis to classify form images.

[0003] As a document classification system technique, Patent Document 1 discloses a document classification system technique that analyzes image information of evidence, classifies the evidence, determines a storage destination, and stores the image information of the evidence. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 136778 Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, if it were possible to further utilize the text data scanned after documentary evidence has been read using OCR, it would be possible to provide convenient services to customers. For example, if an invoice, which is an example of documentary evidence, could be scanned using OCR and subsequent processing, such as payment processing, could be performed, which would improve convenience for customers, and such a function was in demand.

[0006] Therefore, this disclosure describes a technology that enables invoices to be read using OCR processing and payment processing to be performed from the read text data. [Means for solving the problem]

[0007] According to one embodiment of the present disclosure, there is provided a program to be executed by a computer having a processor and a memory, wherein the memory stores image data of an invoice related to a user, and the program causes the processor to execute the following steps: reading the image data of the invoice using OCR processing to generate text data; classifying the generated text data by item listed on the invoice to generate categorized text data; determining whether items necessary for processing payment for the invoice exist based on the classified categorized text data; and, if the determination result indicates that all necessary items exist, generating payment data for processing payment for the invoice. [Effects of the Invention]

[0008] According to the present disclosure, an invoice is scanned using OCR processing, and the scanned text data is used to determine whether there are any items necessary for payment processing, and payment data for payment processing is generated, thereby improving convenience for customers. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 10 is a diagram showing an example of image data of a bill to be read in the present disclosure. [Figure 2] 1 is a block diagram showing the overall configuration of a bill reading system 1 according to a first embodiment. [Figure 3] 3 is a block diagram showing a functional configuration of the terminal device 10 of FIG. 2. FIG. [Figure 4] 3 is a block diagram showing the functional configuration of the server 20 of FIG. 2. FIG. [Figure 5] FIG. 5 is a diagram showing an example of the data structure of a transaction database 2022 in FIG. 4. [Figure 6] FIG. 5 is a diagram illustrating an example of the data structure of an accounting database 2023 in FIG. 4. [Figure 7] 10 is a flowchart showing an example of the flow of payment data generation processing performed by the bill reading system 1. [Figure 8] 10 is a flowchart showing an example of the flow of payment data transmission processing by the bill reading system 1. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The following description and drawings are examples for explaining the present invention, and appropriate omissions and simplifications have been made for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural. In the following description, the same components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0011] <Summary> The invoice reading system according to the present disclosure will be described below. The invoice reading system according to the present disclosure is a system that reads image data of an invoice using OCR processing and determines from the read text data whether items necessary for payment processing are present. The invoice reading system according to the present disclosure is configured to infer and complete the necessary items if the necessary items for payment processing are not present. Furthermore, if the necessary items for payment processing are present or if the necessary items have been completed, the invoice reading system according to the present disclosure generates payment data for payment processing and performs the payment processing. The invoice reading system according to the present disclosure is a system provided as a web service, for example, via a cloud server or the like, known as SaaS (Software as a Service).

[0012] The invoice reading system according to the present disclosure also manages transaction data for users, such as businesses and sole proprietors, and provides accounting services. The accounting services provided to users involve managing cash transactions such as purchases, sales, and borrowings, recording (entering) the transactions in ledgers (journals and general ledgers), creating trial balances as closing procedures at designated accounting periods, and preparing financial statements such as income statements and balance sheets.

[0013] FIG. 1 is a diagram illustrating an example of image data of an invoice scanned by the present disclosure. The image data scanned by the information processing system according to the present disclosure is image data of invoice P1, as shown in FIG. 1. Invoice P1 is a document that serves as the basis for the image data of the invoice to be processed by OCR. Examples of scanned items include title A1, billing company name A2, billing company name A3, billing address A4, invoice issue date A5, subject A6, item A7, quantity A8, itemized invoice amount A9, subtotal A10, consumption tax A11, and total A12. This invoice scanning system extracts coordinate information, for example, with the upper left corner of invoice P1 as the origin, from the information indicated by the dashed lines within each scanned item shown in FIG. 1 as area data. Furthermore, character strings contained in this area data are scanned and extracted using OCR processing.

[0014] By the way, if it were possible to read invoice P1 using OCR processing and process payments from the read text data, it would be possible to provide convenient services to customers. In order to process payments, it is necessary to extract the items necessary for processing payments from each read item on invoice P1 and create payment data.

[0015] Therefore, the invoice reading system according to the present disclosure reads image data of an invoice, for example invoice P1, using OCR processing, and determines from the read text data whether there are any items necessary for processing the payment. If there are any items necessary for processing the payment, the invoice reading system according to the present disclosure generates payment data for processing the payment.

[0016] In addition, in the example of invoice P1, the items required for payment processing include the billing company name A3, item A7, and total amount A12. However, information on the payment due date is also required for payment processing. Therefore, in the example of invoice P1, payment data cannot be created.

[0017] Therefore, the bill reading system according to the present disclosure estimates and completes the necessary items when the necessary items for payment processing are not present. Then, when the necessary items for payment processing are completed, the bill reading system according to the present disclosure generates payment data for payment processing.

[0018] With the above configuration, payment data can be automatically generated from the image data of the invoice and payment processing can be performed, making it possible to provide convenient services to customers, which leads to improved convenience for customers.

[0019] <First Embodiment> The following describes the bill reading system 1. In the following description, for example, when the terminal device 10 accesses the server 20, the server 20 responds with information for generating a screen on the terminal device 10. The terminal device 10 generates and displays a screen based on the information received from the server 20.

[0020] <1 Overall configuration of bill reading system 1> FIG. 2 is a block diagram showing the overall configuration of the invoice reading system 1 according to the first embodiment. As shown in FIG. 2, the invoice reading system 1 includes a plurality of terminal devices (terminal device 10A and terminal device 10B are shown in FIG. 2; hereinafter, they may be collectively referred to as "terminal device 10"), a server 20, and an external server 30. The terminal devices 10, the server 20, and the external server 30 are connected to each other via a network 80 so that they can communicate with each other. The network 80 is configured as a wired or wireless network. In this embodiment, the server 20 is a web server (including a cloud server) and exchanges information with the terminal device 10 via web pages. The external server 30 is also a web server (including a cloud server) and exchanges information with the server 20. Note that a web page browser for viewing web pages is installed on the terminal device 10, but a dedicated application for providing the services of the server 20 may also be installed so that the pages can be viewed using the dedicated application.

[0021] The terminal device 10 is a device operated by each user. Here, a user is a person who uses the terminal device 10 to perform payment processing, which is a function of the bill reading system 1, and is, for example, an employee of a business company or a sole proprietor. The terminal device 10 is realized by a desktop personal computer (PC), a laptop PC, or the like. Alternatively, the terminal device 10 may be, for example, a tablet compatible with a mobile communication system, a mobile terminal such as a smartphone, or the like.

[0022] The terminal device 10 is communicatively connected to the server 20 via a network 80. The terminal device 10 is connected to the network 80 by communicating with communication devices such as a wireless base station 81 conforming to communication standards such as 4G, 5G, and LTE (Long Term Evolution), and a wireless LAN router 82 conforming to a wireless LAN (Local Area Network) standard such as IEEE (Institute of Electrical and Electronics Engineers) 802.11. As shown as a terminal device 10B in FIG. 2 , the terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage unit 16, and a processor 19.

[0023] The communication IF 12 is an interface for inputting and outputting signals so that the terminal device 10 can communicate with external devices. The input device 13 is an input device (e.g., a keyboard, a touch panel, a touch pad, a pointing device such as a mouse, etc.) for receiving input operations from a user. The output device 14 is an output device (e.g., a display, a speaker, etc.) for presenting information to a user. The memory 15 is for temporarily storing programs and data processed by the programs, etc., and is a volatile memory such as a DRAM (Dynamic Random Access Memory). The storage unit 16 is a storage device for saving data, such as a flash memory or an HDD (Hard Disc Drive). The processor 19 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0024] The server 20 is a device that reads image data of an invoice, which is a function of the invoice reading system 1, using OCR processing and determines from the read text data whether the items necessary for payment processing are present. If the items necessary for payment processing are not present, the server 20 guesses and completes the necessary items. Furthermore, if the items necessary for payment processing are present or the necessary items have been completed, the server 20 generates payment data for payment processing and performs the payment processing.

[0025] The server 20 is a computer connected to a network 80. The server 20 includes a communication IF 22, an input / output IF 23, a memory 25, a storage 26, and a processor 29.

[0026] The communication IF 22 is an interface for inputting and outputting signals so that the server 20 can communicate with external devices. The input / output IF 23 functions as an interface with an input device for receiving input operations from a user and an output device for presenting information to the user. The memory 25 is for temporarily storing programs and data processed by the programs, etc., and is a volatile memory such as a DRAM (Dynamic Random Access Memory). The storage 26 is a storage device for saving data, such as a flash memory or an HDD (Hard Disc Drive). The processor 29 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0027] <1.1 Configuration of the terminal device 10> FIG. 3 is a block diagram showing the functional configuration of the terminal device 10 of FIG. 2. As shown in FIG. 3, the terminal device 10 includes multiple antennas (antenna 111, antenna 112), wireless communication units (first wireless communication unit 121, second wireless communication unit 122) corresponding to the respective antennas, an operation reception unit 130 (including a keyboard 131 and a mouse 132), an audio processing unit 140, a microphone 141, a speaker 142, a display 150, a camera 160, a storage unit 170, and a control unit 180. The terminal device 10 also has functions and configurations not specifically shown in FIG. 3 (e.g., a battery for storing power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.). As shown in FIG. 3, the blocks included in the terminal device 10 are electrically connected by a bus or the like. The terminal device 10 shown in FIG. 3 will be described as a mobile terminal such as a tablet or smartphone compatible with a mobile communication system, but may also be a stationary personal computer (PC) or a laptop PC.

[0028] The antenna 111 emits a signal emitted by the terminal device 10 as a radio wave. The antenna 111 also receives a radio wave from space and provides the received signal to the first radio communication unit 121.

[0029] The antenna 112 emits a signal emitted by the terminal device 10 as a radio wave. The antenna 112 also receives a radio wave from space and provides the received signal to the second radio communication unit 122.

[0030] The first wireless communication unit 121 performs modulation / demodulation processing and the like for transmitting and receiving signals via the antenna 111 so that the terminal device 10 can communicate with other wireless devices. The second wireless communication unit 122 performs modulation / demodulation processing and the like for transmitting and receiving signals via the antenna 112 so that the terminal device 10 can communicate with other wireless devices. The first wireless communication unit 121 and the second wireless communication unit 122 are communication modules including a tuner, an RSSI (Received Signal Strength Indicator) calculation circuit, a CRC (Cyclic Redundancy Check) calculation circuit, a high-frequency circuit, etc. The first wireless communication unit 121 and the second wireless communication unit 122 perform modulation / demodulation and frequency conversion of wireless signals transmitted and received by the terminal device 10, and provide the received signals to the control unit 180.

[0031] The operation reception unit 130 has a mechanism for receiving input operations from the user in accordance with the control of the control unit 180. Specifically, the operation reception unit 130 includes a keyboard 131 and a mouse 132. Note that the operation reception unit 130 may be configured as a touch screen that detects the position of the user's contact on the touch panel, for example, by using a capacitive touch panel.

[0032] The keyboard 131 accepts input operations by the user of the terminal device 10. The keyboard 131 is a device for inputting characters, and outputs input character information to the control unit 180 as an input signal.

[0033] The mouse 132 accepts input operations by the user of the terminal device 10. The mouse 132 is a pointing device for selecting an object displayed on the display 150, and outputs position information of the selected object on the screen and information indicating that a button has been pressed as input signals to the control unit 180.

[0034] The audio processing unit 140 modulates and demodulates audio signals under the control of the control unit 180. The audio processing unit 140 modulates a signal provided from the microphone 141 and provides the modulated signal to the control unit 180. The audio processing unit 140 also provides the audio signal to the speaker 142. The audio processing unit 140 is realized, for example, by a processor for audio processing. The microphone 141 accepts audio input and provides an audio signal corresponding to the audio input to the audio processing unit 140. The speaker 142 converts the audio signal provided from the audio processing unit 140 into audio and outputs the audio to the outside of the terminal device 10.

[0035] Display 150 displays data such as images, videos, and text under the control of control unit 180. Display 150 is realized by, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0036] Camera 160 is a device that receives light with a light receiving element and outputs the received light as a captured image in accordance with the control of control unit 180. Camera 160 may be configured, for example, by an imaging device such as a digital camera or a video camera.

[0037] Storage unit 170 is configured with memory 15, such as a flash memory, and storage unit 16, and stores data and programs used by terminal device 10. In one aspect, storage unit 170 stores user information 171.

[0038] The user information 171 is information about a user who uses the terminal device 10 to perform payment processing, which is a function of the bill reading system 1. The user information includes information that identifies the user (user ID), the name or title of the business operator or person in charge, etc.

[0039] The control unit 180 is configured by, for example, the processor 19, and controls the operation of the terminal device 10 by reading a program stored in the storage unit 170 and executing instructions included in the program. The control unit 180 performs the functions of an input operation reception unit 181, a transmission / reception unit 182, a notification control unit 183, and a data processing unit 184 by operating in accordance with a program (not shown) that is an application that is stored in advance in the storage unit 170 of the terminal device 10 and installed, for example.

[0040] The input operation receiving unit 181 performs processing to receive input operations by the user via an input device such as the keyboard 131 .

[0041] The transmitting / receiving unit 182 performs processing for the terminal device 10 to transmit and receive data to and from an external device such as the server 20 in accordance with a communication protocol.

[0042] The notification control unit 183 performs processing to present information to the user. The notification control unit 183 performs processing to display a display image on the display 150, processing to output sound from the speaker 142, and the like.

[0043] The data processing unit 184 performs calculations on the data that the terminal device 10 has received as input in accordance with a program, and outputs the calculation results to a memory or the like.

[0044] <1.2 Functional configuration of server 20> Fig. 4 is a block diagram showing the functional configuration of the server 20 of Fig. 2. As shown in Fig. 4, the server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0045] The communication unit 201 performs processing for the server 20 to communicate with external devices.

[0046] The storage unit 202 is composed of, for example, a memory 25 and a storage 26, and stores data and programs used by the server 20. The storage unit 202 stores an image data folder 2021, a transaction database 2022, an accounting database 2023, and the like.

[0047] The image data folder 2021 is a data folder (storage area) for holding image data of invoices. The image data folder 2021 is a data folder provided in a predetermined storage area within the storage unit 202, and is configured, for example, so that image data of invoices can be stored in any order by the user. The image data folder 2021 does not need to be divided into subfolders or the like, but may be divided, for example, by billing party or by time period. Furthermore, the image data of invoices stored in the image data folder 2021 may be in any data format, such as JPEG, PNG, BMP, TIFF, GIF, EPS, or SVG.

[0048] The transaction database 2022 is a database for storing data (transaction data) related to transactions such as purchases of various goods and sales, various payments, etc. by users of the invoice reading system 1, specifically information related to deposits and withdrawals at the user business operators. Details will be described later.

[0049] The accounting database 2023 is a database for holding accounting information relating to transactions by users of the invoice reading system 1, specifically, journal entry information relating to companies, sole proprietors, etc. Details will be described later.

[0050] The control unit 203 performs the functions shown in various modules, such as a receiving control module 2031, a sending control module 2032, an image data reading module 2033, a text data classification module 2034, a payment data determination module 2035, a payment data complement module 2036, a user presentation module 2037, a payment data generation module 2038, and a payment data sending module 2039, by the processor 29 of the server 20 performing processing according to the program.

[0051] The reception control module 2031 controls the process by which the server 20 receives signals from external devices in accordance with a communication protocol.

[0052] The transmission control module 2032 controls the process in which the server 20 transmits signals to external devices in accordance with a communication protocol.

[0053] The image data reading module 2033 controls the process of reading image data of invoices stored in the image data folder 2021 using OCR processing to generate text data. For example, the image data reading module 2033 may perform OCR processing on each item, such as billing company name A2, billing company name A3, billing address A4, etc., in the example of FIG. 1, based on location information indicating the location of the read items in the image data of the invoice, to generate text data. Alternatively, the image data reading module 2033 may read all of the items written in the image data of the invoice P1 (full-text OCR processing). For example, the image data reading module 2033 identifies the type of character based on the attribute information of the read items. For example, if the read character is shaped like a vertical bar, and the character type defined in the attribute information is a number, it is read as "1," and if it is an English letter (alphabet), it is read as a lowercase "l."

[0054] The image data reading module 2033 may determine the character determination and reading process described above using a machine learning model for estimating character recognition. Such a machine learning model may be generated by machine learning performed as a function of the server 20, or may be acquired as a machine learning model generated as a result of machine learning performed by another device. Such machine learning may be performed by supervised machine learning using predetermined training data, unsupervised machine learning, or deep learning. Furthermore, a process of merging the results of learning by further machine learning with the reading results by the image data reading module 2033 may be performed.

[0055] The image data reading module 2033 reads the image data of invoices stored in any order in the image data folder 2021, but the order in which the image data of the invoices is read can be arbitrary, and can be the order of the image data file names, the order of the dates, or, if divided into subfolders, the order of the subfolder names.

[0056] The image data reading module 2033 stores the generated text data in a predetermined area (not shown) of the storage unit 202.

[0057] The text data classification module 2034 controls the process of generating classified text data by classifying the text data generated by the image data reading module 2033 into each item described on the invoice. For example, the text data classification module 2034 classifies the text data read from the image data of the invoice into each item, such as billing company name A2, billing recipient company name A3, billing address A4, etc. in the example of Fig. 1, and generates classified text data for the corresponding item name.

[0058] When the image data reading module 2033 performs OCR processing for each item, the text data classification module 2034 classifies the text data for each item by assigning an item name to it. Also, when the image data reading module 2033 generates text data by full-text OCR processing, the text data classification module 2034 classifies the text data by interpreting the wording of the text data based on item information indicating the items described on the invoice, and assigns item names to the classification results. The text data classification module 2034 may determine the classification of the text data as described above using a machine learning model for estimating the items described in the text data.

[0059] The text data classification module 2034 stores the generated classified text data in a predetermined area (not shown) of the storage unit 202.

[0060] The payment data determination module 2035 controls the process of determining whether items necessary for processing a payment for an invoice are present, based on the categorized text data classified by the text data classification module 2034. The payment data determination module 2035, for example, determines whether items preset as necessary items are present in the categorized text data. For example, the payment data determination module 2035 may determine whether biller information (billing company name A2 in the example of FIG. 1), item information (item A7 in the example of FIG. 1), amount information (total amount A12 in the example of FIG. 1), and due date information are present in the categorized text data as necessary items. This is because payment data must identify the payee (billing source), item, payment amount, and payment due date. The items necessary for processing a payment may differ depending on, for example, the payee (billing source), item, etc., or may be preset for each payee (billing source), item, etc.

[0061] Furthermore, the payment data determination module 2035 may determine whether the classified text data is based on an invoice stored in the image data folder 2021, and if the determination result is that it is based on an invoice, determine whether the above-mentioned required items are present. This is because, for example, if the image data folder 2021 stores not only image data of invoices but also image data of various types of supporting documents, it is first necessary to determine whether the classified text data is based on an invoice.

[0062] If the determination result of the payment data determination module 2035 is that some of the items necessary for payment processing are missing, the payment data complement module 2036 makes an estimate for those missing items and controls the process of complementing them. For example, if biller information (biller's company name A2 in the example of Figure 1), item information (item A7 in the example of Figure 1), and amount information (total amount A12 in the example of Figure 1) are present in the categorized text data, but due date information is missing, the payment data complement module 2036 makes an estimate for the due date information and complements it.

[0063] The payment data completion module 2036, for example, references past payment data for the biller and makes inferences about necessary items. In the example of FIG. 1, the payment data completion module 2036 references past payment data, and if the payment due date to the biller is three months after the invoice issue date (invoice issue date A5 in the example of FIG. 1), it infers that the same is true for the categorized text data, and completes the date three months after the invoice issue date as the payment due date. The payment data completion module 2036 may use a machine learning model for completing items described in categorized text data to complete the above items.

[0064] The user presentation module 2037 presents the items completed by the payment data completion module 2036 to the user and controls the process of accepting input of approval or correction from the user. The user presentation module 2037 transmits information on the items completed by the payment data completion module 2036 to the terminal device 10 via the communication unit 201. The terminal device 10 receives the information on the completed items via the transmission / reception unit 182, and displays the information on the items completed by the processing of the notification control unit 183 on the display 150 to present it to the user.

[0065] Furthermore, when the terminal device 10 receives an input of approval or correction from the user through processing by the input operation receiving unit 181, it transmits the input information of approval or correction from the user via the transmitting / receiving unit 182. The user presentation module 2037 receives and accepts the input information of approval or correction from the user via the communication unit 201.

[0066] If the payment data determination module 2035 determines that all items necessary for processing the payment for the invoice are present, the payment data generation module 2038 controls the process of generating payment data for processing the payment for the invoice. The payment data for processing the payment is data that is sent to the external server 30, which is the server of the financial institution, in order to pay the invoice. The payment data is, for example, in the format of the financial institution performing the payment process (for example, the Zengin format), but it may also be payment data for credit cards or other cashless payments.

[0067] In addition, the payment data generation module 2038 generates payment data for processing the payment for the invoice even when the judgment result of the payment data judgment module 2035 is that the items necessary for processing the payment for the invoice do not exist and the payment data completion module 2036 has completed the items necessary for processing the payment.

[0068] The payment data transmission module 2039 controls the process of transmitting the payment data generated by the payment data generation module 2038 to the external server 30, which is a server of a financial institution. The payment data transmission module 2039 transmits the payment data, for example, via the communication unit 201. The payment data transmission module 2039 determines the financial institution (which may be a credit card operator or other cashless payment operator) that will perform the payment process based on the payment data, etc., and transmits the data to the external server 30 of the relevant financial institution, etc.

[0069] <2 Data Structure> The data structures referenced by the invoice reading system 1 will be described below with reference to Figures 5 and 6. Figure 5 is a diagram showing an example of the data structure of the transaction database 2022 in Figure 4. Figure 6 is a diagram showing an example of the data structure of the accounting database 2023 in Figure 4.

[0070] As shown in FIG. 5, each record in the transaction database 2022 includes an item "customer ID," an item "customer name," an item "detailed transaction information," and the like.

[0071] The item "Customer ID" is information that identifies each business operator that is a customer of the invoice reading system 1.

[0072] The item "Customer Name" is information indicating the name of a business partner managed by the invoice reading system 1, and stores, for example, the name of a corporation or the name of a sole proprietor.

[0073] The "detailed transaction information" item is management information about transactions managed by the invoice reading system 1, and specifically includes the items "date," "summary," "item," and "amount." This "detailed transaction information" item is information registered by users such as businesses and personnel about daily transactions, and is configured so that the latest information is added as a record in chronological order, for example, by the "date" item.

[0074] The item "Date" is information about the date on which a transaction managed by the invoice reading system 1 occurred.

[0075] The item "Summary" is information indicating whether a transaction managed by the invoice reading system 1 is an expenditure (withdrawal) or an income (deposit); for example, if it is an expenditure, "Expense" information is stored, and if it is an income, "Income" information is stored.

[0076] The item "item" is information indicating the specific details of goods, services, etc., for a transaction managed by the invoice reading system 1.

[0077] The item "amount" is information indicating the amount of deposits and withdrawals for transactions managed by the bill reading system 1.

[0078] The server 20 adds a record to the item "detailed transaction information" in the transaction database 2022 as it receives information about the transaction from the user.

[0079] As shown in FIG. 6, each record in the accounting database 2023 includes an item "user ID," an item "user name," an item "detailed journal data information," and the like.

[0080] The item "user ID" is information that identifies each user or company that registers transaction information managed by the invoice reading system 1.

[0081] The item "user name" is the name of the company (company name) or sole proprietorship of the user who is registering the transaction information managed by the invoice reading system 1.

[0082] The item "Detailed journal data information" is journal data that records transactions of deposits and withdrawals such as purchases and sales managed by the invoice reading system 1, and specifically includes the items "Date," "Debit / Credit," "Summary," and "Amount." This "Detailed journal data information information" is information that is journalized based on information registered by a user who is a business operator about daily transactions and registered as accounting information, and is configured so that the latest information is added as a record in chronological order, for example, in the order of the "Date" item.

[0083] The item "Date" is information indicating the date on which a transaction managed by the invoice reading system 1 occurred.

[0084] The item "Debit / Credit" is information that indicates whether the journal entry data is a debit or a credit. For example, if it is a debit, "Debit" information is stored, and if it is a credit, "Credit" information is stored. Debits and credits are items to be recorded in double-entry bookkeeping, which is currently commonly used for recording in ledgers.

[0085] The "Summary" item is information that indicates the account title of the journal entry data. Account titles are information that indicates the details of the expense items used in entering daily transactions, and for example, as shown in Figure 5, details such as "Purchases," "Accounts Payable," and "Suspended Consumption Tax" are stored.

[0086] The item "Amount" is information that indicates the amount of the journal entry data that has been journalized. The item "Amount" stores the amounts of "Purchases," "Accounts Payable," "Suspended Consumption Tax," and other examples of account items indicated by the item "Summary."

[0087] The server 20 performs journalization based on the information about the transaction and registers it as accounting information, updating the item "detailed journalization data information" in the accounting database 2023.

[0088] <3 operations> Hereinafter, the payment data generation process and the payment data transmission process performed by the bill reading system 1 in the first embodiment will be described with reference to FIGS.

[0089] FIG. 7 is a flowchart showing an example of the flow of the payment data generation process performed by the bill reading system 1.

[0090] In step S121, the image data reading module 2033 of the server 20 generates text data by reading the image data of the invoice stored in the image data folder 2021 through OCR processing. In step S121, for example, OCR processing is performed on each item, such as billing company name A2, billing destination company name A3, billing destination address A4, etc. in the example of Figure 1, or on all the items listed on the invoice P1, to generate text data.

[0091] In step S122, the text data classification module 2034 of the server 20 classifies the text data generated in step S121 by each item described on the invoice to generate classified text data. In step S122, for example, the text data read from the image data of the invoice is classified by each item, such as billing company name A2, billing recipient company name A3, billing address A4, etc. in the example of Figure 1, and generates classified text data for the corresponding item name.

[0092] In step S123, the payment data determination module 2035 of the server 20 determines whether or not there are items necessary for processing the payment for the invoice based on the categorized text data classified in step S122. In step S123, for example, it determines whether or not there are items preset as items necessary for processing the payment in the categorized text data.

[0093] In step S124, if the determination result in step S123 is that some of the items necessary for payment processing are missing, the payment data complement module 2036 of the server 20 makes an inference about and complements the missing items. In step S124, for example, if biller information (biller's company name A2 in the example of Figure 1), item information (item A7 in the example of Figure 1), and amount information (total amount A12 in the example of Figure 1) are present in the classified text data but due date information is missing, the due date information is inferred and complemented.

[0094] In step S125, the user presentation module 2037 of the server 20 presents the items completed in step S124 to the user and accepts input of approval or correction from the user. In step S125, information on the completed items is transmitted to the terminal device 10 via the communication unit 201.

[0095] In step S115, the transmitting / receiving unit 182 of the terminal device 10 receives the information on the completed item transmitted in step S125. The notification control unit 183 displays the received information on the (completed) item on the display 150 to present it to the user. In addition, the input operation receiving unit 181 of the terminal device 10 receives an input of approval or correction from the user for the completed item, and transmits the input information of approval or correction from the user via the transmitting / receiving unit 182.

[0096] In step S126, if the determination result in step S123 is that all items necessary for processing the payment for the invoice are present, the payment data generation module 2038 of the server 20 generates payment data for processing the payment for the invoice. Also, in step S126, if the determination result in step S123 is that items necessary for processing the payment for the invoice are not present and the items necessary for processing the payment are supplemented in step S124, the payment data generation module 2038 of the server 20 generates payment data for processing the payment for the invoice.

[0097] As described above, the invoice reading system 1 reads image data of an invoice using OCR processing and determines from the read text data whether the items necessary for payment processing are present. If the items necessary for payment processing are present, the invoice reading system 1 generates payment data for payment processing. The invoice reading system 1 also generates payment data for payment processing even if the items necessary for payment processing are not present but the items necessary for payment processing are completed. Therefore, payment data can be automatically generated from image data of an invoice and used to process payments, making it possible to provide convenient services to customers. This can lead to improved convenience for customers.

[0098] FIG. 8 is a flowchart showing an example of the flow of the payment data transmission process performed by the bill reading system 1.

[0099] In step S221, the payment data transmission module 2039 of the server 20 transmits the payment data generated in step S126 to the external server 30, which is a server of a financial institution. In step S221, the payment data is transmitted via the communication unit 201 to the external server 30 of the financial institution or the like.

[0100] In step S231, upon receiving the payment data, the external server 30 performs payment (settlement) processing in accordance with the payment data.

[0101] As described above, the bill reading system 1 transmits payment data for performing payment processing to an external server 30 such as a financial institution, and causes the payment processing to be performed. This allows the payment processing to be performed automatically, making it possible to provide convenient services to customers.

[0102] <Summary> As described above, according to this embodiment, image data of an invoice is read using OCR processing, and the presence of items necessary for payment processing is determined from the read text data. If the necessary items for payment processing are present, payment data for payment processing is generated. Therefore, payment data can be automatically generated from image data of an invoice and payment processing can be performed, making it possible to provide convenient services to customers. This can lead to improved convenience for customers.

[0103] Furthermore, according to this embodiment, image data of an invoice is scanned using OCR processing, and the scanned text data is used to determine whether the necessary items for payment processing are present. If the necessary items for payment processing are not present, the necessary items are supplemented and payment data for payment processing is generated. Therefore, since the necessary items for payment processing can be supplemented from the image data of the invoice and payment data can be automatically generated and payment processing can be performed, it is possible to provide convenient services to customers. This can lead to improved convenience for customers.

[0104] Furthermore, according to this embodiment, payment data for performing payment processing is transmitted to a financial institution, a settlement institution, etc., and payment processing is performed. This allows payment processing to be performed automatically, making it possible to provide convenient services to customers.

[0105] Although the embodiments of the disclosure have been described above, they can be implemented in various other forms and can be implemented with various omissions, substitutions, and modifications. These embodiments and modifications, as well as those with omissions, substitutions, and modifications, are included in the technical scope of the claims and their equivalents.

[0106] <Additional Notes> The matters described in the above embodiments will be supplemented below.

[0107] (Supplementary Note 1) A program to be executed by a computer having a processor 29 and a memory 25, wherein the memory 25 stores image data (2021) of an invoice related to a user, and the program causes the processor 29 to execute the following steps: a step (S121) of reading the image data of the invoice using OCR processing to generate text data; a step (S122) of classifying the generated text data by item listed on the invoice to generate categorized text data; a step (S123) of determining whether there are items necessary for processing payment for the invoice based on the classified categorized text data; and a step (S126) of generating payment data for processing payment for the invoice if the determination result shows that all necessary items are present.

[0108] (Appendix 2) A program described in (Appendix 1), in which, in the step of generating text data, image data of an invoice is read by OCR processing from a storage area (2021) in which image data including image data of multiple invoices is stored in no particular order.

[0109] (Appendix 3) A program described in (Appendix 1), in which, in the determining step, it is determined whether the classified text data is based on an invoice, and if the determination result is that it is based on an invoice, it is determined whether the items necessary to process payment for the invoice exist.

[0110] (Appendix 4) The program described in (Appendix 1), wherein in the determining step, it is determined whether the necessary items in the classification text data include biller information, item information, amount information, and date information.

[0111] (Appendix 5) A program described in (Appendix 1), in which, in the step of generating payment data, payment data corresponding to the format of the financial institution that processes the payment for the invoice is generated, and the program further executes a step (S221) of sending the payment data to the server of the financial institution.

[0112] (Appendix 6) The program described in any of (Appendix 1) to (Appendix 5) further executes a step (S124) of inferring and completing the missing required items when the determination result shows that some of the required items do not exist, and generates payment data based on the completed items in the step of generating payment data.

[0113] (Appendix 7) A program as described in (Appendix 6), in which, in the step of completing the necessary items, past payment data for the biller is referenced and inferences are made regarding the necessary items based on one or more of the biller information, item information, and amount information in the classified text data.

[0114] (Supplementary Note 8) The program according to (Supplementary Note 6), further comprising: a step (S125) of presenting the completed items to the user and receiving input of approval or correction from the user.

[0115] (Appendix 9) An information processing device comprising a control unit 203 and a memory unit 202, wherein the memory unit 202 stores image data (2021) of an invoice related to a user, and the control unit 203 executes the steps of: reading the image data of the invoice using OCR processing to generate text data (S121); classifying the generated text data into items described on the invoice to generate categorized text data (S122); determining based on the classified categorized text data whether there are any items necessary for processing payment for the invoice (S123); and, if the determination result indicates that all necessary items are present, generating payment data for processing payment for the invoice (S126).

[0116] (Appendix 8) A method to be executed by a computer having a processor 29 and a memory 25, wherein the memory 25 stores image data (2021) of an invoice related to a user, and the method comprises the steps of: the processor 29 reading the image data of the invoice using OCR processing to generate text data (S121); classifying the generated text data into items listed on the invoice to generate categorized text data (S122); determining based on the classified categorized text data whether there are any items necessary for processing the payment for the invoice (S123); and, if the determination result indicates that all necessary items are present, generating payment data for processing the payment for the invoice (S126). [Explanation of symbols]

[0117] 1: Invoice reading system 10: Terminal device 10A: Terminal equipment 10B: Terminal device 13: Input device 14: Output device 15: Memory 16: Storage section 19: Processor 20: Server 25: Memory 26: Storage 29: Processor 30: External server 201: Communications Department 202: Storage section 203: Control unit 2021: Image data folder 2022: Transaction Database 2023: Accounting Database 2031: Receiving control module 2032: Transmission control module 2033: Image data reading module 2034: Text data classification module 2035: Payment data determination module 2036: Payment data completion module 2037: User presentation module 2038: Payment data generation module 2039: Payment data transmission module

Claims

1. A program to be executed by a computer having a processor and a memory, The memory stores image data of invoices and image data other than invoices related to the user, The program causes the processor to: a step of reading the image data of the invoice in a storage area of the memory, in which image data including image data of the plurality of invoices is stored in no particular order, by OCR processing to generate text data; a step of classifying the generated text data by each item described on the invoice to generate classified text data; a step of determining whether the classified text data is based on the invoice based on the classified text data, and if the determination result is that the classified text data is based on the invoice, determining whether there are items necessary for performing payment processing for the invoice; If the determination result indicates that all of the required items are present, generating payment data corresponding to the format of a financial institution that processes payment for the invoice; transmitting the payment data to an external server of a financial institution that processes payment for the invoice.

2. 2. The program according to claim 1, wherein the step of determining whether or not biller information, item information, amount information, and due date information are present in the classification text data as the necessary items.

3. The program further comprises: If the determination result indicates that some of the necessary items are not present, a step of inferring and completing the missing necessary items is executed; 3. The program according to claim 1, wherein in the step of generating the payment data, the payment data is generated based on the supplemented items.

4. 4. The program according to claim 3, wherein in the step of completing the necessary items, past payment data for the biller is referenced based on one or more of biller information, item information, and amount information in the classified text data, and an inference is made regarding the necessary items.

5. The program further comprises:

4. The program according to claim 3, further comprising a step of presenting the completed items to a user and receiving input of approval or correction from the user.

6. An information processing device including a control unit and a storage unit, the storage unit stores image data of invoices related to users and image data other than invoices; The control unit a step of reading the image data of the invoice in a storage area in the storage unit, in which image data including image data of the plurality of invoices is stored in no particular order, by OCR processing to generate text data; a step of classifying the generated text data by each item described on the invoice to generate classified text data; a step of determining whether the classified text data is based on the invoice based on the classified text data, and if the determination result is that the classified text data is based on the invoice, determining whether there are items necessary for performing payment processing for the invoice; If the determination result indicates that all of the required items are present, generating payment data corresponding to the format of a financial institution that processes payment for the invoice; transmitting the payment data to an external server of a financial institution that processes payment for the invoice.

7. 1. A method to be executed by a computer having a processor and a memory, comprising: The memory stores image data of invoices and image data other than invoices related to the user, The method further comprises the processor: a step of reading the image data of the invoice in a storage area of the memory, in which image data including image data of the plurality of invoices is stored in no particular order, by OCR processing to generate text data; a step of classifying the generated text data by each item described on the invoice to generate classified text data; a step of determining whether the classified text data is based on the invoice based on the classified text data, and if the determination result is that the classified text data is based on the invoice, determining whether there are items necessary for performing payment processing for the invoice; If the determination result indicates that all of the required items are present, generating payment data corresponding to the format of a financial institution that processes payment for the invoice; transmitting the payment data to an external server of a financial institution that processes payment for the invoice.

Images