Program, information processing device, and method

Abstract

To provide a technique capable of reading a bill by OCR processing and performing payment processing from the read text data.SOLUTION: A server of a bill reading system causes a processor to perform the steps of: storing image data (2021) of a bill related to a user and generating text data by reading the image data of the bill by OCR processing as the function (S121); generating classification text data by classifying the generated text data per bill description item (S122); determining whether items necessary to perform payment processing for the bill exist on the basis of the classified classification text data (S123); and when the determination results indicate that all necessary items exist, generating payment data for performing payment processing for the bill (S126).SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to a program, an information processing device, and a method. [Background technology]

[0002] Optical character recognition (OCR) is a well-known technology that uses an optical reader such as a camera or scanner to recognize documents printed on paper forms. There is also a known technology that classifies form images by image analysis.

[0003] As a technology for a document classification system, Patent Document 1 discloses a technology for a document classification system that analyzes image information of evidence, classifies the evidence, determines a storage destination, and stores the image information of the evidence. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 136778 Summary of the Invention [Problem to be solved by the invention]

[0005] By the way, if it were possible to further utilize the text data read after scanning evidential documents using OCR processing, it would be possible to provide convenient services to customers. For example, if an invoice, which is an example of evidential documents, could be scanned using OCR processing and subsequent processing, such as payment processing, could be performed, which would improve convenience for customers, and so such a function was in demand.

[0006] Therefore, this disclosure describes a technology that enables invoices to be read using OCR processing and payment processing to be performed from the read text data. [Means for solving the problem]

[0007] According to one embodiment of the present disclosure, there is provided a program to be executed by a computer having a processor and a memory, wherein the memory stores image data of an invoice related to a user, and the program causes the processor to execute the steps of: reading the image data of the invoice using OCR processing to generate text data; classifying the generated text data by items listed on the invoice to generate categorized text data; determining whether items necessary for processing the payment for the invoice are present based on the classified categorized text data; and, if the determination result indicates that all necessary items are present, generating payment data for processing the payment for the invoice. Effect of the Invention

[0008] According to the present disclosure, an invoice is read using OCR processing, and from the read text data, it is determined whether there are any items necessary for processing the payment, and payment data for processing the payment is generated, which makes it possible to improve convenience for customers. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing an example of image data of an invoice to be read in the present disclosure. [Diagram 2] 1 is a block diagram showing the overall configuration of a bill reading system 1 according to a first embodiment. [Diagram 3] 3 is a block diagram showing a functional configuration of the terminal device 10 of FIG. 2. [Figure 4] 3 is a block diagram showing a functional configuration of the server 20 of FIG. 2. [Diagram 5] FIG. 5 is a diagram showing an example of a data structure of a transaction database 2022 in FIG. 4. [Figure 6] FIG. 5 is a diagram showing an example of the data structure of an accounting database 2023 in FIG. 4. [Figure 7] 10 is a flowchart showing an example of the flow of a payment data generation process performed by the invoice reading system 1. [Figure 8] 10 is a flowchart showing an example of the flow of a payment data transmission process performed by the invoice reading system 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. The following description and drawings are examples for explaining the present invention, and are omitted and simplified as appropriate for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural. In the following description, the same components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0011] <Summary> The invoice reading system according to the present disclosure will be described below. The invoice reading system according to the present disclosure is a system that reads image data of an invoice by OCR processing and determines whether or not items necessary for performing payment processing are present from the read text data. The invoice reading system according to the present disclosure is configured to infer and supplement the necessary items when the items necessary for performing payment processing are not present. Furthermore, when the items necessary for performing payment processing are present or when the necessary items have been supplemented, the invoice reading system according to the present disclosure generates payment data for performing payment processing and performs the payment processing. The invoice reading system according to the present disclosure is a system provided by a so-called SaaS (Software as a Service) as a Web service, for example, by a cloud server or the like.

[0012] The invoice reading system according to the present disclosure manages transaction data for users, such as companies and individual business owners, and provides accounting services. The accounting services provided to users include managing cash transactions such as purchases, sales, and borrowings, recording (entering) transactions in ledgers (journal entries, general ledgers), creating trial balance sheets as settlement processing at a specified accounting period, and creating financial statements such as income statements and balance sheets.

[0013] FIG. 1 is a diagram showing an example of image data of an invoice read in the present disclosure. Image data read by the information processing system according to the present disclosure is image data of an invoice P1, shown as an example in FIG. 1. The invoice P1 is a form on which the image data of the invoice to be subjected to OCR processing is based, and examples of read items include a title A1, a billing company name A2, a billing company name A3, a billing address A4, an invoice issue date A5, a subject A6, an item A7, a quantity A8, an itemized billing amount A9, a subtotal A10, a consumption tax amount A11, and a total A12. In this invoice reading system, the position information shown by the dashed line in FIG. 1 within each read item shown in FIG. 1 is used as area data, and coordinate information with the upper left corner of the invoice P1 as the origin is extracted. In addition, character strings included in this area data are read and extracted by OCR processing.

[0014] By the way, if it were possible to read the invoice P1 using OCR processing and perform payment processing from the read text data, it would be possible to provide a convenient service to customers. In order to perform payment processing, it is necessary to extract items required for payment processing from each read item of the invoice P1 and create payment data.

[0015] Therefore, the invoice reading system according to the present disclosure reads image data of an invoice, for example invoice P1, using OCR processing, and determines whether the read text data contains items necessary for payment processing. If the items necessary for payment processing are present, the invoice reading system according to the present disclosure generates payment data for payment processing.

[0016] In addition, in the example of invoice P1, the items required for payment processing include the billing company name A3, item A7, and total amount A12. However, the items required for payment processing also include information on the payment due date. Therefore, in the example of invoice P1, payment data cannot be created.

[0017] Therefore, when the items necessary for performing payment processing are not present, the bill reading system according to the present disclosure infers and completes the necessary items. Then, when the items necessary for performing payment processing are completed, the bill reading system according to the present disclosure generates payment data for performing payment processing.

[0018] With the above configuration, payment data can be automatically generated from the image data of the invoice and payment processing can be performed, making it possible to provide convenient services to customers, which leads to improved convenience for customers.

[0019] <Embodiment 1> The following describes the invoice reading system 1. In the following description, for example, when the terminal device 10 accesses the server 20, the server 20 responds with information for generating a screen on the terminal device 10. The terminal device 10 generates and displays a screen based on the information received from the server 20.

[0020] <1 Overall configuration of invoice reading system 1> FIG. 2 is a block diagram showing the overall configuration of the invoice reading system 1 according to the first embodiment. As shown in FIG. 2, the invoice reading system 1 includes a plurality of terminal devices (terminal device 10A and terminal device 10B are shown in FIG. 2. Hereinafter, they may be collectively referred to as "terminal device 10"), a server 20, and an external server 30. The terminal device 10, the server 20, and the external server 30 are connected to each other via a network 80 so as to be able to communicate with each other. The network 80 is configured as a wired or wireless network. In this embodiment, the server 20 is a Web server (including a cloud server) and exchanges information with the terminal device 10 by using Web pages. The external server 30 is a Web server (including a cloud server) and exchanges information with the server 20. Note that a Web page browser for viewing Web pages is installed in the terminal device 10, but a dedicated application for providing the services of the server 20 may be installed and configured to be viewable by the dedicated application.

[0021] The terminal device 10 is a device operated by each user. Here, a user is a person who uses the terminal device 10 to perform payment processing, which is a function of the bill reading system 1, and is, for example, an employee of a business enterprise or a sole proprietor. The terminal device 10 is realized by a stationary PC (Personal Computer), a laptop PC, or the like. Alternatively, the terminal device 10 may be, for example, a tablet compatible with a mobile communication system, a mobile terminal such as a smartphone, or the like.

[0022] The terminal device 10 is communicatively connected to the server 20 via a network 80. The terminal device 10 is connected to the network 80 by communicating with communication devices such as a wireless base station 81 compatible with communication standards such as 4G, 5G, and LTE (Long Term Evolution), and a wireless LAN router 82 compatible with wireless LAN (Local Area Network) standards such as IEEE (Institute of Electrical and Electronics Engineers) 802.11. As shown as the terminal device 10B in FIG. 2, the terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage unit 16, and a processor 19.

[0023] The communication IF 12 is an interface for inputting and outputting signals so that the terminal device 10 can communicate with an external device. The input device 13 is an input device (e.g., a keyboard, a touch panel, a touch pad, a pointing device such as a mouse, etc.) for receiving an input operation from a user. The output device 14 is an output device (a display, a speaker, etc.) for presenting information to a user. The memory 15 is for temporarily storing a program and data processed by the program, etc., and is a volatile memory such as a DRAM (Dynamic Random Access Memory). The storage unit 16 is a storage device for saving data, and is, for example, a flash memory or a HDD (Hard Disc Drive). The processor 19 is hardware for executing an instruction set described in a program, and is composed of an arithmetic unit, a register, a peripheral circuit, etc.

[0024] The server 20 is a device that reads image data of an invoice, which is a function of the invoice reading system 1, by OCR processing, and determines whether or not items necessary for payment processing are present from the read text data. If items necessary for payment processing are not present, the server 20 guesses and supplements the necessary items. Furthermore, if items necessary for payment processing are present or if the necessary items have been supplemented, the server 20 generates payment data for payment processing and performs the payment processing.

[0025] The server 20 is a computer connected to a network 80. The server 20 includes a communication IF 22, an input / output IF 23, a memory 25, a storage 26, and a processor 29.

[0026] The communication IF 22 is an interface for inputting and outputting signals so that the server 20 can communicate with an external device. The input / output IF 23 functions as an interface with an input device for receiving an input operation from a user and an output device for presenting information to a user. The memory 25 is for temporarily storing programs and data processed by the programs, etc., and is a volatile memory such as a DRAM (Dynamic Random Access Memory). The storage 26 is a storage device for saving data, such as a flash memory or a HDD (Hard Disc Drive). The processor 29 is hardware for executing an instruction set described in a program, and is composed of an arithmetic unit, a register, a peripheral circuit, etc.

[0027] <1.1 Configuration of the terminal device 10> FIG. 3 is a block diagram showing a functional configuration of the terminal device 10 of FIG. 2. As shown in FIG. 3, the terminal device 10 includes a plurality of antennas (antenna 111, antenna 112), wireless communication units (first wireless communication unit 121, second wireless communication unit 122) corresponding to the respective antennas, an operation reception unit 130 (including a keyboard 131 and a mouse 132), a voice processing unit 140, a microphone 141, a speaker 142, a display 150, a camera 160, a storage unit 170, and a control unit 180. The terminal device 10 also has functions and configurations (e.g., a battery for holding power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.) that are not particularly shown in FIG. 3. As shown in FIG. 3, each block included in the terminal device 10 is electrically connected by a bus or the like. The terminal device 10 shown in FIG. 3 will be described as a mobile terminal such as a tablet or a smartphone compatible with a mobile communication system, but may be a stationary PC (Personal Computer) or a laptop PC.

[0028] The antenna 111 emits a signal generated by the terminal device 10 as a radio wave. The antenna 111 also receives a radio wave from space and provides the received signal to the first wireless communication unit 121.

[0029] The antenna 112 radiates a signal generated by the terminal device 10 as a radio wave. The antenna 112 also receives the radio wave from space and provides the received signal to the second radio communication unit 122.

[0030] The first wireless communication unit 121 performs modulation / demodulation processing and the like for transmitting and receiving signals via the antenna 111 so that the terminal device 10 can communicate with other wireless devices. The second wireless communication unit 122 performs modulation / demodulation processing and the like for transmitting and receiving signals via the antenna 112 so that the terminal device 10 can communicate with other wireless devices. The first wireless communication unit 121 and the second wireless communication unit 122 are communication modules including a tuner, a Received Signal Strength Indicator (RSSI) calculation circuit, a Cyclic Redundancy Check (CRC) calculation circuit, a high-frequency circuit, and the like. The first wireless communication unit 121 and the second wireless communication unit 122 perform modulation / demodulation and frequency conversion of wireless signals transmitted and received by the terminal device 10, and provide the received signals to the control unit 180.

[0031] The operation reception unit 130 has a mechanism for receiving an input operation from a user in response to the control of the control unit 180. Specifically, the operation reception unit 130 includes a keyboard 131 and a mouse 132. Note that the operation reception unit 130 may be configured as a touch screen that detects the position of the user's contact with the touch panel, for example, by using a capacitive touch panel.

[0032] The keyboard 131 accepts input operations by the user of the terminal device 10. The keyboard 131 is a device for inputting characters, and outputs input character information to the control unit 180 as an input signal.

[0033] The mouse 132 accepts input operations by the user of the terminal device 10. The mouse 132 is a pointing device for selecting an object displayed on the display 150, and outputs position information of an object selected on the screen and information indicating that a button has been pressed to the control unit 180 as input signals.

[0034] The audio processing unit 140 modulates and demodulates an audio signal under the control of the control unit 180. The audio processing unit 140 modulates a signal provided from the microphone 141 and provides the modulated signal to the control unit 180. The audio processing unit 140 also provides the audio signal to the speaker 142. The audio processing unit 140 is realized by, for example, a processor for audio processing. The microphone 141 accepts audio input and provides an audio signal corresponding to the audio input to the audio processing unit 140. The speaker 142 converts the audio signal provided from the audio processing unit 140 into audio and outputs the audio to the outside of the terminal device 10.

[0035] Display 150 displays data such as images, videos, and text under the control of control unit 180. Display 150 is realized by, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0036] Camera 160 is a device that receives light with a light receiving element and outputs the light as a captured image in accordance with the control of control unit 180. Camera 160 may be configured with an imaging device such as a digital camera or a video camera.

[0037] Storage unit 170 is configured with memory 15, such as a flash memory, and storage unit 16, and stores data and programs used by terminal device 10. In one aspect, storage unit 170 stores user information 171.

[0038] The user information 171 is information about a user who uses the terminal device 10 to perform payment processing, which is a function of the bill reading system 1. The user information includes information for identifying the user (user ID), the name or title of the business operator or person in charge, etc.

[0039] The control unit 180 is constituted by, for example, the processor 19, and controls the operation of the terminal device 10 by reading a program stored in the storage unit 170 and executing instructions included in the program. The control unit 180 fulfills the functions of an input operation receiving unit 181, a transmission / reception unit 182, a notification control unit 183, and a data processing unit 184 by operating in accordance with a program (not shown) that is an application that is stored in advance in the storage unit 170 of the terminal device 10 and installed, for example.

[0040] The input operation receiving unit 181 performs processing for receiving input operations by a user via an input device such as the keyboard 131 .

[0041] The transmission / reception unit 182 performs processing for the terminal device 10 to transmit and receive data to and from an external device such as the server 20 in accordance with a communication protocol.

[0042] The notification control unit 183 performs a process of presenting information to the user. The notification control unit 183 performs a process of causing the display 150 to display a display image, a process of causing the speaker 142 to output a sound, and the like.

[0043] The data processing unit 184 performs calculations on the data that the terminal device 10 accepts as input in accordance with a program, and outputs the calculation results to a memory or the like.

[0044] <1.2 Functional configuration of server 20> Fig. 4 is a block diagram showing a functional configuration of the server 20 of Fig. 2. As shown in Fig. 4, the server 20 fulfills the functions of a communication unit 201, a storage unit 202, and a control unit 203.

[0045] The communication unit 201 performs processing for the server 20 to communicate with external devices.

[0046] The storage unit 202 is composed of, for example, a memory 25 and a storage 26, and stores data and programs used by the server 20. The storage unit 202 stores an image data folder 2021, a transaction database 2022, an accounting database 2023, and the like.

[0047] The image data folder 2021 is a data folder (storage area) for holding image data of invoices. The image data folder 2021 is a data folder provided in a predetermined storage area in the storage unit 202, and is configured to store image data of invoices in random order by the user, for example. The image data folder 2021 does not need to be divided into subfolders or the like, but may be divided, for example, by invoice source or time period. Invoice image data stored in the image data folder 2021 may be in any data format, such as jpeg, png, bmp, tiff, gif, eps, svg, etc.

[0048] The transaction database 2022 is a database for holding data (transaction data) on transactions such as purchases of various goods and sales, various payments, etc., by users of the invoice reading system 1, specifically, information on deposits and withdrawals at businesses that are users. Details will be described later.

[0049] The accounting database 2023 is a database for holding accounting information relating to transactions by users of the invoice reading system 1, specifically, journal entry information relating to companies, sole proprietorships, etc. Details will be described later.

[0050] The control unit 203 performs functions as various modules, such as a receiving control module 2031, a transmitting control module 2032, an image data reading module 2033, a text data classification module 2034, a payment data determination module 2035, a payment data complement module 2036, a user presentation module 2037, a payment data generation module 2038, and a payment data transmission module 2039, by the processor 29 of the server 20 performing processing in accordance with a program.

[0051] The reception control module 2031 controls the process in which the server 20 receives a signal from an external device in accordance with a communication protocol.

[0052] The transmission control module 2032 controls the process in which the server 20 transmits signals to external devices in accordance with a communication protocol.

[0053] The image data reading module 2033 controls the process of reading the image data of the invoice stored in the image data folder 2021 by OCR processing to generate text data. For example, the image data reading module 2033 may perform OCR processing for each item of the billing company name A2, billing company name A3, billing address A4, etc. in the example of FIG. 1 based on the position information indicating the position of the read item of the image data of the invoice to generate text data, or may read all the items written in the image data of the invoice P1 (full text OCR processing). The image data reading module 2033 identifies the type of character based on, for example, the attribute information of the read item. For example, if the read character has a vertical bar shape, if the type of character defined in the attribute information is a number, it is determined to be "1" and read, and if it is an English letter (alphabet), it is determined to be a lowercase "l" and read.

[0054] The image data reading module 2033 may determine the above-mentioned character determination and reading process using a machine learning model for estimating character recognition. Such a machine learning model may be generated by machine learning performed as a function of the server 20, or a machine learning model generated as a result of machine learning performed by another device may be acquired. Such machine learning may be performed by supervised machine learning using predetermined teacher data, unsupervised machine learning, or deep learning. In addition, a process of merging the learning result by further machine learning with the reading result by the image data reading module 2033 may be performed.

[0055] The image data reading module 2033 reads the image data of invoices stored in any order in the image data folder 2021, but the order in which the image data of the invoices is read can be arbitrary, and can be the order of the image data file names, the order of the dates, or, if divided into subfolders, the order of the subfolder names.

[0056] The image data reading module 2033 stores the generated text data in a predetermined area (not shown) of the storage unit 202.

[0057] The text data classification module 2034 controls the process of generating categorized text data by classifying the text data generated by the image data reading module 2033 for each item described on the invoice. For example, the text data classification module 2034 classifies the text data read from the image data of the invoice for each item, such as billing company name A2, billing recipient company name A3, billing address A4, ... in the example of Fig. 1, and generates categorized text data for the corresponding item name.

[0058] When the image data reading module 2033 performs OCR processing for each item, the text data classification module 2034 classifies the text data for each item by assigning an item name to the text data. When the image data reading module 2033 generates text data by full-text OCR processing, the text data classification module 2034 classifies the text data by interpreting the wording of the text data based on item information indicating the items described on the invoice, and assigns item names to the classification results. The text data classification module 2034 may determine the classification of the text data as described above using a machine learning model for estimating the items described in the text data.

[0059] The text data classification module 2034 stores the generated classified text data in a predetermined area (not shown) of the storage unit 202.

[0060] The payment data determination module 2035 controls a process for determining whether or not there are items necessary for performing payment processing for an invoice, based on the categorized text data classified by the text data classification module 2034. The payment data determination module 2035, for example, determines whether or not items previously set as necessary items exist in the categorized text data. The payment data determination module 2035 may determine, for example, whether or not there are biller information (biller company name A2 in the example of FIG. 1), item information (item A7 in the example of FIG. 1), amount information (total amount A12 in the example of FIG. 1), and due date information in the categorized text data, as necessary items. This is because the payment data needs to specify the payee (biller), item, payment amount, and due date. Note that the items necessary for performing payment processing may differ depending on, for example, the payee (biller), item, etc., or may be set in advance for each payee (biller), item, etc.

[0061] Furthermore, the payment data determination module 2035 may determine whether the classified text data is based on an invoice stored in the image data folder 2021, and if the determination result is that the classified text data is based on an invoice, determine whether the above-mentioned required items are present. This is because, for example, in a case where the image data folder 2021 stores not only image data of invoices but also image data of various types of vouchers, it is necessary to first determine whether the classified text data is based on an invoice.

[0062] When the determination result of the payment data determination module 2035 is that some of the items necessary for payment processing are missing, the payment data complementation module 2036 makes an inference about the missing items and controls the process of complementing them. For example, when biller information (biller company name A2 in the example of Figure 1), item information (item A7 in the example of Figure 1), and amount information (total amount A12 in the example of Figure 1) are present in the categorized text data, but due date information is missing, the payment data complementation module 2036 makes an inference about the due date information and complements it.

[0063] The payment data complement module 2036, for example, refers to past payment data for the biller and infers necessary items. In the example of Fig. 1, the payment data complement module 2036 refers to past payment data, and if the due date for payment to the biller is three months after the invoice issue date (invoice issue date A5 in the example of Fig. 1), it infers that the same is true for the categorized text data, and complements the date three months after the invoice issue date as the payment due date. The payment data complement module 2036 may use a machine learning model for complementing the items described in the categorized text data to complement the above items.

[0064] The user presentation module 2037 presents the items complemented by the payment data complement module 2036 to the user and controls the process of accepting input of approval or correction from the user. The user presentation module 2037 transmits information on the items complemented by the payment data complement module 2036 to the terminal device 10 via the communication unit 201. The terminal device 10 receives the information on the complemented items via the transmission / reception unit 182, and displays the information on the items complemented by the process by the notification control unit 183 on the display 150 to present it to the user.

[0065] Furthermore, when the terminal device 10 receives an input of approval or correction from the user in the process by the input operation receiving unit 181, the terminal device 10 transmits the input information of approval or correction from the user via the transmitting / receiving unit 182. The user presentation module 2037 receives and accepts the input information of approval or correction from the user via the communication unit 201.

[0066] The payment data generation module 2038 controls the process of generating payment data for performing payment processing for an invoice when the determination result of the payment data determination module 2035 indicates that all items necessary for performing payment processing for the invoice are present. The payment data for performing payment processing is data to be sent to the external server 30, which is the server of the financial institution, in order to perform payment for the invoice. The payment data corresponds, for example, to the format (e.g., all-bank format) of the financial institution performing the payment processing, but may also be payment data corresponding to credit card or other cashless payment.

[0067] In addition, the payment data generation module 2038 also generates payment data for processing the payment for an invoice when the payment data determination module 2035 determines that the items necessary for processing the payment for the invoice do not exist and the payment data complementation module 2036 complements the items necessary for processing the payment.

[0068] The payment data transmission module 2039 controls the process of transmitting the payment data generated by the payment data generation module 2038 to the external server 30, which is a server of a financial institution. The payment data transmission module 2039 transmits the payment data, for example, via the communication unit 201. The payment data transmission module 2039 determines the financial institution performing the payment process (which may be a credit card or other cashless payment management institution) from the payment data, etc., and transmits the data to the external server 30 of the relevant financial institution, etc.

[0069] <2 Data Structure> The data structures referenced by the invoice reading system 1 will be described below with reference to Fig. 5 and Fig. 6. Fig. 5 is a diagram showing an example of the data structure of the transaction database 2022 in Fig. 4. Fig. 6 is a diagram showing an example of the data structure of the accounting database 2023 in Fig. 4.

[0070] As shown in FIG. 5, each record in the transaction database 2022 includes an item "customer ID", an item "customer name", an item "detailed transaction information", and the like.

[0071] The item “business partner ID” is information for identifying each business entity that is a business partner of the invoice reading system 1.

[0072] The item "client name" is information indicating the name of a client company managed by the invoice reading system 1, and stores, for example, the name of a corporation or the name of a sole proprietor.

[0073] The item "transaction detail information" is management information regarding transactions managed by the invoice reading system 1, and specifically includes the items "date", "summary", "item", and "amount". This item "transaction detail information" is information registered by users such as business operators and personnel in charge regarding daily transactions, and is configured so that the latest information is added as a record in chronological order, for example, by the item "date".

[0074] The item "Date" is information on the date on which a transaction managed by the invoice reading system 1 occurred.

[0075] The item "Summary" is information indicating whether a transaction managed by the invoice reading system 1 is an expenditure (withdrawal) or income (deposit); for example, if it is an expenditure, "expense" information is stored, and if it is income, "income" information is stored.

[0076] The item “item” is information indicating the specific details of goods, services, etc., regarding a transaction managed by the invoice reading system 1.

[0077] The item “amount” is information indicating the amount of deposits and withdrawals for transactions managed by the invoice reading system 1.

[0078] The server 20 adds a record to the item “transaction detail information” of the transaction database 2022 as the server 20 receives information about the transaction from the user.

[0079] As shown in FIG. 6, each record of the accounting database 2023 includes an item "user ID," an item "user name," an item "journal entry data detail information," and the like.

[0080] The item “user ID” is information for identifying each user, company, etc. who registers transaction information to be managed by the invoice reading system 1.

[0081] The item “user name” is the name of the business (company name) or the name of the individual business owner who is the user who registers the transaction information managed by the invoice reading system 1.

[0082] The item "journal entry data detail information" is journal entry data that records deposit and withdrawal transactions such as purchases and sales managed by the invoice reading system 1, and specifically includes the items "date", "debit / credit", "summary", and "amount". This item "journal entry data detail information" is information that is journalized and registered as accounting information based on information registered by a user who is a business owner regarding daily transactions, and is configured so that the latest information is added as a record in chronological order of the item "date", for example.

[0083] The item “Date” is information indicating the date on which a transaction managed by the invoice reading system 1 occurred.

[0084] The item "Debit / Credit" is information that indicates whether the journal entry data is a debit or credit. For example, if it is a debit, "debit" information is stored, and if it is a credit, "credit" information is stored. Debit and credit are items to be recorded in double-entry bookkeeping, which is currently commonly used for recording in ledgers.

[0085] The item "Summary" is information that indicates the account heading of the journal entry data. Account headings are information that indicates the contents of the expense items used in entering daily transactions, and for example, as shown in Figure 5, the contents of "Purchases," "Accounts Payable," "Suspended Consumption Tax," etc. are stored.

[0086] The item "Amount" is information indicating the amount of the journal entry data that has been journalized. The item "Amount" stores amounts such as "Purchases," "Accounts Payable," and "Suspended Consumption Tax," which are examples of account items indicated by the item "Description."

[0087] The server 20 performs accounting based on the information about the transaction and updates the item "detailed accounting data information" in the accounting database 2023 as it registers the information as accounting information.

[0088] <3 operations> Hereinafter, the payment data generation process and the payment data transmission process performed by the bill reading system 1 in the first embodiment will be described with reference to FIGS.

[0089] FIG. 7 is a flowchart showing an example of the flow of the payment data generation process performed by the invoice reading system 1.

[0090] In step S121, the image data reading module 2033 of the server 20 reads the image data of the invoice stored in the image data folder 2021 by OCR processing to generate text data. In step S121, for example, OCR processing is performed on each of the items in Fig. 1, such as the billing company name A2, the billing company name A3, the billing address A4, ..., or on all the items written on the invoice P1, to generate text data.

[0091] In step S122, the text data classification module 2034 of the server 20 classifies the text data generated in step S121 into each item described on the invoice to generate classified text data. In step S122, for example, the text data read from the image data of the invoice is classified into each item, such as billing company name A2, billing destination company name A3, billing address A4, ... in the example of Fig. 1, and classified text data of the corresponding item name is generated.

[0092] In step S123, the payment data determination module 2035 of the server 20 determines whether or not there are items necessary for performing payment processing for the invoice, based on the categorized text data classified in step S122. In step S123, for example, it is determined whether or not there are items preset as items necessary for performing payment processing in the categorized text data.

[0093] In step S124, if the determination result in step S123 is that some of the items necessary for payment processing are not present, the payment data complement module 2036 of the server 20 makes an inference about and complements the missing items. In step S124, for example, if biller information (biller company name A2 in the example of FIG. 1), item information (item A7 in the example of FIG. 1), and amount information (total amount A12 in the example of FIG. 1) are present in the classified text data, but due date information is not present, the due date information is inferred and complemented.

[0094] In step S125, the user presentation module 2037 of the server 20 presents the items completed in step S124 to the user and accepts input of approval or correction from the user. In step S125, information on the completed items is transmitted to the terminal device 10 via the communication unit 201.

[0095] In step S115, the transmitting / receiving unit 182 of the terminal device 10 receives the information on the complemented items transmitted in step S125. The notification control unit 183 displays the received information on the (complemented) items on the display 150 to present it to the user. In addition, the input operation receiving unit 181 of the terminal device 10 receives an input of approval or correction from the user for the complemented items, and transmits the input information of approval or correction from the user via the transmitting / receiving unit 182.

[0096] In step S126, if the result of the determination in step S123 is that all items necessary for processing the payment for the invoice are present, the payment data generation module 2038 of the server 20 generates payment data for processing the payment for the invoice. Also, in step S126, if the result of the determination in step S123 is that items necessary for processing the payment for the invoice are not present and the items necessary for processing the payment are supplemented in step S124, the payment data generation module 2038 of the server 20 generates payment data for processing the payment for the invoice.

[0097] As described above, the invoice reading system 1 reads the image data of the invoice using OCR processing, and determines from the read text data whether the items necessary for performing payment processing are present. If the items necessary for performing payment processing are present, the invoice reading system 1 generates payment data for performing payment processing. The invoice reading system 1 also generates payment data for performing payment processing when the items necessary for performing payment processing are not present but the items necessary for performing payment processing have been supplemented. Therefore, since payment data can be automatically generated from the image data of the invoice and payment processing can be performed, it becomes possible to provide a convenient service to the customer. This can lead to improved convenience for customers.

[0098] FIG. 8 is a flow chart showing an example of the flow of payment data transmission processing performed by the invoice reading system 1.

[0099] In step S221, the payment data transmission module 2039 of the server 20 transmits the payment data generated in step S126 to the external server 30, which is a server of a financial institution. In step S221, the payment data is transmitted via the communication unit 201 to the external server 30 of the financial institution or the like.

[0100] In step S231, upon receiving the payment data, the external server 30 performs payment (settlement) processing in accordance with the payment data.

[0101] As described above, the invoice reading system 1 transmits payment data for carrying out the payment process to an external server 30 such as a financial institution, and causes the payment process to be carried out. This allows the payment process to be carried out automatically, making it possible to provide convenient services to customers.

[0102] <Summary> As described above, according to this embodiment, image data of an invoice is read by OCR processing, and it is determined from the read text data whether or not items necessary for performing payment processing are present. If items necessary for performing payment processing are present, payment data for performing payment processing is generated. Therefore, since payment data can be automatically generated from image data of an invoice and payment processing can be performed, it is possible to provide a convenient service to customers. This can lead to improved convenience for customers.

[0103] Furthermore, according to this embodiment, image data of an invoice is read by OCR processing, and it is determined from the read text data whether items necessary for payment processing are present. If the items necessary for payment processing are not present, the items necessary for payment processing are supplemented and payment data for payment processing is generated. Therefore, since the items necessary for payment processing can be supplemented from the image data of the invoice, payment data can be automatically generated, and payment processing can be performed, making it possible to provide a convenient service to customers. This can lead to improved convenience for customers.

[0104] Furthermore, according to this embodiment, payment data for carrying out the payment process is transmitted to a financial institution, a settlement institution, etc., and the payment process is carried out. This allows the payment process to be carried out automatically, making it possible to provide convenient services to customers.

[0105] Although the disclosed embodiments have been described above, they can be implemented in various other forms and can be implemented with various omissions, substitutions, and modifications. These embodiments and modifications, as well as omissions, substitutions, and modifications, are included in the technical scope of the claims and their equivalents.

[0106] <Additional Notes> The matters described in the above embodiments will be supplemented below.

[0107] (Supplementary Note 1) A program to be executed by a computer having a processor 29 and a memory 25, the memory 25 storing image data (2021) of an invoice related to a user, the program causing the processor 29 to execute the following steps: reading the image data of the invoice using OCR processing to generate text data (S121); classifying the generated text data by items listed on the invoice to generate categorized text data (S122); determining based on the classified categorized text data whether items necessary for processing the payment for the invoice exist (S123); and, if the determination result indicates that all necessary items exist, generating payment data for processing the payment for the invoice (S126).

[0108] (Appendix 2) The program described in (Appendix 1), in which, in the step of generating text data, image data of an invoice is read by OCR processing from a storage area (2021) in which image data including image data of multiple invoices is stored in no particular order.

[0109] (Appendix 3) A program as described in (Appendix 1), in which in the determining step, it is determined whether the classified text data is based on an invoice, and if the determination result is that it is based on an invoice, it is determined whether items necessary to process payment for the invoice are present.

[0110] (Appendix 4) The program described in (Appendix 1), wherein in the determining step, it is determined whether the necessary items in the classification text data include information on the biller, item, amount, and date.

[0111] (Appendix 5) A program as described in (Appendix 1), in which, in a step of generating payment data, payment data corresponding to the format of a financial institution that processes payment for invoices is generated, and the program further executes a step (S221) of transmitting the payment data to a server of the financial institution.

[0112] (Appendix 6) The program according to any one of (Appendix 1) to (Appendix 5), further comprising, when the determination result shows that some of the required items are not present, executing a step (S124) of inferring and completing the missing required items, and generating payment data based on the completed items in a step of generating payment data.

[0113] (Appendix 7) The program described in (Appendix 6), in which in the step of completing the necessary items, past payment data to the biller is referenced and an inference is made regarding the necessary items based on one or more of the biller information, item information, and amount information in the classified text data.

[0114] (Supplementary Note 8) The program according to (Supplementary Note 6), further comprising a step (S125) of presenting the completed items to the user and accepting input of approval or correction from the user.

[0115] (Supplementary Note 9) An information processing device comprising a control unit 203 and a memory unit 202, the memory unit 202 stores image data (2021) of an invoice related to a user, and the control unit 203 executes the following steps: a step of reading the image data of the invoice by OCR processing to generate text data (S121); a step of classifying the generated text data by items described on the invoice to generate categorized text data (S122); a step of determining whether items necessary for processing payment for the invoice exist based on the classified categorized text data (S123); and, if the determination result indicates that all necessary items exist, a step of generating payment data for processing payment for the invoice (S126).

[0116] (Appendix 8) A method to be executed by a computer having a processor 29 and a memory 25, wherein the memory 25 stores image data (2021) of an invoice related to a user, and the method includes the steps of: the processor 29 reading the image data of the invoice using OCR processing to generate text data (S121); classifying the generated text data by items listed on the invoice to generate categorized text data (S122); determining based on the classified categorized text data whether items necessary for processing the payment for the invoice exist (S123); and, if the determination result indicates that all necessary items exist, generating payment data for processing the payment for the invoice (S126). [Explanation of symbols]

[0117] 1: Invoice reading system 10: Terminal device 10A: Terminal device 10B: Terminal device 13: Input device 14: Output device 15: Memory 16: Storage section 19: Processor 20: Server 25: Memory 26: Storage 29: Processor 30: External server 201: Communications Department 202: Storage section 203: Control section 2021: Image data folder 2022: Trade Database 2023: Accounting Database 2031: Receiving control module 2032: Transmission control module 2033: Image data reading module 2034: Text data classification module 2035: Payment data judgment module 2036: Payment data completion module 2037: User presented module 2038: Payment data generation module 2039: Payment data transmission module

Claims

1. A program to be executed by a computer having a processor and a memory, The memory stores image data of an invoice relating to a user; The program causes the processor to: A step of reading the image data of the invoice by OCR processing to generate text data; A step of classifying the generated text data by items described on the invoice to generate classified text data; A step of determining whether there are any items necessary for performing payment processing for the invoice based on the classified text data; If the determination result shows that all of the required items are present, generating payment data for performing payment processing for the invoice.

2. 2. The program according to claim 1, wherein in the step of generating the text data, the image data of the invoice is read by OCR processing from a storage area in which image data including image data of a plurality of the invoices is stored in no particular order.

3. 2. The program according to claim 1, wherein in the determining step, it is determined whether the classified text data is based on the invoice, and if the determination result is that the classified text data is based on the invoice, it is determined whether items necessary to perform payment processing for the invoice are present.

4. 2. The program according to claim 1, wherein said determining step determines whether or not biller information, item information, amount information, and due date information are present in said classification text data as said necessary items.

5. In the step of generating payment data, the payment data is generated in accordance with a format of a financial institution that performs payment processing for the invoice; The program further comprises: The program of claim 1 , further comprising the step of transmitting the payment data to a server of the financial institution.

6. The program further comprises: If the determination result indicates that some of the necessary items are not present, executing a step of inferring and supplementing the missing necessary items; 6. The program according to claim 1, wherein in the step of generating the payment data, the payment data is generated based on the supplemented items.

7. 7. The program according to claim 6, wherein in the step of completing the necessary items, a prediction is made about the necessary items by referring to past payment data to the biller based on any one or more of biller information, item information, and amount information in the classified text data.

8. The program further comprises:

7. The program according to claim 6, further comprising a step of presenting the completed items to a user and receiving an input of approval or correction from the user.

9. An information processing device including a control unit and a storage unit, The storage unit stores image data of an invoice related to a user, The control unit is A step of reading the image data of the invoice by OCR processing to generate text data; A step of classifying the generated text data by items described on the invoice to generate classified text data; A step of determining whether there are any items necessary for performing payment processing for the invoice based on the classified text data; and if the determination result indicates that all of the necessary items are present, generating payment data for performing payment processing for the invoice.

10. 1. A method for performing a process on a computer having a processor and a memory, comprising: The memory stores image data of an invoice relating to a user; The method further comprises the processor: A step of reading the image data of the invoice by OCR processing to generate text data; A step of classifying the generated text data by items described on the invoice to generate classified text data; A step of determining whether there are any items necessary for performing payment processing for the invoice based on the classified text data; if the determination result indicates that all of the required items are present, generating payment data for performing payment processing for the invoice.

Images