Receipt processing device, receipt processing method, and program

Abstract

Conventionally, inspection of receipt images has been insufficient. [Solution] A receipt processing device 1 equipped with an image receiving unit 121 that receives receipt images, a number acquisition means 1321 that acquires the registration number from the receipt image received by the image receiving unit 121, a number inspection means 1331 that checks whether the registration number exists in a registration number database, and a receipt processing unit 134 that performs processing according to the inspection results of the number inspection means 1331 enables appropriate inspection of receipt images.

Description

[Technical Field]

[0001] The present invention relates to a receipt processing device that inspects receipt images. [Background technology]

[0002] Conventionally, there has been a technique for inspecting received receipt images to determine whether there are any duplicate receipts (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2006-511896 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the prior art, inspection of receipt images was insufficient. [Means for solving the problem]

[0005] The receipt processing device of the first invention is a receipt processing device that includes an image receiving unit that receives a receipt image, a number acquisition means that acquires a registration number from the receipt image received by the image receiving unit, a number inspection means that checks whether the registration number exists in a registration number database, and a receipt processing unit that performs processing according to the inspection results of the number inspection means.

[0006] This configuration allows inspection of the registration number of the receipt image.

[0007] Furthermore, compared to the first invention, the receipt processing device of the second invention further comprises an amount acquisition means for acquiring the product price and consumption tax amount using a receipt image, and the receipt processing unit comprises a price change means for acquiring an altered product price by adding part or all of the consumption tax amount acquired by the amount acquisition means to the product price if the number acquisition means does not acquire a registration number from the receipt image or if the number inspection means obtains an inspection result that the registration number does not exist in the registration number database.

[0008] This configuration allows appropriate processing to be performed according to the inspection results regarding the registration number of the receipt image.

[0009] Furthermore, in the receipt processing device of the third invention, in contrast to the second invention, the receipt processing section is a receipt processing device that includes product price storage means for storing the changed product prices.

[0010] This configuration allows appropriate processing to be performed according to the inspection results regarding the registration number of the receipt image.

[0011] In addition, the receipt processing device of the fourth invention is a receipt processing device that, compared to the first invention, further comprises an image inspection unit that performs inspection on the receipt image accepted by the image acceptance unit, and a result output unit that outputs the results of the inspection in the image inspection unit.

[0012] This configuration allows for inspection of receipt images.

[0013] In addition, the receipt processing device of the fifth invention is a receipt processing device in which, compared to the fourth invention, the image inspection unit is equipped with a four-corner inspection means that inspects whether the four corners of the receipt are present in the receipt image accepted by the image acceptance unit.

[0014] With this configuration, it is possible to check whether the receipt image contains the entire receipt.

[0015] In addition, the receipt processing device of the sixth invention is a receipt processing device that, compared to the fourth invention, further comprises an issue date acquisition means for acquiring the issue date of the receipt from the receipt image accepted by the image acceptance unit, and a period inspection means for acquiring the period elapsed since the issue date acquired by the issue date acquisition means and inspecting whether the elapsed period satisfies the fraud conditions.

[0016] With this configuration, it is possible to check whether the issue date obtained from the receipt image is correct.

[0017] In addition, the receipt processing device of the seventh invention is a receipt processing device that, compared to the first invention, further comprises a type acquisition means for acquiring an image type indicating whether the receipt image accepted by the image acceptance unit is an image of an electronic receipt or an image of a paper receipt, and a type storage means for storing the image type acquired by the type acquisition means.

[0018] With this configuration, the type of receipt image can be obtained. [Effects of the Invention]

[0019] The receipt processing device according to the present invention allows appropriate inspection of receipt images. [Brief explanation of the drawings]

[0020] [Figure 1] Conceptual diagram of receipt system A in embodiment 1 [Figure 2] Block diagram of the receipt system A [Figure 3] Block diagram of the receipt processing device 1 [Figure 4] A flowchart illustrating an example of the operation of the receipt processing device 1 [Figure 5] A flowchart illustrating an example of the image inspection process [Figure 6] Flowchart illustrating an example of information acquisition processing [Figure 7] Flowchart illustrating an example of a content verification process [Figure 8]A flowchart illustrating an example of the receipt process [Figure 9] A diagram showing an example of the receipt management table [Figure 10] FIG. 10 shows an example of the receipt image. [Figure 11] A diagram showing an example of the screen [Figure 12] Block diagram of the computer system DETAILED DESCRIPTION OF THE INVENTION

[0021] Hereinafter, embodiments of a receipt processing device and the like will be described with reference to the drawings. Note that components with the same reference numerals in the embodiments perform similar operations, and therefore repeated description may be omitted.

[0022] (Embodiment 1) In this embodiment, a receipt processing device will be described that acquires a registration number from a receipt image and checks whether the registration number exists.

[0023] In this embodiment, a receipt processing device will be described that performs different processing depending on whether a registration number is written and exists, or whether the registration number is not written or does not exist.

[0024] In this embodiment, a receipt processing device will be described that processes a receipt by including all or part of the consumption tax as the product price when the registration number is not written or does not exist.

[0025] In this embodiment, a receipt processing device that inspects a receipt image as an image will be described. The inspection as an image includes, for example, inspecting the resolution or inspecting whether the four corners of the receipt are present.

[0026] In this embodiment, a receipt processing device that checks the period of time that has elapsed since the receipt was issued will be described.

[0027] In this embodiment, a receipt processing device will be described that checks whether a receipt image is an electronic image or a scanned paper image, and stores and manages the type of image.

[0028] In this specification, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc.

[0029] Furthermore, in this specification, selecting or determining information Z means obtaining information Z, obtaining a pointer to information Z, obtaining the ID of information Z, setting a flag on information Z, etc., and it is sufficient if information Z can be accessed.

[0030] 1 is a conceptual diagram of a receipt system A according to this embodiment. The receipt system A includes a receipt processing device 1 and a terminal device 2.

[0031] Fig. 2 is a block diagram of the receipt system A according to this embodiment. Fig. 3 is a block diagram of the receipt processing device 1.

[0032] The receipt processing device 1 includes a storage unit 11, a reception unit 12, a processing unit 13, and an output unit 14. The storage unit 11 includes a receipt management unit 111. The reception unit 12 includes an image reception unit 121 and an instruction reception unit 122. The processing unit 13 includes an image inspection unit 131, an information acquisition unit 132, a content inspection unit 133, and a receipt processing unit 134. The image inspection unit 131 includes a resolution inspection means 1311 and a four-corner inspection means 1312. The information acquisition unit 132 includes a number acquisition means 1321, an amount acquisition means 1322, an issue date acquisition means 1323, and a type acquisition means 1324. The content inspection unit 133 includes a number inspection means 1331 and a period inspection means 1332. The receipt processing unit 134 includes an image storage unit 1341, a price change unit 1342, a product price storage unit 1343, a consumption tax storage unit 1344, and a type storage unit 1345. The output unit 14 includes a result output unit 141.

[0033] The terminal device 2 includes a terminal storage unit 21, a terminal reception unit 22, a terminal processing unit 23, a terminal transmission unit 24, a terminal reception unit 25, and a terminal output unit 26.

[0034] Various types of information are stored in the storage unit 11 that constitutes the receipt processing device 1. The various types of information include, for example, receipt information, which will be described later.

[0035] The receipt management unit 111 stores one or more pieces of receipt information. Receipt information is information related to receipts. It is preferable that the receipt information includes a receipt image, but it does not have to include one. Receipt information includes, for example, one or more pieces of element information written on a receipt. Receipt information includes, for example, one or more pieces of element information obtained by performing character recognition processing on a receipt image. The element information may be information obtained based on information obtained by performing character recognition processing on a receipt image. Receipt information corresponds, for example, to one or more inspection results. The element information is, for example, the issue date, product price, consumption tax amount, registration number, payee, and image type.

[0036] The inspection result is the result of inspection processing performed by the image inspection unit 131 or the content inspection unit 133, which will be described later. The inspection processing may be a judgment processing. The inspection result may be a judgment result.

[0037] The reception unit 12 receives various instructions and information. For example, the reception unit 12 receives various instructions and information from the terminal device 2. The various instructions and information include, for example, a receipt image (described later), a receipt application instruction (described later), and an acquisition instruction (described later).

[0038] The image receiving unit 121 receives a receipt image. For example, the image receiving unit 121 receives a receipt image from the terminal device 2. For example, the image receiving unit 121 receives a receipt application instruction.

[0039] A receipt application instruction is an instruction to apply for a receipt. The receipt application instruction includes a receipt image. It is preferable that the receipt application instruction includes a user identifier. The user identifier is information that identifies a user. The user identifier is, for example, a user ID, email address, telephone number, or name.

[0040] Here, acceptance typically refers to the reception of information transmitted via a wired or wireless communication line, but may also be a concept that includes the reception of information input from an input device such as a keyboard, mouse, or touch panel, or the reception of information read from a recording medium such as an optical disk, magnetic disk, or semiconductor memory.

[0041] The instruction receiving unit 122 receives various instructions. The instruction receiving unit 122 receives various instructions from the terminal device 2. The various instructions are, for example, acquisition instructions. The acquisition instruction is an instruction to acquire receipt information.

[0042] The processing unit 13 performs various types of processing, such as processing performed by an image inspection unit 131, an information acquisition unit 132, a content inspection unit 133, and a receipt processing unit 134.

[0043] The image inspection unit 131 inspects the receipt image received by the image reception unit 121. This inspection is called image inspection. The image inspection includes, for example, a resolution inspection, a corner inspection, and a type inspection, which will be described later.

[0044] The resolution inspection means 1311 performs a resolution inspection on the receipt image accepted by the image accepting unit 121. The resolution inspection is an inspection to see if the receipt image satisfies a resolution condition. The resolution condition is a condition related to the resolution of the receipt image. For example, the resolution condition is that the resolution is equal to or greater than a threshold. Here, if the resolution condition is satisfied, the resolution inspection means 1311 obtains an inspection result indicating that the resolution of the receipt image is normal, and if the resolution condition is not satisfied, the resolution inspection means 1311 obtains an inspection result indicating that the resolution of the receipt image is not normal (abnormal).

[0045] The four-corner inspection means 1312 inspects the four corners of the receipt image received by the image receiving unit 121. In other words, the four-corner inspection means 1312 inspects whether the four corners of the receipt are present in the receipt image received by the image receiving unit 121. The four-corner inspection means 1312 obtains an inspection result indicating that the receipt is normal if the four corners of the receipt are present in the receipt image, and that the receipt is abnormal if the four corners of the receipt are not present in the receipt image.

[0046] The four-corner inspection means 1312 performs the four-corner inspection using, for example, an image processing method, a machine learning method, or a generative AI method. Note that the method for the four-corner inspection is not important. Note that the four-corner inspection can also be said to be an inspection of whether the receipt image contains the entire receipt. (1) Image processing method

[0047] The four-corner inspection means 1312, for example, extracts the outline of a rectangle in the receipt image, and if none of the pixels that make up the outline of the extracted rectangle are at the edge of the receipt image, determines that the four corners of the receipt are present in the receipt image, and if any of the pixels are at the edge of the receipt image, determines that the four corners of the receipt are not present in the receipt image.

[0048] The four corner inspection means 1312, for example, extracts the outline of a rectangle in the receipt image, and if it can detect the outline of a rectangle in the receipt image, it determines that the four corners of the receipt are present in the receipt image, and if it cannot detect the outline of a rectangle in the receipt image, it determines that the four corners of the receipt are not present in the receipt image.

[0049] The four corner inspection means 1312 may perform character recognition on the receipt image, obtain one or more character strings, and if it detects that the character strings are interrupted, determine that the receipt image does not contain the entire receipt. (2) Machine learning method

[0050] The four corner inspection means 1312, for example, provides the receipt image and learning model received by the image receiving unit 121 to a machine learning prediction processing module, executes the module, and obtains a prediction result that either the four corners of the receipt are present in the receipt image ("true") or the four corners of the receipt are not present in the receipt image ("false").

[0051] The above learning model is a model obtained by providing a set of training data containing one or more positive examples, which are receipt images in which the four corners of the receipt are present (images in which the entire receipt is visible), and one or more negative examples, which are receipt images in which the four corners of the receipt are not present (images in which part of the receipt is cut off), to a machine learning learning module, and then running the module.

[0052] The learning model is information acquired by a learning unit (not shown) performing the above-described machine learning learning process, and is information used in the machine learning prediction process. The learning model may also be called a learner, a classifier, a classification model, or the like. The machine learning algorithm may be any algorithm, such as deep learning, random forest, decision tree, or SVM. For machine learning, various machine learning functions, such as the TensorFlow (registered trademark) library, the random forest module of the R language, fastText, or TinySVM, or various existing libraries, may be used. (3) Generative AI method

[0053] The four-corner inspection means 1312 performs the four-corner inspection using, for example, a generation AI. The four-corner inspection means 1312 passes a receipt image to the generation AI, and gives the generation AI a prompt such as "Please answer whether the four corners of the receipt are present in the receipt image" or "Please answer whether the entire receipt is visible in the receipt image," and obtains the inspection results of the four-corner inspection from the generation AI.

[0054] The information acquisition unit 132 acquires character strings displayed in the receipt image. The character strings may include, for example, the registration number, the product price, the consumption tax amount, and the receipt issuance date. The product price is usually the price of the product, but it may also be the price of the service.

[0055] The information acquisition unit 132 acquires one or more character strings appearing in the receipt image by, for example, a method using character recognition processing, a method using machine learning, or a method using generation AI. (1) Character recognition processing method

[0056] The information acquisition unit 132 performs character recognition processing on the receipt image received by the image reception unit 121, for example, and acquires one or more character strings.

[0057] When the information acquisition unit 132 acquires two or more character strings, the information acquisition unit 132 acquires, for example, the class of each of the two or more character strings. A class is the type of character string. Examples of classes include the issue date, product price, consumption tax amount, registration number, and payee. (2) Machine learning method

[0058] For example, the information acquisition unit 132 provides the receipt image and learning model received by the image reception unit 121 to a machine learning prediction processing module, executes the module, and acquires a character string that is a prediction result from the receipt image.

[0059] The above learning model is obtained by providing a set of training data containing receipt images and one or more character strings to be acquired to a machine learning learning module, running the module, etc. The machine learning algorithm can be any algorithm, such as deep learning, random forest, decision tree, or SVM. (3) Generative AI method

[0060] For example, the information acquisition unit 132 passes the receipt image accepted by the image acceptance unit 121 to the generation AI, gives the generation AI a prompt (e.g., "Please extract <character string> from the image."), and acquires a character string corresponding to the <character string> from the generation AI. The <character string> may be, for example, a registration number, a product price, or a consumption tax amount.

[0061] The number acquisition means 1321 acquires a registration number from the receipt image received by the image reception unit 121. The registration number is a number that is notified to a business that wishes to be registered as a qualified invoice issuing business when the business submits an "application for registration as a qualified invoice issuing business" to the director of the tax office that has jurisdiction over the place of tax payment and is registered by the director of the tax office.

[0062] If the registration number is not included in the receipt image, the number obtaining means 1321 cannot obtain the registration number.

[0063] The number acquisition unit 1321 acquires the registration number appearing in the receipt image by, for example, a method using character recognition processing, a method using machine learning, or a method using generation AI. Note that the algorithm by which the number acquisition unit 1321 acquires the registration number does not matter. (1) Character recognition processing method

[0064] The number acquisition means 1321, for example, performs character recognition on the receipt image and acquires the character string "'T' + 13-digit numeric string." This character string is the registration number. The number acquisition means 1321 may also acquire the character string "'T' + 13-digit numeric string" from the character string acquired by the information acquisition unit 132.

[0065] The number obtaining means 1321 obtains, for example, a character string of the class "registration number" from among the character strings obtained from the receipt image. (2) Machine learning method

[0066] The number acquisition means 1321 acquires the registration number from the receipt image using machine learning prediction processing.

[0067] The number acquisition means 1321 acquires, for example, by machine learning prediction processing, coordinate information where the registration number is expressed from the receipt image (for example, the top left coordinates (x1, y1) and bottom right coordinates (x2, y2) of the rectangle), or the registration number.

[0068] The number acquisition means 1321, for example, provides the receipt image and the learning model to a module that performs machine learning prediction processing, executes the module, and acquires coordinate information or a registration number. Note that the coordinate information is one or more coordinates that identify a rectangle in the receipt image.

[0069] The learning model here is a model obtained by providing two or more sets of training data to a module that performs machine learning learning processing. As mentioned above, the machine learning algorithm here can be any of deep learning, random forest, decision tree, SVM, etc.

[0070] The training data here includes, for example, a receipt image and coordinate information where the registration number appears, or includes a receipt image and the registration number.

[0071] When the number acquisition means 1321 acquires coordinate information through machine learning prediction processing, it acquires the registration number, which is a character string at the position indicated by the coordinate information, from the receipt image. (3) Generative AI method

[0072] The number acquisition means 1321 uses the generation AI to acquire the registration number from the receipt image. The type acquisition means 1324, for example, passes the receipt image to the generation AI and provides a prompt such as "Please answer the registration number in the receipt image! The registration number is a string that begins with the alphabet "T" and is followed by a 13-digit number after the "T"," and acquires the registration number from the generation AI. The prompt may include the receipt image.

[0073] The amount obtaining means 1322 obtains the product price and consumption tax amount from the receipt image. The processing of the amount obtaining means 1322 can be performed using known technology. The amount obtaining means 1322 obtains the amount displayed in the receipt image using, for example, a character recognition method, a machine learning method, or a generation AI method. The amount is, for example, the consumption tax amount and the payment price or product price. The algorithm used by the amount obtaining means 1322 to obtain the amount is not important. (1) Character recognition processing method

[0074] The amount obtaining means 1322 obtains, for example, from one or more character strings obtained as a result of character recognition processing on a receipt image, the numeric string following "￥" or the largest numeric string as the numerical value of "product price + consumption tax amount (payment price)". The amount obtaining means 1322 obtains, for example, from one or more character strings obtained as a result of character recognition processing on a receipt image, the numeric string closest to the character string "consumption tax" and which is not the payment price as the consumption tax amount.

[0075] The amount obtaining means 1322 obtains one or more character string classes obtained as a result of character recognition processing on a receipt image, for example, and obtains two character strings (here, numerical values) whose class is "amount." Then, of the two character strings, it obtains the smaller numerical value, which is the consumption tax amount, and the larger numerical value, which is the payment price or product price. (2) Machine learning method

[0076] The amount obtaining unit 1322 obtains the consumption tax amount and the payment price from the receipt image by, for example, machine learning prediction processing. The amount obtaining unit 1322 may obtain the product price instead of the payment price.

[0077] The amount acquisition means 1322, for example, provides a receipt image and a learning model to a module that performs machine learning prediction processing, executes the module, and acquires coordinate information where the consumption tax amount is displayed in the receipt image (for example, the upper left coordinates (x3, y3) and lower right coordinates (x4, y4) of a rectangle). Next, the amount acquisition means 1322 acquires, for example, the consumption tax amount, which is a character string at the position indicated by the coordinate information, from the receipt image. Note that the learning model here is a model acquired by providing two or more pieces of training data containing the receipt image and coordinate information of the consumption tax amount to a module that performs machine learning learning processing.

[0078] The amount acquisition means 1322, for example, provides a receipt image and a learning model to a module that performs machine learning prediction processing, executes the module, and acquires coordinate information where the payment price is displayed from the receipt image (for example, the upper left coordinates (x5, y5) and lower right coordinates (x6, y6) of a rectangle). Next, the amount acquisition means 1322 acquires, for example, the payment price, which is a character string at the position indicated by the coordinate information, from the receipt image. The learning model is a model acquired by providing two or more pieces of training data having the receipt image and the coordinate information of the payment price to a module that performs machine learning learning processing.

[0079] The amount obtaining means 1322, for example, provides a receipt image and a learning model to a module that performs machine learning prediction processing, executes the module, and obtains the consumption tax amount from the receipt image. Note that the learning model here is a model obtained by providing two or more pieces of training data containing a receipt image and the consumption tax amount to a module that performs machine learning learning processing.

[0080] The amount obtaining means 1322, for example, provides a receipt image and a learning model to a module that performs machine learning prediction processing, executes the module, and obtains the payment price from the receipt image. Note that the learning model here is a model obtained by providing two or more pieces of training data containing a receipt image and a payment price to a module that performs machine learning learning processing.

[0081] As mentioned above, the machine learning algorithm used here can be any algorithm, such as deep learning, random forest, decision tree, SVM, etc. (3) Generative AI method

[0082] The amount obtaining means 1322 obtains the payment price or consumption tax amount from the receipt image using, for example, the generation AI. For example, the amount obtaining means 1322 passes the receipt image to the generation AI and gives a prompt such as "Please answer the 'payment price' and 'consumption tax amount' in the receipt image!" to the generation AI, and obtains the "payment price" and "consumption tax amount" from the generation AI.

[0083] The issue date acquisition means 1323 acquires the issue date of the receipt from the receipt image accepted by the image acceptance unit 121. The issue date acquisition means 1323 acquires the registration number displayed in the receipt image by, for example, a method using character recognition processing, a method using machine learning, or a method using generation AI. Note that the algorithm used by the issue date acquisition means 1323 to acquire the issue date is not important. (1) Character recognition processing method

[0084] The issue date acquisition means 1323 acquires the issue date of the receipt, which is a character string corresponding to the date type, from one or more character strings obtained as a result of character recognition processing on the receipt image, for example. (2) Machine learning method

[0085] The issue date acquisition means 1323 acquires the issue date of the receipt from the receipt image, for example, by machine learning prediction processing. The number acquisition means 1321, for example, provides the receipt image and the learning model to a module that performs machine learning prediction processing, executes the module, and acquires coordinate information or the issue date. Note that the coordinate information here is information on the coordinates where the issue date of the receipt is displayed.

[0086] The learning model here is a model obtained by providing two or more sets of training data, each containing a receipt image and coordinate information or an issue date, to a module that performs machine learning learning processing. As mentioned above, the machine learning algorithm here can be any of deep learning, random forest, decision tree, SVM, etc.

[0087] When the issue date acquisition means 1323 acquires coordinate information through machine learning prediction processing, it acquires the issue date, which is a character string at the position indicated by the coordinate information, from the receipt image. (3) Generative AI method

[0088] The issue date acquisition means 1323 acquires the issue date of the receipt from the receipt image using, for example, the generation AI. The issue date acquisition means 1323 passes the receipt image to the generation AI and gives a prompt such as "Please tell me the issue date of the receipt in the receipt image!" to the generation AI, and acquires the issue date of the receipt from the generation AI.

[0089] The type acquisition means 1324 performs a type inspection on the receipt image accepted by the image acceptance unit 121 and acquires the image type. The image type is information that identifies the type of receipt image. The image type is information that indicates whether the receipt image is an image of an electronic receipt (for example, "1") or an image of a paper receipt (for example, "0"). The image type is, for example, a flag.

[0090] The type acquisition unit 1324 acquires the image type of the receipt image by, for example, an image recognition processing method, a machine learning method, or a generation AI method. Note that the type acquisition unit 1324 may use any algorithm to acquire the image type. (1) Image recognition processing method

[0091] The type acquisition means 1324 performs image recognition processing on the receipt image to acquire the image type.

[0092] The type acquisition unit 1324 performs character recognition processing on the receipt image, for example, and if handwritten characters are detected, it determines that the receipt image is an image of a paper receipt.

[0093] The type acquisition means 1324, for example, extracts the contour of the receipt image, and if the contour is a rectangle (rectangle or square) without distortion, it checks that the receipt image is an image of an electronic receipt, and if the contour is distorted, it checks that the image is an image of a paper receipt. (2) Machine learning method

[0094] The type acquisition unit 1324 performs type inspection by machine learning prediction processing and acquires the image type.

[0095] The type acquisition means 1324, for example, provides the receipt image and the learning model to a module that performs machine learning prediction processing, executes the module, and acquires the image type.

[0096] The learning model here is a model obtained by providing two or more sets of training data, each containing a receipt image and an image type, to a module that performs machine learning learning processing. As mentioned above, the machine learning algorithm here can be any of deep learning, random forest, decision tree, SVM, etc. (3) Generative AI method

[0097] The type acquisition means 1324 uses the generation AI to acquire the image type from the receipt image. For example, the type acquisition means 1324 passes the receipt image to the generation AI and gives a prompt (for example, "If the receipt image is an image of an electronic receipt, answer '1'; if it is an imaged image of a paper receipt, answer '0'!") to the generation AI, and acquires the image type (here, "1" or "0") from the generation AI.

[0098] The content inspection unit 133 inspects whether the content of one or more character strings acquired from the receipt image is valid. The character strings may be, for example, a registration number or the date of issue of the receipt. Such one or more character strings are usually information acquired by the information acquisition unit 132.

[0099] The number checking means 1331 checks whether the registration number acquired by the number acquiring means 1321 exists in a registration number database (hereinafter referred to as "DB" where appropriate) and acquires the inspection result. The inspection result is, for example, "the registration number exists (e.g., "1")" or "the registration number does not exist (e.g., "0")."

[0100] The registration number database is usually stored on a server (not shown). The server is a device external to the receipt processing device 1. The receipt processing device 1 and the server can communicate via a network such as the Internet. The server is, for example, a server of the National Tax Agency, and is a device that stores a registration number database that manages two or more registration numbers.

[0101] The period checking means 1332 obtains the period that has elapsed since the date of issue obtained by the date of issue obtaining means 1323, and checks whether the period satisfies the fraudulent condition. The fraudulent condition is a fraudulent condition related to the receipt application date. For example, the fraudulent condition is that the period of time is equal to or longer than a threshold value. The threshold value is, for example, 68 days or 2 months, but is not critical. The period of time that has elapsed is usually the period from the date of issue of the receipt to the current day.

[0102] The receipt processing unit 134 processes the received receipt image. The receipt processing unit 134 processes the receipt corresponding to the received receipt image. The content of the processing can vary.

[0103] The receipt processing unit 134 performs processing according to the inspection result of the number inspection means 1331. If the inspection result of the number inspection means 1331 is different, the receipt processing unit 134 performs different processing.

[0104] For example, if the inspection result of the number inspection means 1331 is "the registration number does not exist" or "the registration number is not written" on the receipt image, the receipt processing unit 134 outputs an error message indicating that the registration number does not exist. The error message is output, for example, by sending it to the terminal device 2. "The registration number does not exist" means that the registration number written on the receipt does not exist in the DB.

[0105] For example, if the inspection result of the number inspection means 1331 is "a registration number exists," the receipt processing unit 134 creates receipt information containing the various information acquired by the information acquisition unit 132 and a receipt image, and stores the receipt information in the receipt management unit 111. "A registration number exists" means that the registration number written on the receipt exists in the DB.

[0106] The image storage means 1341 stores the receipt image received by the image reception unit 121 in the receipt management unit 111. It is preferable that the image storage means 1341 stores the receipt image received by the image reception unit 121 in the receipt management unit 111 in association with a user identifier associated with the receipt image.

[0107] If the number checking means 1331 obtains an inspection result indicating that the registration number does not exist in the registration number database or that the registration number is not written on the receipt, the price changing means 1342 obtains a changed product price by adding part or all of the consumption tax amount obtained by the amount obtaining means 1322 to the product price. In such a case, the price changing means 1342 obtains a changed consumption tax amount by subtracting part or all of the consumption tax amount obtained by the amount obtaining means 1322 from the consumption tax amount.

[0108] For example, if the number verification means 1331 obtains a verification result indicating that the registration number does not exist in the registration number database or that the registration number is not written on the receipt, the price modification means 1342 obtains a modified product price by adding a certain percentage of the consumption tax amount obtained by the amount acquisition means 1322 to the product price. The certain percentage is, for example, 20%, 50%, or 100%. In such a case, the price modification means 1342 obtains a modified consumption tax amount by subtracting the certain percentage of the consumption tax amount obtained by the amount acquisition means 1322 from the consumption tax amount.

[0109] For example, if the number verification means 1331 obtains an inspection result indicating that the registration number does not exist in the registration number database or that the registration number is not written on the receipt, and if the receipt is issued before the first threshold deadline or before the first threshold deadline, the price modification means 1342 obtains an updated product price by adding a first percentage of the consumption tax amount obtained by the amount acquisition means 1322 to the product price; if the receipt is issued after the first threshold deadline or after the first threshold deadline, the price modification means 1342 obtains an updated product price by adding a second percentage of the consumption tax amount obtained by the amount acquisition means 1322 to the product price. The first threshold deadline is, for example, January 1, 2025. In such cases, the price modification means 1342 obtains an updated consumption tax amount by subtracting the amount of consumption tax obtained by the amount acquisition means 1322 that has been passed on to the product price from the consumption tax amount.

[0110] When the number checking means 1331 obtains an inspection result that the registration number does not exist in the registration number database or that the registration number is not written on the receipt, the product price storing means 1343 stores the changed product price obtained by the price changing means 1342. When the number checking means 1331 obtains an inspection result that the registration number is not written on the receipt, this is usually the case when the number obtaining means 1321 did not obtain the registration number from the receipt image.

[0111] When the number checking means 1331 obtains a check result indicating that the registration number exists in the registration number database, the product price storing means 1343 stores the product price obtained by the amount obtaining means 1322. The product price storing means 1343 stores the product price in the receipt management unit 111, for example.

[0112] If the number verification means 1331 obtains an inspection result indicating that the registration number does not exist in the registration number database or that the registration number is not written on the receipt, the consumption tax accumulation means 1344 accumulates the revised consumption tax amount, which is the consumption tax amount obtained by subtracting the amount added to the revised product price obtained by the price change means 1342 from the consumption tax amount obtained by the amount acquisition means 1322. If the number verification means 1331 obtains an inspection result indicating that the registration number exists in the registration number database, the consumption tax accumulation means 1344 accumulates the consumption tax amount obtained by the amount acquisition means 1322. The consumption tax accumulation means 1344 accumulates the consumption tax amount in the receipt management unit 111, for example.

[0113] The type storage means 1345 stores the image type acquired by the type acquisition means 1324. The type storage means 1345, for example, stores the image type in the receipt management unit 111. It is preferable that the type storage means 1345 stores the image type in association with the receipt image.

[0114] The output unit 14 outputs various types of information, such as test results and receipt information.

[0115] Here, output means, for example, transmission to terminal device 2, but it may also be a concept that includes display on a display, projection using a projector, printing on a printer, sound output, transmission to a device other than terminal device 2, storage on a recording medium, and handing over processing results to other processing devices or other programs.

[0116] The result output unit 141 outputs the result of the inspection in the image inspection unit 131. The result output unit 141 outputs the result of the inspection in the content inspection unit 133.

[0117] Various types of information are stored in the terminal storage unit 21 included in the terminal device 2. The various types of information include, for example, a user identifier and a receipt image.

[0118] The terminal reception unit 22 receives various information, instructions, etc. The various information, instructions, etc. include, for example, receipt images, receipt application instructions, and acquisition instructions.

[0119] The means for inputting various information and instructions may be any means, such as a touch panel, keyboard, mouse, or menu screen.

[0120] The terminal processing unit 23 performs various types of processing. For example, the various types of processing are processing to convert received information, instructions, etc. into information, instructions, etc. with a structure to be transmitted. For example, the various types of processing are processing to convert received information into information with a structure to be output.

[0121] The terminal transmission unit 24 transmits various information, instructions, etc. to the receipt processing device 1. The various information, instructions, etc. are, for example, a receipt image, a receipt application instruction, and an acquisition instruction.

[0122] The terminal receiving unit 25 receives various types of information from the receipt processing device 1. The various types of information include, for example, receipt information, inspection results, or various types of screen information. Screen information is information used to configure a screen. Screen information is written in, for example, HTML, XML, or the like, but the means for realizing it is not important.

[0123] The terminal output unit 26 outputs various types of information, such as receipt information, inspection results, and various screens.

[0124] Here, output is a concept that includes displaying on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, and delivery of processing results to other processing devices or other programs.

[0125] The storage unit 11, receipt management unit 111, and terminal storage unit 21 are preferably non-volatile recording media, but may also be realized as volatile recording media.

[0126] There is no restriction on the process by which information is stored in the storage unit 11 etc. For example, information may be stored in the storage unit 11 etc. via a recording medium, information transmitted via a communication line etc. may be stored in the storage unit 11 etc., or information input via an input device may be stored in the storage unit 11 etc.

[0127] The reception unit 12, the image reception unit 121, and the instruction reception unit 122 are preferably realized by wireless or wired communication means, but may also be realized by a means for receiving broadcasts, a device driver for an input means such as a touch panel or keyboard, or control software for a menu screen.

[0128] The processing unit 13, image inspection unit 131, information acquisition unit 132, content inspection unit 133, receipt processing unit 134, resolution inspection means 1311, corner inspection means 1312, type acquisition means 1324, number acquisition means 1321, amount acquisition means 1322, issue date acquisition means 1323, number inspection means 1331, period inspection means 1332, image storage means 1341, price change means 1342, product price storage means 1343, consumption tax storage means 1344, and type storage means 1345 are typically realized by a processor, memory, etc. The processing procedures of the processing unit 13, etc. are typically realized by software, and the software is recorded on a recording medium such as a ROM. However, they may also be realized by hardware (dedicated circuitry). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0129] The output unit 14 and the result output unit 141 are preferably realized by wireless or wired communication means, but may also be realized by driver software for an output device such as a display or speaker, or by a combination of driver software for an output device and an output device.

[0130] The terminal reception unit 22 can be realized by a device driver for an input means such as a touch panel or a keyboard, or control software for a menu screen.

[0131] The terminal transmitting unit 24 is usually realized by a wireless or wired communication means, but may also be realized by a broadcasting means.

[0132] The terminal receiving unit 25 is usually realized by a wireless or wired communication means, but may also be realized by a means for receiving broadcasts.

[0133] The terminal output unit 26 may or may not include an output device such as a display, a speaker, etc. The terminal output unit 26 may be realized by driver software for an output device, or by a combination of driver software for an output device and the output device, etc.

[0134] Next, an example of the operation of the receipt processing device 1 will be described with reference to the flowchart of FIG.

[0135] (Step S401) The image receiving unit 121 determines whether or not a receipt application instruction including a receipt image has been received. If a receipt application instruction has been received, the process proceeds to step S402; if a receipt application instruction has not been received, the process proceeds to step S410. Note that the receipt application instruction here corresponds to a user identifier.

[0136] (Step S402) The processing unit 13 acquires a receipt image from the receipt application instruction.

[0137] (Step S403) The processing unit 13 acquires a user identifier associated with the receipt application instruction.

[0138] (Step S404) The image inspection unit 131 inspects the receipt image. An example of such image inspection processing will be described with reference to the flowchart in FIG.

[0139] (Step S405) The processing unit 13 determines whether or not the result of the image inspection process is an error. If there is an error, the process proceeds to step S406, and if there is no error, the process proceeds to step S407.

[0140] (Step S406) The processing unit 13 obtains an error message indicating that the result of the image inspection process is an error. The result output unit 141 outputs the error message. The process returns to step S401. The error message may be, for example, "The resolution is insufficient," or "The four corners of the receipt are not visible. Please take the photo again so that the four corners are visible."

[0141] (Step S407) The information acquisition unit 132 acquires two or more pieces of information from the receipt image. An example of such information acquisition processing will be described with reference to the flowchart in FIG.

[0142] (Step S408) The content inspection unit 133 inspects the information acquired in step S407. An example of such content inspection processing will be described with reference to the flowchart in FIG.

[0143] (Step S409) The receipt processing unit 134 performs receipt processing in response to the receipt application instruction accepted in step S401. Return to step S401. An example of receipt processing will be described using the flowchart in FIG.

[0144] (Step S410) The instruction receiving unit 122 determines whether or not an acquisition instruction has been received. If an acquisition instruction has been received, the process proceeds to step S411, and if an acquisition instruction has not been received, the process returns to step S401.

[0145] (Step S411) The processing unit 13 acquires one or more receipt information items that match the conditions of the acquisition instruction received in step S410 from the receipt management unit 111. The processing unit 13 searches for receipt information items from the receipt management unit 111 using the conditions of the acquisition instruction.

[0146] (Step S412) The output unit 14 outputs one or more pieces of receipt information acquired in step S411. Return to step S401.

[0147] In the flowchart of FIG. 4, the process ends when the power is turned off or an interrupt occurs to end the process.

[0148] Next, an example of the image inspection process in step S404 will be described with reference to the flowchart in FIG.

[0149] (Step S501) The resolution checking means 1311 acquires the resolution of the receipt image.

[0150] (Step S502) The resolution inspection means 1311 determines whether the resolution acquired in step S501 matches the resolution conditions. If the resolution conditions are met, the process proceeds to step S503. If the resolution conditions are not met, the process proceeds to step S508.

[0151] (Step S503) The four-corner inspection means 1312 inspects whether the receipt image has the four corners of a receipt. Examples of such four-corner inspection processing have been described above. As described above, the four-corner inspection processing method can be, for example, a method using image processing, a method using machine learning, a method using generation AI, etc., and the method of four-corner inspection is not important.

[0152] (Step S504) If the result of the four corner inspection process in step S503 is that the entire receipt is shown in the receipt image, proceed to step S505, and if the entire receipt is not shown, proceed to step S507.

[0153] (Step S505) The image inspection unit 131 acquires information that the result of the image inspection is "normal."

[0154] (Step S506) The type acquisition means 1324 acquires the image type of the receipt image. The process returns to the upper level process. As mentioned above, examples of methods for acquiring the image type include image recognition processing, machine learning, and generation AI, and the algorithm is not important.

[0155] (Step S507) The image inspection unit 131 receives an error message indicating that the entire receipt is not visible in the receipt image. The process proceeds to step S509.

[0156] (Step S508) The image inspection unit 131 receives an error message indicating that the resolution of the receipt image is inappropriate.

[0157] (Step S509) The image inspection unit 131 acquires information that the result of the image inspection is "abnormal," and returns to the upper processing.

[0158] Next, an example of the information acquisition process in step S407 will be described with reference to the flowchart in FIG.

[0159] (Step S601) The number acquisition unit 1321 acquires the registration number written on the receipt image. As described above, examples of the process for acquiring the registration number written on the receipt image include a method using character recognition processing, a method using machine learning, a method using generation AI, etc., and the algorithm is not important.

[0160] (Step S602) The amount obtaining means 1322 obtains the amount listed on the receipt image. In this case, the amount is the product price and consumption tax. When the amount obtaining means 1322 obtains the payment price and consumption tax amount listed on the receipt image, it is preferable to calculate the product price using the formula "payment price - consumption tax amount." As mentioned above, examples of processes for obtaining the amount listed on the receipt image include, for example, a method using character recognition processing, a method using machine learning, a method using generation AI, etc., and the algorithm is not important.

[0161] (Step S603) The issue date acquisition means 1323 acquires the issue date written on the receipt image. As described above, examples of the process for acquiring the issue date written on the receipt image include a method using character recognition processing, a method using machine learning, a method using generation AI, etc., and the algorithm is not important. The issue date is the date the receipt was issued.

[0162] (Step S604) acquires other information written on the receipt image. The process returns to the upper level process. An example of the process for acquiring other information written on the receipt image is the same as the process described above. The other information is, for example, the payee.

[0163] Next, an example of the content inspection process in step S408 will be described with reference to the flowchart in FIG.

[0164] (Step S701) The number checking means 1331 searches a registration number database (DB) using the registration number acquired by the number acquiring means 1321 as a key, and determines whether or not the registration number exists in the DB.

[0165] If the number obtaining means 1321 does not obtain the registration number, the process proceeds to step S704.

[0166] (Step S702) If the registration number exists in the DB, the process proceeds to step S703, and if not, the process proceeds to step S704.

[0167] (Step S703) The number checking means 1331 assigns "OK (for example, "1")" to the variable "number checking result".

[0168] (Step S704) The number checking means 1331 assigns "NG (for example, "0")" to the variable "number checking result".

[0169] (Step S705) The period checking means 1332 obtains today's date from a clock (not shown).

[0170] (Step S706) The period checking means 1332 acquires the period elapsed from the issue date, which is the difference between the issue date acquired by the issue date acquiring means 1323 and today's date.

[0171] (Step S707) The period checking means 1332 judges whether or not the elapsed period satisfies the invalidity condition. If the invalidity condition is satisfied, the process proceeds to step S709, and if the invalidity condition is not satisfied, the process proceeds to step S708.

[0172] (Step S708) The period checking means 1332 assigns "OK (for example, "1")" to the variable "period checking result." The process returns to the upper level process.

[0173] (Step S709) The period checking means 1332 assigns "NG (for example, "0")" to the variable "period checking result." The process returns to the upper level process.

[0174] Next, an example of the receipt processing in step S409 will be described with reference to the flowchart in FIG.

[0175] (Step S801) The price change means 1342 acquires the amount acquired by the amount acquisition means 1322. The amount includes the amount of consumption tax. The amount includes the total amount of the product price and the amount of consumption tax, or the product price.

[0176] (Step S802) If the result of the number check is "NG", the process proceeds to step S803, and if the result of the number check is "OK", the process proceeds to step S806.

[0177] (Step S803) The price modification means 1342 modifies the consumption tax amount obtained in step S801 to obtain a new consumption tax amount. The modified consumption tax amount is, for example, a fixed percentage (e.g., 0.8 times, 0.5 times, or 0 times) of the consumption tax obtained in step S801.

[0178] (Step S804) The price change means 1342 uses the total amount of the product price and the consumption tax amount acquired in step S801 or the product price to acquire a product price increased by the amount of the reduced consumption tax.

[0179] (Step S805) The result output unit 141 outputs an error message indicating that the registration number does not exist. Note that the fact that the registration number does not exist usually means that the registration number written on the receipt does not exist in the DB, but it may also mean that the registration number is not written on the receipt.

[0180] (Step S806) If the period inspection result is "NG", the process proceeds to step S807, and if the period inspection result is "OK", the process proceeds to step S808.

[0181] (Step S807) The result output unit 141 outputs an error message to the effect that the receipt is invalid because the issue date is too old.

[0182] (Step S808) The receipt processing unit 134 determines whether the receipt information accumulation condition is met. If the accumulation condition is met, the process proceeds to step S809; if the accumulation condition is not met, the process returns to the upper level process. Note that an example of an accumulation condition is that the period inspection result is "OK." Alternatively, step S808 may be omitted and the process may proceed from step S807 to S809.

[0183] (Step S809) The receipt processing unit 134 acquires a receipt image.

[0184] (Step S810) The receipt processing unit 134 acquires the image type.

[0185] (Step S811) The receipt processing unit 134 acquires the product price, consumption tax amount, and the like.

[0186] (Step S812) The receipt processing unit 134 generates receipt information including a receipt image, image type, product price, consumption tax amount, and the like.

[0187] (Step S813) The receipt processing unit 134 stores the receipt information configured in step S812 in the receipt management unit 111 in association with the user identifier.

[0188] (Step S814) The result output unit 141 outputs a message that the receipt has been registered, and the process returns to the upper level process.

[0189] A specific example of the operation of the receipt system A in this embodiment will be described below.

[0190] The receipt management table shown in FIG. 9 is stored in the receipt management unit 111 of the receipt processing device 1. The receipt management table is a table for managing receipt information. The receipt management table includes "ID," "user identifier," "receipt information," and "received date." "Receipt information" includes "issue date," "product price," "consumption tax amount," "registration number," "payee," "receipt image," "image type," and "issue date error." A "-" in the "registration number" indicates "NULL," meaning no data is entered. "issue date error" is a flag indicating whether the period elapsed since the receipt's issue date meets the invalidity condition. An issue date error of "1" indicates that the period elapsed meets the invalidity condition (is invalid). An issue date error of "0" indicates that the period elapsed does not meet the invalidity condition. "received date" is the date the receipt image was received. "receipt image" is the file name of the receipt image stored in the receipt management unit 111.

[0191] In the above situation, the following two specific examples will be explained. Specific Example 1 is a case where a photo (receipt image) that does not show the four corners of a receipt is uploaded from the terminal device 2 to the receipt processing device 1. Specific Example 2 is a case where a receipt image with a non-existent registration number printed on it is uploaded from the terminal device 2 to the receipt processing device 1.

[0192] (Example 1) It is assumed that a user U identified by a user identifier "U003" takes the photo shown in FIG. 10 and transmits a receipt application instruction including the photo (receipt image) from the terminal device 2 to the receipt processing device 1.

[0193] The image receiving unit 121 of the receipt processing device 1 receives a receipt application instruction that includes a receipt image (FIG. 10).

[0194] Next, the image inspection unit 131 performs the image inspection process described above. That is, the resolution inspection means 1311 acquires the resolution of the receipt image (FIG. 10) and determines that the resolution satisfies the resolution condition. Meanwhile, the four-corner inspection means 1312 performs the above-described four-corner inspection on the receipt image (FIG. 10) and determines that the four corners are not included.

[0195] Then, the four-corner inspection means 1312 receives an error message indicating that the four corners are not included.

[0196] Next, the result output unit 141 transmits the error message to the terminal device 2.

[0197] The terminal device 2 receives and outputs the error message. An example of such a screen is shown in Fig. 11. In Fig. 11, an error message 1101 is output.

[0198] (Example 2) Next, assume that user U takes a photo showing the four corners of the receipt in Figure 10. Then, assume that the user selects the "re-upload" button 1102 in Figure 11 and uploads the photo showing the four corners to the receipt processing device 1.

[0199] Next, the image receiving unit 121 of the receipt processing device 1 receives a receipt application instruction including the receipt image, which is the photograph.

[0200] Next, the image inspection unit 131 performs the image inspection process described above. That is, the resolution inspection means 1311 acquires the resolution of the receipt image and determines that the resolution satisfies the resolution conditions. Furthermore, the four-corner inspection means 1312 performs the above-described four-corner inspection on the received receipt image and determines that the four corners are included.

[0201] Also, it is assumed that the number acquisition means 1321 acquires the registration number "T1234567890123" from the receipt image. Next, the amount acquisition means 1322 acquires the "payment price (product price + consumption tax)" of "2900" printed on the receipt image and the "consumption tax amount" of "263" printed on the receipt image. Furthermore, the issue date acquisition means 1323 acquires the issue date "June 14, 2024" printed on the receipt image. Furthermore, the information acquisition unit 132 acquires other information (for example, the payee).

[0202] Next, the number checking means 1331 searches the registration number database (DB) using the registration number acquired by the number acquiring means 1321 as a key, and determines that the registration number does not exist in the DB. Then, the number checking means 1331 assigns "NG (e.g., "0")" to the variable "number checking result."

[0203] The period checking means 1332 obtains "0", the elapsed period from the issue date, which is the difference between the issue date (June 14, 2024) obtained by the issue date obtaining means 1323 and today's date (here, "June 14, 2024"). Next, the period checking means 1332 determines that the elapsed period "0" does not satisfy the invalid condition (elapsed period >= 68 days). The period checking means 1332 assigns "OK (e.g., "1")" to the variable "period check result".

[0204] Next, the price change means 1342 acquires the total amount of the product price and consumption tax acquired by the amount acquisition means 1322, "2900", and the consumption tax amount, "263". Next, the price change means 1342 acquires the consumption tax amount, "210", obtained by changing the acquired consumption tax amount, "263". Note that the changed consumption tax amount is assumed to be a fixed percentage of the consumption tax (here, 0.8 times). Next, the price change means 1342 acquires the changed product price, "2690", by subtracting the consumption tax amount, "210", from the total amount of the product price and consumption tax, "2900".

[0205] Furthermore, the result output unit 141 transmits to the terminal device 2 an error message to the effect that the registration number does not exist.

[0206] The terminal device 2 receives and outputs the error message.

[0207] Next, the receipt processing unit 134 acquires a receipt image. The type acquisition means 1324 also acquires image type "0", which is the type of receipt image. The receipt processing unit 134 also acquires the product price "2690", the consumption tax amount "210", the payee "XX Company", etc. Next, the receipt processing unit 134 creates receipt information including the receipt image, image type, product price, consumption tax amount, etc. Next, the receipt processing unit 134 associates the receipt information with the user identifier "U003" and stores it in the receipt management table (Figure 9). This receipt information is the record "ID=401" in Figure 9.

[0208] As described above, according to this embodiment, it is possible to inspect a receipt image. In particular, according to this embodiment, it is possible to inspect the registration number of the receipt image. Furthermore, according to this embodiment, it is possible to inspect whether the receipt image contains the entire receipt. Furthermore, according to this embodiment, it is possible to inspect whether the issue date obtained from the receipt image is correct.

[0209] Furthermore, according to this embodiment, appropriate processing can be performed in accordance with the inspection results regarding the registration number of the receipt image.

[0210] Furthermore, according to this embodiment, the type of receipt image can be obtained.

[0211] In this embodiment, the receipt processing device 1 may perform only some of the two or more inspections. In other words, the receipt processing device 1 may only inspect the registration number of the receipt image. The receipt processing device 1 may only inspect whether the receipt image contains the entire receipt.

[0212] Furthermore, according to this embodiment, the receipt processing device 1 may acquire the image type of the receipt image without performing an inspection, and store the image type in pairs with the receipt image or information that can be acquired from the receipt image.

[0213] Furthermore, the processing in this embodiment may be implemented by software. This software may be distributed by software download, etc. This software may also be recorded on a recording medium such as a CD-ROM and distributed. This also applies to the other embodiments in this specification. The software that implements the receipt processing device 1 in this embodiment is the following program. In other words, this program causes a computer to function as an image receiving unit that receives a receipt image, a number acquisition means that acquires a registration number from the receipt image received by the image receiving unit, a number inspection means that inspects whether the registration number exists in a registration number database, and a receipt processing unit that performs processing according to the inspection results of the number inspection means.

[0214] FIG. 12 is a block diagram of a computer system 300 that executes the programs described in this specification to realize the receipt processing device 1 and the like according to the various embodiments described above.

[0215] In FIG. 12, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0216] 12, computer 301 includes, in addition to CD-ROM drive 3012, MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as a boot-up program, RAM 3016 connected to MPU 3013 for temporarily storing instructions of application programs and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data. Although not shown here, computer 301 may further include a network card for providing connection to a LAN.

[0217] A program that causes the computer system 300 to execute the functions of the receipt processing device 1 and the like of the above-described embodiment may be stored on a CD-ROM 3101, inserted into the CD-ROM drive 3012, and then transferred to the hard disk 3017. Alternatively, the program may be sent to the computer 301 via a network (not shown) and stored on the hard disk 3017. The program is loaded into RAM 3016 when executed. The program may also be loaded directly from the CD-ROM 3101 or the network.

[0218] The program does not necessarily include an operating system (OS) or third-party programs that cause the computer 301 to execute the functions of the receipt processing device 1 of the above-described embodiment. The program need only include instructions that call appropriate functions (modules) in a controlled manner to achieve the desired results. How the computer system 300 operates is well known, and a detailed description will be omitted.

[0219] In addition, in the above program, the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmission step (processing that can only be performed by hardware).

[0220] The computer that executes the program may be a single computer or a plurality of computers, that is, it may perform centralized processing or distributed processing.

[0221] Furthermore, in each of the above embodiments, it goes without saying that two or more communication means present in one device may be physically realized by one medium.

[0222] Furthermore, in each of the above embodiments, each process may be realized by centralized processing in a single device, or may be realized by distributed processing in a plurality of devices.

[0223] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention. [Industrial Applicability]

[0224] As described above, the receipt processing device 1 according to the present invention has the effect of being able to properly inspect receipt images, and is useful as a server that manages receipt information, etc. [Explanation of symbols]

[0225] A Receipt System 1. Receipt processing device 2. Terminal Device 11 Storage area 12 Reception 13 Processing section 14 Output section 21 Terminal storage section 22 Terminal Reception 23 Terminal processing section 24 Terminal transmitter 25 Terminal receiving section 26 Terminal Output Unit 111 Receipt Management Department 121 Image Reception Department 122 Instruction Reception Department 131 Imaging Examination Department 132 Information Acquisition Department 133 Content Inspection Department 134 Receipt Processing Unit 141 Result output section 1311 Resolution Testing Method 1312 Four corner inspection means 1321 Number Acquisition Method 1322 Amount acquisition means 1323 Issue date acquisition method 1324 type acquisition means 1331 Number Verification Method 1332 Period inspection means 1341 Image storage means 1342 Price Change Method 1343 Commodity Price Accumulation Instrument 1344 Consumption tax accumulation means 1345 types of storage means

Claims

1. an image receiving unit that receives a receipt image; a number obtaining means for obtaining a registration number from the receipt image received by the image receiving unit; a registration number database for managing two or more registration numbers, and a number checking means for checking whether the registration number exists in the registration number database held by an external server; a receipt processing unit that performs processing according to the inspection result of the number inspection unit; The number acquisition means Perform character recognition on the receipt image and obtain the registration number, which is "'T' + 13-digit numeric string," from the receipt image; or A set of training data having a receipt image and coordinate information where the registration number is displayed is provided to a machine learning learning module, the module is executed, the acquired learning model and the receipt image are provided to a machine learning prediction module, the module is executed, the coordinate information where the registration number is displayed is acquired, and the registration number is obtained as a character string at the position indicated by the coordinate information, or Provide a set of training data having receipt images and registration numbers to a machine learning learning module, execute the module, provide the acquired learning model and the receipt images to a machine learning prediction module, execute the module, and acquire the registration numbers; or Passing the receipt image and a prompt for obtaining a registration number to a generation AI, and obtaining the registration number from the generation AI; If the number checking means determines that the registration number does not exist in the registration number database, the receipt processing unit: An error message will be displayed stating that the registration number does not exist, or acquiring a revised product price by adding a part or all of the consumption tax amount acquired using the receipt image to the product price acquired using the receipt image; an image inspection unit that inspects the receipt image received by the image reception unit; a result output unit that outputs the results of the inspection in the image inspection unit; an issue date acquisition means for acquiring the issue date of the receipt from the receipt image accepted by the image acceptance unit; a period checking means for checking whether or not the elapsed period from the issue date acquired by the issue date acquisition means to today satisfies an illegal condition; The issue date acquisition means performing character recognition processing on the receipt image to obtain the date of issue, which is a character string corresponding to a date type; or A set of training data having a receipt image and coordinate information where the issue date is displayed is provided to a machine learning learning module, the module is executed, the acquired learning model and the receipt image are provided to a machine learning prediction module, the module is executed, the coordinate information where the issue date is displayed is acquired, and the issue date is acquired as a character string at the position indicated by the coordinate information, or Provide a set of training data having receipt images and issuance dates to a machine learning learning module, execute the module, provide the acquired learning model and the receipt images to a machine learning prediction module, execute the module, and acquire the issuance dates; or Passing the receipt image and a prompt for obtaining the issue date to a generation AI, and obtaining the issue date from the generation AI; The receipt processing device further comprises a result output unit that outputs the result of the inspection by the period inspection means.

2. The method further includes: obtaining a price of a product and a consumption tax amount using the receipt image; The receipt processing unit 2. A receipt processing device as described in claim 1, further comprising a price change means for acquiring a modified product price by adding part or all of the consumption tax amount acquired by the amount acquisition means to the product price when the number acquisition means does not acquire the registration number from the receipt image, or when the number inspection means obtains an inspection result indicating that the registration number does not exist in the registration number database.

3. The receipt processing unit 3. A receipt processing device according to claim 2, further comprising product price storage means for storing said changed product prices.

4. an image inspection unit that inspects the receipt image received by the image reception unit; 2. The receipt processing device according to claim 1, further comprising a result output unit that outputs the results of the inspection in the image inspection unit.

5. The image inspection unit 5. The receipt processing device according to claim 4, further comprising a four-corner inspection means for inspecting whether or not the four corners of the receipt are present in the receipt image received by the image receiving unit.

6. a type acquiring means for acquiring an image type indicating whether the receipt image accepted by the image accepting unit is an image of an electronic receipt or an image of a paper receipt; 2. The receipt processing device according to claim 1, further comprising: a type storage means for storing the image type acquired by the type acquisition means.

7. A receipt processing method, wherein all of the processes performed by the receipt processing device according to any one of claims 1 to 6 are performed using a computer.

8. Computer, A program for causing the receipt processing device according to any one of claims 1 to 6 to function as the receipt processing device.

Images