Inspection apparatus, inspection method, and program

Abstract

To solve the problem that, in conventional inspection apparatuses, flyers containing incorrect information may be distributed.SOLUTION: An inspection apparatus 1 capable of preventing flyers with incorrect information from being distributed, comprises: an image receiving unit 121 for receiving a flyer image including two or more pieces of product image information including a flyer product image and one or more character string images related to a product; an image analysis unit 131 for analyzing the flyer image and obtaining an analysis result having the flyer product image included in each of two or more pieces of product image information and one or more recognized character strings that are the result of recognizing the character string images; an inspection unit 132 for inspecting, for each of the two or more analysis results, whether or not there is a product record corresponding to the product image included in the analysis result and one or more recognized character strings included in the analysis result by referring to a product master 1111 having two or more product records each having the product image and one or more product character strings, and obtaining inspection results; and an output unit 14 for outputting the inspection results.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an inspection apparatus for inspecting a flyer image and the like.

Background Art

[0002] Conventionally, there has been an apparatus for assisting in the creation of flyers and catalogs, particularly an apparatus for assisting in the creation of flyers and catalogs having variations consisting of a plurality of editions such as a Kanto edition and a Kansai edition (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, it has not been possible to prevent the distribution of flyers with incorrect content.

Means for Solving the Problems

[0005] The inspection apparatus of the first invention of the present invention includes an image reception unit that receives a flyer image including two or more pieces of product image information including a flyer product image and one or more character string images related to the product, an image analysis unit that analyzes the flyer image and obtains an analysis result having a flyer product image included in each of the two or more pieces of product image information and one or more recognized character strings that are the result of recognizing the character string image, a product master having two or more product records having a product image and one or more product character strings, and for each of the two or more analysis results, referring to the product master to check whether there is a product record corresponding to the product image included in the analysis result and one or more recognized character strings included in the analysis result, and obtaining an inspection result, and an output unit that outputs the inspection result.

[0006] With such a configuration, it is possible to prevent the distribution of flyers with incorrect content.

[0007] Further, in the inspection apparatus of the second invention, with respect to the first invention, the image analysis unit includes: character recognition means for performing character recognition processing on the flyer image to obtain two or more recognition character strings; product image acquisition means for acquiring two or more flyer product images from the flyer image; and correspondence means for associating the recognition character string with the flyer image using the string position information of each of the two or more recognition character strings in the flyer image and the image position information of the flyer product image in the flyer image. The inspection unit includes: image inspection means for performing an image inspection process of obtaining the similarity between each of the two or more flyer product images included in the flyer image and the product image in the product master and determining whether the similarity satisfies the image determination condition; string inspection means for performing a string inspection process of determining whether one or more recognition character strings corresponding to each of the two or more flyer product images are product records in the product master and exist in the product records including the product image; and inspection result acquisition means for obtaining an inspection result using the result of the image inspection process and the result of the string inspection process.

[0008] With such a configuration, it is possible to prevent the distribution of flyers with incorrect content.

[0009] Further, in the inspection apparatus of the third invention, with respect to the first invention, the product information includes a character string image indicating the price of the product, the product record includes a product character string indicating the price of the product, and the inspection unit is an inspection apparatus that performs inspections corresponding to both the case where the recognition character string indicating the price of the product is a character string including consumption tax and the case where it does not include consumption tax.

[0010] With such a configuration, by performing an inspection considering consumption tax on the price of the product, it is possible to prevent the distribution of flyers with incorrect prices of the products described.

[0011] In addition, for the third invention, the inspection apparatus of the fourth invention is such that the product record includes a product character string indicating the first amount including consumption tax of the product and a product character string indicating the second amount not including consumption tax, and the inspection unit acquires an inspection result indicating an abnormality regarding the amount of the product only when the recognition character string indicating the amount of the product is different from both the product character strings indicating the first amount and the second amount.

[0012] With such a configuration, by performing an inspection considering consumption tax on the amount of the product, it is possible to prevent the distribution of flyers with incorrect product amounts described therein.

[0013] In addition, for the first invention, the inspection apparatus of the fifth invention is such that, among two or more attributes corresponding to the product record of the product master, string judgment conditions are stored in association with one or more of each attribute, and the inspection unit determines whether the product character string and the recognition character string corresponding to one or more of each attribute satisfy the string judgment conditions, and acquires an inspection result.

[0014] With such a configuration, it is possible to prevent the distribution of flyers with incorrect contents.

[0015] In addition, for any one of the first to fifth inventions, the inspection apparatus of the sixth invention is such that the flyer image received by the image reception unit is associated with a user identifier, and the inspection unit selects, from two or more product masters associated with the user identifier, the product master associated with the same user identifier as the user identifier associated with the flyer image, performs an inspection using the product master, and acquires an inspection result.

[0016] With such a configuration, by using the product master corresponding to each user, it is possible to prevent the distribution of flyers with incorrect contents.

[0017] Further, in the inspection apparatus of the seventh invention, for any one of the first to fifth inventions, the flyer image received by the image reception unit is associated with a master identifier, and each of the inspection units selects, from two or more product masters associated with the master identifier, a product master associated with the same master identifier as the master identifier associated with the flyer image, and performs an inspection using the product master to obtain an inspection result.

[0018] With such a configuration, by using an appropriate product master, it is possible to prevent the distribution of flyers with incorrect content.

[0019] Further, in the inspection apparatus of the eighth invention, for any one of the first to seventh inventions, the inspection unit is an inspection apparatus that obtains an inspection result including a product image or a product character string of a target product record detected as abnormal and product image information included in the flyer image.

[0020] With such a configuration, the inspection result of the flyer can be presented to the user in an easy-to-understand manner.

Advantages of the Invention

[0021] According to the inspection apparatus of the present invention, it is possible to prevent the distribution of flyers with incorrect content.

Brief Description of the Drawings

[0022]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Mode for Carrying Out the Invention

[0023] Hereinafter, embodiments of an inspection apparatus and the like will be described with reference to the drawings. In the embodiments, components denoted by the same reference numerals perform the same operations, and thus the description may be omitted again.

[0024] (Embodiment 1) In the present embodiment, an inspection apparatus will be described that refers to a product master having two or more records including a product image and a product character string, inspects errors in the received flyer image, and outputs an inspection result. The flyer image is usually an image of a flyer, but may also be an image such as a catalog, and is interpreted broadly. The flyer image is, for example, JPEG, GIF, etc., but its format does not matter. The flyer image may be, for example, an image on a web page. The flyer image is an image on which information of two or more products is posted.

[0025] Also, in the present embodiment, a inspection device that performs an inspection considering the presence or absence of consumption tax on the price of a product in a flyer image will be described.

[0026] Also, in the present embodiment, an inspection device that determines whether it is normal or not based on the attribute of a product character string will be described.

[0027] Also, in the present embodiment, an inspection device that uses a product master corresponding to a user who transmitted a flyer image to inspect an error in the flyer image and outputs an inspection result will be described.

[0028] Furthermore, in the present embodiment, an inspection device that uses a product master identified by a received master identifier to inspect an error in the flyer image and outputs an inspection result will be described.

[0029] Note that in this specification, the fact that information X is associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not limited. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc.

[0030] Also, in this specification, selecting or determining information Z means obtaining information Z, obtaining a pointer to information Z, obtaining the ID of information Z, setting a flag for information Z, etc., as long as information Z can be accessed.

[0031] FIG. 1 is a conceptual diagram of an information system A in the present embodiment. The information system A includes an inspection device 1 and one or two or more terminal devices 2.

[0032] The inspection device 1 is a device for inspecting a flyer image. The inspection device 1 is, for example, a cloud server or an ASP server, and its type does not matter. When the inspection device 1 is a terminal, it can be considered that the terminal device 2 is unnecessary for the information system A, or the inspection device 1 also serves as the terminal device 2.

[0033] The terminal device 2 is a terminal used by the user. The user is a person who wishes to inspect the flyer image. The terminal device 2 is, for example, a so-called personal computer, smartphone, or tablet terminal, and its type does not matter.

[0034] Figure 2 is a block diagram of the information system A in the present embodiment.

[0035] The inspection device 1 includes a storage unit 11, a reception unit 12, a processing unit 13, and an output unit 14. The storage unit 11 includes a product management unit 111. The product management unit 111 includes one or more product masters 1111. The reception unit 12 includes an image reception unit 121. The processing unit 13 includes an image analysis unit 131 and an inspection unit 132. The image analysis unit 131 includes a character recognition means 1311, a product name determination means 1312, a product image acquisition means 1313, and a correspondence means 1314. The inspection unit 132 includes an image inspection means 1321, a character string inspection means 1322, and an inspection result acquisition means 1323.

[0036] The terminal device 2 includes a terminal storage unit 21, a terminal reception unit 22, a terminal processing unit 23, a terminal transmission unit 24, a terminal reception unit 25, and a terminal output unit 26.

[0037] Various types of information are stored in the storage unit 11 that constitutes the inspection device 1. The various types of information are, for example, product masters 1111 and various conditions. The various conditions are, for example, image judgment conditions and character string judgment conditions. Needless to say, the various conditions may also be embedded in the program.

[0038] The image determination condition is a condition for determining whether a flyer product image, which is an image of a product in a flyer image, is normal or abnormal. The image determination condition is, for example, that the similarity between the flyer product image and the product image corresponding to the flyer product image is equal to or greater than a threshold value. The product image is an image in the product master 1111.

[0039] The character string determination condition is a condition for determining whether a character string image, which is an image showing a character string in a flyer image, is normal or abnormal. The character string determination condition is, for example, that the recognized character string, which is the result of character recognition of the character string image, matches the product character string corresponding to the recognized character string. The character string determination condition is, for example, that the similarity between the recognized character string, which is the result of character recognition of the character string image, and the product character string corresponding to the recognized character string is equal to or greater than a threshold value. The product character string is a character string in the product master 1111.

[0040] It is preferable that each of two or more character string determination conditions is associated with different attributes of the product record.

[0041] The product management department 111 manages the product master 1111. One or two or more product masters 1111 are stored in the product management department 111.

[0042] The product master 1111 has two or more product records. A product record is a set of information about a product. A product record has a product image and one or more product character strings. A product image is an image of a product. A product character string is an attribute value of a product. A product character string is a character string related to a product. One or more product character strings are, for example, a product name, a price, and a product description. The product character strings included in the product record have, for example, a first amount and a second amount. Both the first amount and the second amount are the prices of the product in the product master 1111. The first amount is the price including consumption tax. The second amount is the price excluding consumption tax.

[0043] The product master 1111 is usually a database. The product master 1111 can be, for example, a table in a relational database, an object in an object-oriented database, a CSV file, etc., regardless of its structure.

[0044] The reception unit 12 receives various instructions and information. The various instructions and information are, for example, a start instruction, a flyer image, and the product master 1111. The start instruction is an instruction to start the inspection of the flyer image. The start instruction usually has a flyer image or information (e.g., a link) for acquiring the flyer image.

[0045] The image reception unit 121 receives the flyer image. The image reception unit 121 receives, for example, a start instruction.

[0046] Here, reception usually refers to the reception of information transmitted via a wired or wireless communication line, but it may also be a concept including the reception of information input from input devices such as a keyboard, a mouse, and a touch panel, and the reception of information read from recording media such as an optical disk, a magnetic disk, and a semiconductor memory.

[0047] It is preferable that a user identifier or a master identifier is associated with the flyer image received by the image reception unit 121. The flyer image received by the image reception unit 121 is the flyer image to be inspected.

[0048] The user identifier is information for identifying a user. The user identifier is, for example, a user ID, an email address, a phone number, or a name. The master identifier is information for identifying the product master 1111. The master identifier is the ID of the product master 1111 or the name of the product master 1111.

[0049] The flyer image includes two or more pieces of product image information. In the flyer image, two or more pieces of product image information are exposed. Product image information is an image of information related to a product. The product image information includes a flyer product image and one or more string images. The flyer product image is the product image within the flyer image. The product image can be an image of the product taken or an illustration image of the product, etc., regardless of its creation process. The string image is an image of a string of information related to the product.

[0050] The processing unit 13 performs various processes. The various processes are, for example, the processes performed by the image analysis unit 131 and the inspection unit 132.

[0051] The image analysis unit 131 analyzes the flyer image received by the image reception unit 121 and obtains two or more analysis results. The analysis results include a flyer product image and one or more recognized strings. The flyer product image is an image of the product included in two or more pieces of product image information contained in the flyer image. The recognized string is the string as the result of recognizing the string image included in the flyer image.

[0052] The character recognition means 1311 performs character recognition processing on the flyer image and obtains two or more recognized strings. Usually, the character recognition means 1311 also obtains string position information corresponding to two or more recognized strings from the flyer image. The string position information is information for specifying the relative position of the recognized string in the flyer image. The string position information is, for example, the upper left coordinate (x1, y1) and the lower right coordinate (x2, y2) of the recognized string. The string position information is, for example, the upper left coordinate (x1, y1) of the recognized string.

[0053] The product name determination means 1312 determines two or more product names from the two or more recognized strings obtained by the character recognition means 1311.

[0054] The technology for determining whether a recognized string is a product name is a technology for determining the class of a string and is a known technology. The process of determining whether a recognized string is a product name is, for example, to search through the product master 1111 using the string as a key, and when it exists, to determine whether the attribute of the string indicates the attribute of "product name".

[0055] The flyer product image acquisition means 1313 acquires two or more flyer product images from the flyer image received by the image reception unit 121. The flyer product image acquisition means 1313 usually also acquires image position information corresponding to each of the two or more flyer product images from the flyer image. The image position information is information for specifying the relative position of the flyer product image within the flyer image. The image position information is, for example, the upper left coordinates (x1, y1) and the lower right coordinates (x2, y2) of the flyer product image. The image position information is, for example, the upper left coordinates (x1, y1) of the flyer product image.

[0056] The process of acquiring flyer product images from a flyer image can be performed using "One-Stage One-Shot Object Detection (OS2D)" or the like, and such a process is a known technology.

[0057] The correspondence means 1314 associates one or more recognized strings with the flyer image using the string position information of each of the two or more recognized strings in the flyer image and the image position information of the flyer product image in the flyer image.

[0058] The inspection unit 132 refers to the product master 1111 and inspects whether there is a product record corresponding to the product image and one or more recognized strings included in the analysis result for each of the two or more analysis results, and obtains an inspection result. The inspection unit 132, for example, obtains the inspection result "normal" when a product record exists for each analysis result, and obtains the inspection result "abnormal" when a product record does not exist. The inspection unit 132, for example, obtains the inspection result "normal" or "abnormal" for each of the two or more analysis results.

[0059] The inspection unit 132 preferably performs inspections corresponding to both the case where the recognition string indicating the price of the product is a string including consumption tax and the case where it is a string not including consumption tax. Note that the price of the product is the price.

[0060] For example, the inspection unit 132 obtains an inspection result indicating an abnormality regarding the price of the product only when the recognition string indicating the price of the product is different in both the first price and the product string indicating the second price.

[0061] The inspection unit 132 determines whether the product string and the recognition string corresponding to each of one or more attributes satisfy the string determination condition, and obtains an inspection result.

[0062] The string determination condition corresponding to the attribute "product name" is, for example, "match". The string determination condition corresponding to the attribute "price" is, for example, "match either the first price or the second price". The string determination condition corresponding to the attribute "product description" is, for example, "the similarity is equal to or greater than the threshold value".

[0063] For example, the inspection unit 132 selects, from two or more product masters 1111 associated with the user identifier, the product master 1111 associated with the same user identifier as the user identifier associated with the flyer image, performs an inspection using the product master 1111, and obtains an inspection result.

[0064] For example, the inspection unit 132 selects, from two or more product masters 1111 associated with the master identifier, the product master 1111 associated with the same master identifier as the master identifier associated with the flyer image, performs an inspection using the product master 1111, and obtains an inspection result.

[0065] For example, the inspection unit 132 obtains an inspection result including the product image or the product string of the product record of the target detected as abnormal and the product image information included in the flyer image.

[0066] The inspection unit 132 obtains an inspection result including, for example, the product string of the product record of the target detected as abnormal and the recognition string obtained from the flyer image.

[0067] The image inspection means 1321 performs an image inspection process. For example, for each of two or more flyer product images included in the flyer image, the image inspection means 1321 obtains the similarity between the flyer product image and the product image in the product master 1111, and performs an image inspection process of determining whether the similarity satisfies the image determination condition.

[0068] Here, the flyer product image and the product image for which the image inspection means 1321 obtains the similarity are images corresponding to the same product name. The flyer product image and the product image for which the similarity is obtained are corresponding images. The product image corresponding to the flyer product image is, for example, the string corresponding to the flyer product image, and is the product image paired with the product name determined by the product name determination means 1312 in the product master 1111.

[0069] The string inspection means 1322 performs a string inspection process. For each of one or more recognition strings corresponding to two or more flyer product images, the string inspection means 1322 determines whether the recognition string and the corresponding product string satisfy the string determination condition. For example, the string inspection means 1322 performs a string inspection process of determining whether one or more recognition strings corresponding to two or more flyer product images exist in the product records in the product master 1111 and in the product records including product images.

[0070] The inspection result acquisition means 1323 obtains an inspection result using the result of the image inspection process and the result of the string inspection process.

[0071] It is preferable for the inspection result acquisition means 1323 to obtain an inspection result for each of two or more analysis results. It is preferable for the inspection result acquisition means 1323 to obtain an inspection result for each of two or more analysis results and for each of the flyer product image and one or more string images.

[0072] The inspection result acquisition means 1323 normally acquires the inspection result "normal" only when both the result of the image inspection process and the result of the character string inspection process are "normal".

[0073] The output unit 14 outputs the inspection result acquired by the inspection unit 132.

[0074] Here, the output usually means transmission to the terminal device 2, but it may also be a concept including display on a display, projection using a projector, printing by a printer, sound output, storage in a recording medium, delivery of a processing result to another processing device or another program, etc.

[0075] Various types of information are stored in the terminal storage unit 21 that constitutes the terminal device 2. The various types of information are, for example, a flyer image, a user identifier, a master identifier, and a product master 1111. Note that the information in the terminal storage unit 21 may be information input by the user and temporarily stored.

[0076] The terminal reception unit 22 receives various types of information, instructions, etc. The various types of information, instructions, etc. are, for example, a start instruction and a product master 1111. The start instruction has, for example, a user identifier or a master identifier.

[0077] The input means for various types of information, instructions, etc. may be anything, such as a touch panel, a keyboard, a mouse, or a menu screen.

[0078] The terminal processing unit 23 performs various types of processing. The various types of processing are, for example, processing to convert the received information, instructions, etc. into information, instructions, etc. having a structure for transmission. The various types of processing are, for example, processing to convert the received information into information having a structure for output.

[0079] The terminal transmission unit 24 transmits various types of information, instructions, etc. to the inspection device 1. The various types of information, instructions, etc. are, for example, a product master, a start instruction, a flyer image, a user identifier, or a master identifier.

[0080] The terminal receiving unit 25 receives various types of information from the inspection device 1. The various types of information are, for example, inspection results.

[0081] The terminal output unit 26 outputs various types of information. The various types of information are, for example, inspection results.

[0082] The storage unit 11, the product management unit 111, and the product master 1111 are preferably non-volatile recording media, but can also be realized with volatile recording media.

[0083] The process of storing information in the storage unit 11 or the like is not limited. For example, information may be stored in the storage unit 11 or the like via a recording medium, information transmitted via a communication line or the like may be stored in the storage unit 11 or the like, or information input via an input device may be stored in the storage unit 11 or the like.

[0084] The reception unit 12 and the image reception unit 121 are preferably realized by wireless or wired communication means, but may also be realized by means of receiving broadcasts, device drivers for input means such as touch panels and keyboards, control software for menu screens, etc.

[0085] The processing unit 13, the image analysis unit 131, the inspection unit 132, the character recognition means 1311, the product name determination means 1312, the product image acquisition means 1313, the correspondence means 1314, the image inspection means 1321, the character string inspection means 1322, the inspection result acquisition means 1323, and the terminal processing unit 23 can usually be realized from a processor, a memory, etc. The processing procedures of the processing unit 13 etc. are usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may also be realized by hardware (dedicated circuit). Note that the processor is a CPU, MPU, GPU, etc., and its type is not limited.

[0086] The output unit 14 can be realized by wireless or wired communication means, but may also be realized by driver software for the output device or driver software for the output device and the output device etc.

[0087] The terminal reception unit 22 can be implemented by a device driver of an input means such as a touch panel or a keyboard, control software of a menu screen, or the like.

[0088] The terminal transmission unit 24 is usually implemented by wireless or wired communication means, but may be implemented by a broadcast means.

[0089] The terminal reception unit 25 is usually implemented by wireless or wired communication means, but may be implemented by a means for receiving a broadcast.

[0090] The terminal output unit 26 may or may not be considered to include output devices such as a display and a speaker. The terminal output unit 26 can be implemented by driver software of an output device or driver software of an output device and an output device or the like.

[0091] Next, an operation example of the inspection apparatus 1 will be described with reference to the flowchart of FIG. 3.

[0092] (Step S301) The image reception unit 121 determines whether or not it has received a start instruction from the terminal device 2. If it has received the start instruction, it proceeds to step S302, and if it has not received it, it proceeds to step S307. Here, it is preferable that the start instruction has a user identifier or a master identifier.

[0093] (Step S302) The processing unit 13 acquires the leaflet image included in the start instruction received in step S301.

[0094] (Step S303) The processing unit 13 acquires the product master 1111 from the product management unit 111. The processing unit 13 acquires, for example, the product master 1111 that pairs with the user identifier or the master identifier included in the start instruction.

[0095] (Step S304) The processing unit 13 uses the product master 1111 obtained in step S303 to inspect the flyer image obtained in step S302. An example of such inspection processing will be described with reference to the flowcharts of FIGS. 4 and 9.

[0096] (Step S305) The inspection unit 132 composes the inspection results to be output. An example of such result composition processing will be described with reference to the flowchart of FIG. 11.

[0097] (Step S306) The output unit 14 transmits the inspection results composed in step S305 to the terminal device 2.

[0098] (Step S307) The reception unit 12 determines whether the product master 1111 has been received from the terminal device 2. If the product master 1111 has been received, the process proceeds to step S308; if not, the process returns to step S301. Here, the product master 1111 may be associated with a user identifier or a master identifier.

[0099] (Step S308) The processing unit 13 stores the product master 1111 received in step S307 in the product management unit 111. The process returns to step S301. Here, the processing unit 13 may store the product master 1111 in the product management unit 111 in association with a user identifier or a master identifier. Also, the processing unit 13 may generate a unique master identifier.

[0100] Note that in the flowchart of FIG. 3, the process ends due to a power-off or a processing end interrupt.

[0101] Next, a first example of the inspection processing in step S304 will be described with reference to the flowchart of FIG. 4.

[0102] (Step S401) The character recognition means 1311 performs character recognition processing on the flyer image and obtains one or more recognition character strings. The character recognition means 1311 usually obtains one or more recognition character strings in association with the character string position information.

[0103] (Step S402) The product name determination means 1312 determines one or more recognized character strings corresponding to the product name from among the one or more recognized character strings acquired in Step S401. An example of such product name acquisition processing will be described using the flowcharts of FIGS. 5 and 6.

[0104] (Step S403) The inspection unit 132 substitutes 1 for the counter i.

[0105] (Step S404) The inspection unit 132 determines whether the i-th product name exists among the product names acquired in Step S402. If the i-th product name exists, it proceeds to Step S405; if not, it returns to the upper-level process.

[0106] (Step S405) The inspection unit 132 performs image inspection processing. An example of the image inspection processing will be described using the flowchart of FIG. 7.

[0107] (Step S406) The inspection unit 132 acquires one or more recognized character strings corresponding to the i-th product name from among the recognized character strings acquired in Step S401. Here, the one or more acquired recognized character strings may include the recognized character string indicating the i-th product name.

[0108] The inspection unit 132, for example, uses the string position information corresponding to the recognized character string corresponding to the i-th product name and the string position information of each recognized character string to obtain the distance between the recognized character string corresponding to the i-th product name and each recognized character string, and acquires one or more recognized character strings whose distance is within or less than the threshold value.

[0109] The inspection unit 132, for example, uses the string position information corresponding to the recognized character string corresponding to the i-th product name and the string position information of each recognized character string to obtain one or more recognized character strings existing at positions consecutive to the recognized character string corresponding to the i-th product name.

[0110] Incidentally, the process in which the inspection unit 132 determines two or more recognition character strings of one product specified by the i-th product name is a process of grouping two or more recognition character strings using the string position information corresponding to the recognition character strings. Such grouping process may be performed by a method other than those described above. Such grouping process can be realized by known techniques.

[0111] (Step S407) The inspection unit 132 substitutes 1 for the counter j.

[0112] (Step S408) The inspection unit 132 determines whether the j-th recognition character string exists among the one or more recognition character strings acquired in Step S406. If the j-th recognition character string exists, the process proceeds to Step S409; if not, the process proceeds to Step S411.

[0113] (Step S409) The inspection unit 132 performs a string inspection process. An example of the string inspection process will be described with reference to the flowchart of FIG. 8. Incidentally, the string inspection process is an inspection of whether one recognition character string is correct.

[0114] (Step S410) The inspection unit 132 increments the counter j by 1. The process returns to Step S408.

[0115] (Step S411) The inspection unit 132 increments the counter i by 1. The process returns to Step S404.

[0116] Next, a first example of the product name acquisition process in Step S402 will be described with reference to the flowchart of FIG. 5. The first example of the product name acquisition process is the case where the product master 1111 is used.

[0117] (Step S501) The inspection unit 132 substitutes 1 for the counter i.

[0118] (Step S502) The inspection unit 132 determines whether the i-th recognition character string to be inspected exists. If the i-th recognition character string exists, it proceeds to step S503; if it does not exist, it returns to the upper-level process. The recognition character string to be inspected is the recognition character string obtained in step S401.

[0119] (Step S503) The inspection unit 132 acquires the i-th recognition character string to be inspected.

[0120] (Step S504) The inspection unit 132 assigns 1 to the counter j.

[0121] (Step S505) The inspection unit 132 determines whether the product character string of the j-th product name exists in the product master 1111. If the product character string of the j-th product name exists, it proceeds to step S506; if it does not exist, it proceeds to step S509. It is assumed that the inspection unit 132 has previously held information for specifying the attribute name or storage location of the product name in the product master 1111.

[0122] (Step S506) The inspection unit 132 acquires the product character string of the j-th product name from the product master 1111.

[0123] (Step S507) The inspection unit 132 determines whether the i-th recognition character string acquired in step S503 matches the product character string acquired in step S506. If they match, it proceeds to step S507; if they do not match, it proceeds to step S510.

[0124] The conditions for matching are, for example, that the two character strings are identical, but they may also be that the similarity between the two character strings is greater than or equal to a threshold value, or the distance between the two character strings is within or less than a threshold value, or the two character strings are in a synonym relationship, etc. Various conditions for matching are conceivable.

[0125] (Step S508) The inspection unit 132 temporarily stores the i-th recognition character string or the like to be inspected in a buffer (not shown). The recognition character string or the like is, for example, a recognition character string and position information. The recognition character string or the like may be only the recognition character string.

[0126] (Step S509) The inspection unit 132 increments the counter i by 1. It returns to step S502.

[0127] (Step S510) The inspection unit 132 increments the counter j by 1. It returns to step S505.

[0128] Next, the second example of the product name acquisition process in step S402 will be described with reference to the flowchart of FIG. 6. The second example of the product name acquisition process is the case where machine learning is used.

[0129] (Step S601) The inspection unit 132 assigns 1 to the counter i.

[0130] (Step S602) The inspection unit 132 determines whether the i-th recognition character string to be inspected exists. If the i-th recognition character string exists, it proceeds to step S603; if it does not exist, it returns to the upper-level process.

[0131] (Step S603) The inspection unit 132 acquires the i-th recognition character string to be inspected.

[0132] (Step S604) The inspection unit 132 acquires the learning model in the storage unit 11. The learning model is, for example, information obtained by performing a learning process of machine learning using one or more positive examples including product names and one or more negative examples including information that is not a product name. Such a learning model is used in the prediction process of binary classification machine learning for obtaining information indicating whether it is a product name.

[0133] The learning model is information obtained by performing learning processing of machine learning using two or more pieces of teacher data including, for example, a character string and its class. Such a learning model is used for prediction processing of multi-class classification machine learning that acquires information indicating whether it is any of three or more classes including being a product name.

[0134] The learning model may be referred to as a learning device, a classifier, a classification model, etc. The machine learning algorithm is not limited, such as deep learning, random forest, decision tree, SVM, etc. Also, for machine learning, for example, various machine learning functions such as the TensorFlow (registered trademark) library, the random forest module of the R language, fastText, TinySVM, etc., and various existing libraries can be used.

[0135] (Step S605) The inspection unit 132 gives the i-th recognized character string and the learning model to the prediction processing of machine learning and acquires a prediction result. The prediction result is, for example, information indicating whether the i-th recognized character string is a product name, or information indicating the class of the i-th recognized character string.

[0136] (Step S606) The inspection unit 132 determines whether the prediction result corresponds to "product name". If it corresponds to "product name", it proceeds to step S607, and if it does not correspond to "product name", it proceeds to step S608.

[0137] (Step S607) The inspection unit 132 temporarily stores the i-th recognized character string and the like of the inspection target in a buffer (not shown). The recognized character string and the like are, for example, a recognized character string and position information. The recognized character string and the like may be only the recognized character string.

[0138] (Step S608) The inspection unit 132 increments the counter i by 1. It returns to step S602.

[0139] Next, an example of the image inspection process in step S405 will be described using the flowchart of FIG. 7.

[0140] (Step S701) The inspection unit 132 acquires the flier product image to be inspected from the flier image. Note that the technology for acquiring the flier product image, which is an image of a product, from the flier image is possible, for example, by using the software "os2d". The flier product image to be inspected is the flier product image that pairs with the image position information corresponding to the string position information corresponding to the i-th product name in step S404.

[0141] (Step S702) The inspection unit 132 acquires the product image to be used for inspection from the product master 1111. The product image to be used for inspection is the product image corresponding to the i-th product name in step S404.

[0142] (Step S703) The inspection unit 132 acquires the similarity between the flier product image acquired in step S701 and the product image acquired in step S702. Note that the technology for acquiring the similarity between two images is a known technology.

[0143] (Step S704) The inspection unit 132 determines whether the similarity acquired in step S703 satisfies the image determination condition. If it satisfies the image determination condition, it proceeds to step S705, and if it does not satisfy the image determination condition, it proceeds to step S706.

[0144] (Step S705) The inspection unit 132 adds "Image: OK" to the target inspection result. Returns to the upper-level process. "Image: OK" indicates that the inspection result of the flier product image was normal.

[0145] (Step S706) The inspection unit 132 adds "Image: NG" to the target inspection result. Returns to the upper-level process. "Image: NG" indicates that the inspection result of the flier product image was abnormal.

[0146] Next, an example of the character string inspection process in step S409 will be described using the flowchart in FIG. 8.

[0147] (Step S801) The inspection unit 132 acquires the recognition character string of the inspection target. The recognition character string of the inspection target is the j-th recognition character string in step S408 or the recognition character string in step S911.

[0148] (Step S802) The inspection unit 132 determines the attribute of the recognition character string acquired in step S801.

[0149] The inspection unit 132 acquires the attribute (class) of the recognition character string, for example, by the prediction process of machine learning. More specifically, the inspection unit 132 gives the recognition character string acquired in step S801 and the learning model to the prediction module of machine learning, executes the prediction module, and acquires the class of the recognition character string. The learning model is the data obtained by giving two or more pieces of teacher data having a character string and a class to the learning module of machine learning and executing the learning module. The process of acquiring the class of the character string may be other processes, and this process is based on known techniques.

[0150] (Step S803) The inspection unit 132 acquires, from the product master 1111, the product character string which is the attribute value corresponding to the attribute acquired in step S802 and is the product record to be targeted.

[0151] The product record to be targeted is, for example, the record including the i-th product name in step S404 or the record corresponding to the i-th analysis result in step S903.

[0152] (Step S804) The inspection unit 132 acquires, from the storage unit 11, the character string determination condition corresponding to the attribute acquired in step S802.

[0153] (Step S805) The inspection unit 132 determines whether the recognition character string acquired in step S801 and the product character string acquired in step S803 satisfy the character string determination condition acquired in step S804. If the character string determination condition is satisfied, it proceeds to step S806; if not, it proceeds to step S807.

[0154] (Step S806) The inspection unit 132 adds "Target character string: OK" to the target inspection result, and returns to the upper-level process. "Target character string: OK" indicates that the inspection result of the recognition character string corresponding to the attribute obtained in Step S802 is normal.

[0155] (Step S807) The inspection unit 132 adds "Target character string: NG" to the target inspection result, and returns to the upper-level process. "Target character string: NG" indicates that the inspection result of the recognition character string corresponding to the attribute obtained in Step S802 is abnormal.

[0156] Note that in the flowchart of FIG. 8, the character string determination condition may be "match" regardless of the attribute. Also, in the flowchart of FIG. 8, the inspection unit 132 may determine whether the recognition character string to be inspected exists in the target product record without determining the attribute of the recognition character string. If it exists, "Target character string: OK" is added to the target inspection result, and if it does not exist, "Target character string: NG" is added to the target inspection result, etc.

[0157] Next, a second example of the inspection process in Step S304 will be described with reference to the flowchart of FIG. 9.

[0158] (Step S901) The image analysis unit 131 analyzes the flyer image and obtains two or more analysis results. An example of such an analysis process will be described with reference to the flowchart of FIG. 10.

[0159] (Step S902) The image analysis unit 131 assigns 1 to the counter i.

[0160] (Step S903) The image analysis unit 131 determines whether the i-th analysis result exists in the analysis results obtained in Step S901. If the i-th analysis result exists, it proceeds to Step S904, and if it does not exist, it returns to the upper-level process.

[0161] (Step S904) The product name determination means 1312 acquires one or more product names from among the analysis results obtained in Step S901. An example of such product name acquisition processing was described using the flowcharts of FIGS. 5 and 6.

[0162] (Step S905) The inspection unit 132 determines whether a product name was acquired in Step S904. If a product name was acquired, it proceeds to Step S906; if not, it proceeds to Step S907.

[0163] (Step S906) The inspection unit 132 adds "Product name: OK" to the i-th inspection result. It proceeds to Step S908. Note that "Product name: OK" indicates that the product name in the i-th analysis result is normal.

[0164] (Step S907) The inspection unit 132 adds "Product name: NG" to the i-th inspection result. Note that "Product name: NG" indicates that the product name in the i-th analysis result is abnormal.

[0165] (Step S908) The image inspection means 1321 performs image inspection processing on the flyer product image included in the i-th analysis result. An example of the image inspection processing was described using the flowchart of FIG. 7.

[0166] (Step S909) The character string inspection means 1322 acquires one or more recognized character strings other than the product name from the i-th analysis result.

[0167] (Step S910) The character string inspection means 1322 assigns 1 to the counter j.

[0168] (Step S911) The character string inspection means 1322 determines whether the j-th recognized character string exists among the recognized character strings acquired in Step S909. If the j-th recognized character string exists, it proceeds to Step S912; if not, it proceeds to Step S914.

[0169] (Step S912) The string inspection means 1322 performs a string inspection on the j-th recognized string. An example of such a string inspection was described using the flowchart of FIG. 8.

[0170] (Step S913) The string inspection means 1322 increments the counter j by 1. Return to step S911.

[0171] (Step S914) The image analysis unit 131 increments the counter i by 1. Return to step S903.

[0172] Next, an example of the analysis process in step S901 will be described using the flowchart of FIG. 10.

[0173] (Step S1001) The character recognition means 1311 performs character recognition processing on the flyer image and obtains two or more recognized strings and the like. The recognized strings and the like are, for example, recognized strings and string position information. Further, the image analysis unit 131 groups two or more recognized strings and the like obtained by the character recognition means 1311 into two or more recognized strings existing at adjacent positions using the string position information. The grouped one or two or more recognized strings are called a string set. Note that the process of grouping two or more recognized strings corresponding to one product is not limited.

[0174] (Step S1002) The image analysis unit 131 detects two or more flyer product images from the flyer image. The detection of the flyer product image is, for example, to obtain the flyer product image and the image position information for specifying the position of the image. The detection of the flyer product image is, for example, to obtain the image position information for specifying the position of the flyer product image.

[0175] (Step S1003) The image analysis unit 131 substitutes 1 for the counter i.

[0176] (Step S1004) The image analysis unit 131 determines whether the i-th flyer product image exists. If the i-th flyer product image exists, it proceeds to step S1005; if not, it returns to the upper-level process.

[0177] (Step S1005) The image analysis unit 131 acquires the image position information corresponding to the i-th flyer product image.

[0178] (Step S1006) The image analysis unit 131 assigns 1 to the counter j.

[0179] (Step S1007) The image analysis unit 131 determines whether the j-th string set exists. If the j-th string set exists, it proceeds to step S1008; if not, it proceeds to step S1011.

[0180] (Step S1008) The image analysis unit 131 acquires the string position information for specifying the position of the j-th string set.

[0181] (Step S1009) The correspondence means 1314 determines whether the i-th flyer product image and the j-th string set are associated using the string position information acquired in step S1008 and the image position information acquired in step S1005. If they are associated, it proceeds to step S1010; if not, it proceeds to step S1012.

[0182] (Step S1010) The correspondence means 1314 associates the i-th flyer product image and the j-th string set. As a result, one analysis result is constituted. The one analysis result has the i-th flyer product image and the j-th string set.

[0183] (Step S1011) The image analysis unit 131 increments the counter i by 1. It returns to step S1004.

[0184] (Step S1012) The image analysis unit 131 increments the counter j by 1. It returns to step S1007.

[0185] In the flowchart of FIG. 10, the process of obtaining two or more analysis results from the flyer image may be other processes.

[0186] Next, an example of the result configuration process in step S305 will be described using the flowchart of FIG. 11.

[0187] (Step S1101) The inspection result acquisition means 1323 acquires a flyer image.

[0188] (Step S1102) The inspection result acquisition means 1323 assigns 1 to the counter i.

[0189] (Step S1103) The inspection result acquisition means 1323 determines whether there is an inspection result corresponding to the i-th analysis result (which may also be referred to as the i-th inspection result). If there is an inspection result corresponding to the i-th analysis result, it proceeds to step S1104; if not, it proceeds to step S1110.

[0190] (Step S1104) The inspection result acquisition means 1323 acquires the inspection result corresponding to the i-th analysis result.

[0191] (Step S1105) The inspection result acquisition means 1323 determines whether "NG" exists in the inspection result corresponding to the i-th analysis result. If "NG" exists, it proceeds to step S1106; if "NG" does not exist, it proceeds to step S1109.

[0192] (Step S1106) The inspection result acquisition means 1323 acquires the information corresponding to "NG" from the product master 1111.

[0193] (Step S1107) The inspection result acquisition means 1323 acquires the position information of the information corresponding to "NG" in the flyer image. The position information is either string position information or image position information.

[0194] (Step S1108) The inspection result acquisition means 1323 adds a mark indicating that it was "NG" at the position indicated by the position information acquired in Step S1107.

[0195] (Step S1109) The inspection result acquisition means 1323 sets the counter i to 1 and increments it. Return to Step S1102.

[0196] (Step S1110) The inspection result acquisition means 1323 constructs the information to be output. The information to be output is, for example, the flyer image with the mark added and the information corresponding to "NG" acquired in Step S1106.

[0197] Next, in the character string inspection of FIG. 8, an example of the inspection operation when the attribute of the recognized character string is "price" will be described using the flowchart of FIG. 12.

[0198] (Step S1201) The character string inspection means 1322 acquires a numerical value from the recognized character string.

[0199] (Step S1202) The character string inspection means 1322 acquires one or two product character strings corresponding to the price from the product master 1111. Here, when two product character strings are acquired, the two product character strings are the first amount including consumption tax of the product and the second amount not including consumption tax. When one product character string is acquired, it is usually unknown whether the one product character string includes consumption tax or not.

[0200] (Step S1203) The character string inspection means 1322 determines whether a product character string that matches the numerical value acquired in Step S1201 is included in the product character strings acquired in Step S1202. If it is included, go to Step S1204; if not, go to Step S1205.

[0201] (Step S1204) The character string inspection means 1322 adds "price: OK" to the target inspection result. Return to the upper-level process.

[0202] (Step S1205) The character string inspection means 1322 determines whether the numerical value obtained in Step S1201 is a tax-included numerical value. If it is a tax-included numerical value, it proceeds to Step S1206; if it is not a tax-included numerical value, it proceeds to Step S1209.

[0203] Note that the character string inspection means 1322 acquires a recognition character string that pairs with the character string position information of the position adjacent to the position indicated by the character string position information of the recognized character string that is a numerical value, and determines whether the recognized character string contains the character string "tax-included". If the character string inspection means 1322 includes "tax-included", it determines that the recognized character string is a tax-included numerical value.

[0204] (Step S1206) The character string inspection means 1322 calculates "the product character string obtained in Step S1202 × (1 + consumption tax rate)". Note that the consumption tax rate is a constant, for example, "0.1".

[0205] (Step S1207) The character string inspection means 1322 determines whether the numerical value calculated in Step S1206 or Step S1210 matches the numerical value obtained in Step S1201. If they match, it proceeds to Step S1204; if they do not match, it proceeds to Step S1208.

[0206] (Step S1208) The character string inspection means 1322 adds "Price: NG" to the target inspection result. Returns to the upper-level process.

[0207] (Step S1209) The character string inspection means 1322 determines whether the numerical value obtained in Step S1201 is a tax-excluded numerical value. If it is a tax-excluded numerical value, it proceeds to Step S1210; if it is not a tax-excluded numerical value, it proceeds to Step S1211.

[0208] The character string inspection means 1322 acquires a recognition character string that pairs with the character string position information at a position adjacent to the position indicated by the character string position information of the recognized character string which is a numerical value, and determines whether or not the recognized character string includes the character string "excluding tax". When the character string inspection means 1322 includes "excluding tax", it determines that the recognized character string is a numerical value excluding tax.

[0209] Also, when going to step S1211, it is a case where it cannot be determined whether the numerical value acquired in step S1201 is a numerical value including tax or a numerical value excluding tax.

[0210] (Step S1210) The character string inspection means 1322 calculates "the product character string acquired in step S1202 / (1 + consumption tax rate)". It goes to step S1207.

[0211] (Step S1211) The character string inspection means 1322 calculates "the product character string acquired in step S1202 × (1 + consumption tax rate)".

[0212] (Step S1212) The character string inspection means 1322 determines whether or not the numerical value calculated in step S1211 matches the numerical value acquired in step S1201. If they match, it goes to step S1204, and if they do not match, it goes to step S1213.

[0213] (Step S1213) The character string inspection means 1322 calculates "the product character string acquired in step S1202 / (1 + consumption tax rate)".

[0214] (Step S1214) The character string inspection means 1322 determines whether or not the numerical value calculated in step S1213 matches the numerical value acquired in step S1201. If they match, it goes to step S1204, and if they do not match, it goes to step S1208.

[0215] Hereinafter, a specific operation example of the information system A in the present embodiment will be described.

[0216] Assume that User A, identified by the user identifier "User A", inputs the product master 1111 containing information on the product group to be used in the flyer into the terminal device 2. Next, the terminal reception unit 22 of the terminal device 2 receives the product master 1111. Next, the terminal processing unit 23 reads out the user identifier "User A" in the terminal storage unit 21 and configures the product master 1111 paired with the user identifier "User A". Next, the terminal transmission unit 24 transmits the product master 1111 paired with the user identifier "User A" to the inspection device 1.

[0217] The image reception unit 121 of the inspection device 1 receives the product master 1111 paired with the user identifier "User A". Next, the processing unit 13 generates a unique master identifier "TA1". Next, the processing unit 13 accumulates the received product master 1111 in the product management unit 111 in pairs with the user identifier "User A" and the master identifier "TA1". Such a product master 1111 is 1301 in FIG. 13. Assume that a large number of other product masters 1111 are also accumulated in the product management unit 111 by the same processing as above. As shown in 1302 and 1303, two or more product masters 1111 may be accumulated in the product management unit 111 in association with one user identifier.

[0218] Next, assume that User A inputs a flyer image for which inspection is desired into the terminal device 2. Next, the terminal reception unit 22 of the terminal device 2 receives the flyer image. Next, the terminal processing unit 23 reads out the user identifier "User A" in the terminal storage unit 21 and configures a start instruction including the flyer image paired with the user identifier "User A". Next, the terminal transmission unit 24 transmits the start instruction to the inspection device 1. The image of the flyer image is shown in FIG. 14.

[0219] Next, the image reception unit 121 of the inspection device 1 receives the start instruction from the terminal device 2. Next, the processing unit 13 acquires the flyer image included in the received start instruction.

[0220] Next, the processing unit 13 acquires the user identifier "User A" from the start instruction. Next, the processing unit 13 acquires the product master 1111 (1301 in FIG. 13) paired with the user identifier "User A" from the product management unit 111.

[0221] Next, the processing unit 13 inspects the flyer image (FIG. 14) using the product master 1111 (1301 in FIG. 13) as follows.

[0222] First, the character recognition means 1311 of the image analysis unit 131 performs character recognition processing on the flyer image (FIG. 14) and acquires one or more recognized character strings. Here, the image analysis unit 131 acquires recognized character strings such as "HAPPY", "Father's", "DAY",..., "Berry's Cheesecake", "Kiwi Compote",..., "￥720 (tax included)",..., "￥1,500 (tax included)", etc.

[0223] Next, the product name determination means 1312 determines, from the acquired recognized character strings, the recognized character string with the highest similarity to each of the plurality of attribute values of the "product name" attribute of the product master 1111 (1301 in FIG. 13). Then, the product name determination means 1312 acquires each of the two or more determined recognized character strings as the product name.

[0224] Next, the product image acquisition means 1313 acquires, from the flyer image (FIG. 14), image position information which is position information for specifying regions of a plurality of flyer product images. Note that such processing is possible by "OS2D".

[0225] Next, the product image acquisition means 1313 uses each of the plurality of image position information to acquire a plurality of flier product images from the flier image (FIG. 14). Next, the image inspection means 1321 determines, for each of the plurality of flier product images, the product image with the highest similarity to the flier product image that is equal to or greater than the threshold value from the product master 1111. Note that when the product image acquisition means 1313 acquires the similarity between two images, each of the two images is vectorized, and the similarity between the two vectors is acquired. The process of vectorizing an image is possible by "Contrastive Language-Image Pretraining (CLIP)". Also, here, when the image inspection means 1321 cannot determine a product image with a similarity equal to or greater than the threshold value from the product master 1111, the image inspection means 1321 obtains the inspection result "NG" for the flier product image.

[0226] Next, the correspondence means 1314 calculates, for each recognition string other than the product name, the distance from the string of the product name and the distance from the flier product image associated with the product name. Next, the correspondence means 1314 associates, for each recognition string other than the product name, the recognition string with the set of the product name and the flier product image for which the sum of the two acquired distances is the smallest. Through the above processing, two or more analysis results, which are a set of the string set that is a set of recognition strings and the flier product images, can be obtained. Note that the process of analyzing the flier image and obtaining two or more analysis results may be other processes.

[0227] Next, the string inspection means 1322 determines, for each of the two or more analysis results and for each recognition string other than the product name included in the analysis result, whether the recognition string satisfies the string determination condition corresponding to the attribute with respect to the attribute value of the record targeted by the product master 1111 according to the operation of the flowchart in FIG. 8, and obtains the inspection result for each recognition string. Note that when the recognition string is "price", it is preferable for the string inspection means 1322 to perform the processing according to the flowchart in FIG. 12.

[0228] Next, the processing unit 13 constructs the inspection result to be output using the inspection result acquired by the inspection unit 132. Here, the processing unit 13 acquires, for each product name, the recognition character string (detection) detected from the flyer image, the attribute name (item) in the product master 1111, the attribute value (master) in the product master 1111, and the inspection result (judgment) in association with each other. Also, the processing unit 13 acquires the flyer image. Then, the processing unit 13 constructs screen information having the acquired flyer image and the information for each acquired product name.

[0229] Next, the output unit 14 transmits the configured screen information to the terminal device 2.

[0230] Next, the terminal receiving unit 25 of the terminal device 2 receives the screen information including the inspection result from the inspection device 1. Next, the terminal processing unit 23 constructs the screen to be output from the received screen information. Next, the terminal output unit 26 outputs the screen. An example of such a screen is shown in FIG. 15.

[0231] As described above, according to the present embodiment, it is possible to prevent the distribution of flyers with incorrect content.

[0232] Also, according to the present embodiment, by performing an inspection considering consumption tax on the price of the product, it is possible to prevent the distribution of flyers with incorrect product prices described therein.

[0233] Also, according to the present embodiment, by using the product master corresponding to each user, it is possible to prevent the distribution of flyers with incorrect content.

[0234] Also, according to the present embodiment, by using an appropriate product master, it is possible to prevent the distribution of flyers with incorrect content.

[0235] Furthermore, according to the present embodiment, the inspection result of the flyer can be presented to the user in an easy-to-understand manner.

[0236] Note that the processing in this embodiment may be implemented by software. And this software may be distributed by software download or the like. Also, this software may be recorded on a recording medium such as a CD-ROM and distributed. Note that this also applies to other embodiments in this specification. The software that realizes the inspection device 1 in this embodiment is a program as follows. That is, this program enables a computer that can access a product master having two or more product records each having a product image and one or more product character strings to receive a flyer image including two or more product image information including a flyer product image and one or more character string images related to the product through an image reception unit, analyze the flyer image, and obtain an analysis result having the flyer product image included in each of the two or more product image information and one or more recognized character strings that are the results of recognizing the character string images through an image analysis unit, refer to the product master, and for each of the two or more analysis results, inspect whether there is a product record corresponding to the product image included in the analysis result and one or more recognized character strings included in the analysis result, and obtain an inspection result, and it is a program for functioning as an output unit that outputs the inspection result.

[0237] Also, FIG. 16 shows the appearance of a computer that executes the program described in this specification and realizes the inspection device 1 and the like of the various embodiments described above. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 16 is an overview diagram of this computer system 300, and FIG. 17 is a block diagram of the system 300.

[0238] In FIG. 16, the computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0239] In FIG. 17, in addition to the CD-ROM drive 3012, the computer 301 includes an MPU 3013, a bus 3014 connected to the CD-ROM drive 3012 and the like, a ROM 3015 for storing programs such as a boot-up program, a RAM 3016 connected to the MPU 3013 for temporarily storing instructions of application programs and providing a temporary storage space, and a hard disk 3017 for storing application programs, system programs, and data. Here, although not shown, the computer 301 may further include a network card for providing connection to a LAN.

[0240] A program for causing the computer system 300 to execute the functions of the inspection apparatus 1 and the like of the above-described embodiment may be stored in the CD-ROM 3101, inserted into the CD-ROM drive 3012, and further transferred to the hard disk 3017. Alternatively, the program may be transmitted to the computer 301 via a network (not shown) and stored in the hard disk 3017. The program is loaded into the RAM 3016 when executed. The program may be loaded directly from the CD-ROM 3101 or the network.

[0241] The program does not necessarily include an operating system (OS) or a third-party program for causing the computer 301 to execute the functions of the inspection apparatus and the like of the above-described embodiment. The program only needs to include only the part of the instructions that calls appropriate functions (modules) in a controlled manner so as to obtain a desired result. How the computer system 300 operates is well-known, and a detailed description is omitted.

[0242] Note that in the above program, in steps such as the step of transmitting information and the step of receiving information, processes performed by hardware, for example, processes performed by a modem or an interface card in the transmission step (processes that can only be performed by hardware) are not included.

[0243] Also, the computer that executes the above program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

[0244] Also, in each of the above embodiments, it goes without saying that two or more communication means existing in one device may be physically realized by one medium.

[0245] Also, in each of the above embodiments, each process may be realized by being centrally processed by a single device, or may be realized by being distributedly processed by a plurality of devices.

[0246] Needless to say, the present invention is not limited to the above embodiments, and various modifications are possible, and those are also included in the scope of the present invention.

Industrial Applicability

[0247] As described above, the inspection device according to the present invention has an effect of preventing the distribution of flyers with incorrect content, and is useful as an inspection device or the like.

Explanation of Signs

[0248] A Information system 1 Inspection device 2 Terminal device 11 Storage unit 12 Reception unit 13 Processing unit 14 Output unit 21 Terminal storage unit 22 Terminal reception unit 23 Terminal processing unit 24 Terminal transmission unit 25 Terminal reception unit 26 Terminal output unit 111 Product management unit 121 Image reception unit 131 Image analysis unit 132 Inspection unit 1111 Product master 1311 Character recognition means 1312 Product name determination means 1313 Product image acquisition means 1314 Corresponding means 1321 Image inspection means 1322 Character string inspection means 1323 Inspection result acquisition means

Claims

1. An image receiving unit that receives a flyer image including two or more pieces of product image information including a flyer product image and one or more string images related to the product; An image analysis unit that analyzes the flyer image and obtains an analysis result having the flyer product image included in each of the two or more pieces of product image information and one or more recognized strings that are the result of recognizing the string image; Referring to a product master having two or more product records having a product image and one or more product strings, and for each of the two or more analysis results, checking whether there is a product record corresponding to the product image included in the analysis result and one or more recognized strings included in the analysis result, and obtaining an inspection result; an inspection unit; An inspection apparatus comprising an output unit that outputs the inspection result.

2. The image analysis unit includes: Character recognition means for performing character recognition processing on the flyer image to obtain two or more recognized strings; Product image acquisition means for acquiring two or more flyer product images from the flyer image; Corresponding means for associating the recognized string with the flyer image using the string position information of the recognized string in the flyer image and the image position information of the flyer product image in the flyer image for each of the two or more recognized strings; The inspection unit includes: For each of the two or more flyer product images included in the flyer image, an image inspection means for obtaining the similarity between the flyer product image and the product image in the product master, and determining whether the similarity satisfies an image determination condition; performing an image inspection process; String inspection means for performing a string inspection process for determining whether one or more recognized strings corresponding to each of the two or more flyer product images exist in a product record in the product master and in the product record including the product image; The inspection apparatus according to claim 1, further comprising inspection result acquisition means for obtaining the inspection result using the result of the image inspection process and the result of the string inspection process.

3. The product information includes a string image indicating the price of the product, The product record includes a product string indicating the price of the product, The inspection unit includes: The inspection apparatus according to claim 1, which performs inspections corresponding to both the case where the recognized string indicating the price of the product is a string including consumption tax and the case where it is a string not including consumption tax.

4. The product record includes a product string indicating the first price including consumption tax of the product and a product string indicating the second price not including consumption tax, The inspection unit includes: The inspection apparatus according to claim 3, wherein an inspection result indicating an abnormality in the price of the product is obtained only when the recognition character string indicating the price of the product is different from both the product character strings indicating the first price and the second price.

5. Among two or more attributes corresponding to the product record included in the product master, string judgment conditions are stored in association with each of one or more of the attributes. The inspection unit determines whether the product character string and the recognition character string corresponding to each of the one or more attributes satisfy the string judgment condition, and obtains an inspection result. The inspection apparatus according to claim 1.

6. The flyer image received by the image reception unit is associated with a user identifier. The inspection unit selects, from two or more product masters associated with the user identifier, a product master associated with the same user identifier as the user identifier associated with the flyer image, and performs the inspection using the selected product master to obtain the inspection result. The inspection apparatus according to any one of claims 1 to 5.

7. The flyer image received by the image reception unit is associated with a master identifier. The inspection unit selects, from two or more product masters associated with the master identifier, a product master associated with the same master identifier as the master identifier associated with the flyer image, and performs the inspection using the selected product master to obtain the inspection result. The inspection apparatus according to any one of claims 1 to 5.

8. The inspection unit obtains an inspection result including the product image or the product character string of the product record of the target for which an abnormality has been detected and the product image information included in the flyer image. The inspection apparatus according to any one of claims 1 to 7.

9. An inspection method implemented by a product master having two or more product records having a product image and one or more product character strings, an image reception unit, an image analysis unit, an inspection unit, and an output unit, an image reception step in which the image reception unit receives a flyer image including two or more pieces of product image information including a flyer product image and one or more character strings related to the product; an image analysis step in which the image analysis unit analyzes the flyer image and obtains an analysis result having the flyer product image included in each of the two or more pieces of product image information and one or more recognition character strings that are the results of recognizing the character string images. An inspection step in which the inspection unit refers to a product master having two or more product records each having a product image and one or more product character strings, and for each of the two or more analysis results, checks whether there is a product record corresponding to the product image included in the analysis result and the one or more recognized character strings included in the analysis result, and obtains an inspection result; An inspection method comprising: an output step in which the output unit outputs the inspection result. **Claim 10** A computer capable of accessing a product master having two or more product records each having a product image and one or more product character strings, An image reception unit that receives a flyer image including two or more pieces of product image information including a flyer product image and one or more character string images related to the product; An image analysis unit that analyzes the flyer image and obtains an analysis result having a flyer product image included in each of the two or more pieces of product image information and one or more recognized character strings that are the results of recognizing the character string images; An inspection unit that refers to the product master, checks whether there is a product record corresponding to the product image included in the analysis result and the one or more recognized character strings included in the analysis result for each of the two or more analysis results, and obtains an inspection result; A program for functioning as an output unit that outputs the inspection result.

Images