Authenticity Determination Support Information Creation Assistance Device

Abstract

This authenticity determination support information generation assist device comprises: an object image acquisition means for acquiring, as an object image, an image photographed of a surface of a medicinal product package where a medicinal product name is stated; a frontalized image generation means for extracting the surface region from the acquired object image and generating a frontalized image through keystone correction of same; a medicinal product name recognition means for recognizing a medicinal product name from the generated frontalized image by means of character recognition; a legitimate medicinal product name search means for using the recognized medicinal product name to search for a similar medicinal product name as a legitimate medicinal product name from an authorized medicinal product name list; and a legitimate product image search means for, by means of image search using the found legitimate medicinal product name as a keyword, searching for a legitimate product image from image information in which legitimate medicinal product names are associated with legitimate product images, the legitimate product images being the result of photographing surfaces of legitimate medicinal product packages where the medicinal product names thereof are stated.

Description

Technical Field

[0001] The present invention relates to a counterfeit determination support information creation assistance device, a counterfeit determination support information creation assistance method, and a recording medium.

Background Art

[0002] In countries such as Africa, it is said that about 40% of the pharmaceuticals distributed in the market are counterfeit drugs, which has become a serious social problem. Many of the counterfeit drugs are of poor quality, and in some cases, users can even determine whether a drug is counterfeit or not just by comparing the image of the pharmaceutical package with that of a genuine product. Therefore, in some countries such as Kenya, experts create and distribute leaflets that illustrate and explain with text the information to support users in determining whether a drug is genuine or counterfeit.

[0003] On the other hand, Patent Document 1 describes displaying information to support users in determining whether a product is genuine or counterfeit. Specifically, based on a barcode or the like attached to a product or the like, counterfeit checkpoints regarding the product or the like are acquired from a database and displayed on the screen of a mobile terminal. As a result, users can grasp important points when determining whether a product is fake or genuine, that is, when determining authenticity.

[0004] In addition, the following are some of the documents that describe technologies related to the present invention.

[0005] Patent Document 2 describes that in a prescription order processing system for preventing incorrect prescription of drugs based on a prescription order, the entered drug name is searched in a database of similar drug names to check for the presence of similar drug names.

[0006] Patent Document 3 describes performing character recognition on an image of a label of a general pharmaceutical product, extracting text such as a product name and ingredients, and searching a pharmaceutical database.

[0007] Patent Document 4 describes an inspection support system that assists in the inspection of dispensing based on prescription data, and when displaying the results of image inspection processing, it detects and displays the difference between the captured image of the tablet and the original image of the tablet. [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2003-150488 [Patent Document 2] Japanese Patent Publication No. 2006-163541 [Patent Document 3] Japanese Patent Publication No. 2016-151876 [Patent Document 4] Japanese Patent Publication No. 2020-075172 [Overview of the project] [Problems that the invention aims to solve]

[0009] Regarding pharmaceuticals, it is desirable to store information in a database to assist users in determining authenticity, and to provide this information to users who need it, so that users can determine whether a drug is genuine or counterfeit themselves. However, in order to achieve this, experts with specialized knowledge of pharmaceutical authenticity must create the information to assist users in determining authenticity in advance. If such information is created entirely by hand, it will be extremely time-consuming and costly.

[0010] The object of the present invention is to provide an assistance device for creating information to support the determination of the authenticity of pharmaceuticals, which solves the above-mentioned problem, namely, the problem that it takes a great deal of time and cost to create information that supports users in determining the authenticity of pharmaceuticals. [Means for solving the problem]

[0011] An assisting device for creating authenticity determination support information according to one embodiment of the present invention is: A target image acquisition means that acquires an image of the side of a pharmaceutical package on which the name of the pharmaceutical product is written as the target image, A frontal image generation means extracts the region of the surface from the acquired target image, performs trapezoidal correction, and generates a frontal image; A drug name recognition means that performs character recognition from the generated frontal image to recognize the drug name, A legitimate drug name search means that searches for a similar drug name as a legitimate drug name from a list of approved drug names using the recognized drug name, A genuine product image search means searches for genuine product images from image information that associates the genuine product name with the genuine product image of the side of the genuine product packaging on which the genuine product name is written, using the searched genuine product name as a keyword in the image search. It is configured to include the following:

[0012] Furthermore, a method for creating information to assist in authenticity determination according to another embodiment of the present invention is: The image obtained is a photograph of the side of the pharmaceutical packaging that shows the name of the drug, and the target image is obtained from that side. The region of the surface is extracted from the acquired target image, trapezoidal correction is applied, and a frontal image is generated. Character recognition is performed on the generated frontal image to recognize the name of the drug. Using the recognized drug name, a similar drug name is searched for as the official drug name from the list of approved drug names. By performing an image search using the searched legitimate drug name as a keyword, the image of the legitimate product is searched from image information that associates the legitimate drug name with an image of the legitimate product, which is a photograph of the side of the legitimate drug packaging on which the drug name is written. It is structured in this way.

[0013] Furthermore, a computer-readable recording medium according to another embodiment of the present invention is: On the computer, The process involves acquiring an image of the side of the pharmaceutical packaging that contains the name of the drug as the target image, The process involves extracting the region of the surface from the acquired target image, performing trapezoidal correction, and generating a frontal image. A process of performing character recognition on the generated front-facing image to recognize the pharmaceutical product name, a process of searching for a similar pharmaceutical product name from a list of authorized pharmaceutical product names using the recognized pharmaceutical product name as a regular pharmaceutical product name, a process of searching for the regular product image from image information in which the regular product image of the regular pharmaceutical product name and the surface on which the pharmaceutical product name of the regular pharmaceutical packaging body is described are associated by image search using the searched regular pharmaceutical product name as a keyword, is configured to record a program for causing the above to be performed.

[0014] Further, the authenticity determination support system according to another aspect of the present invention is an authenticity determination support system that supports a user in determining the authenticity of an object to be determined, a database that stores authenticity determination support information created using the above authenticity determination support information creation auxiliary device and including an image of an authentic pharmaceutical packaging body taken and an image of a counterfeit of the pharmaceutical packaging body, acquisition means for acquiring an image of the object to be determined taken from a user terminal, collation means for collating the image in the authenticity determination support information stored in the database with the image of the object to be determined taken, presentation means for displaying the detected authenticity determination support information on the screen of the user terminal when, as a result of the collation, authenticity determination support information including an image that matches the image of the object to be determined taken is detected, is configured to include the above.

Advantages of the Invention

[0015] By having the configuration as described above, the present invention can reduce the time and cost when an expert creates information for supporting a user in determining the authenticity of a pharmaceutical product.

Brief Description of the Drawings

[0016] [Figure 1] It is a block diagram of an authenticity determination support information creation auxiliary device according to a first embodiment of the present invention. [Figure 2] This figure shows an example of information stored in the authenticity determination support information database in the first embodiment of the present invention. [Figure 3] This flowchart shows an example of the operation of the authenticity determination support information creation auxiliary device according to the first embodiment of the present invention. [Figure 4] This is a schematic diagram showing an example of authenticity determination support information created using the authenticity determination support information creation auxiliary device according to the first embodiment of the present invention. [Figure 5] This is a schematic diagram showing another example of authenticity determination support information created using the authenticity determination support information creation auxiliary device according to the first embodiment of the present invention. [Figure 6] This is a block diagram showing an example of the configuration of the frontal image generation unit in the first embodiment of the present invention. [Figure 7] This is a block diagram showing an example of the configuration of the drug name recognition unit in the first embodiment of the present invention. [Figure 8] This is a block diagram showing an example of the configuration of the difference detection unit in the first embodiment of the present invention. [Figure 9] This figure shows an example of a CNN used in the difference detection unit in the first embodiment of the present invention. [Figure 10] This is a block diagram of a counterfeit detection support system according to a second embodiment of the present invention. [Figure 11] This is a block diagram showing an example of a server terminal in a second embodiment of the present invention. [Figure 12] This flowchart shows an example of the operation of a server terminal when a support request is sent from a user terminal in a second embodiment of the present invention. [Figure 13] This flowchart shows an example of the operation of the server terminal 10 when authenticity determination support information is transmitted from the expert terminal in the second embodiment of the present invention. [Figure 14] This is a block diagram of an auxiliary device for creating authenticity determination support information according to a third embodiment of the present invention. [Modes for carrying out the invention]

[0017] [First Embodiment] Figure 1 is a block diagram of the Authenticity Determination Support Information Creation Auxiliary Device 1 according to the first embodiment of the present invention. This Authenticity Determination Support Information Creation Auxiliary Device 1 (hereinafter sometimes simply referred to as the auxiliary device) 1 is a device used by experts who create authenticity determination support information, and is used to determine the authenticity of pharmaceutical packaging that is the subject of independent transactions. For example, primary packaging that comes into direct contact with the pharmaceutical, such as containers and PTP sheets, and outer containers or outer wrappers that further package the primary packaging for retail are examples of pharmaceutical packaging. Hereinafter, pharmaceutical packaging may be simply referred to as packaging.

[0018] Referring to Figure 1, the auxiliary device 1 consists of a camera 2, a communication I / F unit 3, an operation input unit 4, a screen display unit 5, a storage unit 6, and an arithmetic processing unit 7.

[0019] Camera 2 is a color or monochrome camera equipped with, for example, a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor with a pixel capacity of several million pixels. The communication I / F unit 3 consists of a data communication circuit and performs data communication with various external devices wirelessly or via wired connection. The operation input unit 4 consists of devices such as a keyboard or mouse and detects the operations of an expert who creates authenticity determination support information using the auxiliary device 1 and outputs it to the calculation processing unit 7. The screen display unit 5 consists of a device such as an LCD (Liquid Crystal Display) and displays various information on the screen in accordance with instructions from the calculation processing unit 7.

[0020] The storage unit 6 consists of one or more storage devices such as a hard disk or memory, and stores processing information and programs 61 necessary for various processes in the arithmetic processing unit 7. The programs 61 are programs that realize various processing processes when read and executed by the arithmetic processing unit 7, and are pre-read from external devices or recording media (not shown) via data input / output functions such as the communication I / F unit 3 and stored in the storage unit 6. The main processing information stored in the storage unit 6 includes an authorized drug name list 62, a genuine drug image list 63, and a genuine / counterfeit determination support information database 64.

[0021] The List of Approved Drugs 62 is a list of drug names approved as pharmaceuticals by national government agencies. The List of Approved Drugs 62 enumerates the official drug names of all approved pharmaceuticals.

[0022] The List of Authentic Drug Images 63 is a list of images of genuine (authentic) packaging. The List of Authentic Drug Images 63 enumerates images of the front of all genuine (authentic) packaging, taken directly from the front. Generally, the drug name is printed in large letters on the front of the packaging. The List of Authentic Drug Images 63 is a list of images of the front of packaging with the drug name printed in such large letters. Furthermore, the List of Authentic Drug Images 63 associates images of genuine packaging with the drug name printed on that packaging. Therefore, by searching the List of Authentic Drug Images 63 using the drug name as a keyword, it is possible to obtain images of the front of genuine packaging with that drug name printed on it, taken directly from the front.

[0023] The authenticity determination support information database 64 is a database that stores information (authenticity determination support information) to assist users in determining the authenticity of packaging. Figure 2 shows an example of the information stored in the authenticity determination support information database. In this example, the authenticity determination support information database 64 consists of multiple authenticity determination support information entries 641. Each authenticity determination support information entry 641 consists of an ID 6411, a drug name 6412, an image of a genuine package 6413, an image of a counterfeit 6414, and explanatory information 6415.

[0024] The ID6411 field contains an ID, such as a number that uniquely identifies the authenticity verification support information. The drug name6412 field contains the name of the legitimate drug. The image6413 field contains an image of the front of the legitimate packaging taken from directly in front. The image6414 field contains an image of the front of the counterfeit packaging taken from directly in front. The explanatory information6415 field contains a diagram and / or text that explains the key points for distinguishing between the image6413 of the genuine packaging and the image6414 of the counterfeit packaging.

[0025] The arithmetic processing unit 7 has one or more processors such as a CPU (Central Processing Unit) and their peripheral circuits, and by reading and executing the program 61 from the storage unit 6, it realizes various processing units by cooperating with the hardware and the program 61. The main processing units realized by the arithmetic processing unit 7 are the target image acquisition unit 71, the frontal image generation unit 72, the drug name recognition unit 73, the legitimate drug name search unit 74, the genuine product image search unit 75, the difference detection unit 76, and the editing unit 77.

[0026] The target image acquisition unit 71 is configured to acquire an image as the target image from the camera 2 or from an external device via the communication I / F unit 3, which is an image of the side of the packaging on which the name of the drug is written.

[0027] The frontal image generation unit 72 is configured to extract the region of the surface on which the drug name is written from the target image acquired by the target image acquisition unit 71, and to generate a frontal image by trapezoidal correction of this extracted region.

[0028] The drug name recognition unit 73 is configured to recognize the drug name by performing character recognition on the frontal image generated by the frontal image generation unit 72.

[0029] The authorized drug name search unit 74 is configured to use the drug name recognized by the drug name recognition unit 73 to search for similar drug names from the approved drug name list 62 as authorized drug names.

[0030] The genuine product image search unit 75 is configured to obtain an image of the side of a genuine package that has the drug name written on it (genuine product image) by searching the genuine drug image list 63 using the genuine drug name searched by the genuine drug name search unit 74 as a keyword. In this example, the genuine drug image list 63 was used as image information that associates the genuine drug name with a genuine product image of the side of a genuine drug package that has the drug name written on it, obtained by image search using the searched genuine drug name as a keyword. However, the image information that associates the genuine drug name with a genuine product image of the side of a genuine drug package that has the drug name written on it is not limited to the genuine drug image list 63. For example, such image information may be from a drug sales e-commerce site used in the second embodiment described later.

[0031] The difference detection unit 76 is configured to detect areas with large differences between the frontal image generated by the frontal image generation unit 72 and the genuine product image acquired by the genuine product image search unit 75. For example, the difference detection unit 76 outputs a frontal image as a detection result, in which areas with large differences from the genuine product image are marked with rectangles or the like.

[0032] The editing unit 77 is configured to edit authenticity determination support information through interactive processing with experts. For example, the editing unit 77 displays the target image acquired by the target image acquisition unit 71, the frontal image generated by the frontal image generation unit 72, the recognition results of the drug name recognition unit 73, the search results of the genuine drug name search unit 74, the search results of the genuine product image search unit 75, and the detection results of the difference detection unit 76 as editing material on the screen display unit 5 for the expert to review, and allows the expert to select or modify the necessary editing material through input from the operation input unit 4. Furthermore, the editing unit 77 edits the authenticity determination support information using the editing material selected or modified by the expert and the text input from the operation input unit 4, and saves it in the authenticity determination support information database 64.

[0033] Figure 3 is a flowchart illustrating an example of the operation of auxiliary device 1. The operation of auxiliary device 1 will be explained below with reference to Figure 3.

[0034] First, the target image acquisition unit 71 of the auxiliary device 1 acquires an image of the side of the packaging that has the drug name written on it, either from the camera 2 or from an external device via the communication I / F unit 3, as the target image (step S1). The acquired target image is displayed on the screen display unit 5 via the editing unit 77. This allows the expert to confirm the content of the target image. If the expert inputs OK from the operation input unit 4 for the displayed target image, or if a certain period of time has elapsed without input, the auxiliary device 1 assumes the target image has been approved and continues processing. On the other hand, if the expert inputs NG from the operation input unit 4 within a certain period of time for reasons such as the packaging not being visible in the target image, the auxiliary device 1 interrupts the processing shown in Figure 3 at this point.

[0035] Next, the frontal image generation unit 72 extracts the area of ​​the surface on which the drug name is written from the target image acquired by the target image acquisition unit 71, and generates a frontal image by trapezoidal correction of this extracted area (step S2). The generated frontal image is displayed on the screen display unit 5 via the editing unit 77. This allows the expert to confirm the contents of the frontal image. If the expert inputs OK from the operation input unit 4 for the displayed frontal image, or if a certain period of time has elapsed without input, the auxiliary device 1 assumes that the frontal image has been approved and continues processing. On the other hand, if the expert inputs NG from the operation input unit 4 within a certain period of time for reasons such as it being clearly not frontal, the auxiliary device 1 interrupts the processing shown in Figure 3 at this point.

[0036] Next, the drug name recognition unit 73 recognizes the drug name by performing character recognition on the frontal image generated by the frontal image generation unit 72 (step S3). The recognized drug name is displayed on the screen display unit 5 via the editing unit 77. This allows the expert to confirm the recognized drug name. If the expert inputs OK for the recognized drug name from the operation input unit 4, or if no input is made and a certain period of time has elapsed, the auxiliary device 1 assumes that the recognized drug name has been acknowledged and continues processing. On the other hand, if the recognized drug name is incorrect and the expert corrects the recognized drug name from the operation input unit 4 within a certain period of time, the auxiliary device 1 adopts the corrected drug name as the recognized drug name and continues processing.

[0037] Next, the authorized drug name search unit 74 searches the list of authorized drug names 62 for similar drug names as authorized drug names using the drug name recognition unit 73 (step S4). The search results for authorized drug names are displayed on the screen display unit 5 via the editing unit 77. This allows experts to confirm the search results for authorized drug names. If the expert inputs OK from the operation input unit 4 for the most similar authorized drug name found, or if no input is made and a certain period of time has elapsed, the auxiliary device 1 assumes that the most similar authorized drug name found has been acknowledged and continues processing. On the other hand, if the most similar authorized drug name is incorrect and the expert corrects the authorized drug name from the operation input unit 4 within a certain period of time, the auxiliary device 1 adopts the corrected authorized drug name as the most similar authorized drug name and continues processing.

[0038] Next, the genuine product image search unit 75 searches the genuine drug image list 63 using the most similar genuine drug name found by the genuine drug name search unit 74 as a keyword, thereby obtaining an image of the side of the genuine packaging on which the drug name is written (genuine product image) (step S5). The search results for genuine product images are displayed on the screen display unit 5 via the editing unit 77. This allows experts to confirm the search results for genuine product images. If the expert inputs OK from the operation input unit 4 for the found most similar genuine product image, or if a certain period of time has elapsed without input, the auxiliary device 1 assumes that the found most similar genuine product image has been acknowledged and continues processing. On the other hand, if the most similar genuine product image is incorrect and the expert selects another genuine product image found within a certain period of time from the operation input unit 4, the auxiliary device 1 adopts the selected genuine product image as the most similar genuine product image and continues processing.

[0039] Next, the editorial department 77 receives an opinion from the operation input unit 4 regarding whether the target image (the image acquired in step S1) displayed on the screen display unit 5 is genuine or counterfeit (step S6). Based on the target image displayed on the screen display unit 5, the frontal view image, the genuine drug name, the normalized image, and their expertise in authenticity determination, the expert determines whether the packaging related to the target image is genuine or counterfeit, and inputs the result of that determination from the operation input unit 4.

[0040] Next, if auxiliary device 1 receives a result indicating that the item is counterfeit (NO in step S7), it proceeds to step S8. If auxiliary device 1 receives a result indicating that the item is genuine (YES in step S7), it skips step S8 and proceeds to step S9.

[0041] In step S8, the difference detection unit 76 detects areas with significant differences between the frontal image generated by the frontal image generation unit 72 and the genuine product image acquired by the genuine product image search unit 75 (step S6). The detection results of the difference detection unit 76 are displayed on the screen display unit 5 via the editing unit 77. This allows experts to confirm the presence and location of areas with significant differences between the frontal image and the genuine product image. Experts can also modify the detection results of the difference detection unit 76 through input operations from the operation input unit 4. For example, if an expert finds a significant difference in an area where the difference detection unit 76's detection results indicated no significant difference, they can add a marking such as a rectangle to the discovered area. Conversely, if an expert finds no significant difference in an area where the difference detection unit 76's detection results indicated a significant difference, they can delete the marking in that area. When, for example, confirmation or completion of modification is input from the expert via the operation input unit 4, the auxiliary device 1 proceeds to the process in step S9.

[0042] In step S9, the editorial department 77 edits authenticity verification support information using the editorial materials selected or modified by experts, as well as the authenticity verification results and text entered from the operation input unit 4, and saves it in the authenticity verification support information database 64 (step S109).

[0043] Next, we will explain a specific example of the authenticity determination support information created using the auxiliary device 1.

[0044] Figure 4 is a schematic diagram showing an example of authenticity determination support information created using the auxiliary device 1. This example is for a case where the target image is an image of counterfeit packaging. In this example, the authenticity determination support information has "007" set in the ID 6411 field and "XYZ" set in the drug name 6412 field. "007" set in the ID 6411 field is, for example, a number automatically assigned by the editorial department 77, or an ID attached to the input target image. "XYZ" set in the drug name 6412 field is the legitimate drug name searched by the legitimate drug name search unit 74 and approved by experts. In addition, the image 6413 field of the genuine packaging is set to a genuine product image searched by the genuine product image search unit 75 and approved by experts. In addition, the image 6414 field of the counterfeit product is set to a frontal view image generated by the frontal view image generation unit 72. Furthermore, item 6415 of the explanatory information contains the detection results detected by the difference detection unit 76 (a frontal view image with areas showing significant differences from the genuine product image marked with rectangles, etc.) and the text entered by an expert: "The counterfeit product has a different display position for HHHH compared to the genuine product." In the image of the genuine product set in item 6413 and the image of the counterfeit product set in item 6414, there are slight differences in the size of the "60" characters and the shape of the graphic between the genuine and counterfeit products. However, these differences are not significant, so they are not marked in the frontal view image of the counterfeit product set in item 6415, nor are they explained in the text entered by the expert. By deliberately omitting differences that are difficult for the average user to understand, the important checkpoints for determining authenticity are highlighted.

[0045] Figure 5 is a schematic diagram showing another example of authenticity determination support information created using auxiliary device 1. This example is one where the target image is an image of genuine packaging. Therefore, in the authenticity determination support information, the ID, genuine drug name, and genuine product image are set in the fields for ID 6411, drug name 6412, and image 6413 of genuine packaging, while the fields for image 6414 of counterfeit product and explanatory information 6415 are NULL.

[0046] As described above, the auxiliary device 1 includes a frontal image generation unit 72 that extracts the area of ​​the side of a pharmaceutical package on which the drug name is written from a target image, performs trapezoidal correction, and generates a frontal image, and a drug name recognition unit 73 that performs character recognition from the generated frontal image to recognize the drug name. Therefore, experts only need to confirm the recognized drug name, and do not need to visually read the drug name from the target image and manually input it from the operation input unit 4. In addition, if the shape deformation of the target image is large, it becomes difficult to apply character recognition or the possibility of misrecognition increases, but since the frontal image generation unit 72 performs trapezoidal correction and generates a frontal image before character recognition, such problems are avoided and character recognition can be performed with high accuracy.

[0047] Furthermore, the auxiliary device 1 includes a legitimate drug name search unit 74 that uses the recognized drug name to search for similar drug names from the approved drug name list 62 as legitimate drug names. Therefore, experts can avoid the trouble of searching for whether a drug is approved themselves.

[0048] Furthermore, the auxiliary device 1 is equipped with a genuine product image search unit 75 that searches for genuine product images by using the genuine drug name as a keyword in an image search, specifically images of the side of the genuine drug packaging that displays the drug name. Therefore, experts can avoid the trouble of performing an image search themselves to obtain genuine images.

[0049] In this way, experts can avoid the trouble of searching for whether a drug is approved or for images of genuine products themselves, allowing them ample time to determine whether the image in question is of genuine packaging or a counterfeit.

[0050] Furthermore, the auxiliary device 1 includes a difference detection unit 76 that detects areas with large differences between the frontal image and the genuine product image. Therefore, experts only need to confirm the detected differences, saving them the time and effort of discovering and drawing the differences themselves. In addition, it is possible to reduce the variation in differences among experts.

[0051] For the reasons described above, using the auxiliary device 1 can reduce the time and cost involved in creating authenticity verification support information.

[0052] Next, we will describe some suitable examples of the main components.

[0053] Figure 6 is a block diagram showing an example configuration of the frontal image generation unit 72. In this example, the frontal image generation unit 72 consists of an edge CNN (Convolutional Neural Network) 721, a plane CNN 722, a watershed 723, an integrated CNN 724, a Hough transform unit 725, and a projection transform unit 726. In this example, the frontal image generation unit 72 inputs the target image to the edge CNN 721, the plane CNN 722, the watershed 723, and the integrated CNN 724.

[0054] Edge CNN 721 detects the edges of the packaging in the input target image and outputs a target image with the edges displayed in red, for example. Surface CNN 722 detects the surface of the packaging in the input target image and outputs a target image with the surface displayed in red, for example. Watershed CNN 723 separates objects such as the packaging from the background in the input target image and outputs a target image with the background displayed in yellow, for example. Integrated CNN 724 outputs a target image with the surface of the packaging in the target image displayed in red, almost entirely, from the target image, the output image of Edge CNN 721, the output image of Surface CNN 722, and the output image of Watershed CNN 723. Hough transform unit 725 takes the output image of Integrated CNN 724 as input, detects the outer contour of the packaging surface, and outputs a target image with the outer contour displayed in red, for example. Projection transform unit 726 takes the output image of Hough transform unit 725 as input, trapezoidally corrects the shape of the packaging surface identified by the outer contour, and generates a frontal image.

[0055] The above describes an example configuration of the frontal image generation unit 72. However, the frontal image generation unit 72 is not limited to the above example configuration. Well-known techniques for trapezoidal correction can be applied to surfaces in the image that are originally rectangular but distorted into trapezoids.

[0056] Figure 7 is a block diagram showing an example configuration of the drug name recognition unit 73. In this example, the drug name recognition unit 73 consists of a string detection unit 731, an OCR (Optical Character Recognition) processing unit 732, a rectangular area calculation unit 733, and a priority setting unit 734.

[0057] The character detection unit 731 receives a frontal view image as input and detects all the bounding rectangles of the character strings in the frontal view image. For example, the character detection unit 731 detects a rectangular area in which similarly sized shapes (characters) are arranged at regular intervals as a single character string. The OCR processing unit 732 performs OCR processing on each rectangular area detected by the character detection unit 731 to recognize the characters and outputs the recognition result. Meanwhile, the rectangle area calculation unit 733 calculates and outputs the area of ​​the rectangular area detected by the character detection unit 731. The prioritization unit 734 outputs a ranking of the recognition results output from the OCR processing unit 732, such that the larger the area of ​​the rectangular area output by the rectangle area calculation unit 733, the higher the ranking.

[0058] The drug name recognition unit 73 in this example focuses on the fact that while various strings of characters may exist on the packaging in addition to the drug name, the drug name is displayed most prominently. In the inventor's verification, the probability that the highest-ranking recognition result was the drug name was almost 100%. This effect is also related to the function of the frontal image generation unit 72. In other words, in an image of the packaging taken from an angle, the size of the strings of characters is distorted, so depending on the orientation of the photograph, the drug name may appear to be the same size as other strings of characters or even smaller. In contrast, frontal images do not suffer from such problems.

[0059] Figure 8 is a block diagram showing an example configuration of the difference detection unit 76. In this example, the difference detection unit 76 comprises a CNN 761, a learning unit 762, and a control unit 763.

[0060] CNN761 is a pre-trained machine learning model that takes a frontal view image generated by the frontal view image generation unit 72, a genuine product image retrieved by the genuine product image retrieval unit 75, and a difference image between these two images (frontal view image and genuine product image) as input to estimate regions with large differences between the frontal view image and the genuine product image. An example of CNN761 is shown in Figure 9. However, CNN761 is not limited to the configuration shown in Figure 9.

[0061] The learning unit 762 is a means of performing machine learning on CNN761 to estimate regions with large differences using various frontalized image and canonical image pairs as training data. Any method can be used to teach regions with large differences between frontalized image and canonical image pairs, such as enclosing or filling in these regions with rectangles.

[0062] The control unit 763 receives the frontalized image generated by the frontalized image generation unit 72 and the genuine product image retrieved by the genuine product image search unit 75, and generates a difference image between these two images (frontalized image and genuine product image). The control unit 763 is also configured to input the input frontalized image, genuine product image, and the generated difference image into a trained CNN 761, thereby obtaining regions with large differences between the frontalized image and the genuine product image from the CNN 761.

[0063] The configuration and operation of auxiliary device 1 have been described above. Next, we will describe a system that uses the authenticity determination support information created using auxiliary device 1 to support the user in determining the authenticity of packaging.

[0064] [Second Embodiment] Figure 10 is a block diagram of a counterfeit determination support system 100 according to a second embodiment of the present invention. This counterfeit determination support system 100 targets packaging for authenticity determination. Referring to Figure 10, the counterfeit determination support system 100 consists of a server terminal 10 and a user terminal 20, which are connected to each other via a network 30 such as the Internet. The user terminal 20 is a terminal used by a user who performs authenticity determination on packaging. The user terminal 20 may be an information processing device such as a smartphone, mobile information terminal, or personal computer having camera functions, communication functions, and display functions. On the other hand, the server terminal 10 is a terminal that provides information to the user terminal 20 to support authenticity determination on packaging in accordance with requests from the user terminal 20. The server terminal 10 may be an information processing device such as a personal computer having communication functions, calculation functions, and storage functions.

[0065] If a user of user terminal 20 wants to check whether a package they are about to purchase is genuine or counterfeit, they use the camera function of user terminal 20 to take an image of the package's appearance. Next, user terminal 20 sends a support request containing the acquired image to server terminal 10 via network 30. Then, user terminal 20 waits for a response from server terminal 10.

[0066] The server terminal 10 has pre-stored authenticity verification support information created by an expert using the aforementioned auxiliary device 1. When the server terminal 10 receives a support request from the user terminal 20 via the network 30, it compares the image of the packaging included in the support request with the images in the pre-stored authenticity verification support information (including both images of genuine packaging and images of counterfeit packaging). Next, if the server terminal 10 finds authenticity verification support information that matches the image of the packaging included in the support request (including at least one of the images of genuine packaging and images of counterfeit packaging), it transmits the detected authenticity verification support information to the user terminal 20.

[0067] When the user terminal 20 receives authenticity verification support information from the server terminal 10, it receives it and displays it on the screen. This allows the user to view not only images of genuine packaging but also images of counterfeit packaging when determining authenticity. Therefore, compared to a situation where the user must determine whether the packaging is genuine or counterfeit based solely on images of genuine packaging, the user can judge the similarity between the images of genuine and counterfeit packaging from a more multifaceted perspective, enabling easier and more accurate authenticity verification. Furthermore, by displaying text and / or diagrams explaining the key points for distinguishing between genuine and counterfeit packaging, the user can perform authenticity verification even more easily and accurately.

[0068] Furthermore, in the authenticity verification support system 100 shown in Figure 10, the expert terminal 40 and the EC (Electronic Commerce) site 50 are connected to the network 30. The expert terminal 40 is a terminal used by an expert who performs authenticity verification of packaging. The expert terminal 40 may be an information processing device such as a smartphone, personal information terminal, or personal computer having camera functions, communication functions, and display functions. Specifically, the expert terminal 40 may be the auxiliary device 1 according to the first embodiment. The EC site 50 is a site that sells packaging. The EC site 50 publishes images of the packaging being sold, associated with product names.

[0069] Next, we will explain in detail the configuration and operation of server terminal 10.

[0070] Figure 11 is a block diagram showing an example of a server terminal 10. Referring to Figure 11, the server terminal 10 consists of a communication interface unit 11, an operation input unit 12, a screen display unit 13, a storage unit 14, and an arithmetic processing unit 15.

[0071] The communication interface unit 11 consists of a data communication circuit and communicates data wirelessly or via wired connection with external devices such as user terminals 20, expert terminals 40, and e-commerce sites 50 via the network 30. The operation input unit 12 consists of devices such as a keyboard and mouse and detects operator operations and outputs them to the calculation processing unit 15. The screen display unit 13 consists of devices such as an LCD and displays various information on the screen in response to instructions from the calculation processing unit 15.

[0072] The storage unit 14 consists of one or more storage devices such as a hard disk or memory, and stores processing information and programs 141 necessary for various processes in the arithmetic processing unit 15. The programs 141 are programs that realize various processing processes when read and executed by the arithmetic processing unit 15, and are pre-read from external devices or recording media (not shown) via data input / output functions such as the communication I / F unit 11 and stored in the storage unit 14. The main processing information stored in the storage unit 14 includes a genuine / counterfeit determination support information database 142, a database of items of unknown authenticity 143, a list of reliable e-commerce sites 144, and a counterfeit drug database 145.

[0073] The authenticity determination support information database 142 is a database constructed using the aforementioned authenticity determination support information database 64, which was previously created by an expert using the auxiliary device 1. The configuration of each authenticity determination support information 641 stored in the authenticity determination support information database 142 is the same as the configuration shown in Figure 2. In the example in Figure 11, the authenticity determination support information database 142 is locally connected to the server terminal 10, but it may also be remotely connected to the server terminal 10.

[0074] The database of items of uncertain authenticity 143 temporarily stores images of packages whose authenticity is uncertain. In the example in Figure 11, the database of items of uncertain authenticity 143 is locally connected to the server terminal 10, but it may also be connected remotely. If an expert confirms that an image stored in the database of items of uncertain authenticity 143 is an image of a genuine package or an image of a counterfeit package, new authenticity determination support information including that image is registered in the authenticity determination support information database 142, and the original image is deleted from the database of items of uncertain authenticity 143.

[0075] The list of reliable e-commerce sites (144) records a list of sites that have been confirmed to be reliable through prior investigation, out of 50 e-commerce sites that sell packaging. The e-commerce sites listed publish images of the packaging they sell, associating them with the product names. Therefore, by performing an internet image search using the product name of a pharmaceutical as a keyword, it is possible to obtain images of genuine packaging provided by these sites. Among the images of genuine packaging provided by these sites, some may match images of genuine packaging registered in the Authenticity Determination Support Information Database (142), while others may not.

[0076] The placebo database 145 stores images of counterfeit packaging in advance. In the example in Figure 11, the placebo database 145 is locally connected to the server terminal 10, but it may also be connected remotely. Among the images of counterfeit packaging stored in the placebo database 145, some may match images of counterfeit packaging registered in the authenticity determination support information database 142, while others may not.

[0077] The arithmetic processing unit 15 has one or more processors such as a CPU and their peripheral circuits, and by reading and executing the program 141 from the storage unit 14, it realizes various processing functions by having the hardware and the program 141 cooperate. The main processing functions realized by the arithmetic processing unit 15 are the acquisition unit 151, the matching unit 152, the presentation unit 153, and the addition unit 154.

[0078] The acquisition unit 151 communicates with the user terminal 20 via the network 30 through the communication I / F unit 11 and is configured to acquire a support request from the user terminal 20 that includes an image of the packaging subject to authenticity determination (hereinafter referred to as the image to be determined).

[0079] The matching unit 152 is configured to compare the image to be judged included in the support request received from the acquisition unit 151 with the images 6413 of genuine packaging and images 6414 of counterfeit packaging among all the authenticity judgment support information 641 stored in the authenticity judgment support information database 142. For image matching, for example, a subject identification engine may be used. The matching unit 152 detects authenticity judgment support information 641 in which at least one of the images 6413 of genuine packaging or the images 6414 of counterfeit packaging matches the image to be judged. The matching unit 152 transmits the matching result, including the ID 6411 of the detected authenticity judgment support information 641, to the presentation unit 153. If no such authenticity judgment support information 641 is detected, the matching unit 152 transmits a matching result to the presentation unit 153 that does not include the ID 6411 of any authenticity judgment support information 641. Furthermore, the matching unit 152 assigns an "unknown authenticity" ID to the target images for which no authenticity determination support information 641 was detected, and stores them in the unknown authenticity database 143.

[0080] The presentation unit 153 is configured to present information to the user of the user terminal 20 that assists in determining the authenticity of the object to be judged. The presentation unit 153 checks whether ID 6411 is included in the matching result transmitted from the matching unit 152. If it is included, it creates support information based on the authenticity determination support information database 142. If it is not included, it creates support information based on the e-commerce site 50 and the counterfeit drug database 145.

[0081] In creating support information based on the authenticity determination support information database 142, the presentation unit 153 reads authenticity determination support information 641 containing the ID 6411 from the authenticity determination support information database 142 using the ID 6411 included in the matching result as a key, and creates support information based on this read authenticity determination support information 641. For example, the presentation unit 153 uses the authenticity determination support information 641 as support information as is.

[0082] In creating support information based on e-commerce sites, the presentation unit 153 first extracts the product name from the image to be judged. Next, the presentation unit 153 performs an internet image search using the extracted product name as a keyword. This image search yields images of genuine packaging sold under the extracted product name. Next, the presentation unit 153 extracts only the images from e-commerce sites listed in the reliable e-commerce site list 144 from the images of packaging obtained through the image search. Next, the presentation unit 153 compares the extracted images with the image to be judged and detects images of genuine packaging that match the image to be judged. Next, the presentation unit 153 creates support information that includes the detected images as examples of images of genuine packaging. If the presentation unit 153 cannot detect any images of genuine packaging that match the image to be judged, it creates support information indicating that fact.

[0083] In creating support information based on the placebo database 145, the presentation unit 153 first compares images of counterfeit packaging stored in the placebo database 145 with the target image to detect images of counterfeit packaging that match the target image. Next, the presentation unit 153 creates support information that includes the detected images of counterfeit packaging as examples. If no images of counterfeit packaging matching the target image are detected, the presentation unit 153 creates support information indicating that fact.

[0084] The display unit 153 transmits the support information created as described above to the user terminal 20 that requested support via the network 30 through the communication I / F unit 11, and displays the support information on the screen of the user terminal 20. Various forms are possible in which the display unit 153 displays the support information on the screen of the user terminal 20. For example, the display unit 153 may simultaneously display an image of a genuine package, an image of a counterfeit package, and explanatory information included in the support information. Alternatively, the display unit 153 may alternately display an image of a genuine package and an image of a counterfeit package included in the support information, and display the explanatory information only when requested by the user.

[0085] The additional unit 154 is configured to enhance the authenticity determination support information database 142 by collecting authenticity determination support information regarding images of unknown authenticity from experts and storing it in the authenticity determination support information database 142. For example, when an image of an item to be determined that has been assigned an authenticity determination ID by the matching unit 152 is newly stored in the authenticity determination database 143, the additional unit 154 transmits the authenticity determination information to the expert terminal 40. The additional unit 154 includes at least the image of an item to be determined that has been assigned an authenticity determination ID in the authenticity determination information.

[0086] Furthermore, when new authenticity determination support information is transmitted from the expert terminal 40, the additional unit 154 receives it and stores it in the authenticity determination support information database 142. Here, the format of the authenticity determination support information transmitted from the expert terminal 40 is the same as the authenticity determination support information 641 shown in Figure 2. However, the ID is set by the expert as an ID for items of unknown authenticity.

[0087] In other words, the expert uses the auxiliary device 1 according to the first embodiment and their own expertise to determine whether the image to be judged received from the server terminal 10 is an image of a genuine package or an image of a counterfeit package. If the expert determines that the image to be judged is an image of a genuine package, they record the authenticity / counterfeiting ID in the ID 6411 field, the name of the genuine package in the drug name 6412 field, record the image to be judged or another image of a genuine package in place of it in the image of a genuine package 6413 field, and create authenticity / counterfeiting support information 641 with the images of a counterfeit 6414 and explanatory information 6415 fields set to NULL. However, if an expert has obtained an image of a counterfeit genuine package through some means, they may set that image of the counterfeit in the Image 6414 field and record text and / or diagrams in the Explanatory Information 6415 field explaining the key points for distinguishing between Image 6413 of a genuine package and Image 6414 of a counterfeit package. Furthermore, if the expert determines that the image to be judged is an image of a counterfeit package, they shall record the Authenticity Undetermined Item ID in the ID 6411 field, record the name of the genuine package and the image of the package that formed the basis for the determination that the package is counterfeit in the Drug Name 6412 and Image 6413 fields, record the image to be judged in the Image 6414 field, and record text and / or diagrams in the Explanatory Information 6415 field explaining the key points for distinguishing between Image 163 of a genuine package and Image 164 of a counterfeit package, thereby creating the Authenticity Determination Support Information 641.

[0088] Next, the expert transmits the authenticity determination support information 641 created as described above from the expert terminal 40 to the server terminal 10 via the network 30. When the addition unit 154 receives this authenticity determination support information 641 through the communication I / F unit 11, it rewrites the ID 6411 item to an ID that can be uniquely identified in the authenticity determination support information database 142 and registers it in the authenticity determination support information database 142. Next, the addition unit 154 deletes the image to be judged that has the original authenticity unknown item ID from the authenticity unknown item database 143.

[0089] Next, we will explain the operation of server terminal 10.

[0090] Figure 12 is a flowchart illustrating an example of the operation of the server terminal 10 when a support request is sent from the user terminal 20 to the server terminal 10. Referring to Figure 12, the acquisition unit 151 of the server terminal 10 receives the support request sent from the user terminal 20 (step S11). Next, the matching unit 152 of the server terminal 10 compares the image to be judged included in the support request with the images 6413 of genuine packaging and 6414 of counterfeit packaging among all the authenticity judgment support information 641 stored in the authenticity judgment support information database 142 (step S12). In step S12, if the matching unit 152 detects authenticity judgment support information 641 in which at least one of the images 6413 of genuine packaging or 6414 of counterfeit packaging matches the image to be judged, it transmits the matching result, including the ID 6411 of the detected authenticity judgment support information 641, to the presentation unit 153. Furthermore, if no authenticity determination support information 641 is detected in step S12, the matching unit 152 transmits a matching result to the presentation unit 153 that does not include the ID 6411 of the authenticity determination support information 641 at all.

[0091] Next, the presentation unit 153 creates information to support the determination of the authenticity of the object to be judged (step S13). In step S13, the presentation unit 153 checks whether ID 6411 is included in the matching result. If it is included, it creates support information based on the authenticity determination support information database 142 in the manner described above. If it is not included, it creates support information based on the EC site 50 and the counterfeit drug database 145 in the manner described above. Next, the presentation unit 153 transmits the created support information to the user terminal 20 that requested support via the network 30 through the communication I / F unit 11, and displays the support information on the screen of the user terminal 20 (step S14).

[0092] Furthermore, the matching unit 152 assigns an unverified authenticity ID to any image to be judged for which no authenticity determination support information 641 was detected in the matching in step S2, and stores it in the unverified authenticity database 143 (step S15). Also, when a new image to be judged is stored in the unverified authenticity database 143, the addition unit 154 transmits unverified authenticity information, including the image to be judged with the assigned unverified authenticity ID, to the expert terminal 40 (step S16). When the expert terminal 40 receives the unverified authenticity information, including the image to be judged, transmitted from the server terminal 10, the expert in the expert terminal 40 is notified of this. The expert creates authenticity determination support information 641 by performing the same processing as described in Figure 3 based on the received image to be judged. The expert then transmits the created authenticity determination support information 641 from the expert terminal 40 to the server terminal 10 via the communication I / F unit.

[0093] Figure 13 is a flowchart illustrating an example of the operation of the server terminal 10 when authenticity determination support information is transmitted from the expert terminal 40 to the server terminal 10. Referring to Figure 13, the add-on unit 154 of the server terminal 10 receives the authenticity determination support information 641 transmitted from the expert terminal 40 (step S21). Next, the add-on unit 154 rewrites the ID 6411 of the received authenticity determination support information from the authenticity unknown item ID to the authenticity determination support information ID and registers it in the authenticity determination support information database 142 (step S22). Next, the add-on unit 154 deletes the image to be determined that had the authenticity unknown item ID before rewriting from the authenticity unknown item database 143 (step S23).

[0094] In this way, the server terminal 10 compares the image to be judged included in the support request sent from the user terminal 20 with the images 6413 of genuine packaging and 6414 of counterfeit packaging among all the authenticity judgment support information 641 stored in the authenticity judgment support information database 142. Next, if the server terminal 10 detects authenticity judgment support information 641 in which at least one of the images 6413 of genuine packaging or 6414 of counterfeit packaging matches the image to be judged, it creates support information based on the detected authenticity judgment support information 641, including the image of genuine packaging, the image of counterfeit packaging, and text and / or images explaining the key points for distinguishing between the two, and displays it on the screen of the user terminal 20. This allows the user to easily and accurately determine authenticity.

[0095] Furthermore, if the server terminal 10 does not find any authenticity determination support information 641 that matches the image to be judged, it creates support information based on the results of searching for images that match the image to be judged from reliable e-commerce sites 50 and a counterfeit drug database 145, and displays it on the screen of the user terminal 20. This allows the user to determine authenticity with a certain degree of accuracy.

[0096] Furthermore, if the server terminal 10 does not find any authenticity determination support information 641 that matches the image to be judged, it saves the image to be judged in the database of items of unknown authenticity 143 and transmits the image to the expert terminal 40. Then, when the expert determines the authenticity of the image to be judged and transmits the authenticity determination support information created by the expert based on the result from the expert terminal 40, the server terminal 10 receives it and adds it to the authenticity determination support information database 142. This allows the authenticity determination support information database to be enriched.

[0097] [Third Embodiment] Figure 14 is a block diagram of an auxiliary device 1000 according to a third embodiment of the present invention.

[0098] Referring to Figure 14, the auxiliary device 1000 comprises a target image acquisition unit 1001, a frontal image generation unit 1002, a drug name recognition unit 1003, a legitimate drug name search unit 1004, and a genuine product image search unit 1005.

[0099] The target image acquisition unit 1001 is configured to acquire an image of the side of the pharmaceutical packaging on which the drug name is written as the target image. The target image acquisition unit 1001 can be configured similarly to, for example, the target image acquisition unit 71 in Figure 1, but is not limited thereto. The frontal image generation unit 1002 is configured to extract the area of ​​the above-mentioned side from the target image acquired by the target image acquisition unit 1001, perform trapezoidal correction, and generate a frontal image. The frontal image generation unit 1002 can be configured similarly to, for example, the frontal image generation unit 72 in Figure 1, but is not limited thereto. The drug name recognition unit 1003 is configured to recognize the drug name by performing character recognition on the frontal image generated by the frontal image generation unit 1002. The drug name recognition unit 1003 can be configured similarly to, for example, the drug name recognition unit 73 in Figure 1, but is not limited thereto. The authorized drug name search unit 1004 is configured to search for a similar drug name as the authorized drug name from the list of authorized drug names using the drug name recognized by the drug name recognition unit 1003. The authorized drug name search unit 1004 can be configured in the same way as, for example, the authorized drug name search unit 74 in Figure 1, but is not limited thereto. The authorized product image search unit 1005 is configured to search for an authorized product image from image information that associates the authorized drug name with an authorized product image, which is a photograph of the side of the authorized drug packaging on which the drug name is written, by using the authorized drug name search unit 1004 as a keyword for image search. The authorized product image search unit 1005 can be configured in the same way as, for example, the authorized product image search unit 75 in Figure 1, but is not limited thereto.

[0100] The auxiliary device 1000, configured as described above, operates as follows: The target image acquisition unit 1001 acquires an image of the side of the pharmaceutical packaging on which the pharmaceutical name is written as the target image. Next, the frontal image generation unit 1002 extracts the area of ​​the aforementioned side from the target image acquired by the target image acquisition unit 1001, performs trapezoidal correction, and generates a frontal image. Next, the pharmaceutical name recognition unit 1003 recognizes the pharmaceutical name by performing character recognition on the frontal image generated by the frontal image generation unit 1002. Next, the legitimate pharmaceutical name search unit 1004 searches for a similar pharmaceutical name as the legitimate pharmaceutical name from the list of approved pharmaceutical names using the pharmaceutical name recognized by the pharmaceutical name recognition unit 1003. Next, the genuine product image search unit 1005 searches for the genuine product image from image information that associates the legitimate pharmaceutical name with a genuine product image of the side of the pharmaceutical packaging on which the pharmaceutical name is written, using the legitimate pharmaceutical name search unit 1004 as a keyword.

[0101] By using the auxiliary device 1 configured and operating as described above, the time and cost required for experts to create information to assist users in determining the authenticity of pharmaceuticals can be reduced. This is because experts only need to confirm the recognized pharmaceutical name, eliminating the need to visually read and manually input the pharmaceutical name from the target image. Furthermore, while significant deformation of the target image shape can make character recognition difficult or increase the likelihood of misrecognition, the frontal image generation unit 1002 performs trapezoidal correction to generate a frontal image before character recognition, thus eliminating such problems and enabling accurate character recognition. Equipped with a genuine pharmaceutical name search unit 1004, experts can avoid the trouble of searching for whether a drug is approved themselves. Additionally, equipped with a genuine product image search unit 1005, experts can avoid the trouble of searching for genuine product images themselves. In this way, experts can avoid the trouble of searching for whether a drug is approved or genuine product images themselves, allowing them ample time to determine whether the target image is a genuine package image or a counterfeit. They can also refer to information such as whether the drug is approved or the content of the genuine product image when making their judgment.

[0102] Although the present invention has been described above with reference to the embodiments described above, the present invention is not limited to the embodiments described above. Various modifications to the configuration and details of the present invention can be made within the scope of the present invention as can be understood by those skilled in the art.

[0103] For example, instead of the CPU mentioned above, a GPU (Graphics Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination of these can be used. [Industrial applicability]

[0104] This invention can be used in the field of determining the authenticity of pharmaceutical packaging. [Explanation of symbols]

[0105] 1. Device for creating information to support authenticity determination. 2 cameras 3. Communication I / F section 4. Operation Input Section 5 Screen display section 6 Memory section 7. Arithmetic Processing Unit

Claims

1. A target image acquisition means that acquires an image of the side of a pharmaceutical package on which the name of the pharmaceutical product is written as the target image, A frontal image generation means extracts the region of the surface from the acquired target image, performs trapezoidal correction, and generates a frontal image; A drug name recognition means that performs character recognition from the generated frontal image to recognize the drug name, A legitimate drug name search means that searches for a similar drug name as a legitimate drug name from a list of approved drug names using the recognized drug name, A genuine product image search means searches for genuine product images from image information that associates the genuine product name with the genuine product image of the side of the genuine product packaging on which the genuine product name is written, using the searched genuine product name as a keyword in the image search. A device for creating information to support authenticity determination, equipped with the following features.

2. The system further includes a difference detection means for detecting areas with large differences between the generated frontal image and the retrieved genuine product image. The device for creating information to support the determination of authenticity, as described in claim 1.

3. The difference detection means has a trained model that takes the generated frontal image, the searched genuine image, and the difference image between these two images as input and performs machine learning to estimate the regions where there are large differences between the frontal image and the genuine image. The device for creating information to support the determination of authenticity, as described in claim 2.

4. The frontal image generation means is An edge CNN for detecting the edges of the packaging shown in the aforementioned target image, A surface CNN for detecting the surface of the package shown in the aforementioned target image, A Watershed object, which separates the object such as the packaged object in the aforementioned target image from the background, An integrated CNN outputs a surface image of the packaging body shown in the target image from the target image, the output image of the edge CNN, the output image of the surface CNN, and the output image of the watershed. A Hough transform unit for detecting the outer contour of the surface image, The system includes a projection transformation unit that trapezoidally corrects the shape of the surface of the packaging body, which is identified by the outer contour, and generates the frontal image. The device for creating information to support the determination of authenticity, as described in claim 1.

5. The aforementioned drug name recognition means is The system calculates the character recognition results and the bounding rectangle area for one or more characters in the frontal view image, and outputs character recognition results prioritized according to the size of the bounding rectangle area. The device for creating information to support the determination of authenticity, as described in claim 1.

6. A computer, The image obtained is a photograph of the side of the pharmaceutical packaging that shows the name of the drug, and the target image is obtained from that side. The region of the surface is extracted from the acquired target image, trapezoidal correction is applied, and a frontal image is generated. Character recognition is performed on the generated frontal image to recognize the name of the drug. Using the recognized drug name, a similar drug name is searched for as the official drug name from the list of approved drug names. By performing an image search using the searched legitimate drug name as a keyword, the image of the legitimate product is searched from image information that associates the legitimate drug name with an image of the legitimate product, which is a photograph of the side of the legitimate drug packaging on which the drug name is written. A method for assisting in the creation of information to support the determination of authenticity.

7. On the computer, The process involves acquiring an image of the side of the pharmaceutical packaging that contains the name of the drug as the target image, The process involves extracting the region of the surface from the acquired target image, performing trapezoidal correction, and generating a frontal image. The process involves performing character recognition on the generated frontal image to recognize the name of the drug, The process involves using the recognized drug name to search for a similar drug name from the list of approved drug names as the official drug name, The process involves searching for images of genuine products from image information that associates the genuine drug name with an image of the genuine product, which is a photograph of the side of the genuine drug packaging on which the drug name is written, using the searched genuine drug name as a keyword in the image search. A program to perform that action.

8. A genuineness determination support system that assists users in determining the authenticity of an object, A database that stores authenticity determination support information, which is created using the authenticity determination support information creation auxiliary device described in any one of claims 1 to 5, and which includes images of genuine pharmaceutical packaging and images of counterfeit pharmaceutical packaging. An acquisition means for acquiring an image of the object to be judged from the user's terminal, A matching means for comparing an image in the authenticity determination support information stored in the database with an image of the object to be determined, If, as a result of the matching, the authenticity determination support information including an image matching the image of the object to be determined is detected, the presenting means displays the detected authenticity determination support information on the screen of the user terminal, A system that supports authenticity determination, including [specific features / features].

Images