Medicine inspection system and program

Abstract

To efficiently inspect each medicine of a plurality of medicine packages.SOLUTION: A medicine inspection system includes an image acquisition unit configured to acquire a medicine package image obtained by photographing each of a plurality of medicine packages in which a medicine is packaged based on a prescription, an inspection unit configured to inspect whether or not the medicine of each of the plurality of medicine packages conforms to the prescription based on the medicine package image of each of the plurality of medicine packages, an output unit configured to output an inspection result of the inspection unit, and a medicine extraction display unit configured to extract and display a medicine package image containing a designated medicine from the medicine package images of the plurality of medicine packages.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a medicine inspection system and a program. [Background technology]

[0002] A prescribed number of medications are packaged in single-dose medicine pouches and provided to patients. In this case, each medicine pouch is inspected to determine whether it is as prescribed. A system that supports this inspection process compares a photographed image of each medicine pouch with a master image of the medicine, recognizes the medicine in each medicine pouch, and determines whether the recognized medicine matches the prescription. The system displays the judgment results and an image of each medicine pouch on a screen, allowing an inspector (e.g., a pharmacist) to visually inspect the medicine.

[0003] For example, paragraphs 0265-0271, Figures 29-32, paragraphs 0393-0401, and Figures 58-64 of the specification of Japanese Patent No. 5196337 describe examples of displaying inspection results from the drug inspection process. In this example, while the drug inspection process is being performed, each sachet in the continuum of sachets is displayed in thumbnail format in order of inspection completion. Clicking on a thumbnail image displays an individual inspection result display window. This window lists the name, image, and quantity of the prescribed drug, as well as the detection results for the drug. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5196337 Summary of the Invention [Problem to be solved by the invention]

[0005] The present application discloses an inspection system and program for efficiently inspecting each drug in a plurality of medicine packets. [Means for solving the problem]

[0006] A medicine inspection system in an embodiment of the present invention includes an image acquisition unit that acquires medicine package images of multiple medicine packages in which medicines are packaged based on a prescription, an inspection unit that inspects whether the medicine in each of the multiple medicine packages is as prescribed based on the medicine package image of each of the multiple medicine packages, an output unit that outputs the inspection results of the inspection unit, and a medicine extraction and display unit that extracts and displays a medicine package image containing a specified medicine from the medicine package images of the multiple medicine packages. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a functional block diagram showing an example of the configuration of a medicine inspection system according to an embodiment of the present invention. [Figure 2] 2 is a flowchart showing an example of the operation of the medicine inspection system 10 shown in FIG. [Figure 3] FIG. 10 is a diagram showing an example of a screen displaying a medicine image including a specified medicine. [Figure 4] FIG. 10 is a diagram showing an example of an individual screen that displays extracted medicine package images individually for each medicine package. [Figure 5] 2 is a functional block diagram showing an example of the configuration of a master management unit 6 in FIG. 1. FIG. [Figure 6] FIG. 10 is a diagram showing an example of a screen for accepting an instruction to change a master image. [Figure 7] 10 is a flowchart showing an example of a medicine recognition process. [Figure 8] 10A and 10B are diagrams showing examples of a print image G6 and an image G7 of a transparent medicine. [Figure 9] FIG. 10 is a diagram showing an example of dots in areas where medicines in a medicine area match. [Figure 10] 10A and 10B are diagrams showing examples of a print image G9 and an image G10 of a transparent medicine. [Figure 11] FIG. 10 is a diagram showing an example of a region R4 that matches a drug region R5. DETAILED DESCRIPTION OF THE INVENTION

[0008] (Configuration 1) A medicine inspection system in an embodiment of the present invention includes an image acquisition unit that acquires medicine package images of multiple medicine packages in which medicines are packaged based on a prescription, an inspection unit that inspects whether the medicine in each of the multiple medicine packages is as prescribed based on the medicine package image of each of the multiple medicine packages, an output unit that outputs the inspection results of the inspection unit, and a medicine extraction and display unit that extracts and displays a medicine package image containing a specified medicine from the medicine package images of the multiple medicine packages.

[0009] The medicine inspection system of configuration 1 inspects whether the medicine recognized in each medicine package image of multiple medicine packages is as prescribed and outputs the inspection results. Furthermore, it is possible to extract and display medicine package images containing a specified medicine from the medicine package images of multiple medicine packages. In addition to understanding the system's judgment results, a user (e.g., an inspector) can continuously visually inspect the medicine package images containing the specified medicine. This makes it possible to efficiently inspect each medicine in multiple medicine packages.

[0010] The drug extraction display unit may extract and display a drug package image containing a drug specified by a user from the drug package images of the multiple drug packages. This allows the user to specify the drug they wish to check. For example, the drug extraction display unit may display information about the drug contained in the multiple drug packages (e.g., drug name, drug master image, etc.) and accept the drug specification from the user. Note that the form of the drug extraction display unit is not limited to a form in which the user specifies the drug. For example, a drug inspection system may determine the specified drug based on prescription data.

[0011] The medicine extraction display unit may extract a plurality of medicine package images containing a specified medicine from the medicine package images of the plurality of medicine packages and display a list of only the extracted medicine package images, thereby allowing the user to check and compare the plurality of medicine package images containing the specified medicine.

[0012] The medicine extraction display unit may display a plurality of medicine package images each containing a designated medicine, and may highlight an area of ​​the designated medicine in each medicine package image, thereby making it easier for the user to identify the designated medicine in each medicine package.

[0013] The drug extraction display unit may extract a drug package image including the specified drug based on information about the drug to be packaged in each of the multiple drug packages indicated by prescription data. Alternatively, the drug extraction display unit may extract a drug package image including the specified drug based on the drug recognized in each of the multiple drug package images. That is, a drug package image including the specified drug may be extracted based on at least one of the drug in each drug package indicated by prescription data and the drug in each drug package recognized in each drug package image. Note that prescription data is data indicating a prescription from which the multiple drug packages were packaged.

[0014] (Configuration 2) In the above configuration 1, the multiple medicine packages may be a group of medicine packages in which medicines are packaged based on multiple prescriptions. The medicine extraction display unit may extract and display a medicine package image containing a specified medicine from the medicine package images of the multiple medicine packages in which medicines are packaged based on the multiple prescriptions. This allows the user to efficiently check the specified medicine for multiple medicine packages containing multiple prescriptions. The multiple prescriptions may include prescriptions for multiple patients. For example, a medicine package image containing a specified medicine may be extracted and displayed from a consecutive group of medicine packages containing multiple types of medicines for multiple patients. This makes it easier for the user to inspect and confirm whether the consecutive groups of medicines for multiple patients are being packaged according to prescription on a medicine-by-medicine basis. The medicine extraction unit may display the medicines contained in the multiple medicine packages in which medicines are packaged based on multiple prescriptions and receive a medicine designation from the user.

[0015] (Configuration 3) In the above configuration 1 or 2, the multiple medicine packages may include medicine packages in which medicines manually dispensed in a packaging machine are packaged. The medicine extraction display unit may extract and display medicine package images including the manually dispensed medicines from the medicine package images of the multiple medicine packages. This allows the user to efficiently check medicine images including the manually dispensed and packaged medicines. For example, the medicine extraction unit may display the manually dispensed medicines included in the multiple medicine packages and accept the user's designation of the medicines.

[0016] The packaging machine may be equipped with a manual drug supply device (manual supply unit) that supplies manually dispensed drug packets to the packaging device (packaging section) of the packaging machine. The manual drug supply device may be configured, for example, to include a tray having a plurality of boxes partitioned in a lattice pattern, each of which contains one packet of manually dispensed drug, and to supply the manually dispensed drug by sequentially opening the bottom plates of the boxes. The prescription data may include information indicating the manually dispensed drug among the prescribed drug packets. In this case, the drug extraction display section can identify the manually dispensed drug based on the prescription data.

[0017] The drug extraction display unit may display images of the multiple drug packages, which are a group of drug packages containing drugs packaged based on multiple prescriptions, including drug packages containing manually dispensed drugs. The drug extraction display unit may extract and display images of drug packages containing the manually dispensed drugs from images of the multiple drug packages containing drugs packaged based on multiple prescriptions. This allows the user to efficiently check manually dispensed drugs (e.g., drugs dispensed by multiple prescriptions) in multiple drug packages containing drugs packaged based on multiple prescriptions. For example, the drug extraction unit may display manually dispensed drugs contained in multiple drug packages containing drugs packaged based on multiple prescriptions and accept designation of the drug from the user.

[0018] (Configuration 4) In any of the above configurations 1 to 3, the medicine extraction display unit may display the extracted medicine package images on a single screen. By displaying the extracted medicine package images together on a single screen in this manner, the user can more easily perform quick and accurate visual inspection compared to when the images are displayed individually and sequentially. Specifically, a plurality of medicine package images containing a specified medicine may be extracted, and the extracted medicine package images may be displayed as a list arranged on a single screen.

[0019] (Configuration 5) In any of the above configurations 1 to 3, the drug extraction display unit may display each of the extracted and displayed drug package images on the same screen as information on the drug to be contained in the drug package of each drug package image indicated in the prescription. This makes it easier for the user to check whether the drug is as prescribed in each drug package image containing a specified drug. For example, when inspecting a series of consecutive drug package images containing multiple types of drugs for multiple patients, the user can easily inspect and check whether each drug in the drug package images containing the specified drugs is as prescribed. As an example, the drug extraction display unit may sequentially display each of the extracted and displayed drug package images on the same screen as information on the drug to be contained in the drug package of each drug package image indicated in the prescription.

[0020] (Configuration 6) A drug inspection system in an embodiment of the present invention includes an image acquisition unit that acquires drug images of drugs dispensed based on a prescription, a drug recognition unit that recognizes the drugs included in the drug images using drug master data, a prescription determination unit that determines whether the drugs recognized by the drug recognition unit are as prescribed, and an output unit that outputs the determination result of the prescription determination unit. The medicine inspection system may further include a master image change receiving unit that displays a list of medicines registered in the master data that require a change to their master images due to a change in appearance, along with the changed master images of each medicine, and receives input of a change instruction for the master image of each medicine.

[0021] This allows the master image to be changed in response to a change in the appearance of the drug at an appropriate timing based on a user's instruction. The reception of an instruction to change the master image may be, for example, reception of approval by the user to change the master image. The drug inspection system may further include a master data update unit. The master data update unit updates the master data used in the drug recognition unit in response to the master image change instruction received by the master change reception unit. This updates the master image of the drug whose master data is to be changed to a master image of the drug's appearance after the change.

[0022] The medicine inspection system may include a master registration reception unit instead of or in addition to the master change reception unit. The master registration reception unit may display master information of unregistered medicines to be registered in master data to the user and accept input of master registration instructions (e.g., user approval of medicine master registration) from the user. For example, when the number of unregistered medicine groups reaches a predetermined number, the master registration reception unit may notify the user of this and accept input of master registration instructions. This allows the user to issue master registration instructions (e.g., master registration approval) for an appropriate number of unregistered medicine groups. The master registration reception unit may determine the display order of unregistered medicines based on the quantity of each unregistered drug included in the prescription to be inspected in the medicine inspection system. For example, it may display unregistered medicine groups in descending order of the number of tablets included in the prescription and accept master registration instructions for each unregistered drug. The master data update unit adds (i.e., registers) the master information of the specified unregistered drug to the master data in accordance with the master registration instructions accepted by the master registration reception unit. The medicine recognition unit is capable of performing medicine recognition processing using master information of registered medicines.

[0023] The master information of the unregistered drug may be supplied as master data to the drug inspection system from a master data supplier. For example, when master information of a new drug is added to the mother master, the mother master including the master information of the added drug may be distributed to the drug inspection system. The master information of the added drug in the mother master is recorded in the drug inspection system as master information of the unregistered drug.

[0024] The image acquisition unit of the drug inspection system may acquire drug images captured by an inspection device including a camera for capturing images of drugs, and the drug recognition unit may recognize drugs in the drug images using master data including color data indicating the color of the drug based on the image captured by the inspection device. This allows drugs included in the drug images captured by the inspection device to be recognized using color data indicating the drug color specific to the inspection device. This stabilizes recognition performance. For example, when multiple inspection devices share master data, the master information for at least some of the drugs in the master data may include color data specific to each inspection device. In other words, the master information for at least some of the drugs registered in the master data may include master information common to multiple inspection devices and color data recorded for each inspection device.

[0025] If the drugs indicated in the prescription to be inspected include drugs for which master information is not registered in the master data, the image acquisition unit may acquire and store in a storage unit all drug images of the drugs dispensed based on the prescription. In this case, for drugs for which master information is available, the drug recognition unit may recognize the drugs included in the drug images using the master data, and the prescription determination unit may determine whether the recognized drugs are as prescribed. After the master information for drugs for which master information is not registered is registered in the master data, the drug recognition unit may recognize the drugs included in the drug images stored in the storage unit using the master data, and the prescription determination unit may determine whether the drugs recognized by the drug recognition unit are as prescribed. In this way, if the prescription to be inspected includes drugs for which master information is not registered, drug images can be saved, and after registering the master information, the drug recognition and prescription determination processes can be performed using the saved drug images. This eliminates the need to re-photograph drug images, enabling efficient inspection.

[0026] If the prescription that is the basis of the drug image acquired by the image acquisition unit includes a drug whose master information is not registered in the master data (an unregistered master drug), the drug inspection system may notify the user that this unregistered master drug is a drug for which master information should be registered.The master registration acceptance unit may also display this unregistered master drug to the user and accept input of a master registration instruction from the user (for example, approval of drug master registration by the user).In addition, a screen displaying a list of prescriptions displayed by the drug inspection system may display a message indicating that, for prescriptions that included unregistered master drugs, the master information of the unregistered master drug was registered in the master data after the drug image was acquired.

[0027] The medicine inspection system may output a list of unregistered medicines whose master information is not registered in the master data and for which master images need to be taken with the photography device of the inspection device. The user can use the information in the output list to efficiently prepare the medicines that need to be photographed.

[0028] In another embodiment of the present invention, the medicine inspection system includes an inspection unit that inspects medicines dispensed based on prescriptions based on images of the medicines to determine whether the medicines are as prescribed, and a display control unit that displays a list of prescriptions and displays a reception screen that accepts the selection of prescriptions to be inspected by the inspection unit. The prescription list displayed by the display control unit includes information on the start date of each prescription. This allows the user to have the medicine inspection system perform inspection processing, for example, by prioritizing prescriptions with the closest start date.

[0029] The drug inspection system may display drug images that have been inspected and accept user comments for each drug image. The entered comments are recorded in association with the drug image. When a drug image is displayed, if there is an associated comment, the drug image is displayed in a manner that allows the user to refer to the comment. Furthermore, when the drug image is displayed, a mark indicating comment entry may be added to the drug screen on which a comment has been entered. For example, a button for accepting comment entry may be provided on an individual drug package screen that individually displays each of multiple drug package images that are the subject of inspection. In this case, clicking the button displays a comment entry form. Note that the individual drug package screen may also have a button for accepting input of the results of visual inspection of the drug package image. For example, for a drug package image that has been visually rejected, the user can enter comments on the reason for the rejection, the fact that the image was photographed with an evidence camera, and other information.

[0030] A medicine inspection system in another embodiment of the present invention includes an inspection unit that inspects whether the medicine in each of multiple medicine packages is as prescribed, based on medicine package images captured of each of multiple medicine packages containing medicines packaged based on a prescription. The inspection unit may acquire packaging paper type data indicating the type of packaging paper used for the medicine packages, and switch image processing for the medicine package image depending on the type of packaging paper indicated by the packaging paper type data. This allows image processing to be performed according to the type of packaging paper used for the medicine package being inspected. This eliminates the need for a user to switch between different packaging paper types.

[0031] In another embodiment of the present invention, a medicine inspection system includes an image acquisition unit that acquires a medicine image of a medicine dispensed based on a prescription, a medicine recognition unit that recognizes the medicine included in the medicine image, and a prescription determination unit that determines whether the medicine recognized by the medicine recognition unit is as prescribed. The medicine image may include a backlight image of the medicine illuminated from behind.

[0032] The drug recognition unit may include a drug area detection unit that detects a drug area using a converted image obtained by binarizing the backlight image with a reference color component. This improves the accuracy of detecting a drug area of ​​a transparent drug. As a result, inspection accuracy is improved. The reference color component may be, for example, one of red, green, or blue. As an example, the drug area may be detected using a converted image obtained by binarizing the backlight image with a red component.

[0033] The drug recognition unit may detect a drug region based on the rear face image and recognize the drug by determining a drug that matches the drug region. The drug recognition unit can determine whether or not there is a false detection based on the number of dots in an area where the drug in the drug region matches. The drug recognition unit can determine whether or not there is a false detection based on the number of average gray values ​​in an area where the drug in the drug region matches. The drug recognition unit can determine whether or not there is a false detection based on the overlap between the drug region and an area where the drug matches. Detection accuracy can be improved by performing at least one of the above three false detection determinations.

[0034] The present invention also includes a program for executing at least a portion of the processing of the above-described medicine inspection system, and a non-transitory storage medium storing the program. The medicine inspection program in the present invention causes a computer to execute an image acquisition process for acquiring a medicine package image obtained by photographing each of a plurality of medicine packages into which medicines are packaged based on a prescription, an inspection process for inspecting whether the medicine in each of the plurality of medicine packages is as prescribed based on the medicine package image of each of the plurality of medicine packages, an output process for outputting the inspection results of the inspection process, and a medicine extraction and display process for extracting and displaying a medicine package image containing a specified medicine from the medicine package images of the plurality of medicine packages.

[0035] The medicine inspection device according to an embodiment of the present invention is a conveying device that conveys a group of medicine packets each containing a medicine. the conveying device; a photographing device that photographs the group of medicine packets being conveyed by the conveying device; a control unit that controls the conveying device and the photographing device; and an inspection unit that uses the images of the group of medicine packets photographed by the photographing device to inspect whether the medicines contained in each of the group of medicine packets are as prescribed.

[0036] The control unit of the medicine inspection device may cause the conveying device to convey the group of medicine packets so that marked medicine packets with marks on their surfaces are within the imaging range of the imaging device, and adjust the imaging conditions of the imaging device based on images obtained by the imaging device photographing the marked medicine packets. This allows the imaging conditions of the imaging device to be adjusted to conditions suitable for detecting marks on the medicine packets. This improves the detection accuracy of the marks on the medicine packets. The imaging conditions may be, for example, exposure time. The marks may be, for example, marks indicating a change or start of a prescription in the group of medicine packets.

[0037] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0038] (System configuration example 1) FIG. 1 is a functional block diagram showing an example configuration of a medicine inspection system in this embodiment. The medicine inspection system 10 shown in FIG. 1 is a system that inspects whether tablets dispensed based on a prescription are as prescribed. The medicine inspection system 10 acquires medicine package images from a photographing device 21 that photographs each of multiple medicine packages in which medicines are individually packaged, and determines whether the medicine recognized in the medicine package image is as prescribed. The medicine inspection system 10 is implemented by a computer built into or connected to an inspection device that includes the photographing device 21. The medicine inspection system 10 can access a recording unit 11 that records master data.

[0039] In the example shown in FIG. 1, the inspection device includes a photographing device 21 and a conveying device 22. The conveying device 22 conveys a series of multiple medicine packages (a group of sachets) P. The photographing device 21 photographs each of the multiple medicine packages P conveyed by the conveying device 22. As shown in FIG. 1, the inspection device may also include a display 23 and an input device 24. The medicine inspection system 10 can display medicine package images and inspection results on the display 23. The medicine inspection system 10 can accept input from a user via the input device 24. The input device 24 may be composed of, for example, at least one of a touch panel of the display 23, a keyboard, a mouse, a button, or other operator.

[0040] Medicine inspection system 10 includes an image acquisition unit 1, an inspection unit 20, an output unit 4, a medicine extraction and display unit 5, and a master management unit 6. The inspection unit 20 includes a medicine recognition unit 2 and a prescription determination unit 3. The inspection unit 20 uses medicine package images of each of the multiple medicine packages into which medicines are packaged based on a prescription to inspect whether the medicine in the medicine package is as prescribed. The inspection unit can also be referred to as an inspection processing unit. The image acquisition unit 1 acquires medicine package images obtained by photographing the medicine package with a photographing device 21. The image acquisition unit 1 makes the medicine package image data available to the computer of medicine inspection system 10. For example, the image acquisition unit 1 receives medicine package image data from the photographing device 21 and stores it in the computer's memory. In the example of FIG. 1, the photographing device 21 photographs each of the multiple medicine packages transported by the transporting device 22. The image acquisition unit 1 acquires a medicine package image for each medicine package.

[0041] The medicine recognition unit 2 uses master data to recognize the medicine included in the medicine package image. The master data is recorded in advance in the recording unit 11. The medicine recognition unit 2 may recognize the medicine included in the medicine image using master information of the medicine to be inspected (i.e., the prescribed tablet). The master data includes master information for each medicine that may be inspected. The master information includes information indicating the conditions for recognizing the medicine. The medicine recognition unit 2 acquires master information for the medicine included in the prescription to be inspected. The master information may be, for example, a master image of the medicine. The master image includes, for example, at least one piece of information of the shape, size, color, marking, identification code, and printing of the medicine (tablet). The data format of the master information is not particularly limited. The master information may be, for example, image data (a collection of pixel values) indicating the appearance of the medicine, or values ​​indicating the feature quantities of the medicine.

[0042] The drug recognition unit 2 recognizes an area in a drug image that matches the conditions indicating the master information as the drug indicated by that master information. For example, the drug recognition unit 2 performs a matching process on a medicine package image using a master image, and when it detects an area that matches the master image, it determines that the drug in that master image is present. The drug recognition unit 2 can determine at least one of the number of tablets or the drug type in the target image.

[0043] The prescription determination unit 3 determines whether the drug recognized by the drug recognition unit 2 is as prescribed. The prescription determination unit 3 makes this determination by comparing the recognized drug with the drug indicated in the prescription data. The prescription data is recorded in advance in the recording unit 11. The prescription data is data including at least one of the quantity and drug type of the prescribed drug. For example, the prescription data may include information indicating at least one of the quantity and drug type of the prescribed drug in each drug image (for example, each dose). The prescription determination unit 3 compares either the quantity or drug type of the recognized drug with the prescription data, or both. The prescription determination unit 3 may determine whether the quantity of the drug is as prescribed, or may determine whether both the quantity and drug type of the drug are as prescribed. Alternatively, the user may be able to select whether the prescription determination unit 3 determines only the quantity of the drug, or whether it determines the quantity and drug type.

[0044] The output unit 4 outputs the judgment result of the prescription judgment unit 3, i.e., the inspection result. The output unit 4 may display on the display 23, as the judgment result of the prescription judgment unit 3, information indicating whether the drug in each medicine package image is as prescribed or not. The output unit 4 may display on the display 23 the judgment result of each medicine package, i.e., each medicine package image, together with the medicine package image. The output unit 4 may also display the recognition result of the medicine recognition unit 2 for each medicine package image. Note that the output unit 4 is not limited to displaying an image on the display 23. For example, the output unit 4 may output sound from a speaker or the like, or may print the result on a printer or the like. The judgment result may also be output by recording the data on a recording medium or transmitting the data to another system.

[0045] Output unit 4 may output each medicine package image in association with the prescribed medicine and the judgment result by prescription judgment unit 3. In this case, medicine inspection system 10 may accept input of the visual inspection result of visually inspecting the medicine package image of each medicine package from the user. In this case, the visual inspection result of each medicine package is recorded in recording unit 11 in association with each medicine package image.

[0046] The medicine extraction display unit 5 extracts and displays a medicine package image containing a specified medicine from the medicine package images of multiple medicine packages. The medicine extraction display unit 5 displays a list of medicines contained in the multiple medicine packages and accepts a medicine specification from the user. The medicine extraction display unit 5 extracts and displays a medicine package image containing a medicine specified by the user from the list of medicines from the medicine package images of multiple medicine packages.

[0047] The images of each medicine package displayed as the inspection result of the inspection process (e.g., the judgment result of the prescription judgment unit 3) are often displayed in the order in which they were captured. When multiple consecutive groups of medicine packages (groups of individual packets) are photographed, the images of each medicine package are displayed in the order in which the consecutive groups of medicine packages are connected. The inspector visually checks the displayed images of each medicine package in order to confirm whether the medicine in each medicine package is as prescribed. Prescription medicines are often packaged in single doses for each date and method of use. Therefore, the medicines in the medicine packages displayed as the inspection result of the inspection process have a regularly repeated pattern, making it easy to visually detect abnormalities. However, when inspecting a group of medicine packages containing multiple prescriptions with short dosing periods, the medicines in consecutive medicine packages may not repeat regularly. For example, medicines in a group of medicine packages in which multiple prescription medicines are packaged often contain different medicines in each package. When a user visually inspects the images of such a group of medicine packages in order, they will check for abnormalities in the medicine packages, whose contents vary from package to package. Compared to inspecting a group of medicine packets containing regular individual medicine packets, the burden and time required for verification increases on the user.

[0048] By providing the medicine inspection system 10 with the medicine extraction display unit 5 that extracts and displays a medicine package image containing a specified medicine from medicine package images of multiple medicine packages, the user can efficiently check the same medicine (specified medicine). This enables efficient inspection. It also makes it easier for the user to notice abnormalities in the medicine. As an example, the medicine extraction display unit 5 may extract and display a medicine package image containing a specified medicine from multiple medicine packages into which medicines dispensed in multiple prescription distribution are packaged. This improves the efficiency of inspecting medicines dispensed in multiple prescription distribution. In multiple prescription distribution, medicines for multiple prescriptions (e.g., multiple patients) are dispensed into a tray in a single manual distribution. For example, the medicine extraction display unit 5 may display a list of manually dispensed medicines from multiple medicine packages and accept a user's selection.

[0049] The master management unit 6 manages the master data used for recognizing medicines by the medicine inspection system 10. For example, the master management unit 6 updates the master data (registers, changes, and deletes master information) in response to input from a user or automatically.

[0050] The medicine inspection system 10 can be configured with one or more computers. The processes of the image acquisition unit 1, medicine recognition unit 2, prescription determination unit 3, output unit 4, medicine extraction and display unit 5, and master management unit 6 can be realized by the computer's processor executing a predetermined program recorded in memory. Programs for causing the processor to execute each of these processes and non-transitory recording media on which they are recorded are also included in embodiments of the present invention. The recording unit 11 may be configured with memory or storage accessible by a computer. The medicine inspection system 10 may further include functional units. For example, the computer constituting the medicine inspection system 10 may function as a controller for controlling the imaging device 21 and the transport device 22.

[0051] Fig. 2 is a flowchart showing an example of the operation of the medicine inspection system 10 shown in Fig. 1. In Fig. 2, the medicine inspection system 10 acquires prescription data indicating the prescription of tablets to be inspected (S1). The prescription data may be acquired, for example, by reading information attached to a sachet using the photographing device 21 provided in the inspection device or another reader (for example, a barcode reader, IC tag reader, etc.), or the medicine inspection system may acquire the prescription data from another system (for example, a packaging machine, prescription system, or receipt computer, etc.). The acquired prescription data may be recorded in the recording unit 11, for example.

[0052] The medicine inspection system 10 acquires master data (S2). As the master data, master information of the medicine indicated in the prescription data is acquired. The master data is recorded in the recording unit 11, for example.

[0053] Image acquisition unit 1 acquires an image of each of the multiple medicine packages photographed by photographing device 21 (S3). For example, multiple consecutive medicine packages containing medicines packaged based on one or more prescriptions are transported by transport device 22, and photographed one package at a time by photographing device 21. Image acquisition unit 1 acquires medicine package images of the multiple medicine packages corresponding to one or more prescriptions, and records them in recording unit 11 in association with the prescription information. In multiple consecutive medicine packages, a medicine package corresponding to the start of one prescription may be affixed with a next prescription mark indicating the start of the prescription. Medicine inspection system 10 can recognize the range of one prescription in multiple consecutive medicine packages by reading the next prescription mark on the medicine package.

[0054] Inspection unit 20 executes an inspection process to determine whether the drug in each medicine package image of multiple medicine packages corresponding to one or multiple prescriptions is as prescribed (S4). The inspection process for the medicine packages is executed repeatedly for all medicine packages of the multiple medicine packages, that is, until a NO determination is made in S5. The inspection process includes a process in which medicine recognition unit 2 recognizes the drug in each medicine package image using master data, and a process in which prescription determination unit 3 determines whether the drug recognized in each medicine package image is as prescribed.

[0055] The output unit 4 outputs the inspection results of the inspection process of S4 (S6). For example, each medicine package image may be displayed in association with the judgment result. As an example, a list screen displaying a list of medicine package images for multiple medicine packages and an individual screen displaying an image of each medicine package individually may be displayed as the inspection results. When one medicine package is selected on the list screen, an individual screen for that medicine package may be displayed. In addition to the medicine package image, the individual screen displays prescription information, inspection results (judgment results), and a button for accepting input of visual inspection results. The prescription information includes information indicating the drug that should be contained in that medicine package. Input of visual inspection results is accepted on the individual screen for each medicine package (S7). The user can visually check the individual screen for each medicine package and enter the visual inspection results.

[0056] The drug extraction display unit 5 displays a drug list that accepts drug designations (S8). Each drug in the drug list is displayed in a specifiable manner. Drug designations can be accepted from the user by displaying the drug list. The drug extraction display unit 5 displays a list of drugs contained in multiple drug packages corresponding to one or multiple prescriptions. The timing and display screen for displaying the drug list are not particularly limited, and for example, the drug list may be displayed on a list screen of inspection results. Note that the form in which drug designations from the user are accepted is not limited to displaying a drug list. For example, designation by text input of drug names, etc., designation by reading drug codes, or designation of extraction conditions may be accepted. The designation of extraction conditions can be conditions related to the drug injection method, shape, and other characteristics, such as manual drug, half tablet, etc.

[0057] When a drug is designated (YES in S9), the drug extraction display unit 5 extracts and displays a drug package image containing the designated drug from the drug package images of multiple drug packages (S10). FIG. 3 is a diagram showing an example of a screen displaying a drug image containing the designated drug. FIG. 3(a) is a display example when one drug (for example, "Loxonin 60ms") is designated from drug list L1. Drug package images Y1 containing the designated drug K1 are displayed side by side on one screen. In each drug package image Y1, the area of ​​the designated drug K1 is highlighted. The highlighting of the area of ​​drug K1 is a display that indicates the area of ​​drug K1 in a manner that allows it to be visually distinguished from the areas of other drugs. For example, the outer edge of the area of ​​drug K1 is displayed with a line that is different from the outer edges of the areas of the other drugs. FIG. 3(b) is a display example when another drug (for example, "Selbex CP") is designated from drug list L1. In this example, drug package images Y2 containing the designated drug K2 are also displayed side by side on one screen. In this way, since only the medicine packages containing the specified medicine are displayed in a list, the user can immediately find, for example, any medicine packages that are incorrectly packaged with a different medicine from the specified medicine, and can efficiently check whether there are any abnormalities with the specified medicine.

[0058] The drugs in the drug list L1, i.e., the drugs designated by the user, may be, for example, all drugs contained in a multiple drug package. Alternatively, among the drugs contained in multiple drug packages, drugs that have been manually dispensed may be displayed in the drug list L1 and designated by the user. This allows images of drugs packaged in multiple drug packages to be extracted and displayed, including drugs that have been manually dispensed. In this case, the user can efficiently check for abnormalities, such as whether the manually dispensed drugs differ from the prescription. For example, the prescription data may include information indicating whether the prescribed drugs should be manually dispensed. In this case, the drug extraction and display unit 5 can identify manually dispensed drugs in multiple drug packages based on the prescription data. The drug inspection system 10 may acquire data indicating manually dispensed drugs among the prescription drugs to be inspected from the packaging machine.

[0059] Medicine extraction display unit 5 can extract and display a medicine package image of a medicine package containing a specified medicine from medicine package images of multiple medicine packages corresponding to multiple prescriptions. For example, it can extract and display a medicine package image of a medicine package containing a specified medicine from medicine package images of multiple medicine packages containing medicines dispensed in a multiple prescription spread.

[0060] For example, multiple prescription dispensing may be in the form of sequentially dispensing medications for multiple patients with the same dosing period based on the prescriptions of multiple different patients, which are manually dispensed consecutively for each patient and then sequentially packaged. As a specific example, medications for patient A's morning, B's morning, C's morning, A's afternoon, B's afternoon, C's afternoon, A's evening, B's evening, and C's evening may be manually dispensed into consecutive boxes for the number of days prescribed, and then packaged in consecutive medicine packets in that order. As another example, multiple prescription dispensing may be in the form of sequentially dispensing medications for all dosing periods for multiple patients based on the prescriptions of multiple different patients, which are manually dispensed consecutively for each dosing period and then packaged in consecutive medicine packets. As a specific example, medications for all days and dosing periods for patient A alone may be manually dispensed into consecutive boxes for each dosing period, and then medications for all days and dosing periods for patient B may be manually dispensed into consecutive boxes for each dosing period, and then these may be packaged in consecutive medicine packets. Such multi-prescription packaging is carried out mainly for the purpose of forming a package that makes it easy for caregivers to distribute medication to a large number of residents in a nursing home or the like, for example.

[0061] FIG. 4 is a diagram showing an example of an individual screen that displays medicine package images extracted by the medicine extraction display unit 5 individually for each medicine package. The screen of FIG. 4 is displayed, for example, when one of the extracted and displayed medicine package images Y1 or Y2 shown in FIG. 3 is clicked. The screen of FIG. 3 displays the judgment result for one medicine package (sachet), i.e., the judgment result for one dose of the medicine for the prescription being inspected. The screen includes prescription information H1, information H2 indicating the number of the medicine package in the prescription, prescription judgment result H3, medicine package image J1, medicine package forwarding button B1, submenu button B2, and visual inspection result input button B3. In medicine package image J1, the area of ​​the specified medicine is surrounded by a dotted line, and the area is displayed in a manner that allows the user to confirm this area. This allows the user to efficiently confirm the specified medicine. The user can view the medicine package image J1, prescription information H1, and prescription determination result H3 on the individual screen, visually confirm whether the medicine in the medicine package image J1 is as prescribed, and click button B3 to enter the visual confirmation result (visual inspection result).

[0062] (Example of Master Management Unit Configuration) Fig. 5 is a functional block diagram showing an example configuration of the master management unit 6 in Fig. 1. The master management unit 6 reflects master information provided from the mother master or master information generated based on images of medicines captured by the photography device 21 in the master data used in the medicine inspection system 10. In the example of Fig. 5, the master management unit 6 includes a master change reception unit 61, a master registration reception unit 62, a master data update unit 63, a color data addition unit 64, and a master information reception unit 65.

[0063] The master image change receiving unit 61 displays, among the medicines registered in the master data, medicines for which a master image change is required in accordance with a change in the medicine appearance as medicines to be changed, and receives input of a master image change instruction from the user.

[0064] When the appearance (e.g., shape or printing) of a medicine is changed, a master image (changed master image) showing the changed appearance of the medicine is supplied to the medicine inspection system from the master data supplier. For example, the latest master data is distributed as a mother master from the manufacturer of the medicine inspection system. To reflect the distributed changes in the master data used in the medicine inspection system 10, the user must visually confirm and approve the change. The user may not notice the distribution of the mother master. Furthermore, when a changed master image accompanying a change in the appearance of a medicine is provided, the user may already be using the original medicine, and there may be no need to change the master image. As such, the timing of switching the adopted medicine to the changed medicine differs depending on the user, and each user must make a decision. Therefore, the master change receiving unit 61 may display a list screen of the changed master images distributed as the mother master and accept an instruction to change the master image on that screen. This allows the user to efficiently issue an instruction (approval) to change the master image. Furthermore, the user can visually confirm the changed master image before entering an instruction to change the master image. Therefore, it is possible to accept the input of the change instruction and at the same time accept the user's approval of the master image change. The master data update unit 63 updates the master data. The master data update unit 63 updates the master image of the drug to be changed, for which the user has instructed to change, to the changed master image.

[0065] FIG. 6 is a diagram showing an example of a screen that accepts an instruction to change a master image. In the example of FIG. 6, master information H4 before the change and master information H5 after the change for each drug to be changed are displayed. The master information includes a confirmation image, master images HM4 and HM5, and the update date. Check boxes are provided corresponding to each drug. Clicking the confirm button B4 inputs an instruction to change the master image of the drug whose check box is checked. Note that the displayed master information and the user interface tool that accepts the change instruction are not limited to the example of FIG. 6.

[0066] The master registration receiving unit 62 may display master information of unregistered drugs to be registered in the master data to the user and may receive a master registration instruction from the user. In response to the master registration instruction received by the master registration receiving unit 62, the master data updating unit 63 adds (i.e., registers) the master information of the instructed unregistered drug to the master data.

[0067] Master information to be added to the master data is supplied from a master data supplier. For example, a mother master containing the additional master information may be automatically downloaded to the drug inspection system. The master information to be added is recorded, for example, in the recording unit 11 as master information for unregistered drugs. In this way, master images of unregistered drugs are accumulated in the drug inspection system 10. To register and use master images of unregistered drugs in the master data, the user must visually inspect the product. As the number of accumulated unregistered drugs increases, the effort and time required for user approval becomes enormous. Furthermore, without user approval, the master information will not be registered in the master data, and inspection processing for unregistered drugs will remain unable to be performed.

[0068] Therefore, for example, when the number of unregistered drug groups reaches a predetermined number, the master registration reception unit 62 may notify the user of this and accept input of a master registration instruction. For example, the notification is made by displaying a pop-up message indicating that there are a predetermined number of unregistered drugs (unapproved drugs). A user operation on the pop-up may trigger the display of a master registration instruction reception screen. The timing of the notification is not particularly limited, but may be, for example, when prescription data for a prescription to be inspected is received or when automatic master maintenance processing is performed periodically.

[0069] On the master registration instruction reception screen, multiple unregistered drugs may be sorted and arranged in descending order of the number of tablets in the prescription. For example, multiple unregistered drugs may be sorted in descending order of the number of tablets included in prescriptions that have not yet undergone inspection processing. On the reception screen, master information for each of multiple unregistered drugs is displayed. It may also be possible to input registration instructions for each unregistered drug. Alternatively, a list of unregistered drugs may be displayed, and when an unregistered drug is selected from the list, a master registration reception screen for the selected unregistered drug may be displayed.

[0070] The color data adding unit 64 adds color data indicating the color of the drug, which is generated based on the color of the drug in the drug image captured by the photographing device 21, to the master data. The color data is included in the master data as color data specific to the inspection device including the photographing device 21. The color data specific to the inspection device including the photographing device 21 is used in the recognition process of the drug in the drug image captured by the photographing device 21. In other words, the drug recognition unit 2 recognizes the drug in the drug image captured by the inspection device using master data including color data indicating the color of the drug based on the image captured by the inspection device. The color data may be, for example, inside-package color data indicating the color of the inside of the drug package, which is generated based on the color of the drug in the drug package image.

[0071] The master data can be shared by multiple inspection devices. That is, the drug inspection systems of multiple inspection devices can use common master data. In this case, the common master data is used to recognize drugs for drug images captured by each of the multiple inspection devices. Each inspection device includes an imaging device 21 and a transport device 22. The master data may include color data unique to each of the multiple inspection devices. Specifically, at least some of the drugs included in the master data may include master information common to the multiple inspection devices and color data unique to each inspection device. That is, the color data of drugs can be managed for each inspection device in the master data common to the multiple inspection devices. For example, the common master data may include color data for each inspection device. For example, the color data may be managed in a file separate from other files of common master information.

[0072] When drugs indicated in a prescription to be inspected (prescribed drugs) include drugs whose master information is not registered in the master data (drugs not registered in master data), the drug inspection system 10 can record drug images and, after registering the master information, execute the inspection process using the recorded drug images. As a specific example, when prescribed drugs include drugs not registered in master data, the image acquisition unit 1 may acquire drug images of all drugs dispensed based on the prescription and record them in the recording unit 11. In this case, for drugs among the prescribed drugs that have master data, the drug recognition unit 2 may recognize the drugs included in the drug images using the master data, and the prescription determination unit 3 may determine whether the recognized drugs are as prescribed. Then, drug recognition and prescription determination are not performed for drugs not registered in master data, and the inspection process is temporarily terminated. Once the process is terminated, the drug inspection system 10 notifies the user that the drugs not registered in master data are drugs for which master data should be registered. This notification may be executed, for example, by displaying a function button on a portion of the screen listing the prescriptions to be inspected. In this case, a master data registration instruction may be received together with the notification. For example, when the function button is clicked, a screen for accepting a master information registration instruction may be displayed. On this screen, master information of the medicine not yet registered in the master is displayed, and a registration instruction (approval) is accepted from the user.

[0073] When the user inputs a registration instruction (approval) for a drug not registered in master data, the master information for the drug not registered in master data is registered in the master data. On the prescription list screen, after recording drug images, the prescriptions for which the drug not registered in master data has been registered are displayed so that the user can understand them. When the user selects the prescription on the prescription list screen and instructs inspection processing, a drug recognition process is executed using the recorded drug image and master data, and a prescription judgment process is executed to determine whether the recognized drug is as prescribed. This allows inspection processing to be executed on the drug image of a prescription that includes a drug not registered in master data, and then, after the drug not registered in master data has been registered, inspection processing can be executed again using the recorded drug image. This eliminates the need to transport a group of drug packages using the transport device 22 and photograph each drug package using the photographing device 21 for the second inspection process. This reduces the user's work time.

[0074] During the period from when the manufacture and sale of a new drug begins until the master information for that drug is added to the mother master, a user may adopt the new drug. When a user adopts a drug that is not even registered in the mother master, the drug inspection system 10 can generate a master image of the drug based on an image of the adopted drug taken by the photographing device 21. To acquire a master image by photographing in this manner, the drug inspection system 10 outputs to the user a list of drugs that are not registered in the master and for which a master image needs to be photographed by the photographing device 21. The list may be displayed on a display or printed out by a journal printer, for example. For example, a button for displaying unregistered drugs may be provided on a portion of the prescription list screen.

[0075] After reviewing this list, the user prepares the medicines, places them on the transport device 22, and has the camera 21 take a photograph. In this case, as an example, the user installs a camera jig on the transport device 22 and brings it into the shooting range of the camera 21. By placing the medicine on the camera jig and taking a photograph, an image of the medicine suitable for the master image can be captured. The camera jig and the process of constructing a master image based on images captured by the camera are described in, for example, Figures 37, 38, and 39, paragraphs 0124-0126, and paragraphs 0286-0311 of Japanese Patent No. 5196337, which are incorporated herein by reference. The medicine inspection system 10 generates a master image based on the captured image and records it in the recording unit 11 as a master image of the unregistered medicine.

[0076] (Example of process switching depending on packaging paper) The medicine inspection system 10 can acquire packaging paper type data indicating the type of packaging paper used for the medicine package and execute processing according to the packaging paper type. By switching processing depending on the type of packaging paper being inspected, the user does not need to perform a switching operation, thereby improving usability. The packaging paper type data can be acquired, for example, from a packaging machine or a higher-level system such as a prescription system or receipt computer. For example, the packaging paper type data may be included in prescription data indicating the prescription being inspected that is acquired by the medicine inspection system 10.

[0077] Packaging paper types are distinguished based on at least one of the material and dimensions of the packaging paper. Examples of packaging paper materials include cellopoly (cellophane and polyethylene), glassine, and PET (polyethylene terephthalate). Alternatively, the material of the packaging paper may be indicated by its characteristics (e.g., transparency, heat-sealability, surface roughness, biodegradability, etc.). Packaging paper dimensions that contribute to distinguishing packaging paper types may be, for example, the dimensions of the seal (seal width in the conveying direction or seal width perpendicular to the conveying direction), the length in the conveying direction between the seals, or the width of the packaging paper perpendicular to the conveying direction.

[0078] For example, the inspection unit 20 may switch the image processing for the medicine package image depending on the type of packaging paper indicated by the packaging paper type data. As an example, the image processing may be switched between packaging paper of a type with a seal width shorter than a predetermined value and packaging paper of a type with a seal width longer than a predetermined value. The image processing to be switched may be, for example, boundary detection processing that detects the upstream and downstream boundaries of at least one medicine package in the image of the captured medicine package group P. For example, biodegradable packaging paper is less likely to wrinkle on its surface than other types of packaging paper, which tends to improve the imaging conditions and increase the success rate of image processing. When the packaging paper type is biodegradable packaging paper, the inspection process may be performed using image processing specifically for biodegradable packaging paper. This improves the efficiency of image processing. As another example, when the packaging paper type is a specific type (e.g., PET packaging paper), a warning message may be displayed indicating that the inspection process cannot be performed.

[0079] (Example of drug recognition processing: transparent drug detection) The inspection device shown in Fig. 1 includes a conveying device 22, a photographing device 21, lighting devices 41A and 41B, and an inspection system 10. Lighting device 41B is a front lighting device that irradiates light from the front of the medicine packet group P as seen from the photographing device 21. The front lighting device may be, for example, a dome light. Lighting device 41B is a back lighting device (backlight) that irradiates light from the back of the medicine packet group P as seen from the photographing device 21.

[0080] The image acquisition unit 1 can acquire, as drug images, an image of a drug illuminated from the front (front-illuminated image) and an image of a drug illuminated from the back (back-illuminated image). In this case, the drug recognition unit 2 can detect the drug area using the back-illuminated image. A non-transparent drug will appear as a black area in the back-illuminated image because a shadow will be created when light is shone on it from the back. The black area is detected as the drug area. In this case, a partial image in which the area other than the drug area of ​​the front-illuminated image is masked may be compared with master information, and the drug that matches this partial image may be determined, thereby recognizing the drug.

[0081] Light from the backlighting passes partially through transparent medicines. This means that detection accuracy may be reduced when detecting black areas such as non-transparent medicines. Therefore, the detection accuracy of transparent medicines can be improved by detecting medicine areas using a converted image obtained by binarizing and converting the backlighting image using the red component. This makes it possible to detect dark transparent medicines with high accuracy.

[0082] FIG. 7 is a flowchart showing an example of drug recognition processing. In the example of FIG. 7, the drug recognition unit 2 converts a back-illuminated image G1 of a drug into an image (converted image G2) in which only the red component is binarized (S11). The red component is red in RGB (red, green, blue), which are the reference colors of an image. In this conversion, red, yellow, and white are converted to white, and black remains black. The converted image may be subjected to processing such as contrast enhancement. Furthermore, instead of binarizing the red component, the blue component or green component may be binarized. Furthermore, the converted image may be an image (e.g., a difference image) obtained by combining an image in which the back-illuminated image is binarized using the red component and an image in which the back-illuminated image is binarized using the blue component of the back-illuminated image.

[0083] The medicine recognition unit 2 extracts a medicine area using the converted image (S12). For example, in the converted image, a low-brightness area where the brightness is lower than the surrounding area or below a threshold is detected as a medicine area. Image G3 in FIG. 7 shows a detected medicine area. Note that an area obtained by shaping a low-brightness area into a shape that has the characteristics of a medicine may also be detected as a medicine area. Image G5 in FIG. 7 shows an example where a medicine area is detected in a backlit image rather than in a converted image. In image G5, an area of ​​transparent medicine is not detected as a medicine area.

[0084] The drug recognition unit 2 extracts the drug area detected in S12 from the front illumination image G4 (S13). In S13, for example, a mask image in which the area other than the drug area in the front illumination image G4 is masked becomes an image in which the drug area has been extracted. The drug is recognized by matching this extracted image of the drug area with a master image. For example, a drug having a master image that matches the drug area is recognized as the drug in that drug area.

[0085] (Example of drug recognition processing: False positive countermeasures) The medicine recognition unit 2 may perform a medicine recognition process after performing edge enhancement processing to at least one of the back-illuminated image and the front-illuminated image to make edges more prominent. Such preprocessing increases the possibility of erroneously detecting printed areas or other non-medicine areas as medicine areas. Therefore, by performing at least one of the following three processes, the possibility of erroneous detection can be reduced.

[0086] (First process) The drug recognition unit 2 can determine whether a detection error has occurred based on the number of dots in the area where the drug in the drug region matches. For example, the backlight 41B may be configured to emit dot-shaped light. In this case, a dot-shaped pattern appears in the area of ​​the backlight image where light from the backlight 41B that has passed through the drug reaches. Here, the dot-shaped pattern refers to a pattern in which circular dots sufficiently smaller than the drug are arranged regularly, i.e., at a predetermined pitch. The light source 41B that emits the dot-shaped light may include, for example, a light source and a plate that transmits the light from the light source in a dot-shaped pattern. Alternatively, the light source 41B may include light-emitting diodes arranged in a dot matrix. For example, the backlight means having a dot-shaped illumination area described in Japanese Patent No. 5196337 (see FIG. 24, paragraphs 0248 and 0249) can be used as the light source 41B.

[0087] Figure 8 shows an example of a printed image G6 and a transparent medicine image G7 in a backlighting image using backlighting, which emits dot-like light. Because light from behind the transparent medicine penetrates the transparent medicine, a black area F1 remains at the outline of the transparent medicine, similar to the outline of the printed portion "B." This can lead to the print area being mistakenly recognized as a medicine area during medicine area detection. The inventors noted that while dots are difficult to clearly see in the image of transparent medicine, especially dark transparent medicine, dots are clearly visible in areas of fuzzy letters like "B" and "O." Misdetection can be avoided by determining a medicine area as a text area rather than a medicine area if a predetermined number of dots or more are detected in the medicine area. As shown in Figure 9, the number of dot features in the backlighting image corresponding to the portion R1 that matches the medicine in the medicine area in the frontlighting image G8 is counted. If a predetermined number of dot features are detected, it can be determined that the print has been misdetected.

[0088] (Second process) Some types of packaging paper for packaging medicines are transparent on both sides, while others have at least a portion of one side colored. For example, one side of the packaging paper may include a white band or solid white area, with a portion or front colored white for printing text information. In such cases, it is preferable to place the packaging paper (medicine package) with the colored side facing the backlighting image and the transparent side facing the frontlighting image. This results in a dark transparent medicine against the white background, resulting in a whiter, higher average gray value than an image in which the transparent medicine is illuminated against the transparent packaging paper background. Focusing on this point, the medicine recognition unit 2 can determine whether a detection error has occurred based on the average gray value of the area where the medicine matches. Figure 10 shows an example of an image G9 of printed text and an image G10 of a transparent medicine in a frontlighting image. As shown in Figure 10, the color of the transparent medicine and the solid white portion of the printed text (white-out area) are significantly different. The solid white part of the print is closer to white than the transparent medicine, meaning that the average gray value is higher. If the average gray value (average brightness value on the grayscale) of the part R1 in the medicine area that matches the medicine in the front illumination image is above a certain level, it can be determined that the print has been misdetected.

[0089] (Third Processing) The drug recognition unit 2 can determine whether a detection error has occurred based on the overlap between the drug region and the drug-matching region. In particular, it can determine whether a drug is a foreign object or not for black or nearly black drugs. Figure 11 shows an example of a drug region R5 and a drug-matching region R4 in a front-illuminated image G11. In the example of Figure 11, due to preprocessing to emphasize edges, a false detection occurs in which another edge is detected on top of the other drug. This is because not only the edge of the correct outline of the black drug but also an edge is detected in the text or engraved information on the surface of the drug, leading to the false detection that another drug is present. In other words, it is falsely detected as if another tablet of another drug is placed on top of the black drug. To avoid this false detection, the drug recognition unit 2 can determine a false detection if the proportion of the overlap between the drug region and the drug-matching region in the drug region is less than a threshold. As a specific example, the drug recognition unit 2 compares the overlapping portion between the drug area (where the drug is present, area R5 in the example of FIG. 11) extracted from the rear image and the area matching the drug master image in the front image (area R4 in the example of FIG. 11) with the drug area (R5). If the ratio of this overlapping portion to the drug area is smaller than a threshold, the drug recognition unit 2 determines that the detection is erroneous.

[0090] (Automatic backlight adjustment) The shutter speed of the photographing device 21 is adjusted to an appropriate shutter speed by repeatedly photographing the screen of the conveying device 22 or the medicine package with a mark printed thereon while the backlight is on. If the light intensity of the backlight decreases, this affects the accuracy of detecting medicines and marks. However, because the shutter speed is not adjusted frequently, the detection of a decrease in light intensity may be delayed. Therefore, the medicine inspection system 10 can detect a decrease in the light intensity of the backlight by photographing the screen of the conveying device 22 while automatically turning on the backlight during times when inspections are not being performed. When the medicine inspection system 10 detects a decrease in light intensity, it can adjust the shutter speed and change the setting to a shutter speed appropriate for the decreased light intensity.

[0091] The medicine inspection system 10 may adjust the shutter speed for each packaging machine that outputs the next prescription mark, and set (record) a different shutter speed for each packaging machine. This allows each group of medicine packets from multiple packaging machines to be photographed at an appropriate shutter speed. The shutter speed adjustment may be performed, for example, at the start of the inspection process or when a decrease in the light intensity of the backlight is detected.

[0092] (Marking control) The inspection device may be equipped with a marking unit that marks the surfaces of the group of medicine packets transported by the transport device 22 with a pen. The marking unit has a pen and an actuator that controls the position of the pen. The medicine inspection system 10 controls the marking unit. For example, the interval and number of strokes at which the pen marks the medicine packets are controlled. As an example, the medicine inspection system 10 can control the number of pen strokes for one medicine packet. This makes it possible to mark a single pen to convey multiple meanings. In other words, the information displayed by the marking can be varied. For example, for medicines that require special emphasis and visual inspection, marking with three strokes makes it possible to differentiate it from marking with one stroke.

[0093] (Output of number of medicine packets when packaging is interrupted) When packaging based on one prescription is interrupted before completion, medicine inspection system 10 can photograph the group of medicine packets that have been packaged to some extent, record the medicine packet images, and output interrupted prescription information including the number of medicine packets packaged up until the interruption. The packaging interruption may be notified to medicine inspection system 10 from the packaging machine, for example. Alternatively, medicine inspection system 10 may determine whether packaging has been interrupted by comparing the number of medicine packets for which image acquisition unit 1 has acquired medicine packet images for one prescription with the number of medicine packets indicated by the prescription data for that prescription.

[0094] The output interrupted prescription information may include, for example, prescription identification information that identifies the prescription and interruption point information that indicates the number of drug packets packaged before packaging was interrupted, i.e., the number of packets at which packaging was interrupted. The output format of the interrupted prescription information is not particularly limited, and for example, a paper with a code (e.g., a barcode or two-dimensional code) indicating the interrupted prescription information printed thereon may be output from a journal printer. In this case, by reading the code with a reader possessed by the medicine inspection system 10, the medicine inspection system 10 can execute inspection processing from the interrupted point of the prescription where packaging was interrupted. This reduces the time required to redo the work when packaging based on a prescription is interrupted due to a shortage of medicine, etc.

[0095] (User interface for changing settings) The medicine inspection system 10 can change settings related to the control or inspection process of the inspection device in response to user input. If the device's operation method changes frequently, the user's task of changing the settings becomes a heavy burden. Furthermore, if multiple settings need to be changed to change the operation, mistakes are more likely to occur when changing the settings. Therefore, the medicine inspection system 10 may be provided with at least one of the following interfaces for changing settings. An additional button for changing settings may be provided. For example, when the setting change button is pressed, the settings of the medicine inspection system 10 switch to a pre-registered operation setting. Alternatively, when the setting change button is pressed, a code (e.g., a barcode or two-dimensional code) may be output to a journal printer or on a screen. The code includes setting information. When the code is read by a reader provided in the inspection device, the settings switch to the pre-registered operation setting. The reader may be, for example, a code reader mounted on the medicine package insertion side of the transport device 22.

[0096] Alternatively, user-specific operational settings may be recorded for each user recognized by the medicine inspection system 10. For example, when the medicine inspection system 10 recognizes a user, the settings can be automatically switched to settings specific to the recognized user. For example, the user is recognized by the user's input login ID. Alternatively, the medicine inspection system 10 may automatically switch settings according to a predetermined schedule. For example, operation settings corresponding to a predetermined time period may be recorded. The settings are automatically switched to those corresponding to the predetermined time period.

[0097] As an example of operation settings, the prescription determination unit 3 may be configured to switch between a number mode, which determines whether the number of drugs is as prescribed, and a verification mode, which determines whether the drug type is as prescribed. For example, the settings may be automatically changed so that the number mode is used during the daytime when the dispenser and inspector are different, and the verification mode is used during the nighttime when the dispenser and inspector are the same person.

[0098] (Remote inspection function) In the drug inspection system 10, the inspection device may include, as the display 23, multiple displays that are co-located with the inspection device. The multiple displays are positioned so that they can be seen by a user who directly operates the inspection device. The screen of a first display among the multiple displays is referenced from a remote terminal via remote desktop. The screen of this display is transmitted to the remote terminal. When the screen of a display is transmitted to the remote terminal and displayed on the remote terminal, the display may not be displayed. For example, when the display is connected to the remote terminal via VPN, the screen is not displayed while the screen is displayed on the remote terminal. In this case, a user interface for operating the inspection device can be provided on the second display. This allows a user to access the inspection device from a remote terminal and perform an inspection remotely, while another user is operating the inspection device at the same time.

[0099] The user interface provided on the second display may be a user interface that enables at least the minimum necessary display or operation, such as a display regarding an error or operation and / or an operation for recovery or setting in the event of an error. Examples of displays regarding an error or operation on the second display include a warning display regarding the setting status of the marking pen unit, a warning display that the medicine packet is inserted twisted, a warning display that the medicine packet is stuck in the guide that receives the medicine packet paper after inspection, a warning display that the master image for verifying the medicine to be inspected has not been registered, a warning display when the inspection has already been completed after being paused and then resumed, a confirmation display regarding whether to maintain or change the current state when the marking unit can be set to use / not use, an error display when the medicine packet to be inspected cannot be firmly secured, an error display when the perforation at the connection between the medicine packet papers in the medicine packet cannot be detected, and an indication that prescription information is being received.

[0100] For example, the first display can be a display attached to the housing that houses the inspection device's camera and transport device, and the second display can be an external display independent of the housing, which allows the second display to be introduced at low cost.

[0101] The above-described embodiment is an example of the present invention, and the present invention is not limited to this example. Furthermore, the configurations and processing functions described in the above-described embodiment can be selected and combined as desired. [Explanation of symbols]

[0102] 1: Image acquisition unit, 2: Medicine recognition unit, 3: Prescription judgment unit, 4: Output unit, 5: Medicine extraction and display unit, 6: Master management unit, 10: Medicine inspection system, 21: Photography device, 22: Transport device, P: Medicine package group

Claims

1. an image acquisition unit that acquires medicine package images by photographing each of a plurality of medicine packages into which medicines are individually packaged based on a prescription; an inspection unit that inspects whether or not the medicine in each of the plurality of medicine packages is as prescribed based on the medicine package image of each of the plurality of medicine packages; an output unit that outputs the inspection result of the inspection unit; a medicine extraction display unit that extracts and displays a medicine package image containing a specified medicine from the medicine package images of the plurality of medicine packages.

2. The medicine inspection system according to claim 1, The multiple medicine packets are a group of medicine packets in which medicines are individually packaged based on multiple prescriptions, The drug extraction display unit extracts and displays a drug package image including a specified drug from the drug package images of the multiple drug packages in which drugs are packaged based on the multiple prescriptions.

3. The medicine inspection system according to claim 1 or 2, The plurality of medicine packets include medicine packets in which medicines manually dispensed in a packaging machine are packaged, The medicine extraction display unit extracts and displays a medicine package image including the manually dispensed medicine from the medicine package images of the plurality of medicine packages.

4. The medicine inspection system according to claim 1 or 2, The medicine extraction display unit displays the extracted medicine package image on one screen.

5. The medicine inspection system according to claim 1 or 2, A drug inspection system in which the drug extraction display unit displays each of the extracted and displayed drug package images on the same screen as information on the drugs to be contained in the drug package of each drug package image indicated in the prescription.

6. The medicine inspection system according to claim 1 or 2, The inspection unit a medicine recognition unit that recognizes the medicine included in the medicine package image using master data of the medicine; a prescription determination unit that determines whether the medicine recognized by the medicine recognition unit is as prescribed, The drug inspection system further includes a master change reception unit that displays a list of candidate drugs for change that require a change to be made to the master image of the drug due to a change in appearance, along with the changed master image of each candidate drug, among the drugs registered in the master data, and receives input of instructions to change the master image of each candidate drug for change.

7. an image acquisition process for acquiring medicine package images by photographing each of a plurality of medicine packages into which medicines are individually packaged based on a prescription; an inspection process for inspecting whether or not the medicine in each of the plurality of medicine packages is as prescribed based on the medicine package image of each of the plurality of medicine packages; an output process for outputting the inspection results of the inspection process; A medicine extraction and display process for extracting and displaying a medicine package image containing a specified medicine from the medicine package images of the plurality of medicine packages, and a medicine inspection program for causing a computer to execute the process.

Images