Type discrimination device, type discrimination method, and type discrimination program

Abstract

A computer (60) comprises a feature extraction unit (64) and a discrimination unit (65). The feature extraction unit generates, on the basis of an output value obtained by inputting, to a learnt model (84) that has been constructed so as to extract a mark formed on a drug, a captured image (82) obtained by capturing a target drug the type of which is unknown, an extracted mark image (83) in which a mark appearing in the captured image is extracted. The discrimination unit discriminates the type of the target drug on the basis of the matching result between the extracted mark image and a registered mark image registered in advance for each drug type.

Description

[Technical Field]

[0001] The present invention relates to a type discrimination device that discriminates the type of medicine using an image of the medicine, and also to a medicine sorting device that sorts medicines by type. [Background technology]

[0002] Conventionally, multiple types of returned medications have been sorted by type by pharmacists or doctors. The returned medications are either prescribed to various patients or have been dispensed. Therefore, compared to the dispensing task of assembling (packaging) (one or more types) of medications (tablets) for each dosing period from groups of medications (medication cassettes) that have been pre-assembled by medication type in dispensing equipment, etc., based on prescription information for each patient, the number of types of medications that are returned together, including medications prescribed for multiple patients, is extremely large. Therefore, it is highly useful to automatically sort and reuse returned medications.

[0003] In the automatic sorting of medicines, it is necessary to identify the type of medicine from the image of the medicine, and one of the main clues for this type identification is the mark formed on the medicine. The mark can be engraved or printed. The content of the mark also varies, such as figures, symbols, letters, and numbers.

[0004] Patent Document 2 discloses a medicine sorting device that realizes automatic medicine sorting. The medicine sorting device in Patent Document 2 captures images of multiple types of medicine stored in a first storage unit one by one, determines the type of medicine based on the captured image, and sorts the medicines by type into second storage units based on the determination results. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2020-52627 [Patent Document 2] International Publication No. 2018 / 190394 Summary of the Invention [Problem to be solved by the invention]

[0006] It is difficult to accurately identify the type of medicine based on the mark because, as mentioned above, the content of the mark varies and the appearance of the mark in the captured image changes depending on the way the medicine is illuminated during imaging.

[0007] Image processing techniques using machine learning have been known for some time. For example, Patent Document 1 above describes the use of trained models in the medical field, such as for supporting case diagnosis using images. More specifically, Patent Document 1 discloses a verification system that determines the validity of a discrimination result output from an output layer of a trained model in response to medical information being input to the input layer of the trained model, based on information about the calculation results stored in a storage unit and the calculation results output from an intermediate layer of the trained model. This verification system makes it possible to verify the validity of the discrimination result.

[0008] Applying a pre-trained model to drug type discrimination is expected to improve discrimination accuracy. However, when using a pre-trained model to discriminate drug types, there is a problem in that it is difficult to explain the basis for the discrimination results, in other words, it is difficult to present evidence of the discrimination results. For example, when an image of a drug that has not been trained is input into a pre-trained model to discriminate drug types, it is difficult to predict what discrimination results will be output. Furthermore, if the discrimination result is incorrect, evidence of the discrimination result is required in the medical field. However, when using a pre-trained model such as the one described above, it is difficult to present evidence.

[0009] Furthermore, as mentioned above, the content of marks formed on medicines is diverse, so there is also the problem that building a trained model that has learned all of them requires significant costs in terms of money, manpower, time, etc.

[0010] One aspect of the present invention aims to realize a type discrimination device that discriminates the type of medicine, which enables the presentation of evidence of the discrimination results and is capable of handling a variety of marks.

[0011] Furthermore, the medicine sorting device of Patent Document 2 does not limit the number of types of medicines that can be stored in the first storage unit. On the other hand, the number of types of medicines that can be stored in the second storage unit is limited. Therefore, if the first storage unit contains more types of medicines than the second storage unit can contain, some medicines will not be able to be stored in the second storage unit even if their types are identified. The medicine sorting device of Patent Document 2 temporarily waits for medicines that cannot be stored in the second storage unit in a standby position (standby tray) different from the second storage unit before sorting them into the second storage unit.

[0012] In the medicine sorting device of Patent Document 2, after all medicines stored in the first storage unit are transported to the second storage unit or the standby position, the medicines are removed from the second storage unit, for example, by packaging the medicines in the second storage unit. The medicine sorting device of Patent Document 2 then re-identifies the types of medicines that were waiting in the standby position and stores them in the second storage unit by type. If more medicines than the second storage unit can store are waiting in the standby position, there will again be medicines that cannot be stored in the second storage unit even if their types have been identified. In this case, the medicine sorting device of Patent Document 2 will once again have the medicines wait in the standby position (first storage unit or standby tray).

[0013] In this way, when the drug sorting device of Patent Document 2 waits for drugs at the waiting position, it determines the type of drug and repeatedly stores the drugs by type in the second storage section based on the results of the determination until all the drugs waiting at the waiting position are used up.

[0014] Another aspect of the present invention aims to provide a medicine sorting device that can reduce the processing time that occurs when medicines are kept waiting at a waiting position. [Means for solving the problem]

[0015] In order to solve the above problem, a type discrimination device according to one embodiment of the present invention comprises an image generation unit that generates an extracted mark image by extracting a mark that appears in an image of an unknown type of target drug based on an output value obtained by inputting an image of the target drug into a trained model constructed to extract marks formed on the drug, and a discrimination unit that discriminates the type of the target drug based on the result of comparing the extracted mark image generated by the image generation unit with a registered mark image that is registered in advance for each type of drug.

[0016] In order to solve the above problem, a type discrimination method according to one aspect of the present invention is a type discrimination method executed by a type discrimination device, and includes an image generation step of generating an extracted mark image by extracting a mark appearing in an image of a target drug of unknown type based on an output value obtained by inputting an image of the target drug of unknown type into a trained model constructed to extract marks formed on the drug, and a discrimination step of discriminating the type of the target drug based on the result of comparing the extracted mark image generated in the image generation step with a registered mark image registered in advance for each type of drug.

[0017] In order to solve the above problem, a type discrimination program according to one embodiment of the present invention causes a computer to execute an image generation step in which an image of an unknown type of target drug is input into a trained model constructed to extract marks formed on the drug, and based on the output value obtained, an extracted mark image is generated by extracting a mark appearing in the captured image; and a discrimination step in which the type of the target drug is discriminated based on the result of comparing the extracted mark image generated in the image generation step with a registered mark image registered in advance for each type of drug.

[0018] In order to solve the other problem described above, a drug sorting device according to one embodiment of the present invention comprises: a first storage unit having a plurality of first compartments and capable of storing a plurality of types of drugs; a second storage unit having a plurality of second compartments and capable of storing the drugs by type in each of the second compartments; a reading unit that reads information indicating the types of drugs stored in the first storage unit; a counting unit that counts the number of types of drugs stored in the first storage unit based on the information read by the reading unit; an alarm unit that issues an alarm to change the first compartment in which the drugs are to be stored each time the total number of types of drugs counted by the counting unit reaches the number of the second compartments; and a transport unit that transports the drugs stored in the first storage unit to the second storage unit for each of the first compartments and stores the drugs by type in the second storage unit. [Effects of the Invention]

[0019] According to a type determination device, a type determination method, and a type determination program according to an aspect of the present invention, it is possible to present evidence of the determination result and to determine the type based on various marks.

[0020] Furthermore, according to the medicine sorting device according to another aspect of the present invention, it is possible to reduce the processing time that occurs when medicines are kept waiting at the waiting position. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a block diagram showing the overall configuration of a medicine sorting device. [Figure 2] FIG. 1 is a diagram illustrating an example of the configuration of a medicine sorting device. [Figure 3] 1A and 1B are perspective views showing the overall configuration of an imaging unit and an example of a medicine placing table, respectively. [Figure 4] FIG. 10 is a diagram for explaining the rotation of the imaging unit. [Figure 5] 1A and 1B are diagrams showing an example of a captured image and an example of an extraction mark image extracted from the captured image; [Figure 6] 10A and 10B are diagrams showing another example of a captured image and another example of an extraction mark image extracted from the captured image. [Figure 7]10 is a flowchart showing an example of the flow of a determination process. [Figure 8] FIG. 10 is a diagram illustrating an example of training data. [Figure 9] 10A and 10B are diagrams for explaining the packaging operation of the packaging mechanism when packaging normal medicines. [Figure 10] 10A and 10B are diagrams for explaining the packaging operation of the packaging mechanism when packaging medicines that require caution when heated. [Figure 11] 10A to 10C are diagrams showing examples of packaging by the packaging mechanism, examples of printing, and examples of cutting of packaging sheets. [Figure 12] FIG. 1 is a diagram illustrating an example of the configuration of a medicine dispensing system. [Figure 13] FIG. 10 is a diagram showing a state in which a two-dimensional code is printed on a medicine packet packaged by a packaging machine. [Figure 14] 1 is a block diagram showing the overall configuration of a medicine sorting device. [Figure 15] 10 is a flowchart showing an example of a process flow when a medicine is introduced into a first container. [Figure 16] 10 is a flowchart showing an example of the flow of a medicine sorting process. [Figure 17] 10A and 10B are diagrams for explaining a method of capturing an image of the inside of the sorting cup. DETAILED DESCRIPTION OF THE INVENTION

[0022] [Embodiment 1] [Overview of the medicine sorting device 1] First, an overview of the medicine sorting device 1 will be described using Figures 1 and 2. Figure 1 is a block diagram showing the overall configuration of the medicine sorting device 1. Figure 2 is a diagram showing an example of the configuration of the medicine sorting device 1, with 2001 being a perspective view of the medicine sorting device 1 and 2002 being a perspective view showing the basic configuration of the medicine sorting area 2. As shown in Figure 1 and 2001 and 2002 of Figure 2, the medicine sorting device 1 includes the medicine sorting area 2, a touch panel 3, a print output unit 4, and a packaging mechanism 6.

[0023] The medicine sorting device 1 takes an image of each of multiple types of medicine, determines the type of medicine based on the image obtained as a result of the image capture, and sorts the medicines by type. Specifically, this process is performed in the medicine sorting area 2. The medicine sorting area 2 (the internal configuration of the medicine sorting device 1) will be described later. After the medicines sorted by type are visually inspected by the user, they are packaged or returned to the medicine shelf or packaging machine.

[0024] In this embodiment, the multiple types of medicines are medicines that are not contained in a container or are not packaged, and examples of such medicines are tablets or capsules. The multiple types of medicines are also described as returned medicines. These medicines are medicines that are used by a pharmacy or hospital and are returned to the pharmacy or hospital as "returned medicines." However, the multiple types of medicines may also be "brought-in medicines" that may include medicines issued by other pharmacies or hospitals in addition to the used medicines. The medicine sorting device 1 can automatically perform processes from imaging to sorting after the medicines are returned.

[0025] The touch panel 3 accepts various user inputs at the operation unit 31, and displays various images on the display unit 32 (for example, an image showing the progress of drug sorting, an image for visual inspection).

[0026] The print output unit 4 prints a journal representing drug data (e.g., data indicating the drug name, manufacturer, or ingredients) related to the drug after visual inspection according to user input after visual inspection. The drug data may include image data representing a drug-specific image.

[0027] The packaging mechanism 6 packages the sorted medicines. The packaging mechanism 6 is an optional mechanism. When the packaging mechanism 6 is provided in the medicine sorting device 1, the medicine sorting device 1 can perform all processes from sorting returned medicines to packaging after visual inspection all at once. In particular, when medicines are fed into the packaging mechanism 6 by the transport / sorting unit 12, the processes from sorting to packaging can be performed automatically, except for visual inspection.

[0028] A conventional packaging unit of a tablet packaging machine or powder packaging machine can be used as the packaging mechanism 6. In this case, for example, drugs in a sorting cup 141 sorted for each drug type can be packaged into one or more packages.

[0029] It should be noted that throughout this specification, the term "packaging," as used at least in the description of the drug sorting device 1, includes both the meaning of "packaging drugs according to the time of administration based on prescription data" and the meaning of "simply packaging the drugs sorted into the second storage section 14 regardless of the prescription data."

[0030] The medicine sorting device 1 also includes a first RFID (Radio Frequency Identifier) ​​reader / writer unit 5. The first RFID reader / writer unit 5 is provided on the medicine removal side of the base 19, as shown by 2002 in FIG.

[0031] The first RFID reader / writer unit 5 reads data related to the medicines stored in each sorting cup 141, which is stored in an RFID tag (not shown) provided at the bottom of each sorting cup 141 in the second storage section 14. This data includes, for example, data indicating the number of medicines stored, medicine identification data (e.g., GS1 code) for identifying the medicines, data indicating the type of sorting cup 141, and sorting cup identification information (identification information assigned to the RFID tag) for identifying the sorting cup 141. Note that the medicine data (e.g., medicine name) of the medicines stored in the sorting cup 141, image data acquired by the imaging unit 13, etc. are stored in the storage section 80 in association with the sorting cup identification information.

[0032] The data may also include drug data determined by visual inspection (drug data after visual inspection). The drug data after visual inspection may also be written to the RFID tag. The drug data after visual inspection is used when the drugs stored in the corresponding sorting cup 141 are (1) packaged by the packaging mechanism 6 or a packaging machine other than the drug sorting device 1, or (2) returned to the drug shelf. The drug data may also be linked to the sorting cup identification information and stored in the memory unit 80.

[0033] As shown in 2001 in FIG. 2, the medicine sorting device 1 is also provided with an opening / closing shutter 51 and an opening / closing door 52 that enable the medicine take-out side to be opened and closed.

[0034] [Basic configuration of drug sorting area 2] Next, the basic configuration of the medicine sorting area 2 (internal configuration of the medicine sorting device 1) will be described with reference to FIG. 1 and 2002 in FIG.

[0035] 1 and 2, the medicine sorting area 2 mainly comprises, as hardware, a first storage section 11, a transport / sorting unit 12 (sorting section), an imaging unit 13, a second storage section 14, a waiting tray 15, a collection tray 16, a medicine insertion port 17, and a second RFID reader / writer unit 18. Each component except for the transport / sorting unit 12 is mounted on a base 19. The main functions of the transport / sorting unit 12, the imaging unit 13, and the second RFID reader / writer unit 18 will be described in detail in the description of each process below.

[0036] The first storage unit 11 stores a mixture of multiple types of medicines returned by users. In this embodiment, the first storage unit 11 is divided into multiple storage units. In this case, for example, when all of the medicines stored in one storage unit are transported by the transport / sorting unit 12, the medicines stored in a storage unit adjacent to that storage unit become the target for transport. The first storage unit 11 may also be rotatable about the Z axis (the center of the cylindrical shape). In this case, the control unit 60a of the computer 60 (type determination device) may rotate the first storage unit 11, for example, when one storage unit becomes empty, so that the transport / sorting unit 12 can easily retrieve the medicines.

[0037] The second storage section 14 includes a plurality of sorting cups 141 that store medicines sorted by type. The control section 60a determines the type of medicine based on an image of the medicine captured by the imaging unit 13, and determines the sorting cup 141 in which to store the medicine based on the determination result. The medicine is transported to the determined sorting cup 141 by the transporting / sorting unit 12 and stored therein.

[0038] The standby tray 15 is a temporary storage unit where medicines are temporarily placed. For example, when all of the sorting cups 141 are filled with medicines, medicines determined by the control unit 60a to be of a type other than those medicines are temporarily placed on the standby tray 15. In this case, after the medicines are removed from the sorting cups 141, they may be transported from the standby tray 15 to the sorting cups 141.

[0039] In this embodiment, a suspected drug (described later) that is suspected to be a drug may be temporarily placed on the waiting tray 15. When the suspected drug is temporarily placed, the suspected drug is transported from the waiting tray 15 to a predetermined area of ​​the second storage unit 14, for example, according to the determination result of the control unit 60a.

[0040] The collection tray 16 is a storage unit that stores objects whose type cannot be determined by the control unit 60a (e.g., foreign matter other than medicine). Examples of foreign matter other than medicine include pieces of PTP (Press Through Pack) sheets. Pieces of PTP sheets may get mixed into the first storage unit 11 when medicines are returned. The control unit 60a also stores in the collection tray 16 medicines that are registered in the medicine database 81 as medicines to be discarded or medicines that the user wishes to discard (e.g., medicines with old manufacturing dates).

[0041] When the medicine sorting device 1 is equipped with a packaging mechanism 6, the medicine input port 17 is used to transport medicines stored in the second storage section 14 to the packaging mechanism 6 by the transport / sorting unit 12. Naturally, when the medicine sorting device 1 is not equipped with a packaging mechanism 6, the medicine input port 17 is not required.

[0042] As shown in FIG. 1 , the medicine sorting device 1 also includes a computer 60 that controls the above-mentioned components (hardware). The computer 60 includes a control unit 60a and a storage unit 80. The control unit 60a includes a transport control unit 61, a sorting control unit 62, an imaging control unit 63, a feature extraction unit 64, a discrimination unit 65, an operation input unit 66, a display control unit 67, an RFID control unit 68, a printout control unit 69, a registration unit 70, and a packaging control unit 71. Here, the basic processing of the operation input unit 66, the display control unit 67, the RFID control unit 68, the printout control unit 69, and the packaging control unit 71 will be described. The basic processing of the transport control unit 61, the sorting control unit 62, the imaging control unit 63, the feature extraction unit 64, and the discrimination unit 65 will be described in detail in the description of each process below. Other processing will be described in each embodiment below.

[0043] The operation input unit 66 and the display control unit 67 respectively control the operation unit 31 and the display unit 32 of the touch panel 3. The RFID control unit 68 controls the first RFID reader / writer unit 5 and the second RFID reader / writer unit 18. The printout control unit 69 controls the printout unit 4 in accordance with the user input received by the operation input unit 66.

[0044] The registration unit 70 registers the drug data relating to a drug for which the discrimination unit 65 has determined that no drug data corresponding to the image data exists in the drug database 81. Specifically, for a drug for which the discrimination unit 65 has determined that no corresponding drug data exists, the registration unit 70 links the captured image 82 of the drug with the drug data specified by the user and registers the image in the drug database 81.

[0045] When the medicine sorting device 1 is equipped with a packaging mechanism 6, the control unit 60a is equipped with a packaging control unit 71 that controls the packaging mechanism 6. The packaging control unit 71 controls the operation of the packaging mechanism 6. The packaging control unit 71 also controls the transporting and sorting unit 12 to transport the medicines stored in the sorting cups 141 to the medicine inlet 17.

[0046] The medicine sorting device 1 may also include a barcode reader 7. The barcode reader 7 reads, for example, a barcode indicating medicine data of the medicine printed on a medicine packet (packaging paper) in which the packaging mechanism 6 packages the medicines after sorting. This allows the control unit 60a to perform processing based on the read barcode (e.g., displaying the medicine data indicated by the barcode). The packaging mechanism 6 may also include a barcode printing mechanism (not shown) that prints, on the medicine packet, a barcode indicating medicine data of the medicine packaged in the medicine packet.

[0047] The barcode reader 7 may be any reading device capable of reading information indicating drug data printed on a medicine packet or the like. The drug sorting device 1 does not necessarily need to be equipped with a barcode reader 7. When the control unit 60a performs processing based on the above information, the control unit 60a may acquire the above information by communicating with an external device equipped with the barcode reader 7.

[0048] The computer 60 also includes a storage unit 80. The storage unit 80 pre-stores a drug database (drug master) 81 that manages drug data related to multiple types of drugs, and stores captured images 82 and the like as sorting is performed by the drug sorting device 1. The captured image 82 is an image captured by the first camera 131. Specifically, the captured image 82 is an image of an unknown type of target drug that has been removed from the first storage unit 11. The storage unit 80 also stores an extraction mark image 83 generated by the feature extraction unit 64, and a trained model 84 used by the feature extraction unit 64 to generate the extraction mark image 83.

[0049] The various data stored in the storage unit 80 may not necessarily be managed by the storage unit 80, but may be managed by, for example, an external device. In this case, the control unit 60a may acquire the various data from the external device via a communication line such as the Internet, as necessary. The drug database 81 may also be updated by adding new drug data.

[0050] [Overview of processing in medicine sorting device 1] In the medicine sorting device 1, the transporting / sorting unit 12 transports each medicine returned to the first storage section 11 to the imaging unit 13. The imaging unit 13 sequentially captures images of each transported medicine. The control unit 60a identifies the type of each medicine based on the captured images and determines a sorting position in the second storage section 14 for each identified medicine. The transporting / sorting unit 12 transports each medicine to the determined sorting position. Information about the medicines stored in the second storage section 14 is written to the RFID tag of the sorting cup 141, stored in the memory section 80, or displayed on the touch panel 3. After or during the sorting of the medicines, the user operates the touch panel 3 to perform processes such as visual inspection and packaging. Each process will be described in detail below.

[0051] [Drug Delivery Process to Imaging Unit 13] First, the process of transporting medicine from the first container 11 to the imaging unit 13 will be described with reference to FIG. 1 and 2001 in FIG.

[0052] Specifically, the conveying / sorting unit 12 conveys the medicines stored in the first storage section 11 to a receiving area Ar1 (see 3002 in FIG. 3) where the imaging unit 13 receives the medicines. The conveying control section 61 controls the conveying process by the conveying / sorting unit 12.

[0053] The transport and sorting unit 12 includes a second camera 121 , a suction and shutter mechanism 122 , and a transport mechanism 123 .

[0054] The second camera 121 sequentially captures images of the first storage unit 11 to identify the medicine to be transported. The imaging control unit 63 controls the imaging process of the second camera 121. The second camera 121 is provided at the end of the transport / sorting unit 12 (specifically, of the housing including at least the suction / shutter mechanism 122) on the side facing the base 19. The second camera 121 may be provided at the tip of the suction mechanism described below. The imaging control unit 63 analyzes the captured image and determines whether or not the image contains medicine. If it is determined that the image contains medicine, the transport control unit 61, for example, brings the tip closer to the first storage unit 11 and identifies the medicine contained in the image captured at that time as the medicine to be transported.

[0055] The suction / shutter mechanism 122 includes a suction mechanism that suctions the medicine identified as the delivery target, and a shutter mechanism that prevents the medicine suctioned by the suction mechanism from falling. The suction mechanism is provided so as to be movable in the Z-axis direction. The shutter mechanism is provided in front of the end so as to be movable substantially parallel to the XY plane.

[0056] When acquiring a medicine, the suction mechanism extends from the end, adsorbs the identified medicine at its tip, and then returns to the end. In this state, the transport control unit 61 moves the shutter mechanism to a position facing the end and maintains the shutter mechanism in this position (closed state) during medicine transport. The transport control unit 61 moves the suction / shutter mechanism 122 to a position facing the medicine placement stage 133a (see 3002 in Figure 3) of the medicine holding mechanism 133 arranged in the receiving area Ar1, and then moves the shutter mechanism to a position not facing the end (open state). Then, after extending the suction mechanism from the end, the suction state is released, thereby placing the medicine on the medicine placement stage 133a.

[0057] The transport mechanism 123 moves the suction / shutter mechanism 122 in the X-axis and Y-axis directions under the control of the transport control unit 61. This transport mechanism 123 enables the movement of the suction / shutter mechanism 122 when searching for a drug to be transported in the first container 11, or the transport of a drug from the first container 11 to the drug placement table 133a. Also, in the drug sorting process, it enables the transport of drugs from the drug placement table 133a to the second container 14, the waiting tray 15, or the collection tray 16. Note that in the drug sorting process, the sorting control unit 62 controls the transport / sorting unit 12 to transport the drugs placed in the receiving area Ar1 to a predetermined sorting cup 141 in the second container 14 or the waiting tray 15 based on the discrimination result by the discrimination unit 65.

[0058] [Drug Imaging Processing] Next, the medicine imaging process by the imaging unit 13 will be described with reference to FIGS. 1, 2002 in FIG. 2, 3, and 4. 3001 and 3002 in FIG. 3 are perspective views showing the overall configuration of the imaging unit 13, and 3003 in FIG. 3 is a perspective view showing an example of the medicine placing table 133a. 4001 and 4002 in FIG. 4 are diagrams for explaining the rotation of the imaging unit 13. The medicine imaging process is mainly performed by the imaging unit 13 and the imaging control unit 63.

[0059] Specifically, the imaging unit 13 is placed on the medicine placing table 133a and captures an image of a medicine placed in a placement area Ar2 (imaging area) where the medicine to be imaged is placed, as shown at 3002 in Fig. 3. The imaging control unit 63 controls the imaging process by the imaging unit 13, the pivoting movement of the first camera 131 and the illuminator 134, and the movement of the medicine holding mechanism 133. As shown in Figs. 1 and 3, the imaging unit 13 includes the first camera 131 (imaging section), a rotation mechanism 132 (rotating section), the medicine holding mechanism 133 (medicine placing table, moving mechanism), and the illuminator 134 (ultraviolet light irradiation section, visible light irradiation section).

[0060] The first camera 131 captures an image of a drug placed in a placement area Ar2 facing the first camera 131 in order to identify the type of drug in the discrimination unit 65, which will be described later. The drug holding mechanism 133 is a mechanism for holding a drug, and as shown by 3001 and 3002 in FIG. 3, includes a drug placing table (petri dish) 133a, a turning mechanism 133b (moving mechanism), and a shaft 133c connecting the drug placing table 133a and the turning mechanism 133b. The drug placing table 133a is used to place a drug to be imaged. The turning mechanism 133b moves the drug placing table 133a; specifically, it turns the drug placing table 133a relative to the XY plane and turns the shaft 133c in the circumferential direction of the shaft 133c.

[0061] When the medicine transported from the first container 11 is placed on the medicine placing table 133a, the imaging control unit 63 drives the turning mechanism 133b to move the medicine placing table 133a from the receiving area Ar1 to the placing area Ar2. Thereafter, the imaging control unit 63 controls at least the first camera 131 and the illuminator 134 to capture an image of the medicine placed in the placing area Ar2. The captured image is stored in the storage unit 80 as a captured image 82. For example, after the imaging is completed, the imaging control unit 63 drives the turning mechanism 133b to move the medicine placing table 133a on which the imaged medicine is placed from the placing area Ar2 to the receiving area Ar1.

[0062] In this embodiment, two medicine placing tables 133a are provided at the tip (end) of the shaft 133c. The turning mechanism 133b turns the shaft 133c, and when one medicine placing table 133a is placed in the placement area Ar2, the other medicine placing table 133a is placed in the receiving area Ar1. When imaging medicines in the placement area Ar2, the transport / sorting unit 12 transports medicines from the first container 11 to the medicine placing table 133a present in the receiving area Ar1, thereby enabling continuous imaging of the medicines. Note that it is assumed that the medicine placing table 133a does not have any medicines placed on it, for example, after the medicines have been sorted into the second container 14.

[0063] In this embodiment, the medicine placing table 133a is transparent, so that the first camera 131 can take images of the medicines placed on the medicine placing table 133a from multiple directions through the medicine placing table 133a.

[0064] 3, the medicine mounting table 133a may have a generally V-shaped cross section with a recessed bottom. As shown in 3003 of FIG. 3 and FIG. 4, when the medicine mounting table 133a is disposed in the receiving area Ar1 and the disposing area Ar2, the groove direction of the generally V-shaped cross section (extension direction of the shaft portion 133c) is generally parallel to the rotation axis Ay of the imaging mechanism (described later) of the rotation mechanism 132. The bottom of the medicine mounting table 133a does not have to be V-shaped with an acute angle. As shown in 3003 of FIG. 3, the bottom may include a bottom surface portion 133aa and inclined surface portions 133ab inclined from two opposing points of the bottom surface portion 133aa. The shape of the bottom portion need only be such that the information (engraved information or printed information) imprinted or printed on the medicine can be recognized when viewed (photographed) from the back of the medicine placing stand 133a, and that the medicine can be fixed in place.

[0065] If the medicine is a capsule or deformed tablet (e.g., rugby ball-shaped), if the bottom of the medicine mounting table 133a is flat, the medicine will not be aligned on the XY plane, which may make it difficult to obtain a clear image of the medicine (engraved information or printed information). If the cross section is approximately V-shaped, the capsule or deformed tablet fits into the lowest end, and the medicine can be fixed. This makes it easier to obtain a clear image of the medicine. In the case of a tablet, for example, the axis portion 133c may be rotated in the circumferential direction of the axis portion 133c, so that the flat portion (inclined surface portion 133ab) of the medicine mounting table 133a faces the first camera 131, thereby ensuring that the medicine does not move.

[0066] In addition, the rotation mechanism 133b can also vibrate (slightly move or shake) the medicine placing table 133a. In this case, for example, by vibrating and rolling a capsule placed on the medicine placing table 133a, the printed portion of the capsule can be oriented in a predetermined direction (for example, this portion can be made to face the first camera 131 placed in the initial position described below). Furthermore, by the vibration, even if a cylindrical tablet (with a circular bottom) is placed upright on the flat surface, the tablet can be turned sideways (positioned so that the bottom of the tablet faces the flat surface).

[0067] The illuminator 134 emits light to be irradiated onto the medicine when imaging the medicine under the control of the imaging control unit 63. As shown in 3001 in Fig. 3, the illuminator 134 includes a visible light irradiating unit (first irradiating unit 134a and second irradiating unit 134b) that irradiates the medicine with visible light, and an ultraviolet light irradiating unit 134c that irradiates the medicine with ultraviolet light.

[0068] The first irradiating unit 134a and the second irradiating unit 134b irradiate the medicine with white light as visible light. The first irradiating unit 134a is a bar-shaped visible light source (bar illumination), and the second irradiating unit 134b is a ring-shaped visible light source (ring illumination). The first camera 131 receives the visible light emitted from the first irradiating unit 134a or the second irradiating unit 134b and reflected by the medicine, thereby acquiring an image based on the visible light (visible light image). The imaging control unit 63 stores image data representing the visible light image acquired by the first camera 131 in the storage unit 80 as an acquired image 82.

[0069] The ultraviolet light irradiation unit 134c irradiates the drug with ultraviolet light (e.g., light having a peak wavelength of 365 nm or more and 410 nm or less), thereby exciting the components contained in the drug. This causes fluorescence (e.g., light having a peak wavelength of 410 nm or more and 800 nm or less) to be extracted from the drug. The first camera 131 receives the fluorescence emitted from the drug and acquires an image based on the ultraviolet light (ultraviolet light image). The imaging control unit 63 stores image data representing the ultraviolet light image acquired by the first camera 131 in the memory unit 80 as an acquired image 82.

[0070] As shown in FIGS. 3 and 4, the rotation mechanism 132 rotates the first camera 131 so as to revolve around the arrangement area Ar2 (the medicine placing table 133a arranged at that position) where the medicine to be imaged is arranged. The first camera 131 images the medicine arranged in the arrangement area Ar2 from multiple positions to which the rotation mechanism 132 has rotated it. Specifically, the imaging mechanism including the first camera 131 and the illuminator 134 is rotated so as to revolve around the arrangement area Ar2. Therefore, the first camera 131 can image the medicine from multiple directions while maintaining the positional relationship of the first camera 131 and the illuminator 134 with respect to the arrangement area Ar2.

[0071] As shown in FIG. 3 at 3001, the rotation mechanism 132 includes an imaging mechanism drive unit 132a and a power transmission mechanism 132b. The imaging mechanism drive unit 132a generates power for rotating the imaging mechanism around the placement area Ar2. The power transmission mechanism 132b transmits the power generated by the imaging mechanism drive unit 132a to the imaging mechanism. The imaging mechanism drive unit 132a is driven under the control of the imaging control unit 63 to change the position of the imaging mechanism around the placement area Ar2.

[0072] The rotation mechanism 132 rotates the imaging mechanism between an initial position and a position opposite to the initial position. The initial position is a position approximately perpendicular to the placement area Ar2 and above the placement area Ar2. The position opposite to the initial position is a position approximately perpendicular to the placement area Ar2 and below the placement area Ar2. This position can also be said to be a position where the first camera 131 faces the bottom of the medicine placing table 133a located in the placement area Ar2.

[0073] As shown in FIG. 4, the axis that passes through the center of the placement area Ar2 and is parallel to the Z axis is defined as axis Ax0, and the axis that passes through the center of the placement area Ar2 and the center of the imaging mechanism is defined as axis Ax1. The angle between axis Ax0 and axis Ax1 is defined as θ. In this embodiment, the rotation mechanism 132 positions the imaging mechanism at one of θ=0° (initial position), 45°, 135°, and 180°. Reference numeral 4001 in FIG. 4 indicates a case where the imaging mechanism is at θ=0°, and reference numeral 4002 in FIG. 4 indicates a case where the imaging mechanism has rotated from the initial position to a position of θ=45°.

[0074] In this way, by rotating the imaging mechanism around the placement area Ar2, it is possible to image the medicine from multiple directions while the medicine is fixed in the placement area Ar2. Furthermore, even if the medicine (tablet) stands upright after shaking the medicine placing table 133a, information indicated by the markings or the like affixed to the medicine can be obtained by imaging from an oblique direction (θ=45° or 135°).

[0075] It is also possible to fix the imaging mechanism and rotate the drug, thereby capturing images of the drug from multiple directions.

[0076] (imaging position control) Next, an example of position control of the imaging mechanism will be described. The imaging control unit 63 first sets the imaging mechanism to an initial position, and then directs the first camera 131 to the initial position. to The first camera 131 captures an image of the medicine placed in the placement area Ar2 in the first irradiation unit 134a. At this time, the first camera 131 acquires visible light images (two visible light images) based on the visible light from the first irradiation unit 134a and the second irradiation unit 134b, and also acquires an ultraviolet light image based on the ultraviolet light from the ultraviolet light irradiation unit 134c.

[0077] Next, the imaging control unit 63 sets the imaging mechanism to a position opposite to the initial position and causes the first camera 131 to capture an image of the drug placed in the placement area Ar2 at that position, thereby acquiring two visible light images and an ultraviolet light image. The discrimination unit 65 analyzes these six images to determine the type of drug. If the type of drug cannot be identified as one, the imaging control unit 63 causes the first irradiator 134a and the second irradiator 134b to emit visible light at positions θ=45° and 135°, causing the first camera 131 to capture an image of the drug. The discrimination unit 65 analyzes the visible light image at this time to determine the type of drug.

[0078] The position of the imaging mechanism can be controlled in various ways, including but not limited to the above. For example, imaging may be performed from a position opposite to the initial position, and then imaging may be performed from the initial position. Alternatively, a drug discrimination process may be performed based on a visible light image captured from a position of θ=45°, and only if the drug type cannot be identified as a single type may a visible light image be acquired from a position of θ=135°. Alternatively, only ultraviolet light images may be acquired at the initial position and a position opposite to the initial position, and after drug discrimination process based on the ultraviolet light image, a visible light image may be acquired at that position. Alternatively, visible light images and ultraviolet light images may be acquired at all positions.

[0079] [Image processing / discrimination processing] Next, image processing of the image captured by the imaging unit 13 and drug discrimination processing based on the results of the image processing will be described with reference to Figures 1, 5 to 7. The image processing is mainly performed by the feature extraction unit 64, and the discrimination processing is mainly executed by the discrimination unit 65.

[0080] The feature extraction unit 64 generates an extracted mark image 83 by extracting a mark on a medicine that appears in a captured image 82 captured by the first camera 131. More specifically, the feature extraction unit 64 generates an extracted mark image 83 by extracting a mark (more precisely, a mark area in which the mark appears) from the captured image 82 using a trained model 84 that is constructed to extract a mark formed on a medicine. In other words, the feature extraction unit 64 functions as an image generation unit that generates the extracted mark image 83 using the trained model 84. The feature extraction unit 64 can extract a mark from a visible light image, and can also extract a mark from an ultraviolet light image.

[0081] The above-mentioned mark is a mark formed on the surface of the drug. For example, letters (including alphabets and numbers), symbols, designs, pictures, lines (e.g., scribal lines), or code information such as two-dimensional codes formed to indicate the type or manufacturer of the drug are also included in the category of the above-mentioned mark. Furthermore, the method of forming the above-mentioned mark is not particularly limited. For example, the above-mentioned mark may be formed by engraving the surface of the drug or by printing (printing) on ​​the surface of the drug.

[0082] Furthermore, the feature extraction unit 64 may also perform image processing (image analysis) on the captured image 82 (visible light image and / or ultraviolet light image) to extract features of the drug other than the mark appearing in the captured image 82. Here, the drug features are indicators that the discrimination unit 65 uses to discriminate the type of drug. Examples of drug features include the above-mentioned marks (stamps, prints, and scribbles), as well as the size, shape, and representative color of the drug (the color of the area with the stamp or print).

[0083] The feature extraction unit 64 extracts features of the drug other than the mark by performing existing image processing on the captured image 82. The existing image processing may be any processing that can extract a plurality of pixel groups having pixel values ​​different from the pixel values ​​of the background portion, and examples of such processing include known techniques such as OCR (Optical Character Recognition) or pattern matching. The feature extraction unit 64 extracts, for example, the size, shape and representative color of the medicine from the visible light image, and extracts the representative color from the ultraviolet light image.

[0084] (About the trained model) In this embodiment, a semantic segmentation model (hereinafter referred to as an SS model) that is trained to distinguish between pixels that make up a mark in an image of a drug and pixels that make up the background of the mark is used as an example of the trained model 84. By using the SS model as the trained model 84, it becomes possible to accurately extract marks with irregular shapes.

[0085] The SS model is a model based on a convolutional neural network (CNN) and has an encoder-decoder structure to generate an output image of the same size as an input image. The encoder structure is constructed, for example, by a first neural network having an input layer, a hidden layer, and an output layer, and the decoder structure is constructed by a second neural network having the inverse structure of the encoder structure. In other words, the structures of the input layer, hidden layer, and output layer of the decoder structure are the same as the structures of the output layer, hidden layer, and input layer of the encoder structure.

[0086] When a captured image 82 of an unknown type of target drug is input, the SS model calculates a probability value indicating the probability that each pixel constituting the image of the drug shown in the captured image 82 is a pixel constituting a mark. The higher the probability value of a pixel, the more likely it is to be a pixel constituting a mark, and the lower the probability value of a pixel, the more likely it is to be a pixel constituting a background portion. The SS model then outputs data in which the pixels constituting the mark and the pixels constituting the background portion are classified based on the probability value (for example, data in which pixels constituting the background region are classified as 0 and pixels constituting the mark are classified as 1).

[0087] Therefore, the feature extraction unit 64 can generate an extraction mark image 83 by performing image processing on the output value obtained by inputting the captured image 82 captured by the first camera 131 into the SS model. For example, the feature extraction unit 64 may set a pixel whose output value of the SS model is 0 as (R, G, B) = (255, 255, 255) and a pixel whose output value of the SS model is 1 as (R, G, B) = (0, 0, 0). This generates an extraction mark image 83 in which pixels constituting the background region are expressed in white and pixels constituting the mark are expressed in black. In this way, the feature extraction unit 64 can generate an extraction mark image 83 based on the output value of the SS model.

[0088] The trained model 84 may be a model that outputs the probability value as an output value. In this case, the feature extraction unit 64 determines whether the probability value of each pixel output from the trained model 84 is equal to or greater than a predetermined threshold, and determines the pixel value of each pixel based on the determination result. In this way, the feature extraction unit 64 generates an extracted mark image 83 in which pixels that make up the mark and pixels that make up the background are distinguished. The trained model 84 may also be configured to output the extracted mark image 83. In this case, the feature extraction unit 64 may use the output value of the trained model 84 as the extracted mark image 83 as is.

[0089] 5 and 6 are diagrams showing an example of a captured image 82 and examples of extraction mark images 83 and 830 extracted from the captured image 82. The extraction mark image 83 is an image obtained by inputting the captured image 82 into a trained model 84. The extraction mark image 830 is an image obtained by performing existing image processing on the captured image 82 without using the trained model 84. FIG. 5 shows an example of a captured image 82 obtained by capturing a drug with no pattern in the background of the marking, and FIG. 6 shows an example of a captured image 82 obtained by capturing a drug with a pattern in the background of the marking. Note that the "pattern" may be one that appears as a pattern due to the constituent components of the drug, etc.

[0090] When extracting a mark on a medicine, depending on how the light hits the medicine, it can be difficult to distinguish part of the mark from the background area. Conventional image processing such as edge detection or binarization cannot recognize such a part as a mark, resulting in a detection result in which part of the mark is missing, as in extracted mark image 830 in Figure 5. In this regard, the trained model 84 has been trained to detect marks, so it can accurately extract the entire mark, as in extracted mark image 83 in Figure 5.

[0091] In addition, pixel values ​​of particles of pharmaceutical components that appear in the background area of ​​the marking (i.e., the surface of the drug) may be close to the pixel values ​​of the marking. In such cases, conventional image processing may determine that part of the background area is the marking, as in extracted mark image 830 in Figure 6, or may erroneously determine the boundary between the marking and the background area. In this regard, the trained model 84 has been trained on the mark, and therefore can accurately extract the entire marking, as in extracted mark image 83 in Figure 6.

[0092] The discrimination unit 65 compares the extracted mark image 83 generated by the feature extraction unit 64 with registered mark images pre-registered for each type of drug, and discriminates the type of target drug based on the comparison result. Since the registered mark images are registered in the drug database 81, the discrimination unit 65 can make the above discrimination by referring to the drug database 81.

[0093] When comparing the extraction mark image 83 with the registered mark image, a blurred image of the extraction mark image 83 may be used. That is, the discrimination unit 65 may discriminate the type of the target drug based on the result of comparing the blurred image of the extraction mark image 83 with the registered mark image.

[0094] The reason for performing the blurring process is that, as a result of research by the inventors of the present application, it has been found that performing the blurring process improves the accuracy of identifying the type of target drug compared to not performing the blurring process. In other words, the above configuration can improve the accuracy of type identification. When performing the blurring process on the extraction mark image 83, it is desirable that the registration mark image also undergo the same blurring process. Furthermore, the method of the blurring process is not particularly limited, and for example, the blurring process may be a process of applying a smoothing filter to the extraction mark image 83. Examples of smoothing filters include a binomial (binomial distribution type) filter, a median filter, a Gaussian filter, and a bilateral filter. Furthermore, the blurring process can also be performed by simple averaging, k-nearest neighbors, gradient calculation, histogram smoothing, or the like.

[0095] The trained model 84 can also be used when generating a registered mark image. That is, an image of a drug of known type may be input to the trained model 84, and an extracted mark image generated based on the output value output from the trained model 84 may be used as the registered mark image for that drug.

[0096] As described above, the feature extraction unit 64 may extract features of the drug other than the mark. In this case, the features of the drug other than the mark may also be registered in the drug database 81 for each type of drug as master data. The discrimination unit 65 may then compare each feature extracted by the feature extraction unit 64 with the master data to discriminate the type of target drug.

[0097] For example, suppose that the feature extraction unit 64 analyzes the captured image 82 and extracts information indicating the size, shape, and representative color of a drug as drug features. In this case, the discrimination unit 65 can narrow down the types of drugs that match the features extracted by the feature extraction unit 64 based on the sizes, shapes, and representative colors of various drugs registered in the master data. Then, the discrimination unit 65 may identify the type of target drug by comparing the extracted mark image 83 with the registered mark image for each of the narrowed-down types. Note that the features to be used in the narrowing-down and in what order can be determined arbitrarily.

[0098] Furthermore, the discrimination unit 65 may rank candidate drug types based on the degree of match between the extracted drug characteristics and the drug characteristics included in the drug database 81. For example, the discrimination unit 65 may rank the candidate types extracted based on the representative color based on the degree of match between the extracted representative color and the master color data, and then discriminate the types based on other characteristics according to the ranking.

[0099] Furthermore, even if the extracted drug characteristics (target characteristics) are not in the drug database 81, if the drug is estimated to be a drug (tablet or capsule) based on at least a part of the target characteristics, the discrimination unit 65 determines the type of drug as a suspected drug. In this case, the suspected drug can also be sorted into the second storage unit 14 or the waiting tray 15. In this embodiment, the suspected drug may first be temporarily placed on the waiting tray 15.

[0100] The discrimination unit 65 outputs the discrimination result of the type of the target drug to the sorting control unit 62. For example, when the type of the target drug can be identified as one, or when the number of candidates has been narrowed down to a predetermined number or less, the discrimination unit 65 outputs the drug data related to the drug as the discrimination result. In this case, the discrimination unit 65 stores the drug data related to the drug in the storage unit 80 in association with the captured image 82 of the drug.

[0101] When the discrimination unit 65 discriminates that the type of the target drug is a suspected drug, it outputs the characteristics of the drug (the characteristics of the object presumed to be a suspected drug) as the discrimination result. On the other hand, when the discrimination unit 65 discriminates that the drug is registered in the drug database 81 as a drug to be discarded, or when it discriminates that the object contained in the first container 11 is a foreign object other than a drug, it outputs the discrimination result that the drug is not to be sorted.

[0102] (Effects of image processing and discrimination processing of this embodiment) As described above, the content of marks formed on medicines varies, and the appearance of the marks in the captured image 82 changes depending on the way the light hits the medicine when it is captured, making it difficult to accurately identify the type of medicine based on the marks. Therefore, in order to accurately identify the type of medicine, it is necessary to be able to handle the variety of marks and to extract the marks formed on the medicine with high accuracy, regardless of changes in the appearance of the marks in the captured image 82.

[0103] The feature extraction unit 64 uses a trained model 84 constructed to extract marks formed on medicines, thereby generating an extracted mark image 83 in which marks formed on the target medicine are extracted from the captured image 82. Therefore, the feature extraction unit 64 (1) can handle the diversity of marks, (2) is not dependent on changes in the appearance of the mark in the captured image 82, and (3) can accurately extract the mark of the target medicine from the captured image 82 even if the target medicine has a pattern.

[0104] In addition, since the trained model 84 is trained on marks, even if the trained model 84 has not trained a mark, it is possible to extract the mark from the captured image 82 as long as the mark is similar to the trained mark. Furthermore, the feature extraction unit 64 is able to extract the mark from the captured image 82 regardless of the position or orientation of the mark in the captured image 82.

[0105] Furthermore, since it is sufficient to be able to extract marks formed on target medicines using the trained model 84, it is not necessary to use captured images of all types of medicines that can be handled by the medicine sorting device 1 in order to construct such a trained model 84. Therefore, a variety of marks can be extracted with high accuracy using the trained model 84 constructed without incurring significant costs.

[0106] Here, when using a trained model to identify the type of target drug, it is possible to construct a trained model such as the following: Comparative Example 1: A trained model trained by linking drug images with drug types (classification labels). Comparative Example 2: A trained model trained on individual letters, numbers, etc. In this case, the trained model is used to extract letters or numbers, etc. attached to the drug one by one from the drug image, and the type of drug is identified from the combination of the extracted letters or numbers, etc.

[0107] However, in the case of Comparative Example 1, since it is necessary to prepare a large amount of training data for each type of drug and perform training, there is a problem that it requires a great deal of effort to build a model capable of identifying the types of many types of drugs. In addition, since it is difficult to train for all types of drugs, there is a possibility that images of drugs of a type that has not been trained will be input. In such cases, it is difficult to predict what kind of discrimination result the trained model will output. In the case of important discrimination related to patient health, such as discrimination of drug types, the low explainability of the discrimination result is a major negative factor. In addition, it is difficult to have the trained model distinguish between drugs that have only slight differences in their marks as different drugs.

[0108] Furthermore, in Comparative Example 2, there is a possibility that an image of a drug with an untrained mark or letters or numbers in a different font from those that have been trained may be input. In these cases, it is difficult to predict what results the trained model will output. Furthermore, the order of letters or numbers serves as a characteristic for identifying the type of drug. However, the position or orientation of letters or numbers may differ for each drug, and the orientation of the drug in the captured image 82 may also vary, making it difficult to accurately reproduce the order of the detected letters or numbers.

[0109] As such, when using the trained model of Comparative Example 1 or 2 to identify the type of target drug, there is a possibility that the basis for the identification results may lack explanatory power (evidence), and there is also a possibility that the reliability of the identification results may not be guaranteed.

[0110] On the other hand, the discrimination unit 65 of this embodiment performs final discrimination of the type of the target drug based on the result of matching the extracted mark image 83 with the registered mark image. Specifically, the discrimination unit 65 performs final discrimination of the type of drug by matching the mark accurately extracted using the trained model 84 with features other than the mark extracted using existing image processing against the drug database 81. Therefore, the computer 60 provided in the drug sorting device 1 uses the trained model 84 only for at least extracting the mark, and therefore, unlike when the type of target drug is discriminated using the trained model of Comparative Example 1 or 2, the explainability of the basis for the discrimination result can be enhanced. In other words, the drug sorting device 1 makes it possible to present evidence of the discrimination result.

[0111] Furthermore, with regard to markings or score lines, there is a high possibility that the appearance of the markings or score lines in the captured image 82 will change depending on factors such as how light hits the drug during imaging. For drugs with shallow markings or score lines, the appearance is even more likely to change. Therefore, it is difficult to extract the markings or score lines from the captured image 82 with high accuracy using existing image processing. By using the trained model 84, the feature extraction unit 64 can extract the markings or score lines with higher accuracy than when using existing image processing, particularly when extracting the markings or score lines from the captured image 82.

[0112] (Flow of the determination process) 7 is a flowchart showing an example of the flow of the discrimination process (type discrimination method). The control unit 60a acquires a captured image 82 stored in the storage unit 80 (S1). The feature extraction unit 64 inputs the acquired captured image 82 to a trained model 84 (S2). The feature extraction unit 64 inputs the captured image 82 to the trained model 84 to generate an extraction mark image 83 (S3, image generation step). As described above, the extraction mark image 83 may be the result of performing image processing on the output value of the trained model 84, may be the output result of the trained model 84 itself, or may be generated based on the probability value output by the trained model 84.

[0113] Thereafter, the discrimination unit 65 compares the extracted mark image 83 generated by the feature extraction unit 64 with the registered mark image included in the drug database 81 (S4, discrimination step). In addition, in S4, the discrimination unit 65 may perform existing image processing on the acquired captured image 82 to extract features other than the mark of the target drug from the captured image 82, and compare these features with various master data included in the drug database 81. In this way, in S4, the type of the target drug is discriminated based on the comparison result between the extracted mark image 83 generated in S3 and the registered mark image. Thereafter, the discrimination unit 65 outputs a discrimination result of the type of the target drug based on these comparison results (S5).

[0114] (Example of building a trained model) The trained model 84 is constructed, for example, as follows. FIG. 8 is a diagram showing an example of training data 181. The training data 181 is data in which a captured image 182 of a known type of medicine is associated with correct answer data 183 for the captured image 182. The correct answer data 183 is data in which a mark is extracted from the captured image 182. The correct answer data 183 is generated by, for example, specifying the outer edge of the mark in the captured image 182 through user input and distinguishing the inside of the area surrounded by the specified outer edge from the outside of the area. In the example of FIG. 8, the internal area of ​​the mark and the secant line specified in the captured image 182 (i.e., the mark area) from the external area (i.e., the background area) is distinguished by performing binarization processing. A plurality of captured images 182 are prepared, and correct answer data 183 is associated with each captured image 182 to create training data 181, thereby preparing a plurality of training data 181.

[0115] The trained model 84 is constructed by sequentially inputting the teacher data 181 and performing machine learning. The trained model 84 is constructed, for example, by a model generation device (not shown), but is not limited to this and may be constructed, for example, by the computer 60. In this case, the computer 60 also functions as a learning device that constructs the trained model 84 by machine learning using the teacher data 181.

[0116] [Medicine sorting process] Next, the medicine sorting process based on the result of the above-mentioned discrimination process will be described with reference to Fig. 1. The medicine sorting process is mainly performed by the conveying / sorting unit 12 and the sorting control unit 62.

[0117] Based on the discrimination result by the discrimination section 65, the conveying / sorting unit 12 sorts the medicines by type and stores them in the second storage section 14 or the waiting tray 15. The sorting control section 62 controls the conveying / sorting unit 12 to convey the medicines placed in the receiving area Ar1 after the imaging and discrimination process to a predetermined sorting cup 141 in the second storage section 14 or the waiting tray 15 based on the discrimination result.

[0118] When the sorting control unit 62 receives the discrimination result of the medicine, it determines a sorting position for storing the medicine, and stores the discrimination result and the determined sorting position in the memory unit 80 in association with each other. Specifically, the sorting control unit 62 determines whether or not a discrimination result identical to the above discrimination result is stored in the memory unit 80.

[0119] If a discrimination result identical to the above discrimination result is stored, the sorting control unit 62 determines the sorting position to be the sorting cup 141 linked to the stored discrimination result. If the sorting position linked to the stored discrimination result (e.g., suspected drug) is the waiting tray 15, the waiting tray 15 is determined to be the sorting position. On the other hand, if a discrimination result identical to the above discrimination result is not stored, the sorting control unit 62 determines the sorting position to be the sorting cup 141 that does not contain any drugs (the sorting cup 141 that has not been determined as a sorting position). If drugs are stored in all of the sorting cups 141, the waiting tray 15 is determined to be the sorting position. Note that for suspected drugs, the sorting control unit 62 may determine any of the sorting cups 141 included in a predetermined area of ​​the second storage unit 14 as the sorting position, instead of the waiting tray 15.

[0120] Once the sorting control unit 62 has determined the sorting position, it controls the transport mechanism 123, similar to the transport control unit 61, to move the transport / sorting unit 12 above the receiving area Ar1. Similarly to the transport control unit 61, the sorting control unit 62 controls the second camera 121 and the suction / shutter mechanism 122 to adsorb the medicines placed in the receiving area Ar1. Thereafter, the transport mechanism 123 transports the medicines to the determined sorting cup 141 or waiting tray 15. As described above, the shutter mechanism is closed while the medicines are being transported, preventing the medicines from falling into areas other than the determined sorting position (e.g., a sorting cup 141 other than the determined sorting cup 141). After transport, the suction is released, allowing the medicines to be stored in the sorting cup 141 or waiting tray 15. Furthermore, the sorting control unit 62 counts the number of medicines stored in the sorting cup 141, and stores the count in the memory unit 80 in association with the sorting position.

[0121] After the sorting control unit 62 transports the medicines (medicines after identification) on the medicine placing table 133a arranged in the receiving area Ar1 to the sorting position, the transport control unit 61 controls the transport / sorting unit 12 to transport and place the medicines stored in the first storage unit 11 on the now empty medicine placing table 133a. This allows the medicine sorting device 1 to continuously identify the types of medicines.

[0122] In addition, if the sorting control unit 62 receives a discrimination result indicating that the type of medicine could not be identified, the object placed in the receiving area Ar1 after discrimination is a foreign object, and therefore the foreign object is transported to the collection tray 16.

[0123] In this way, the sorting control unit 62 stores all of the items contained in the first storage unit 11 in either the second storage unit 14, the waiting tray 15, or the collection tray 16, regardless of the result of determining the type of medicine. Therefore, even if the type of medicine cannot be identified or if a foreign object has been mixed into the first storage unit 11, the sorting process can be continued without being stopped for that reason.

[0124] In order to remove medicines for which the discrimination process has been completed from the medicine placing table 133a arranged in the receiving area Ar1, the sorting control unit 62 causes the second camera 121 to capture an image of the medicine placing table 133a, thereby narrowing down the positions of the medicines. Furthermore, when transporting medicines stored in the sorting cup 141 to the packaging mechanism 6, the sorting control unit 62 causes the second camera 121 to capture an image of the sorting cup 141, thereby narrowing down the medicines to be transported, in order to remove the medicines from the sorting cup 141.

[0125] In addition, when the number of drugs stored in the sorting cup 141 reaches the upper limit, the sorting control unit 62 stores the drugs to be sorted in an empty sorting cup 141 different from the sorting cup 141, even if the type of drug to be sorted is the same as the type of drug sorted in the sorting cup 141.

[0126] Furthermore, data relating to the medicines stored in the sorting cups 141 by the sorting control unit 62 is stored in an RFID tag provided on the sorting cups 141 by the second RFID reader / writer unit 18.

[0127] The second RFID reader / writer unit 18 writes various data to RFID tags or reads various data stored in RFID tags, similar to the first RFID reader / writer unit 5. The sorting control unit 62 causes the RFID control unit 68 to write data related to the medicines every time a medicine is stored in a sorting cup 141. The second RFID reader / writer unit 18 is provided below the second storage unit 14. Specifically, the second RFID reader / writer unit 18 is provided so as to face the bottom of each sorting cup 141 when reading or writing data related to the medicines stored in the RFID tag of each sorting cup 141.

[0128] [Embodiment 2] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated. The same applies to the following embodiments.

[0129] Among the medicines sorted by the medicine sorting device 1, there are some that are heat-sensitive and may discolor when exposed to heat. These heat-sensitive medicines may discolor due to the heat they receive from the packaging mechanism 6, particularly the heater roller 6b described below, during a series of steps related to the packaging process. In this embodiment, using Figures 9 to 11, a configuration for reducing the possibility of medicines discoloring due to the heat of the heater roller 6b of the packaging mechanism 6 in the medicine sorting device 1 equipped with the packaging mechanism 6 will be described. In embodiment 2, for the sake of simplicity, heat-sensitive medicines will be referred to as "heat-sensitive medicines HMD" and other medicines as "normal medicines MD". Furthermore, when simply referring to "medicines", of The term is used as a general term for regular drug MD and heat-sensitive drug HMD.

[0130] FIG. 9 is a diagram illustrating the packaging operation of the packaging mechanism 6 when packaging a normal medicine MD. 9001 in FIG. 9 is a schematic diagram of the packaging mechanism 6. 9002 in FIG. 9 is a schematic diagram of the heater roller 6b when viewed from the direction of the white arrow R in 9001. FIG. 10 is a diagram illustrating the packaging operation of the packaging mechanism 6 when packaging a heat-sensitive medicine HMD. 10001 in FIG. 10 is a schematic diagram of the packaging mechanism 6. 10002 in FIG. 10 is a schematic diagram of the heater roller 6b when viewed from the direction of the white arrow R in 10001. FIG. 11 shows an example of packaging by the packaging mechanism 6, an example of printing, and an example of cutting a packaging sheet.

[0131] 9 and 10, the packaging mechanism 6 includes a packaging hopper 6a, a heater roller 6b, a moving path 6c, and a shutter mechanism 6e. The heater roller 6b and the shutter mechanism 6e are controlled by a packaging control unit 71.

[0132] Packaging hopper 6a receives the medicines that have been put in through medicine inlet 17 and passed (fall) through transfer path 6c, and guides them to packaging paper PP set on heater roller 6b and at standby position Ar4.

[0133] The heater roller 6b heat-seals portions of the packaging paper PP to package medicines individually. The heater roller 6b is a pair of rollers extending substantially perpendicular to the conveyance direction of the packaging paper PP (the direction indicated by arrow Q1) and arranged to sandwich the packaging paper. The heater roller 6b includes a horizontal fusion heater 6bw for fusing the horizontal fusion portions Cr1 and a vertical fusion heater 6bh for fusing the vertical fusion portions Cr2. As the packaging paper PP is conveyed in the conveyance direction, the heater roller 6b rotates in the direction indicated by arrow Q2 in Figures 9 and 10, fusing the horizontal fusion portions Cr1 and fusing the vertical fusion portions Cr2 at arbitrary intervals.

[0134] The transfer path 6c is provided between the medicine inlet 17 and the packaging hopper 6a, and guides the medicines introduced through the medicine inlet 17 to the packaging hopper 6a. The transfer path 6c also guides the medicines introduced into the medicine inlet 17 to a holding area Ar3 above the shutter 6ea, which will be described later.

[0135] The shutter mechanism 6e functions as a medicine holding section that temporarily holds medicines input from medicine input port 17 in holding area Ar3 until all of the medicines to be included in one package created by the packaging mechanism 6 have been input from medicine input port 17. In other words, the shutter mechanism 6e prevents medicines from being input into the packaging hopper 6a until all of the medicines to be included in one package have been input from medicine input port 17. In this embodiment, the shutter mechanism 6e is provided between the packaging hopper 6a and the transfer path 6c, but is not limited to this and may be provided inside the packaging hopper 6a or the transfer path 6c, for example.

[0136] As shown by 9001 in Fig. 9 and 10001 in Fig. 10, the shutter mechanism 6e includes a shutter 6ea that can be opened and closed. The shutter mechanism 6e also includes a shutter drive unit (not shown). The shutter drive unit controls the opening and closing operation of the shutter 6ea by driving the shutter 6ea. 9001 in Fig. 9 and 10001 in Fig. 10 show a state in which the shutter 6ea is closed.

[0137] [Regarding regular drug MD packaging] First, with reference to Figure 9, the control of the packaging control unit 71 when the medicine to be packaged is a regular medicine MD will be described. When the medicine to be packaged is a regular medicine MD, the packaging control unit 71 controls the packaging mechanism 6 so that the following operations (i) to (iii) are repeated after the last medicine of multiple medicines to be included in one package is input into the medicine input port 17. Note that inputting the last medicine of multiple medicines to be included in one package into the medicine input port 17 means that the last medicine of multiple medicines contained in a certain sorting cup 141 is input into the medicine input port 17. When only one medicine is contained in a certain sorting cup 141, that medicine is the last medicine.

[0138] (i) The packaging control unit 71 controls the heater roller 6b to heat-seal the packaging paper PP and feed the packaging paper PP by one packet in the direction indicated by arrow Q1. This allows the packaging control unit 71 to move the regular medicine MD, which has been waiting at standby position Ar4 until all of the regular medicine MD to be packaged this time have been inserted into the medicine insertion port 17, from standby position Ar4 to the fused area Ar5 while packaging them. (ii) The packaging control unit 71 opens the shutter 6ea, drops all of the regular medicine MD held in the holding area Ar3, and then closes the shutter 6ea again. (iii) After the last regular medicine MD to be included in the next package has been inserted into the medicine insertion port 17, the packaging control unit 71 returns to process (i). When packaging regular medicine MD to be included in the first package, the packaging control unit 71 may skip process (i) and start with process (ii).

[0139] When the above process is performed, as shown in Figure 9, the medicine in the packaging paper PP at standby position Ar4 will wait at standby position Ar4 until the medicine to be packaged next is held on shutter 6ea. Because standby position Ar4 is a position sandwiched between the pair of heater rollers 6b, the medicine at standby position Ar4 is susceptible to the heat from heater roller 6b. Therefore, if the medicine to be packaged is a heat-sensitive medicine HMD, packaging control unit 71 may perform control such as the following.

[0140] [About the packaging of HMD, a medication that requires caution when heated] The control of the packaging control unit 71 when the medicine to be packaged is a heat-caution medicine HMD will be described below with reference to Fig. 10. When the medicine to be packaged is a heat-caution medicine HMD, the packaging control unit 71 controls the packaging mechanism 6 so that the following operation is performed after the last medicine of multiple medicines to be included in one package is inserted into the medicine insertion port 17.

[0141] (iv) When the previously packaged heat-sensitive medicine HMD is in the fused area Ar5, the packaging control unit 71 controls the heater roller 6b to heat-seal the packaging paper PP and feed the packaging paper PP by one package in the direction indicated by arrow Q1. This allows the packaging control unit 71 to treat the next package after the previously packaged heat-sensitive medicine HMD as an empty package containing no medicine.

[0142] (v) The packaging control unit 71 opens the shutter 6ea, drops all the heat-sensitive medicine HMDs held in the holding area Ar3, and then closes the shutter 6ea again, so that the heat-sensitive medicine HMDs to be packaged this time are inserted into the packaging paper PP at the standby position Ar4, which is to be made as the next package after the blank package.

[0143] (vi) The packaging control unit 71 controls the heater roller 6b to heat-seal the packaging paper PP containing the heat-sensitive medicine HMD at the standby position Ar4 while feeding the heat-sensitive medicine HMD by one packet in the direction indicated by arrow Q1, and moving the heat-sensitive medicine HMD to the fused area Ar5. This allows the heat-sensitive medicine HMD inserted into the packaging paper PP at the standby position Ar4 where the heater roller 6b is located to be sent from the standby position Ar4 to the fused area Ar5 without waiting at the standby position Ar4. In other words, while the heat-sensitive medicine HMD to be included in the next package is being inserted through the medicine insertion port 17, the heat-sensitive medicine HMD to be packaged this time is packaged and present in the fused area Ar5, not at the standby position Ar4.

[0144] (vii) In this state, after the last heat-cautionary drug HMD to be included in the next packet is inserted into the drug insertion port 17, the packaging control unit 71 advances the packaging paper PP by one packet in the direction shown by arrow Q1 (i.e., returns to process (iv)). This allows the packaging control unit 71 to make the next packet of heat-cautionary drug HMD packaged this time into an empty packet containing no drug before dropping the next heat-cautionary drug HMD into standby position Ar4.

[0145] When packaging a heat-sensitive drug HMD to be included in the first packet, or when packaging a heat-sensitive drug HMD after packaging a normal drug MD, the packaging control unit 71 may skip step (iv) and start with step (v). The packaging control unit 71 may also create two or more blank packets, not just one, or may create blank packets smaller than the specified dimensions of one packet (e.g., blank packets whose length in the direction indicated by arrow Q1 is half the length of the specified one packet). It is sufficient that a blank packet or space large enough to allow the packaged heat-sensitive drug HMD to move to a position where it is least susceptible to the heat of the heater roller 6b is formed between the packaged heat-sensitive drug HMD and the heater roller 6b (standby position Ar4). The packaging control unit 71 simply moves the packaging paper PP in the direction indicated by arrow Q1 to create the blank packet or space.

[0146] As described above, after dropping the heat-sensitive drug HMD into standby position Ar4, the packaging control unit 71 sends the packaging paper PP for at least one packet in the direction indicated by arrow Q1, packaging the heat-sensitive drug HMD, and sending it to the fused area Ar5. After that, after all of the next heat-sensitive drug HMD have been inserted into the drug insertion port 17, the packaging control unit 71 sends the packaging paper PP for at least one packet in the direction indicated by arrow Q1, creating at least one packet following the previously packaged heat-sensitive drug HMD as a blank package. After that, the packaging control unit 71 drops the heat-sensitive drug HMD held in holding area Ar3 into standby position Ar4, inserting the heat-sensitive drug HMD into the next packet of the blank package.

[0147] When the above process is performed, the packaging paper PP at standby position Ar4 is empty during the standby time until the next medicine to be packaged is held on shutter 6ea, as shown in Figure 10. Furthermore, by the above-mentioned operation (iv) or (vii) of packaging mechanism 6, the packaging paper PP after being heat-sealed is dispensed from packaging mechanism 6 as a packaging sheet including an empty packet, as shown by 11001 in Figure 11.

[0148] After dropping the heat-sensitive drug HMD into the waiting position Ar4, the packaging control unit 71 may feed the packaging paper PP in the direction indicated by the arrow Q1 for at least two packets. That is, after dropping the heat-sensitive drug HMD into the waiting position Ar4, the packaging control unit 71 may not only package the heat-sensitive drug HMD but also create a blank packet as the next package of the heat-sensitive drug HMD. In this case, the packaging control unit 71 can create a blank packet before or while the next heat-sensitive drug HMD is being inserted into the drug insertion port 17, rather than after the next heat-sensitive drug HMD has been inserted into the drug insertion port 17. Furthermore, the packaged heat-sensitive drug HMD can wait at a position farther away from the heater roller 6b by the amount of the blank packet while the next heat-sensitive drug HMD is being inserted into the drug insertion port 17. This further reduces the effect of heat from the heater roller 6b on the packaged heat-sensitive drug HMD.

[0149] (Caution when dividing HMD packets containing heat-sensitive medicines) When heat-sensitive drug HMDs are dispensed from the packaging mechanism 6 as a packaging sheet containing blank packets as described above, a warning may be issued to the user by using printing on the blank packets. Specifically, as shown in 11001 in Fig. 11, specific characters or pictures may be printed on the blank packets by a printing mechanism (not shown) provided in the packaging mechanism 6 in the packaging sheet that packages the heat-sensitive drug HMDs. Specifically, characters such as "heat protection" or "SKIP", or pictures or symbols may be printed to indicate that the blank packets are for heat protection purposes.

[0150] Here, it is preferable that the user carefully visually check the heat-sensitive drug HMD for discoloration before returning it to the drug cassette in the packaging machine or to the drug shelf. By printing in this manner, the user can easily recognize that the drug package in front of the blank package labeled "heat-sensitive" or the like is a heat-sensitive drug HMD, thereby assisting the user in the visual check.

[0151] (About the cutting position of the packaging sheet when packaging HMD medicines with heat warning) The series of packaging sheets fused by heater roller 6b are cut into a predetermined number of packages by a cutting mechanism (not shown) of packaging mechanism 6 for subsequent operations. The cutting mechanism is controlled by packaging control unit 71. 11002 and 11003 in FIG. 11 both show examples of cutting packaging sheets when the set number of packages to be cut is set to 3. The positions of the white arrows in 11002 and 11003 in FIG. 11 indicate the cutting positions.

[0152] In cutting example 1 shown at 11002 in FIG. 11, the cutting mechanism cuts the packaging sheet every three packets from the beginning. A packaging sheet cut to the set number of packets is called a cutting unit. The cutting mechanism cuts the packaging sheet just before drug B, which is the switching position between the heat-sensitive drug HMD (drug A) and the normal drug MD (drug B). In this case, the cutting unit containing drug A includes a blank packet, so the position of the drug packet containing drug A differs depending on the cutting unit.

[0153] On the other hand, in cutting example 2 shown at 11003 in FIG. 11 , the cutting mechanism cuts the packaging sheet every three packets from the beginning. Furthermore, if the beginning of a cutting unit is an empty packet, the cutting mechanism cuts only that empty packet for one packet. Furthermore, the cutting mechanism cuts the packaging sheet just before drug B, which is the switching position between the heat-sensitive drug HMD (drug A) and the normal drug MD (drug B). In this case, all cutting units containing drug A have drug packets containing drug A arranged at both ends, resulting in identical cutting units containing empty packets between them. Cutting according to cutting example 2 ensures that the arrangement of drug A in each cutting unit is identical, improving the appearance of the cutting units dispensed from the packaging mechanism 6. Furthermore, this improves the user's work efficiency during subsequent visual inspection and / or loading into a medicine shelf, etc.

[0154] Cutting example 2 shown in 11003 of FIG. 11 can be applied when the set number of cuts is an odd number of 5 or more, and in this case, the same effect as when the set number of cuts is 3 can be obtained.

[0155] [Embodiment 3] The medicine sorting device 1 of embodiment 1 determines the type of medicine one by one for all medicines stored in the first storage unit 11, and stores each type in the second storage unit 14. If the first storage unit 11 stores more types of medicine than the number of sorting cups 141 arranged in the second storage unit 14, some medicines cannot be stored in the second storage unit 14 even if their types have been determined, and the medicines will be stored in the waiting tray 15. When a medicine is stored in the waiting tray 15, the medicine sorting device 1 determines the type of the medicine again and stores it in the second storage unit 14. Then, every time a medicine that cannot be stored in the second storage unit 14 occurs, the medicine sorting device 1 determines the type of the medicine and attempts to store it in the second storage unit 14.

[0156] In this embodiment, a medicine dispensing system 100 will be described that can store medicines stored in a first storage unit 11 in a second storage unit 14 without waiting for storage in the second storage unit 14.

[0157] [Medicine dispensing system] A description will be given of a medicine dispensing system 100 of this embodiment. Fig. 12 is a diagram showing an example of the configuration of the medicine dispensing system 100. Fig. 13 is a diagram showing a state in which a two-dimensional code is printed on a medicine packet packaged by a packaging machine 110.

[0158] 12, the medicine dispensing system 100 is a system that dispenses various medicines based on, for example, prescription data, and is installed in a medical institution such as a pharmacy or a hospital. The medicine dispensing system 100 includes, for example, a packaging machine 110 and a medicine sorting device 1A.

[0159] Packaging machine 110 is a device that dispenses medicines based on, for example, prescription data. Specifically, packaging machine 110 is a device that packages medicines such as tablets or capsules. When packaging medicines, packaging machine 110 prints information indicating the type of medicine to be packaged as a two-dimensional code on the surface of packaging paper that contains the medicines. Specifically, as shown in FIG. 13, packaging machine 110 prints information indicating the type of medicine to be packaged in medicine packet MP1 as a two-dimensional code Co1 on medicine packet MP1, which packages medicines in packaging paper in units of packets. As shown in FIG. 13, information indicating the type of medicine to be packaged in medicine packet MP2 is printed as a two-dimensional code Co2 on medicine packet MP2, and information indicating the type of medicine to be packaged in medicine packet MP3 is printed as a two-dimensional code Co3 on medicine packet MP3.

[0160] In this embodiment, the information indicating the type of medicine may be, for example, the name of the medicine packaged or a YJ code. In this embodiment, the information indicating the type of medicine is printed on the medicine package as a two-dimensional code (e.g., QR code (registered trademark)), but it may also be printed in other formats, such as a barcode. The information indicating the type of medicine printed on the medicine package is read by the medicine sorting device 1A, as will be described later. Therefore, it is sufficient that the information indicating the type of medicine is printed on the medicine package in a format that can be read by the medicine sorting device 1A.

[0161] However, printing of information indicating the type of drug on the packaging paper does not necessarily have to be performed by packaging machine 110. For example, the printing may be performed by a printing device (not shown) that is separate from packaging machine 110. In this case, for example, packaging machine 110 may transmit information indicating the type of drug contained in the medicine packet to the printing device, and the printing device may print the information on the medicine packet dispensed from packaging machine 110.

[0162] The medicine sorting device 1A is a device that identifies the types of medicines input into the medicine sorting device 1A and sorts the medicines by type, similar to the medicine sorting device 1. The specific configuration of the medicine sorting device 1A will be described later.

[0163] In the medicine dispensing system 100, medicines dispensed by the packaging machine 110 are handed over to patients who have been prescribed the medicines. The medicines handed over to the patients may be returned to the medical institution equipped with the medicine dispensing system 100. The returned medicines are placed into the medicine sorting device 1A by a user (e.g., a medical professional such as a pharmacist) and sorted by type in order to be returned to the packaging machine 110 or a medicine shelf (not shown).

[0164] [Medicine sorting device] Fig. 14 is a block diagram showing the overall configuration of the medicine sorting device 1A. As shown in Fig. 14, the medicine sorting device 1A includes a reading unit 8 and a control unit 60Aa in addition to the components of the medicine sorting device 1. In addition to these components, the first storage unit 11, the second storage unit 14, the conveying / sorting unit 12, the packaging mechanism 6, and the display unit 32 of the touch panel 3 will also be described below.

[0165] As described in the first embodiment, the first storage unit 11 is a storage unit capable of storing multiple types of medicines, and is divided into multiple storage units (specifically, four storage units). In this embodiment, the four storage units into which the first storage unit 11 is divided are referred to as first sections 111a to 111d, respectively. The first sections 111a to 111d may also be collectively referred to as first sections 111. The number of first sections 111 may be any number other than four, as long as it is plural. Furthermore, as described above, the first storage unit 11 is capable of storing medicines packaged by the packaging machine 110.

[0166] As described in the first embodiment, the second storage unit 14 is a storage unit in which a plurality of sorting cups 141 can be arranged, and in which each of the arranged plurality of sorting cups 141 can store medicines by type. Instead of the removable sorting cup 141, the second storage unit 14 may be provided with a plurality of compartments in which medicines can be stored. The sorting cups 141 and the compartments can also be referred to as second compartments in which medicines can be stored by type. In this embodiment, the total number of sorting cups 141 that can be stored in the second storage unit 14 (the number of second compartments; hereinafter, also simply referred to as the total number of sorting cups 141) is 40, but is not limited to this as long as it is a plurality.

[0167] The transporting / sorting unit 12 functions as a transporting section that transports the medicines contained in the first storage section 11 to the second storage section 14 for each first section 111, and stores them in sorting cups 141 by type in the second storage section 14.

[0168] The conveying and sorting unit 12 conveys all of the medicines stored in one first section 111 (e.g., first section 111a) to the second storage section 14 or the waiting tray 15 based on the discrimination result by the discrimination section 65. The conveying and sorting unit 12 stores the medicines whose types can be discriminated by type in the respective sorting cups 141 of the second storage section 14. The conveying and sorting unit 12 stores the medicines whose types cannot be discriminated in the waiting tray 15.

[0169] After completing the sorting process for all of the medicines contained in one first compartment 111, the conveying and sorting unit 12 conveys all of the medicines contained in the sorting cups 141 of the second storage section 14 to the packaging mechanism 6. Thereafter, the conveying and sorting unit 12 starts conveying the medicines contained in another first compartment 111 (e.g., first compartment 111b) to the second storage section 14. This allows the medicines contained in the first compartment 111 to be conveyed to the second storage section 14 while there is space in the second storage section 14.

[0170] The conveying / sorting unit 12 performs the above-described conveying process until the medicines have been removed from all of the first compartments 111. That is, in this embodiment, after the sorting process of the medicines stored in the first compartment 111a is completed and the packaging process of the medicines stored in the second storage unit 14 is completed, the sorting process of the medicines stored in the first compartment 111b is started. Thereafter, after the sorting process of the medicines stored in the first compartment 111b is completed and the packaging process of the medicines stored in the second storage unit 14 is completed, the sorting process of the medicines stored in the first compartment 111c is started. Thereafter, after the sorting process of the medicines stored in the first compartment 111c is completed and the packaging process of the medicines stored in the second storage unit 14 is completed, the sorting process of the medicines stored in the first compartment 111d is started.

[0171] As described in the first embodiment, the packaging mechanism 6 functions as a packaging section that packages the medicines stored in the second storage section 14. The transporting / sorting unit 12 transports all of the medicines stored in one first section 111 to the second storage section 14 or the waiting tray 15, and then transports the medicines stored in the second storage section 14 to the medicine inlet 17. The packaging mechanism 6 then starts packaging the medicines stored in the sorting cups 141 of the second storage section 14. As a result, even if the sorting cups 141 are all filled with medicines, the medicine sorting device 1A can make space in the sorting cups 141 by packaging the medicines. Therefore, the medicine sorting device 1A can transport the next medicine to be sorted to the second storage section 14.

[0172] The reading unit 8 reads information indicating the type of medicine contained in the first container 11. In this embodiment, the reading unit 8 reads a two-dimensional code printed on the medicine packet. Specifically, before the user puts the medicine contained in the medicine packet into the first container 11, the user causes the reading unit 8 to read the two-dimensional code printed on the medicine packet. The reading unit 8 only needs to be able to read the information indicating the type of medicine contained in the medicine packet. For example, if the information printed on the medicine packet is a barcode, the reading unit 8 may be a barcode reader.

[0173] The display unit 32 notifies the user to store the medicine in the first compartment 111 designated as the storage destination of the medicine by the compartment designation unit 72 described later. This allows the medicine sorting device 1A to allow the user to store the medicine in the designated first compartment 111. The display unit 32 also functions as a notification unit that notifies the user to change the first compartment 111 as the storage destination of the medicine every time the total number of types of medicine counted by the counting unit 73 described later reaches the total number of sorting cups 141 (40 in this embodiment).

[0174] The notification to store medicines in the designated first compartment 111 does not necessarily have to be made by the display unit 32. For example, the notification may be made by a sound output unit (not shown) that outputs sound. That is, the medicine sorting device 1A only needs to have a notification unit that notifies various information.

[0175] Furthermore, after the sorting process for all of the medicines contained in one first compartment 111 has been completed, the packaging by the packaging mechanism 6 may not be performed. In this case, after the sorting process is completed, the display unit 32 may notify the user to replace the sorting cup 141 with a new one that does not contain any medicines. Furthermore, the display unit 32 may notify the user to empty the sorting cup 141 arranged in the second storage unit 14. Even in this case, it is possible to make space in the sorting cup 141 before transporting the medicines contained in the next first compartment 111 to the second storage unit 14.

[0176] The control unit 60Aa further includes, for example, a partition designation unit 72, a counting unit 73, and a type number determination unit 74 in addition to the configuration of the control unit 60a.

[0177] The compartment designation unit 72 designates one of the first compartments 111a to 111d as the first compartment 111 for storing the medicine. The compartment designation unit 72 changes the designated first compartment 111 every time the type number determination unit 74 determines that the total number of types of medicine counted by the counting unit 73 (total number of types of medicine) exceeds the total number of sorting cups 141.

[0178] The first compartments 111 designated by the compartment designation unit 72 as the initial positions when the medicine sorting process is started, and the designation order of the first compartments 111 by the compartment designation unit 72 are set in advance. Therefore, the compartment designation unit 72 changes the first compartments 111 to store the medicines in accordance with the predetermined designation order every time it is determined that the total number of types of medicine counted by the counting unit 73 exceeds the total number of sorting cups 141. The compartment designation unit 72 transmits information indicating the designated first compartment 111 to the display control unit 67, for example, every time it receives a determination result from the type number determination unit 74.

[0179] For example, it is assumed that the first compartment 111a is set as the initial location for storing medicines, and the order of designation of the first compartments 111 is set as first compartment 111a, first compartment 111b, first compartment 111c, and first compartment 111d. In this case, in this embodiment, when the medicine sorting process is started, the compartment designation unit 72 designates the first compartment 111a as the location for storing medicines. As a result, the display control unit 67 displays on the display unit 32 an image for notifying the user to store medicines in the first compartment 111a. Thereafter, each time it is determined that the total number of types of medicines counted by the counting unit 73 has reached the total number of sorting cups 141, the compartment designation unit 72 changes the location for storing medicines to the first compartment 111b, first compartment 111c, and first compartment 111d, in that order. As a result, the display control unit 67 displays on the display unit 32 (i) an image for notifying the user to store medicines in the first compartment 111b. (ii) an image for informing the user to store medicine in the first section 111c; and (iii) An image for informing the user to store medicine in the first section 111d is displayed in sequence.

[0180] Counting unit 73 counts the number of types of medicine contained in first container 11 based on the two-dimensional code read by reading unit 8. When reading unit 8 reads the two-dimensional code printed on the medicine packet, it transmits the reading result (information indicating the type of medicine indicated by the two-dimensional code) to counting unit 73. Counting unit 73 counts the number of types of medicine contained in the medicine packet by analyzing the reading result. In other words, because the two-dimensional code is printed on the medicine packet, counting unit 73 can count the number of types of medicine contained in the medicine packet.

[0181] As described above, the user causes the reading unit 8 to read the two-dimensional code printed on the medicine packet before putting the medicine contained in the medicine packet into the first container 11. Therefore, the counting unit 73 can count the number of types of medicine contained in the medicine packet, thereby counting the number of types of medicine to be contained in the first container 11.

[0182] However, if the types of medicine indicated by the acquired two-dimensional code include a medicine that matches a type of medicine that has already been counted, the counting unit 73 excludes the matching medicine from the counting target. For example, consider a case where the medicines to be stored are medicines ME to MG and the medicines that have already been counted (medicines stored in one first compartment 111) are medicines MA to MF. In this case, the counting unit 73 excludes medicines ME and MF from the counting target among the three types of medicines ME to MG, and counts the number of types of medicines to be stored as one type.

[0183] The counting unit 73 updates the total number of types of drugs by adding the number of types of drugs counted this time to the sum of the numbers of types of drugs counted up to that point (total number of types). The counting unit 73 updates the total number of types of drugs until it receives a reset instruction to reset the total number of types of drugs from the number-of-types determination unit 74. The counting unit 73 transmits information indicating the updated total number of types of drugs to the number-of-types determination unit 74. The counting unit 73 temporarily stores the total number of types of drugs.

[0184] The number-of-types determination unit 74 determines whether the total number of types of medicines indicated by the information received from the counting unit 73 reaches the total number of sorting cups 141. The number-of-types determination unit 74 transmits the determination result to the section designation unit 72. This enables the section designation unit 72 to determine whether or not the first section 111 needs to be changed.

[0185] In the first embodiment, the medicine sorting device 1 uses, for at least the medicine marks, an extracted mark image 83 obtained by inputting the captured image 82 into a trained model 84 to determine the type of medicine. However, in the present embodiment, the medicine sorting device 1A may extract the medicine marks from the captured image 82 using existing image processing without using the trained model 84.

[0186] [Processing when storing medicine in the first storage section] Next, an example of the process when medicines are input into the first container 11 will be described with reference to FIG. 15. FIG. 15 is a flowchart showing an example of the process flow. In this example, it is assumed that the first compartment 111a is designated as the destination for storing medicines when the medicine sorting process is started. It is also assumed that the destination for storing medicines is changed in the order of first compartment 111a, first compartment 111b, first compartment 111c, and first compartment 111d.

[0187] When the first compartment 111a is designated as the location where the medicine is to be stored, the control unit 60Aa determines whether the reading unit 8 has read the two-dimensional code printed on the medicine packet. That is, the control unit 60Aa determines whether the two-dimensional code read by the reading unit 8 has been acquired from the reading unit 8 (S11).

[0188] Before putting the medicine contained in the medicine packet into first container 11, the user causes reading unit 8 to read the two-dimensional code printed on the medicine packet. When the user causes reading unit 8 to read the two-dimensional code, control unit 60Aa acquires the two-dimensional code from reading unit 8. Display control unit 67 may cause display unit 32 to display an image prompting the user to cause reading unit 8 to read the two-dimensional code printed on the medicine packet before putting the medicine contained in the medicine packet into first container 11.

[0189] When the control unit 60Aa determines that a two-dimensional code has been acquired (YES in S11), the counting unit 73 counts the number of types of medicine contained in the medicine packet based on the information indicating the type of medicine indicated by the acquired two-dimensional code (S12). That is, the counting unit 73 counts the number of types of medicine to be contained in the first container 11. Thereafter, the counting unit 73 updates the total number of types of medicine by adding the currently counted number of types of medicine to the total number of types of medicine temporarily stored. Note that the control unit 60Aa performs the process of S11 until the two-dimensional code is read by the reading unit 8 and the two-dimensional code is acquired from the reading unit 8 (NO in S11).

[0190] The number-of-types determination unit 74 determines whether the total number of types of medicines received from the counting unit 73 (the total number of types of medicines after update) has reached the total number of sorting cups 141 (S13).

[0191] Consider a case where the type number determination unit 74 determines that the total number of types of medicines does not reach the total number of sorting cups 141 (NO in S13). In this case, the display control unit 67 displays an image on the display unit 32 to notify the user to store medicines in the first section 111 currently designated by the section designation unit 72 (S15). 15 During the process of S15, if the section designation unit 72 designates the first section 111a, the display control unit 67 notifies the user to store the medicine in the first section 111a. After the process of S15, the process returns to S11.

[0192] On the other hand, if the type number determination unit 74 determines that the total number of types of medicines has reached the total number of sorting cups 141 (YES in S13), the section designation unit 72 determines whether or not all of the first sections 111a to 111d have been designated (S14). In this embodiment, the section designation unit 72 determines whether or not the first section 111d has been designated.

[0193] When it is determined that all of the first compartments 111a to 111d have not been designated (NO in S14), the compartment designation unit 72 changes the currently designated first compartment 111 to the first compartment 111 set as the storage destination for the next medicine. That is, the compartment designation unit 72 designates the first compartment 111 as the storage destination for the next medicine (S16). The display control unit 67 displays on the display unit 32 an image for notifying the user to store the medicine in the first compartment 111 as the storage destination for the next medicine designated by the compartment designation unit 72 (S17).

[0194] In this embodiment, when the compartment designation unit 72 has designated one of the first compartments 111a to 111c, it designates one of the first compartments 111b to 111d as the destination for storing the next medicine. For example, when the compartment designation unit 72 has designated the first compartment 111a as the destination for storing the medicine, it designates the first compartment 111b as the destination for storing the next medicine. Then, the display control unit 67 displays on the display unit 32 an image for notifying the user to store the medicine in the first compartment 111b.

[0195] Thereafter, the control unit 60Aa resets the total number of types of medicines temporarily stored in the counting unit 73 (S18), which enables the counting unit 73 to count the total number of types of medicines contained in the newly designated first section 111. After the processing of S18, the process returns to the processing of S11.

[0196] On the other hand, if the section designation unit 72 determines that all of the first sections 111a to 111d have been designated (YES in S14), the control unit 60Aa ends this process. In this embodiment, if the section designation unit 72 has designated the first section 111d, the control unit 60Aa ends this process. This makes it possible to prevent the first section 111d from containing more medicines than the total number of sorting cups 141.

[0197] It should be noted that not all of the first compartments 111a to 111d will necessarily contain medicines. That is, the first container 11 will not necessarily contain more than 160 types of medicines. Therefore, when the control unit 60Aa receives a user input to start the medicine sorting process, it may conclude that there are no more medicines to be contained in the first container 11 and terminate this process. For example, if the number of types of medicines contained in the first container 11 is 40 or less, this process will terminate with medicines contained only in the first compartment 111a.

[0198] In this way, if the number of types of medicines to be stored in the first storage unit 11 does not exceed the total number of sorting cups 141 when added to the total number of types of medicines stored in the specified first section 111, the control unit 60Aa can store medicines in the specified first section 111. That is, the control unit 60Aa can prompt the user to store medicines in the first section 111 until the number of medicines stored in the first section 111 reaches the total number of sorting cups 141. On the other hand, if the number of types of medicines added to the total number of types of medicines exceeds the total number of sorting cups 141, the control unit 60Aa can store medicines in another first section 111 (an empty first section 111). Therefore, the control unit 60Aa can store as many types of medicines as possible in the first section 111 without exceeding the total number of sorting cups 141.

[0199] Here, consider a case where a drug of the same type as the drug to be stored in the first storage unit 11 is already stored in a first compartment 111 different from the first compartment 111 designated as the storage destination for the drug. In this case, the compartment designation unit 72 may designate the first compartment 111 in which the same type of drug is already stored as the storage destination for the drug to be stored in the first storage unit 11.

[0200] For example, after acquiring the two-dimensional code in S11, the control unit 60Aa determines whether the type of medicine indicated by the two-dimensional code matches the type of medicine contained in any of the first compartments 111 specified so far. If the control unit 60Aa determines that the two types of medicine match, the compartment designation unit 72 identifies a first compartment 111 that contains the same type of medicine as the type indicated by the two-dimensional code and designates the first compartment 111 as the destination for containing the medicine. Then, the display control unit 67 notifies the display unit 32 to contain the medicine in the first compartment 111. On the other hand, if the control unit 60Aa determines that the two types of medicine do not match, the counting unit 73 counts the types of medicine in S12.

[0201] [Medicine sorting process] Next, the medicine sorting process executed after medicines are stored in the first storage unit 11 will be described with reference to FIG. 16. FIG. 16 is a flowchart showing an example of the flow of the medicine sorting process. This medicine sorting process may be executed by the control unit 60Aa, for example, when a user input for starting the medicine sorting process is received. In this example, the medicines stored in the first compartment 111 are removed in the order of first compartment 111a, first compartment 111b, first compartment 111c, and first compartment 111d, transported to the second storage unit 14, and stored in sorting cups 141 by type.

[0202] The transport control unit 61 controls the transporting and sorting unit 12 to take out one of the medicines stored in one of the first sections 111 identified as the medicine removal target (S21). The imaging control unit 63 controls the first camera 131 to capture an image of the medicine removed by the transporting and sorting unit 12 (S22). The discrimination unit 65 discriminates the type of the medicine based on the image of the medicine captured by the first camera 131 (S23). The sorting control unit 62 determines a sorting cup 141 (sorting position) to store the medicine whose type has been determined. Then, the sorting control unit 62 controls the transporting and sorting unit 12 to store the medicine in the sorting cup 141 determined as the storage destination for the medicine (S24).

[0203] The transport control unit 61 determines whether all medicines have been removed from the first compartment 111 that was the target for medicine removal (S25). When the transport / sorting unit 12 removes medicines from the first compartment 111, the imaging control unit 63 controls the second camera 121 to capture an image of the first compartment 111 that is the target for medicine removal. The imaging control unit 63 analyzes the image of the first compartment 111 captured by the second camera 121 to determine whether or not the first compartment 111 that is the target for medicine removal contains medicine. If the imaging control unit 63 determines that the first compartment 111 does not contain medicine, the transport control unit 61 determines that all medicines have been removed from the first compartment 111 that is the target for medicine removal.

[0204] If the transport control unit 61 determines that not all the medicines have been removed from the first section 111 from which the medicines are to be removed (NO in S25), the process of S21 is performed because medicines are still contained in the first section 111. On the other hand, if the transport control unit 61 determines that all the medicines have been removed from the first section 111 (YES in S25), the transport control unit 61 and the sorting control unit 62 temporarily suspend the transport and sorting of the medicines. In other words, the transport control unit 61 does not perform the transport process of the medicines contained in the first section 111 that contains the next medicines to be sorted.

[0205] Instead, the packaging control unit 71 controls the transporting / sorting unit 12 to transport the medicines contained in the sorting cups 141 to the medicine inlet 17. Thereafter, the packaging control unit 71 controls the packaging mechanism 6 to package the medicines contained in the sorting cups 141 (S26). The packaging control unit 71 executes the process of S26 for all sorting cups 141 that contain medicines.

[0206] The sorting cup 141 containing the medicine can be identified, for example, by storing the sorting position determined by the sorting control unit 62 in the memory unit 80. Whether or not the medicine is contained in the sorting cup 141 may be determined, for example, by the imaging control unit 63 causing the second camera 121 to capture an image of the sorting cup 141 and analyzing the image of the sorting cup 141.

[0207] The packaging control unit 71 determines whether all of the medicines contained in the second container 14 have been packaged (S27). That is, the packaging control unit 71 determines whether all of the medicines contained in the sorting cups 141 have been packaged for all of the sorting cups 141 containing medicines.

[0208] The packaging control unit 71 performs the process of S26 until all of the medicines contained in the second container 14 have been packaged (NO in S27). On the other hand, if it is determined that all of the medicines contained in the second container 14 have been packaged (YES in S27), the transport control unit 61 determines whether removal of the medicines from all of the first compartments 111 containing medicines has been completed (S28).

[0209] If the transport control unit 61 determines that removal of medicines from all of the first compartments 111 containing medicines has not been completed (NO in S28), the process returns to S21. That is, in this case, the control unit 60Aa can resume the sorting process of medicines contained in the first compartment 111 by removing medicines from the first compartment 111 from which medicines are to be removed next. On the other hand, if the transport control unit 61 determines that removal of medicines from all of the first compartments 111 containing medicines has been completed (YES in S28), the control unit 60Aa ends this medicine sorting process.

[0210] For example, the control unit 60Aa manages the first sections 111 designated by the section designation unit 72, allowing the transport control unit 61 to execute the process of S28. For example, if the last first section 111 designated by the section designation unit 72 is the first section 111c, the transport control unit 61 determines in S28 whether removal of medicines from the first section 111c is complete. For example, if packaging processing has been completed for all medicines contained in the first section 111a, the transport control unit 61 determines that removal of medicines from the first section 111c is not complete. In this case, the transport control unit 61 identifies the first section 111b as the next medicine removal target and removes the medicine from the first section 111b in S21. Thereafter, when packaging processing has been completed for all medicines contained in the first section 111c, the control unit 60Aa ends this medicine sorting process.

[0211] In this way, the control unit 60Aa stores all of the medicines stored in one first section 111 and whose types have been identified in the second storage unit 14. Then, the control unit 60Aa packages all of the medicines stored in the second storage unit 14 and empties all of the sorting cups 141, and then starts the sorting process for the medicines stored in another first section 111. As described above, the first section 111 stores a number of types of medicines equal to or less than the total number of sorting cups 141. Therefore, the control unit 60Aa can store all of the medicines whose types have been identified in the second storage unit 14 without transporting them to the standby tray 15.

[0212] In the present embodiment, the control unit 60Aa executes the packaging process in S26 to empty all of the sorting cups 141, but this is not limited to this. That is, the control unit 60Aa may execute the following process instead of the processes of S26 and S27. For example, the control unit 60Aa may prompt the user to replace the sorting cup 141 with a new one that does not contain any medicine. The control unit 60Aa may also prompt the user to empty the sorting cup 141 placed in the second container 14. Thereafter, when the control unit 60Aa receives a user input indicating that the sorting cup 141 has been emptied, it may execute the process of S28.

[0213] Furthermore, when the transport control unit 61 determines that all medicines have been removed from the first section 111 (YES in S25), the section designation unit 72 may designate the first section 111 as the storage destination for the medicines. Then, the display control unit 67 may notify the user to store the medicines in the first section 111. In this case, even if the sorting process for the medicines stored in the subsequent first section 111 has not been completed, the medicines can be stored in the empty first section 111.

[0214] [Major Effects of the Present Embodiment] As described above, the medicine sorting device 1A in this embodiment issues a notification to change the first compartment 111 that stores the medicines each time the total number of types of medicines counted by the counting unit 73 reaches the total number of sorting cups 141. Therefore, one first compartment 111 can store medicines whose number of types is equal to or less than the total number of sorting cups 141.

[0215] Then, the medicine sorting device 1A transports the medicines stored in the first storage unit 11 to the second storage unit 14 for each first section 111, and sorts the medicines by type in the second storage unit 14. Therefore, in one sorting process, the number of medicines to be sorted does not exceed the number of types that can be stored in the second storage unit 14. Therefore, the medicine sorting device 1A can store the medicines whose types have been determined in the second storage unit 14 by type without having them wait in a standby position (e.g., standby tray 15) different from the second storage unit 14. In other words, the medicine sorting device 1A can reduce the time required for the medicine sorting process by the processing time that would be required if the medicines whose types have been determined were waited in a standby position.

[0216] [Embodiment 4] In the medicine sorting device 1 of embodiment 1, when medicines are to be removed from the sorting cup 141, the imaging control unit 63 causes the second camera 121 to capture an image of the inside of the sorting cup 141. Then, the packaging control unit 71 identifies the medicines to be removed from the image of the inside of the sorting cup 141, and moves the suction mechanism of the suction / shutter mechanism 122 vertically downward (in the -Z axis direction).

[0217] However, the imaging control unit 63 only takes one two-dimensional image. Therefore, the imaging control unit 63 may identify the medicines at the bottom, which are partially overlapping and contained in the sorting cup 141, as the medicines to be removed from the image. In this case, when the suction mechanism removes the medicines at the bottom, the medicines at the top may be splashed out of the sorting cup 141.

[0218] Furthermore, when the suction mechanism suctions the medicines, it may push the end of the medicine from the vertically upward direction (+Z-axis direction). In this case, the medicines may fly out of the sorting cup 141. To reduce this possibility, the packaging control unit 71 reduces the speed at which the suction mechanism descends when moving the suction mechanism vertically downward. However, the packaging control unit 71 uniformly reduces the speed at which the suction mechanism descends from a certain predetermined fixed position, regardless of the amount of medicines contained in the sorting cup 141. Therefore, the speed at which the medicines are removed from the sorting cup 141 may decrease, regardless of the amount of medicines contained in the sorting cup 141.

[0219] Therefore, in this embodiment, when removing medicine from the sorting cup 141, the second camera 121 is made to take images of the inside of the sorting cup 141 from multiple positions, thereby reducing the possibility of removing the medicine from the bottom first and the possibility of the removal speed being reduced.

[0220] FIG. 17 is a diagram illustrating a method for capturing an image of the inside of the sorting cup 141. As shown in FIG. 17, the control unit 60a moves the second camera 121 to a first position Po1, which is a predetermined distance away from the center line CL of the sorting cup 141 from which medicines are extracted. The imaging control unit 63 then captures an image of the inside of the sorting cup 141 at the first position Po1. Next, the control unit 60a moves the second camera 121 from the first position Po1 to a second position Po2, which is a predetermined distance away from the center line CL. The imaging control unit 63 then captures an image of the inside of the sorting cup 141 at the second position Po2. Note that the second camera 121 may capture an image of the inside of the sorting cup 141 at the second position Po2, and then capture an image at the first position Po1. In practice, the second camera 121 moves together with the suction / shutter mechanism 122.

[0221] The control unit 60a calculates the parallax D, which is the difference in image position between the first image and the second image, based on the first image acquired at the first position Po1 and the second image acquired at the second position Po2. A known technique can be used to calculate the parallax D.

[0222] Here, the principle of distance measurement using parallax will be explained. As shown in FIG. 17, (i) an object (ii) incident light IL1 from the object to the second camera 121 located at the first position Po1; (iii) inter-camera distance B; , the first triangle Tr1 is formed by (i) incident light IL1 and (ii) incident light IL2. A second triangle Tr2 is also formed by (iii) a straight line PL that is parallel to the first triangle and passes through the focal position of a lens (not shown) of the second camera 121, and (iv) the imaging surface of the second camera 121. The imaging plane of the second camera 121 represents the parallax D. As shown in Fig. 17, the first triangle Tr1 and the second triangle Tr2 are in a similar relationship. Therefore, due to this similar relationship, the control unit 60a can calculate the distance Z using the formula: inter-camera distance B × focal length F / parallax D.

[0223] The inter-camera distance B refers to the distance between the first position Po1 and the second position Po2. The inter-camera distance B is the movement distance of the second camera 121 and is set in advance. The focal length F is the focal length of the lens provided in the second camera 121 and is an inherent value of the second camera 121 employed. Therefore, the control unit 60a calculates the parallax D as described above and substitutes the parallax D into the above equation, thereby being able to calculate the distance Z from the focal position of the lens provided in the second camera 121 to the target (e.g., the drug MDA).

[0224] As described above, the control unit 60a calculates the distance Z by moving the second camera 121 horizontally to capture images of the medicine from two locations. This allows the medicine sorting device 1 to adjust the speed of descent of the suction mechanism for each medicine contained in the sorting cup 141 based on the distance Z before the medicine is adsorbed. This reduces the possibility of a decrease in the speed at which the medicine is taken out of the sorting cup 141. Furthermore, the suction mechanism can appropriately adsorb the medicine contained in the sorting cup 141.

[0225] Furthermore, the control unit 60a can calculate the distance Z for each medicine contained in the sorting cup 141. Therefore, the suction mechanism can sequentially pick up the medicines from the top. Therefore, by the suction mechanism first removing the medicines from the bottom, the possibility that the medicines from the top will fly out of the sorting cup 141 can be reduced.

[0226] In this way, the control unit 60a can reduce the above-mentioned possibility by simply capturing images of the medicines contained in the sorting cup 141 twice and calculating the distance Z without making any mechanical changes to the medicine sorting device 1. In other words, the medicine sorting device 1 can calculate the distance Z without being equipped with, for example, a distance measuring sensor or a stereo camera. Therefore, the medicine sorting device 1 can calculate the distance Z without incurring the cost of installing a distance measuring sensor or the like to reduce the above-mentioned possibility. Furthermore, the medicine sorting device 1 can calculate the distance Z using a space-saving and simple method.

[0227] In this embodiment, an example has been described in which medicines are removed from the sorting cup 141, but the medicine sorting device 1 can also remove medicines from the first storage unit 11 by using the above-described method for calculating the distance Z. The medicine removal method using the above-described method for calculating the distance Z can also be realized by the medicine sorting device 1A. Furthermore, the medicine removal method is not limited to medicine sorting devices, but can also be applied to a picking system that removes objects from a horizontal plane (XY stage).

[0228] [Software implementation example] The control blocks (particularly the control units 60a, 60Aa) of the medicine sorting devices 1, 1A may be realized by a logic circuit (hardware) formed on an integrated circuit (IC chip) or the like, or may be realized by software.

[0229] In the latter case, the medicine sorting device 1, 1A is equipped with a computer that executes instructions of a program (type discrimination program), which is software that realizes each function. This computer is equipped with, for example, one or more processors, and a computer-readable recording medium that stores the program. The object of the present invention is achieved by the processor in the computer reading and executing the program from the recording medium. For example, a CPU (Central Processing Unit) can be used as the processor. The recording medium can be a "non-transitory tangible medium", for example. For example, in addition to a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, etc. may be used. The computer may further include a RAM (Random Access Memory) for expanding the program. The program may be supplied to the computer via any transmission medium capable of transmitting the program (such as a communication network or broadcast waves). One aspect of the present invention may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[0230] [Function that makes it easier to organize packaged items] The medicine sorting device 1, 1A has a packaging mechanism 6, and medicines that have been sorted are sequentially transported to the packaging mechanism 6, where they are packaged into sorting cups 141 containing medicines of the same type. At this time, the number of medicines contained in each package is fixed, and if a large number of medicines are contained in the sorting cup 141, they are packaged in multiple packages. The packaging process is performed automatically, and a predetermined number of packaged bags are packaged in a strip and discharged from the medicine sorting device 1, 1A. The discharged packets are stored in a storage box or the like (not shown) provided outside the device. Therefore, an operator must remove the packets to find which medicine is located in which position, which is time-consuming and labor-intensive.

[0231] In particular, after being sorted by the medicine sorting device 1, 1A, the medicines of each drug type packaged by the packaging mechanism 6 must be returned to a tablet packaging machine or tablet storage shelf, separate from the medicine sorting device 1, 1A, so it is necessary to remove all the packaging bags containing the same drug type. However, because the packaging bags dispensed by the packaging mechanism 6 are stacked in a storage box, there is a problem in that all the bags must be picked up and visually inspected, and the bag containing the drug must be individually found and removed. In addition, there are cases where packaging bags of the same drug type do not exist in the same packaging bag, and unless other groups of packaging bags are also searched, it is not possible to finally check all the packaging bags thoroughly, which is cumbersome work.

[0232] Therefore, when the packaging operation is completed, the display unit 32 of the touch panel 3 displays the information "Packaging completed" along with a button "Check packaging results," and by touching or clicking this button, the number of sachets packaged in this packaging operation can be displayed for each drug. Furthermore, the information regarding the packaging results displayed on the display unit 32 may be printed on a printed material such as a journal that can be carried when searching for the actual sachets from their storage location.

[0233] Specifically, the name of the drug and the number of sachets packaged are displayed or printed on the same line, and similarly for other drugs, the name of the drug and the number of sachets packaged for each drug are displayed or printed on the printout. This allows the worker to know the number of sachets for each drug after packaging is complete, so when actually removing the sachets for the drug from the stored group of sachets, they only need to find the number of sachets displayed or printed, so there is no need to worry about missing a drug, and as long as they have removed the displayed number of sachets, they can move on to the task of removing the next drug sachets, which shortens the work of removing all sachets from a storage box, etc., compared to conventional methods.

[0234] [Additional Notes] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0235] 6 Packaging mechanism (packaging section) 8 Reading unit 11 First storage section 12 Conveying and sorting unit (conveying section) 14 Second storage section 15 Waiting tray (temporary storage area) 32 Display unit (alert unit) 60 Computer (type determination device) 64 Feature extraction unit (image generation unit) 65 Discrimination part 73 Counting Department 82 Captured images 83 Extraction mark image 84 trained models 111, 111a~111d Section 1 141 Sorting cup (Second Section)

Claims

1. an image generation unit that generates an extracted mark image by extracting a mark shown in a captured image of an unknown target drug based on an output value obtained by inputting the captured image into a trained model constructed to extract a mark formed on the drug; a drug database for managing drug data relating to a plurality of types of drugs; a discrimination unit that discriminates the type of the target drug based on a result of matching the extracted mark image generated by the image generation unit with a registered mark image pre-registered in the drug database for each type of drug, and a result of matching the feature other than the mark of the target drug extracted from the captured image by image processing of the captured image with the feature other than the mark pre-registered in the drug database for each type of drug, The discrimination unit discriminates the type of the target drug without using a trained model, A type discrimination device in which no processing using a trained model is performed between the time the extraction mark image is generated by the image generation unit and the time the discrimination unit discriminates the type of the target drug.

2. The type determination device according to claim 1 , wherein the mark is an engraving or a dividing line.

3. The type discrimination device according to claim 1 or 2, wherein the trained model is a semantic segmentation model trained to distinguish between pixels constituting a mark in an image of a drug and pixels constituting a background portion of the mark.

4. The type discrimination device according to claim 1 , wherein the discrimination unit discriminates the type of the target drug based on the result of comparing an image obtained by blurring the extracted mark image with the registered mark image.

5. A type determination method executed by a type determination device, an image generation step of generating an extracted mark image by extracting a mark shown in a captured image of an unknown target drug based on an output value obtained by inputting the captured image into a trained model constructed to extract a mark formed on the drug; a determination step of determining the type of the target drug based on a result of matching the extracted mark image generated in the image generation step with a registered mark image registered in advance for each type of drug in a drug database that manages drug data on multiple types of drugs, and a result of matching the feature other than the mark of the target drug extracted from the captured image by image processing on the captured image with the feature other than the mark registered in advance for each type of drug in the drug database, In the discrimination step, the type of the target drug is discriminated without using a trained model; A type discrimination method in which no processing using a trained model is performed between the time the extraction mark image is generated in the image generation step and the time the type of the target drug is discriminated in the discrimination step.

6. The type discrimination method described in claim 5, wherein the trained model is a semantic segmentation model trained to distinguish between pixels that constitute a mark in an image of a drug and pixels that constitute a background portion of the mark.

7. On the computer, an image generation step of generating an extracted mark image by extracting a mark shown in a captured image of an unknown target drug based on an output value obtained by inputting the captured image into a trained model constructed to extract a mark formed on the drug; a determination step of determining the type of the target drug based on a result of matching the extracted mark image generated in the image generation step with a registered mark image registered in advance for each type of drug in a drug database that manages drug data on multiple types of drugs, and a result of matching the features other than the mark of the target drug extracted from the captured image by image processing on the captured image with the features other than the mark registered in advance for each type of drug in the drug database, In the discrimination step, the type of the target drug is discriminated without using a trained model; A type discrimination program in which no processing using a trained model is performed between the time the extraction mark image is generated by the image generation step and the time the type of the target drug is discriminated by the discrimination step.

8. The type discrimination program according to claim 7, wherein the trained model is a semantic segmentation model trained to distinguish between pixels constituting a mark in an image of a drug and pixels constituting a background portion of the mark.

Images