Drug Identification System

Abstract

To provide a medicine identification system capable of calculating a quantity of medicines based on a weight value without requiring communication by a weight scale for measuring a weight value of the medicines.SOLUTION: A weight scale 20 includes a weighing stage 22 for supporting medicines 12 and a weight display part 24 for indicating a weight value of the medicines 12. An imaging part 32 images a weight meter image in which both the medicines 12 and a weight display part 24 are shown. A medicine identification part 37 executes identification of the medicines 12 shown in the weight meter image and acquisition of the weight value indicated by the weight display part 24 by executing image recognition of the weight meter image, and calculates a quantity of the medicines 12 on the basis of the result of the identification of the medicines 12, and the acquired weight value.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a medication identification system. [Background technology]

[0002] In pharmacies and other facilities that handle medicines, mechanical identification of medicines is sometimes required. For example, a device that performs mechanical identification of medicines may be used to determine, by a party other than the operator (such as a pharmacist), whether the medicines stocked based on a prescription actually match the prescription contents. Using such a device can prevent operational errors when dispensing medicines to patients.

[0003] In the dispensing inspection device described in Patent Document 1, the drugs to be inspected are imaged by an imaging device, and a central control device determines the names of the drugs to be inspected based on the image information from the imaging device. Furthermore, the drugs to be inspected are weighed by a weighing device, and the central control device determines the appropriateness of the dispensing contents, including the quantity of the drugs to be inspected, using the weighing data from the weighing device. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5725363 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the dispensing inspection device described in Patent Document 1 has a weighing device and an imaging device built into it, which makes the dispensing inspection device large. Since large dispensing inspection devices are difficult to move, the dispensing inspection device is installed in a fixed location. Therefore, the dispensing inspection device described in Patent Document 1 cannot be used unless the user carries the medication to the location of the dispensing inspection device, which reduces convenience.

[0006] Furthermore, even if the weighing device is not incorporated into the dispensing inspection device but is implemented as a separate device from the dispensing inspection device, in order for the central control device to use the weighing data from the weighing device, the weighing device and the dispensing inspection device including the central control device must communicate with each other. Therefore, the weighing device and the dispensing inspection device must be connected by a wired communication cable or configured to be able to send and receive data wirelessly via a wireless communication device. In other words, the weighing device must be an expensive device with some kind of communication function; inexpensive weighing devices without communication functions cannot be used with the dispensing inspection device.

[0007] Furthermore, when the weighing device and the dispensing inspection device are connected via a communication cable, the distance between the weighing device and the dispensing inspection device is limited by the length of the communication cable, making it inconvenient to handle each device. Furthermore, in order to wirelessly transmit and receive data via a wireless communication device, power must be supplied not only to the weighing device and the dispensing inspection device but also to the wireless communication device, making it inconvenient to secure a power source and handle the power supply cable. Furthermore, in either the wired or wireless connection, setting a communication protocol, etc. is required to transmit and receive data between the weighing device and the dispensing inspection device, and performing such communication setting work is a burden on the user.

[0008] In view of the above problems, the present invention aims to provide a drug identification system that can calculate the quantity of drugs based on weight values ​​without the weighing device (weighing scale) that measures the weight values ​​of the drugs performing communication. [Means for solving the problem]

[0009] In order to solve the above problem, one aspect of the present invention provides a drug identification system comprising at least one weighing scale that measures the weight value of the drug to be identified, at least one photographing unit that can photograph an image including the drug, and at least one drug identification unit that identifies the drug appearing in the photographed image, wherein the weighing scale has a weighing platform that can support the drug and a weight display unit that indicates the weight value of the drug supported on the weighing platform, the photographing unit is capable of photographing a drug image that shows both the drug supported on the weighing platform and the weight display unit that displays the weight value of the drug, and the drug identification unit performs image recognition of the drug image to identify the drug appearing in the drug image and obtain the weight value indicated by the weight display unit, and performs a quantity calculation process to calculate the drug appearing in the drug image based on the drug identification result and the obtained weight value.

[0010] Preferably, when there are multiple types of drugs to be identified, the photographing unit photographs an image of an assembly of drugs showing the multiple types of drugs, the drug identification unit performs image recognition of the image of the assembly of drugs to identify the types and quantities of the drugs shown in the image of the assembly of drugs, and the quantity calculation process calculates the quantity of the drugs whose quantity has not been identified in the image recognition of the image of the assembly of drugs.

[0011] It is also preferable that a plurality of the weighing scales are provided, and a plurality of terminal devices each having the photographing unit are provided, and that the weighing photographing image can be captured by photographing any of the weighing scales with the photographing unit of any of the terminal devices.

[0012] It is also preferable that the metered photographed image includes any drug accessories attached to the drug, and the drug identification unit calculates the quantity of the drug in the quantity calculation process by taking into account the weight of the drug accessories.

[0013] It is also preferable that the drug identification system is used together with a medical system of a medical institution, and prescription data indicating information including the name and quantity of the drug written on a prescription issued by a doctor is input into the medical system, and the drug identification unit is able to refer to the prescription data, and in the quantity calculation process, the name and quantity of the drug shown in the weighing image are compared with the information indicated in the prescription data. [Effects of the Invention]

[0014] According to a drug identification system according to one aspect of the present invention, a weighing image is captured by an image capture unit, and the quantity of the drug is calculated by image recognition of the weighing image. Therefore, the weighing scale does not need to communicate with the drug identification unit to calculate the quantity of the drug. Therefore, the user does not need to set up communication between the weighing scale and the drug identification unit, reducing the user's workload. Furthermore, this drug identification system can use an inexpensive weighing scale without communication capabilities.

[0015] Furthermore, when identifying the types and quantities of drugs in a collection of drug images by performing image recognition on the collection of drug images that show multiple types of drugs, drugs whose quantities can be identified solely through image recognition of the collection of drug images without using a weighing scale can be excluded from the drugs placed on the weighing platform when capturing a weighing image. This reduces the number of drug types that are the subject of quantity calculation, and improves the accuracy of quantity calculation based on image recognition of the weighing image.

[0016] Furthermore, even if multiple weighing scales and terminal devices are installed, if it is possible to perform quantity calculation processing by photographing any weighing scale using the photographing unit of any terminal device, the user will not need to perform complex communication settings between multiple weighing scales and terminal devices, thereby reducing the user's workload.

[0017] In addition, when the quantity calculation process calculates the quantity of a drug taking into account the weight of the drug accessories, the user does not need to remove the drug accessories from the drug when placing the drug on the weighing platform, thereby reducing the user's workload.

[0018] In addition, if the name and quantity of the drug shown in the measurement image during the quantity calculation process are compared with the information shown in the prescription data, the user can compare the drug with the prescription data simply by taking a measurement image, thereby reducing the user's workload. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a system configuration diagram showing an example of an embodiment of a medicine identification system according to the present invention. [Figure 2] FIG. 10 is a diagram showing a state in which a metering image is displayed on a terminal display unit. [Figure 3] FIG. 10 is a diagram showing how an aggregated medicine image is captured. [Figure 4] 4 is a diagram showing the state in which the image recognition results of the collected medicine photograph image of FIG. 3 are displayed on the terminal display unit. [Figure 5] FIG. 10 is a diagram showing a state in which a quantity calculation result based on a measurement-captured image is displayed on a terminal display unit. [Figure 6] FIG. 10 is a system configuration diagram showing another embodiment of a medicine identification system. [Figure 7] FIG. 10 is a diagram showing an example of prescription data. [Figure 8] 7 is a diagram showing a state in which the result of matching the identification result with prescription data is displayed on the terminal display unit in the drug identification system of FIG. 6. [Figure 9] 3 is a flowchart showing the processing flow in the medicine identification system of the present invention. [Figure 10] 10 is a flowchart showing the flow of processing in a quantity calculation process. [Figure 11] FIG. 10 is a diagram showing a state in which a plurality of terminal devices and a plurality of weighing scales are provided. DETAILED DESCRIPTION OF THE INVENTION

[0020] (System Configuration) The system configuration diagram in Figure 1 shows an example of an embodiment of a drug identification system 10 according to the present invention. The drug identification system 10 of this embodiment includes a weighing scale 20 and a terminal device 30. As an example of a drug 12 to be identified in this drug identification system 10, Figure 1 shows one PTP sheet containing a set of 10 tablets.

[0021] The weighing scale 20 is an instrument that measures the weight of the drug 12 to be identified in the drug identification system 10, and is, for example, an electronic dispensing balance installed in a dispensing pharmacy. The weighing scale 20 has a weighing table 22 that can support the drug 12. The weighing table 22 in Figure 1 is a dish-shaped member with a flat top surface, and can stably support the drug 12 placed on the top surface of the weighing table 22.

[0022] The weighing scale 20 also includes a weight display unit 24. The weight display unit 24 indicates the weight of the medication 12 supported by the weighing platform 22. In FIG. 1, "3.0 g" is displayed as the weight of the medication 12. The weight display unit 24 is typically configured with an LED segment or an LCD display. Note that the weight display unit 24 may be configured with anything that can visually display the weight of the medication 12. For example, the weight display unit 24 may be configured with a scale plate having markings engraved along the circumferential direction of an arc, and a pointer on the surface of the scale plate that rotates at an angle according to the weight of the medication 12 and points to the scale corresponding to the weight value.

[0023] The terminal device 30 is a device equipped with a photographing unit 32 (such as a camera). Typically, the terminal device 30 is a device that can be carried by a user, such as a tablet device or a smartphone.

[0024] In addition to the photographing unit 32, the terminal device 30 is equipped with a drug identification unit 37, a terminal display unit 38, and a control unit 39. The drug identification unit 37 identifies the drug 12 shown in the image photographed by the photographing unit 32 by image recognition. The terminal display unit 38 displays various information to the user. The control unit 39 controls the operation of the entire terminal device 30, including the photographing unit 32, the drug identification unit 37, and the terminal display unit 38.

[0025] When a tablet device or a smartphone is used as the terminal device 30, a camera included in the terminal device 30 functions as the photographing unit 32. A display panel (for example, a touch panel display that also serves as an input device) included in the terminal device 30 functions as the terminal display unit 38. A microcomputer such as a CPU that controls the operation of the terminal device 30 functions as the control unit 39.

[0026] The drug identification unit 37 may be implemented as hardware or software. For example, the drug identification unit 37 may be implemented as hardware by a microcomputer separate from the control unit 39. The drug identification unit 37 may also be implemented as software by an image processing program executed by the control unit 39. In substance, the drug identification unit 37 may be a storage device that stores the image processing program. When the drug identification unit 37 is implemented as software, any device (e.g., a computer) on which an application that executes each process performed in the drug identification system 10 of this embodiment can be installed can be used as the terminal device 30. In other words, the terminal device 30 is not limited to portable devices such as tablet devices and smartphones, and stationary devices such as conventional dispensing inspection devices can also be used as the terminal device 30.

[0027] (Metrophotography image capture) The photographing unit 32 of the terminal device 30 photographs an area including the medication 12, thereby capturing an image including the medication 12. In a tablet device or smartphone, when photographing, the terminal display unit 38 can display the photographed image that will be obtained as a result of the photographing. In the medication identification system 10 of FIG. 1, the photographing unit 32 photographs the entire weighing scale 20, including the weighing platform 22, as shown in FIG. 2. The image captured of the entire weighing scale 20 includes the medication 12 supported by the weighing platform 22 and the weight display unit 24 displaying the weight of the medication 12. Hereinafter, an image that shows both the medication 12 supported by the weighing platform 22 and the weight display unit 24 displaying the weight of the medication 12, as displayed on the terminal display unit 38 of FIG. 2, will be referred to as a "photographed weighing image."

[0028] (Image Recognition) The captured image taken by the photographing unit 32 is transmitted to the drug identifying unit 37, which performs image recognition of the captured image. The drug identifying unit 37 in Fig. 1 includes an image recognition unit 33 and a calculation unit 35. The image recognition unit 33 has an AI model 34 (which actually is a storage device that stores data of the AI ​​model 34). The calculation unit 35 has a drug information list 36 (which actually is a storage device that stores data of the drug information list 36).

[0029] As will be described in detail later, the image recognition unit 33 uses an AI model 34 to extract the characteristic values ​​of the drug 12 and the weight value of the weight display unit 24 (the numerical value indicating the weight of the drug 12 displayed on the weight display unit 24) from the image captured by the imaging unit 32, and transmits these to the calculation unit 35. Then, based on the characteristic values ​​of the drug 12 and the weight value of the weight display unit 24, the calculation unit 35 refers to the drug information list 36, identifies the name of the drug 12, etc., and calculates the quantity of the drug 12, and transmits the identification result and the quantity to the terminal display unit 38 for display.

[0030] The AI ​​model 34 is a model that describes an input / output relationship in which a photographed image is used as input data and corresponding feature values ​​are returned as output data. When the input data is the above-mentioned weighing photographed image, the AI ​​model 34 can output the weight value indicated by the weight display unit 24 included in the weighing photographed image as output data.

[0031] The AI ​​model 34 of this embodiment also describes the correspondence (here, the correspondence between the shape of numbers and the numerical values ​​they represent) between the weight values ​​indicated by the display on the weight display unit 24. If the weight display unit 24 is included in the weighing-captured image of the input data, the AI ​​model 34 can output the weight value indicated by the weight display unit 24 as output data.

[0032] In this embodiment, the task of extracting feature values ​​(and weight values) from a captured image means that the image recognition unit 33 inputs the captured image into the AI ​​model 34 and obtains the feature values ​​(and weight values) returned as output corresponding to the captured image. Feature values ​​are extracted for each figure included in the captured image. If there are multiple figures of the captured image, as in FIG. 1, feature values ​​corresponding to each figure are extracted separately. In the case of FIG. 1, in addition to the feature values ​​of the drug 12 and the weight value of the weight display unit 24, feature values ​​related to the entire weighing scale 20 are also extracted from the captured weighing image. However, in this embodiment, only the data required by the calculation unit 35 is transmitted to the calculation unit 35. That is, the feature values ​​of the drug 12 and the weight value of the weight display unit 24 are transmitted to the calculation unit 35, but the feature values ​​of the entire weighing scale 20 are not transmitted to the calculation unit 35.

[0033] The drug information list 36 included in the calculation unit 35 lists feature values ​​that the AI ​​model 34 can output and drug identification information (information specifying the type of drug, such as the name of the drug) corresponding to the feature values. The calculation unit 35 identifies the drug 12 included in the photographed image based on a comparison between the drug information list 36 and the feature values ​​of the photographed image extracted by the image recognition unit 33. That is, if a feature value matching the feature value transmitted from the image recognition unit 33 is listed in the drug information list 36, the drug corresponding to that feature value in the drug information list 36 is identified as the drug 12 included in the photographed image. Note that in drug identification based on a comparison between the drug information list 36 and the feature values, even if the feature value extracted from the photographed image does not completely match the feature value listed in the drug information list 36, as long as the feature value is somewhat close (for example, about 0.95 to 1.05 times the listed value), the drug with the somewhat close value may be identified as the drug 12 included in the photographed image.

[0034] (Quantity calculation processing) Furthermore, when the weight value obtained from the aforementioned measurement-captured image is transmitted to the calculation unit 35, the calculation unit 35 performs a process for calculating the quantity of the medication 12 included in the measurement-captured image. The medication information list 36 lists weight values ​​of the medication's counting unit (a unit that serves as a reference for counting the quantity of medication, such as one tablet or one bag) for each type of medication. The calculation unit 35 obtains an identification result for the medication 12 based on the characteristic values ​​of the medication 12 included in the measurement-captured image, and then calculates the quantity of the medication 12 by dividing the weight value obtained from the measurement image by the weight value of the counting unit corresponding to the identification result (weight value indicated by the weight display unit 24 / weight value of the counting unit). In the case of FIG. 1, if the medication 12 is a tablet and weighs 0.3 g per counting unit (one tablet), the result of dividing 3.0 g indicated by the weight display unit 24 by 0.3 g, or "10 tablets," is calculated as the quantity of the medication 12. In addition, if the drug 12 comes with any drug accessories, such as rubber bands for bundling multiple drug packages or tags with instructions on how to take the drug (e.g., one tablet per day), the quantity is calculated taking into account the weight of the drug accessories.

[0035] The identification result of the drug 12 and the calculation result of the quantity are then transmitted to the terminal display unit 38, and the terminal display unit 38 displays the identification result and the calculation result of the quantity to the user. For example, if the name (product name) of the drug 12 is "Tablet B," the identification result information such as "Tablet B, 10 tablets" is displayed to the user.

[0036] As described above, according to the drug identification system 10 of this embodiment, the drug 12 can be identified and the quantity calculated simply by the photographing unit 32 photographing a weighing image including the drug 12 supported on the weighing platform 22 and the weight display unit 24 showing the weight value of the drug 12.

[0037] That is, the quantity of drugs 12 can be calculated based on the weight value even if the weighing scale 20 does not communicate with the terminal device 30. Therefore, the user operating the terminal device 30 can perform drug identification without complex communication settings or operations, reducing the burden on the user. Furthermore, an inexpensive device without communication capabilities can be used as the weighing scale 20, reducing the cost of building the drug identification system 10.

[0038] (Taking collective drug images) For the sake of explanation, Figures 1 and 2 show only one type of medication 12 on a single PTP sheet, but in actual dispensing audits, multiple types of medication may be identified. Furthermore, the same type of medication, especially medication packaged in PTP sheets, may be prepared in multiple stacks. In Figure 3, the same tablet medication 12 as in Figure 1 is present on three PTP sheets, as well as capsule medication 14 on three PTP sheets, one can of canned medication 16, and two tubes of medication 18.

[0039] In such a case, before calculating the weight of each medicine, the photographing unit 32 photographs the plurality of types of medicine together. The photographed image showing the plurality of types of medicine thus photographed is hereinafter referred to as a collective medicine photographed image.

[0040] Of the medicines shown in Figure 3, the medicine identification unit 37 in Figure 1 can easily extract feature values ​​for the canned medicine 16 and the tubed medicine 18 because their entire images are captured in the image of the collected medicines. Therefore, the medicine identification unit 37 can identify and determine the quantity of the canned medicine 16 and the tubed medicine 18 in Figure 3 by image recognition of the collected medicine image, without having to calculate the weight.

[0041] Fig. 4 shows the state in which the image recognition results of the photographed image of the collected medicines in Fig. 3 are displayed on the terminal display unit 38 as an identification result list 40. As shown in Fig. 4, the names and quantities as the identification results are obtained by image recognition, such as "Name: Can C, Quantity: 1" for the medicine in a can 16 and "Name: Tube D, Quantity: 2" for the medicine in a tube 18.

[0042] Furthermore, the names of the capsule drug 14 (name: Capsule A) and the tablet drug 12 (name: Tablet B) have also been identified by image recognition of the PTP sheet shown at the top, and "Identification" column indicating whether or not identification has been completed displays "Completed." Furthermore, the drug identification unit 37 also attempts to determine the quantities of the capsule drug 14 and the tablet drug 12 by image recognition based on the collective drug photograph image.

[0043] However, because capsule medication 14 and tablet medication 12 are prepared on multiple stacked PTP sheets, some PTP sheets other than the top one are not visible in the image of the assembled medication, and the determination of the quantity cannot be completed by image recognition of the image of the assembled medication alone. Therefore, in the identification result list 40 of Figure 4, the "Quantity" column, which indicates whether the determination of the quantities of "Capsule A" and "Tablet B" has been completed, displays "Not yet," indicating that it is not completed.

[0044] For capsule medications 14 and tablet medications 12 for which the result of quantity determination is incomplete, a quantity calculation is performed using weigh scale 20. Figure 5 shows how the quantity of tablet medications 12 equivalent to three PTP sheets is calculated using weigh scale 20. The medications 12 equivalent to three PTP sheets are all placed on weighing platform 22 of weigh scale 20, and an image of the entire weigh scale 20 including medications 12 and weight display unit 24 is captured as a measurement image.

[0045] FIG. 5 shows the drug 12, the weighing scale 20, and the result of calculating the quantity of the drug 12 based on the weighing image displayed on the terminal display unit 38 of the terminal device 30. In FIG. 5, the weight display unit 24 displays the weight of the drug 12 as "9.0 g." Based on this weight value, the quantity of the drug 12 is calculated as "30 tablets" (0.3 g per tablet) through the quantity calculation process described above. As a result, the "Quantity" column for "Tablet B" (drug 12) in the identification result list 40 of FIG. 5 displays "30."

[0046] 5, the quantity of "capsules A" (medicine 14) is not yet determined, but like "tablets B" (medicine 12), the medicine identifying unit 37 can determine the quantity of "capsules A" through quantity calculation processing. That is, a measurement image is taken of three PTP sheets worth of medicines 14 placed together on the weighing platform 22, and the quantity of medicines 14 is calculated through image recognition of the measurement image.

[0047] In this way, the quantity of drugs (here, capsule drugs 14 and tablet drugs 12) for which quantity determination has not been completed in the image recognition of the collective drug photographed image is calculated for each type of drug using a weighing scale 20 by a quantity calculation process based on image recognition of the weighing photographed image.

[0048] According to this method, there is no need to take individual metering images for drugs (here, drugs 16 in cans and drugs 18 in tubes) whose quantities can be determined by image recognition based on a collective drug image, thereby reducing the workload on the user.

[0049] (Use of prescription data) 1 is used in a facility that handles medicines, such as a pharmacy, the medicine identification system 10 is used together with a medical system of a medical institution. Fig. 6 shows another example of an embodiment of the present invention, in which a terminal device 30 can communicate with a medical system 50.

[0050] In the terminal device 30 in Fig. 6, the control unit 39 is capable of communicating with the medical system 50. This communication may be via a wired connection or wireless communication. Except for the communication with the medical system 50, the components of the terminal device 30 in Fig. 6 are the same as those of the terminal device 30 in the drug identification system 10 in Fig. 1.

[0051] Prescription data 52 as shown in FIG. 7 is input to the medical system 50. This prescription data 52 indicates information including the name and quantity written on a prescription issued by a doctor. At a medical institution, a doctor determines the drug and quantity to be prescribed to a patient, and then writes the name and quantity of the drug on a prescription. The information written on the prescription is input to the medical system 50 as prescription data 52. The drug identification unit 37 can refer to the prescription data 52 via the control unit 39.

[0052] In the prescription data 52 of Figure 7, the quantity of a drug named "Capsule A" of type "Capsule" is "30", the quantity of a drug named "Tablet B" of type "Tablet" is "30", the quantity of a drug named "Can C" of type "Can" is "1", and the quantity of a drug named "Tube D" of type "Tube" is "2".

[0053] When medication is provided to a patient, the medication is collected according to the above-mentioned prescription data 52, and a tablet medication 12, a capsule medication 14, a canned medication 16, and a tubed medication 18 are prepared as shown in FIG. 3.

[0054] For the multiple types of medications thus prepared, a collection of medication images is captured, and medication identification and quantity determination are attempted using image recognition based on the collection of medication images. When the terminal device 30 communicates with the medical system 50, the medication identification results obtained by image recognition are compared with the prescription data 52. That is, the medication identification unit 37 references the prescription data 52 and checks whether the medication names obtained as the identification results match the "name" data written in the prescription data 52. The identification results are displayed as an identification result list 40 on the terminal display unit 38 of the terminal device 30. As shown in FIG. 8, if the "names" of the identification results (here, "Capsule A," "Tablet B," "Can C," and "Tube D") match the prescription data 52, the identification result list 40 displays "Completed" in the name comparison column.

[0055] The result of determining the quantity obtained by the image recognition or quantity calculation process is also compared with the prescription data 52. Figure 8 shows the comparison result at the time when, after image recognition of the aggregate drug photographed image as shown in Figure 5, a measurement photographed image of the tablet drug 12 ("tablet B") is taken and a quantity calculation process of the drug 12 is performed based on the measurement photographed image.

[0056] As shown in Figure 8, the quantity collation column for "Tablets B" shows "Completed: 30," which indicates that the quantity of medication 12 to be provided to the patient matches the prescription data 52, and this quantity value is displayed in the identification result list 40. For "Can C" and "Tube D," the collation results and quantities are also displayed as "Completed: 1" and "Completed: 2," respectively.

[0057] At the time of Figure 8, the quantity calculation process for "Capsule A" has not yet been performed, and the quantity matching column displays "Not Yet," indicating that a match with prescription data 52 has not been confirmed. A user such as a pharmacy staff member checks the identification result list 40 and takes a weighing image of the medication for which the quantity matching column displays "Not Yet." In the case of Figure 8, "Capsule A" (medicine 14 in Figure 3) is placed on weighing platform 22 of weighing scale 20, and a weighing image is taken and the quantity calculation process is performed.

[0058] The process of taking measurement images and calculating the quantity is repeated, and when the name matching field and the quantity matching field all show "Completed", it means that the drugs that match the prescription data 52 have been collected, and it is confirmed that the drugs provided to the patient are as prescribed.

[0059] As described above, when the terminal device 30 can communicate with the medical system 50, the drug identification unit 37 refers to the prescription data 52, and the drug identification results and quantity calculation results based on the aggregate drug photographed image or the metered photographed image are compared with the prescription data 52. In this case, the user can compare the drugs with the prescription data 52 simply by taking the aggregate drug photographed image and the metered photographed image, thereby reducing the workload on the user.

[0060] (Processing Procedure) Fig. 9 shows a flowchart of the processing procedure from acquisition of the collected medicine image to completion of determination of the quantity of each medicine in Fig. 3. In step S61, first, the photographing unit 32 acquires (photographs) the collected medicine image.

[0061] In step S62, the drug identification unit 37 performs drug identification processing using image recognition based on the collected drug image, identifies each drug (specifies its name), and also attempts to determine the quantity, and if possible, specifies the quantity of each drug.

[0062] In step S63, the results of the drug identification in step S62 are displayed as the identification result. Here, it is determined whether there are any drugs whose quantities could not be determined (step S64). If the quantities of all drugs have been determined solely by image recognition of the collected drug images (step S64-NO), then the identification and quantity determination of each drug is complete at this point (END).

[0063] 5 and 8, if there is a medicine (e.g., "capsule A") for which the quantity determination has not been completed (step S64-YES), a weighing image is acquired (photographed) in step S65. That is, the medicine for which the quantity determination has not been completed is placed on weighing platform 22 of weighing scale 20, and an image including the medicine and weight display unit 24 is photographed by photographing unit 32.

[0064] Then, in step S66, a quantity calculation process is performed based on the metering-captured image. The flow of the quantity calculation process is shown in Figure 10. First, in step S71, the AI ​​model 34 extracts the feature values ​​of the medicine.

[0065] In step S72, the calculation unit 35 compares the characteristic values ​​of the medicine with the medicine information list 36, and identifies (specifies the name of) the medicine shown in the metered photograph image. In the following step S73, the unit weight of the identified medicine is obtained based on the medicine information list 36.

[0066] Then, in step S74, the weight value displayed on weight display section 24 of the measurement-captured image is extracted. In the following step S75, the quantity of the medicine is calculated from the weight value extracted from the measurement-captured image and the unit weight of the medicine.

[0067] 9 is completed, the identification result is displayed again (step S63). If there are still drugs whose quantities have not been determined in this identification result display (step S64-YES), a measurement image is acquired and a quantity calculation process is performed for each drug whose quantities have not been determined. The acquisition of measurement images and the quantity calculation process are repeated, and when there are no more drugs whose quantities have not been determined (step S64-NO), the identification and quantity determination of each drug is completed.

[0068] In this way, according to the drug identification system 10 of the present invention, it is possible to identify and determine the quantity of each drug simply by placing the drug whose quantity has not been determined on the weighing platform 22 of the weighing scale 20 and repeatedly taking weighing images, thereby reducing the workload on the user.

[0069] In the above embodiment, it has been described that the weight value of each drug counting unit is recorded in the drug information list 36. The weight value of each drug counting unit includes not only the weight of the drug when removed from the drug packaging, but also the package weight of the drug. For example, in the case of tablets, a weight value including the weight of the PTP sheet surrounding each tablet when it is separated and handled as a single tablet is recorded. In addition, in the case of canned drugs, the weight value of the entire can when multiple tablets (e.g., 30 tablets) are sealed in an unopened can is recorded.

[0070] Furthermore, when medication is actually provided to a patient, a "fraction" of medication may occur. For example, capsules, tablets, etc. packaged in PTP sheets may not be provided to a patient in full PTP sheets, but rather in "fractions" of several tablets separated from the PTP sheet. Even when a "fraction" of medication occurs, it is possible to determine the quantity of medication contained in the "fraction" using image recognition. For example, data for a "unit image" of one medication is prepared, and it is possible to determine how many of this unit image the "fraction" image contains. However, determining the quantity of such "fractions" is difficult and may not be accurate enough. With the medication identification system of the present invention, even when such "fractions" occur, more accurate quantity determination is possible by calculating the quantity based on measured photographed images.

[0071] Furthermore, in the above embodiment, image recognition is performed using the AI ​​model 34, but image recognition may also be performed without using the AI ​​model 34. For example, a "sample image" may be prepared for each type of drug, and the drug photographed by image processing may be compared with the "sample image" to identify the drug. Furthermore, with regard to extraction of the weight value displayed on the weight display unit 24, a "sample image" may be prepared for each of the numbers "0123456789," and the weight value may be identified by comparing the image of the weight display unit 24 with the "sample image."

[0072] Furthermore, in the above embodiment, for the sake of explanation, one weighing scale 20 and one terminal device 30 are shown, but as shown in FIG. 11 , multiple weighing scales 20 and multiple terminal devices 30 may be provided. In this case, when executing the quantity calculation process, quantity calculation can be performed based on a weighing image obtained by capturing an image of any weighing scale 20 using the image capture unit 32 of any terminal device 30. In this way, if any weighing scale 20 can be used from any terminal device 30, multiple weighing scales 20 and terminal devices 30, such as tablet devices, installed in facilities such as pharmacies can be effectively utilized. Furthermore, there is no need to configure complex communication settings between multiple devices, which is highly convenient for users.

[0073] Furthermore, when multiple weighing scales 20 are used, all weighing scales 20 do not need to be the same product. For example, as shown in the lower right of FIG. 11 , a weighing scale 25 with different specifications (different product) from the other weighing scales 20 may be used. If the display format of the weight display unit 24 is the same for both weighing scales 20 and 25, the drug identification unit 37 can acquire weight values ​​through image recognition in the same way for both weighing scales 20 and 25. Even if the display format of the weight display unit 24 differs between weighing scales 20 and 25, the weight value can be acquired by performing appropriate image recognition depending on the display format. For example, the drug identification unit 37 may store information on the display format of the weight display unit 24 corresponding to the type of weighing scale 20 or 25 (e.g., the shape of numbers, the position of the decimal point, whether or not weight units are displayed, etc.). The drug identification unit 37 may extract information identifying the type of weighing scale 20 or 25, such as a model number, from the weighing image and perform image recognition according to the display format of the weight display unit 24 corresponding to the identified type.

[0074] Furthermore, if the measurement-captured image includes not only the drug but also any drug accessories attached to the drug, the quantity of the drug may be calculated taking into account the weight of the drug accessories. For example, although not shown in FIG. 3, when multiple PTP sheets are provided to a patient, the multiple PTP sheets may be bundled together with rubber bands. Furthermore, the edges of the PTP sheet may have areas called "ears" where information such as the drug name and serial number is engraved, separate from the area for packaging the drug. Additionally, a tag bearing patient precautions, such as dosage instructions (e.g., one tablet per day), may be attached to the drug. In such cases, information regarding the weight of the drug accessories may be stored in the drug identification unit 37. For example, if the drug accessories are rubber bands, the characteristic values ​​of the rubber bands obtained from the measurement-captured image including the rubber bands and the weight of the rubber bands may be stored. The weight of the drug is then calculated by subtracting the weight of the drug accessories, such as the rubber bands, from the weight displayed by the weight display unit 24. The amount of the drug is calculated by dividing the weight of the drug thus calculated by the weight of the drug's counting unit.

[0075] In the above embodiment, one terminal device 30 includes the photographing unit 32, the drug identification unit 37 (image recognition unit 33 and calculation unit 35), the terminal display unit 38, and the control unit 39. Although the above describes a configuration in which multiple components are integrated into the same hardware, these components may be distributed as separate devices. For example, the photographing unit 32 may be a standalone camera device, and the terminal display unit 38 may be a standalone display device. The drug identification unit 37 may be implemented as software that executes drug identification processing on a cloud server located remotely from the camera device and the display device. Furthermore, the image recognition unit 33 and the calculation unit 35 of the drug identification unit 37 may be separate devices or may be software executed on separate servers. The control unit 39 may be a device (e.g., a computer operated by a user) that controls communication of data (such as captured image data and identification result data) between the photographing unit 32, the drug identification unit 37 (image recognition unit 33 and calculation unit 35), the terminal display unit 38, and the control unit 39, each of which is implemented as separate devices. In this way, even if each component is configured as a separate device, if the components are capable of exchanging data with each other by wired connection, wireless communication, or the like, the set of components as a whole will fulfill the function of the drug identification system 10. Furthermore, if data exchange between the components is possible without being controlled by the control unit 39 (for example, if data exchange between the components occurs without an external command, such as when the photographing unit 39 automatically transmits a photographed image to the drug identification unit 37), the set of components as a whole will fulfill the function of the drug identification system 10 even without the control unit 39. [Explanation of symbols]

[0076] 10 Drug Identification System 12 Drugs 20 Weight scale 22 Weighing platform 24 Weight display section 30 Terminal Equipment 32 Photography Department 33 Image Recognition Unit 35 Arithmetic section 37 Drug Identification Section 38 Terminal display 39 Control Unit 40 Identification result list 50 Healthcare System 52 Prescription Data

Claims

1. A drug identification system comprising at least one weighing scale that measures the weight of a drug to be identified, at least one photographing unit that can photograph an image including the drug, and at least one drug identification unit that identifies the drug in the photographed image, the weighing scale has a scale base capable of supporting the drug and a weight display unit that indicates the weight value of the drug supported by the scale base, The photographing unit is capable of photographing a weighing photograph image in which the medicine supported by the weighing platform and the weight display unit displaying the weight value of the medicine are both photographed, The medicine identifying unit By performing image recognition on the measurement-captured image, the drug shown in the measurement-captured image is identified and the weight value indicated by the weight display unit is acquired; performing a quantity calculation process for calculating the quantity of the medicines shown in the measurement-captured image based on the medicine identification result and the acquired weight value; A drug identification system comprising:

2. When there are multiple types of drugs to be identified, The photographing unit photographs an aggregate medicine photographed image in which a plurality of types of medicines are photographed, The medicine identifying unit identifies the type and quantity of the medicines shown in the collected medicine photographed image by performing image recognition on the collected medicine photographed image, The medicine identification system according to claim 1 , wherein the quantity calculation process calculates the quantity of the medicines for which the identification of the quantity has not been completed in the image recognition of the collected medicine photographed image.

3. A plurality of the weighing scales are provided, and a plurality of terminal devices each having the photographing unit are provided; The drug identification system according to claim 1 , wherein the weighing image can be captured by capturing an image of any one of the weighing scales using the image capturing unit of any one of the terminal devices.

4. The metering photographic image is photographed including any drug attachments attached to the drug, The medicine identification system according to claim 1 , wherein the medicine identifying unit calculates the quantity of the medicine in consideration of the weight of the medicine accessories in the quantity calculation process.

5. The drug identification system is used together with a medical system of a medical institution, Prescription data indicating information including the name and quantity of a drug written on a prescription issued by a doctor is input to the medical system; The medicine identifying unit can refer to the prescription data, The drug identification system according to claim 1 , wherein in the quantity calculation process, the name and quantity of the drug shown in the measurement image are compared with information indicated in the prescription data.

Images