Drug identification device, drug identification method, and program

Abstract

The present invention provides a drug identification device, a drug identification method, and a program capable of, even when there is a drug with a drug surface the same as or similar to an identification target surface of an identification target drug in drug identification using single-sided imaging and single-sided image input, notifying a user of existence of the drug with the same or similar drug surface and enabling the user to determine a correct drug as the identification target drug. The present invention acquires a plurality of drug surface candidates which are candidates for correct answers of an identification target surface of an identification target drug on the basis of an identification target image of the identification target surface, and displays, for a drug type candidate with a drug surface candidate as a quasi-identifiable surface, a first master image of the drug surface candidate and a second master image of the opposite-side surface of the drug surface candidate, and displays an alert on the first master image or at a position associated with the first master image.

Description

Drug identification device, drug identification method, and program

[0001] The present invention relates to a drug identification device, a drug identification method, and a program, and more particularly to a technique for determining a correct drug of a drug to be identified from a photographed image of one side of the drug.

[0002] A drug identification device is known that uses a drug identification AI (artificial intelligence) to identify a photographed image of only one side of a drug to be identified, displays candidate drug types for the drug to be identified on a selectable display, and allows the user to select the correct drug for the drug to be identified (see Patent Document 1).

[0003] International Publication No. 2022 / 163508

[0004] There may be multiple different drugs with identical or similar appearances on their drug surfaces. Consider a drug identification device that uses a drug identification AI to identify drug types from a single-sided image. A certain number of candidates are displayed along with a master image in descending order of score, and the user compares the images to determine the correct drug type. In this case, if drugs with many identical or similar drug surfaces are all trained as a separate class, many drugs that differ from the correct drug may appear in the candidate list during drug type identification, and the correct drug may not appear in the candidate list. In this case, there is a risk that the user may confidently select an incorrect drug that appears in the candidate list. If the selected incorrect drug has a different active ingredient amount or different ingredients than the correct drug, serious consequences may result.

[0005] In response to this, there is a method of excluding all drugs with the same or similar drug faces from the learning process, so that they do not appear as candidates. However, there is a problem in that there are many drugs with the same or similar drug faces, and the number of drugs that cannot be distinguished increases.

[0006] The present invention has been made in consideration of these circumstances, and aims to provide a drug identification device, drug identification method, and program that, in drug identification using single-sided photography and single-sided image input, alerts the user to the fact that there is a drug with an identical or similar drug surface to the surface of the drug to be identified, and allows the user to determine the correct drug for the drug to be identified, even if there is a drug with an identical or similar drug surface.

[0007] In order to achieve the above object, a drug identification device according to the present disclosure includes at least one processor and at least one memory that stores instructions to be executed by the processor, wherein the processor acquires an identification target image obtained by capturing an identification target surface that is either a first surface or a second surface of a drug to be identified among a plurality of drugs, each of which has a first surface and a second surface opposite to the first surface, displays the identification target image on a display, acquires a plurality of drug surface candidates that are drug surface candidates that are the first surface or the second surface of any of the plurality of drugs and are correct candidates for the identification target surface based on the identification target image, displays a list of a plurality of drug type candidates each having a plurality of drug surface candidates on the display, and determines a drug type candidate selected from the displayed list by an input device as the correct drug for the drug to be identified, the drug identification device comprising: N is an integer of 2 or more, the first surface and the second surface of each of the multiple drugs are classified into identifiable surfaces where the number of drugs having a first surface or a second surface that is identical or similar to the drug surface candidate is 0, quasi-identifiable surfaces where the number of drugs having a first surface or a second surface that is identical or similar to the drug surface candidate is 1 or more but less than N, and similar multiple surfaces where the number of drugs having a first surface or a second surface that is identical or similar to the drug surface candidate is N or more, and the processor, for drug type candidates whose drug surface candidates are identifiable surfaces, displays a first master image of the drug surface candidate and a second master image of the surface opposite the drug surface candidate, for drug type candidates whose drug surface candidates are quasi-identifiable surfaces, displays the first master image of the drug surface candidate and a second master image of the surface opposite the drug surface candidate, and displays an alert at a position associated with the first master image or the first master image, and does not display an alert for drug type candidates whose drug surface candidates are similar multiple surfaces.

[0008] According to the drug identification device, in drug identification using single-sided photography and single-sided image input, even if there is a drug with a drug surface that is identical or similar to the surface of the drug to be identified, the user is alerted to the fact that there is a drug with an identical or similar drug surface, and the user can determine the correct drug for the drug to be identified.

[0009] In order to achieve the above object, the drug identification method according to the present disclosure is a drug identification method in which at least one processor acquires an identification target image in which an identification target surface, which is either a first surface or a second surface, of a drug to be identified among a plurality of drugs, each having a first surface and a second surface opposite to the first surface, displays the identification target image on a display, acquires a plurality of drug surface candidates, which are drug surface candidates that are the first surface or the second surface of any of the plurality of drugs and are correct candidates for the identification target surface, based on the identification target image, displays a list of a plurality of drug type candidates, each having a plurality of drug surface candidates, on the display, and determines a drug type candidate selected from the displayed list by an input device as the correct drug for the drug to be identified, wherein N is an integer of 2 or more, The first and second surfaces are classified into identifiable surfaces where the number of drugs having a first surface or a second surface that is identical or similar to the first surface is 0, quasi-identifiable surfaces where the number of drugs having a first surface or a second surface that is identical or similar to the first surface is 1 or more but less than N, and similar multiple surfaces where the number of drugs having a first surface or a second surface that is identical or similar to the first surface is N or more, and the processor, for drug type candidates whose drug surface candidates are identifiable surfaces, displays a first master image of the drug surface candidate and a second master image of the surface opposite the drug surface candidate, for drug type candidates whose drug surface candidates are quasi-identifiable surfaces, displays the first master image of the drug surface candidate and a second master image of the surface opposite the drug surface candidate, and displays an alert at a position associated with the first master image or the first master image, and does not display any drug type candidates whose drug surface candidates are similar multiple surfaces.

[0010] According to the drug identification method, in drug identification using single-sided photography and single-sided image input, even if there is a drug with a drug surface that is identical or similar to the surface of the drug to be identified, the user is alerted to the fact that there is a drug with an identical or similar drug surface, and the user can determine the correct drug for the drug to be identified.

[0011] According to the present invention, in drug identification using single-sided photography and single-sided image input, even if there is a drug with a drug surface that is identical or similar to the surface of the drug to be identified, the user is alerted to the fact that there is a drug with an identical or similar drug surface, and the user can determine the correct drug for the drug to be identified.

[0012] FIG. 1 is a block diagram showing the electrical configuration of a smartphone. FIG. 2 is a block diagram showing the functional configuration of a drug identification device. FIG. 3 is a diagram showing targets for learning and setting a warning flag for each category on the drug surface according to the first embodiment. FIG. 4 is a flowchart showing the steps of the drug identification method according to the first embodiment. FIG. 5 is a diagram showing an example of a GUI for a touch panel display. FIG. 6 is a diagram showing targets for learning, non-display flags, and warning flags for each category on the drug surface according to the second embodiment. FIG. 7 is a flowchart showing the steps of the drug identification method according to the second embodiment. FIG. 8 is a diagram showing the classification of the drug surface of the drug to be identified and the identification flow expected when one side is photographed.

[0013] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0014] <Overview of drug identification device> The drug identification device of the present disclosure is a device that acquires an image of the identification target surface, which is one of the first and second surfaces of a drug (e.g., a tablet) that has a first surface (e.g., a front surface) and a second surface (e.g., a back surface) opposite the first surface, identifies the drug type of the drug to be identified, displays multiple candidate drugs, and allows the user to select the correct drug to be identified from the multiple candidate drugs.

[0015] According to the drug identification device, even if there is a drug having a first surface or a second surface similar to the surface to be identified of the drug to be identified, the user can determine the correct drug of the drug to be identified. Note that in this specification, either the first surface of the drug or the second surface of the drug may be referred to as the "drug surface of the drug."

[0016] Examples of drugs that have similar drug surfaces include drugs with different dosages, drugs with different specifications, drugs with different dosage forms, drugs with different original and generic versions, drugs with different generic versions, and drugs with different formulations.

[0017] The medicine surface of the medicine may be engraved and / or printed. Note that "engraved and / or printed" means "either engraved or printed" or "both engraved and printed."

[0018] Here, "marked" means that identification information is formed by forming a groove, which is a recessed area, on the surface of the medicine. The groove is not limited to being formed by digging into the surface of the medicine, but may also be formed by pressing the surface of the medicine.

[0019] Furthermore, "printed" means that identification information is formed by applying edible ink or the like to the drug surface with or without contact. Here, "applied by printing" is synonymous with "applied by printing."

[0020] The surface of the medicine may have a score line without any distinguishing function, which is a groove formed on the surface of the medicine, for example, to divide the tablet.

[0021] The medication surface of the medication may be plain, with no markings, printing, or scoring lines added.

[0022] As an example, the drug identification device is mounted on a portable terminal device. The portable terminal device includes at least one of a mobile phone, a PHS (Personal Handyphone System), a smartphone, a PDA (Personal Digital Assistant), a tablet computer terminal, a notebook personal computer terminal, and a portable game console. The drug identification device may also be mounted on a fixed desktop computer, a workstation, or configured in combination with a separate digital camera and display. Below, a drug identification device configured on a smartphone will be used as an example and described in detail with reference to the drawings.

[0023] <Electrical Configuration of Smartphone> Fig. 1 is a block diagram showing the electrical configuration of a smartphone 10. As shown in Fig. 1, the smartphone 10 includes a touch panel display 14, a speaker 16, a microphone 18, an in-camera 20, an out-camera 22, a light 24, and a switch 26, as well as a CPU (Central Processing Unit) 28, a wireless communication unit 30, a call unit 32, a memory 34, an external input / output unit 40, a GPS receiving unit 42, and a power supply unit 44.

[0024] The touch panel display 14 includes a display unit that displays images and the like, and a touch panel unit that is disposed in front of the display unit and accepts touch inputs. The display unit is, for example, a color LCD (Liquid Crystal Display) panel, which is an example of a display device.

[0025] The touch panel unit (an example of an "input device") is, for example, a capacitive touch panel provided in a planar form on a light-transmitting substrate body, and having light-transmitting position detection electrodes and an insulating layer provided on the position detection electrodes. The touch panel unit generates and outputs two-dimensional position coordinate information corresponding to a user's touch operation. Touch operations include a tap operation, a long tap operation, a double tap operation, a flick operation, a swipe operation, a drag operation, a pinch-in operation, and a pinch-out operation.

[0026] The speaker 16 is an audio output unit that outputs audio during a call and when playing back a video. The microphone 18 is an audio input unit that inputs audio during a call and when shooting a video. The in-camera 20 is an imaging device that captures videos and still images. The out-camera 22 is an imaging device that captures videos and still images. The light 24 is a light source that emits illumination light when shooting with the out-camera 22, and is configured, for example, by an LED (Light Emitting Diode).

[0027] The switch 26 is an input member that receives instructions from a user. The switch 26 is a push-button switch that turns on when pressed with a finger or the like and turns off when the finger is released due to the restoring force of a spring or the like.

[0028] The CPU 28 is an example of a processor that executes instructions stored in the memory 34. The CPU 28 operates in accordance with the control program and control data stored in the memory 34, and controls all the components of the smartphone 10. The CPU 28 has a mobile communication control function that controls all the components of the communication system to perform voice communication and data communication via the wireless communication unit 30, and an application processing function.

[0029] The CPU 28 also has an image processing function for displaying moving images, still images, text, etc. on the touch panel display 14. This image processing function visually conveys information such as still images, moving images, and text to the user. The CPU 28 also acquires two-dimensional position coordinate information corresponding to a user's touch operation from the touch panel portion of the touch panel display 14. The CPU 28 also acquires an input signal from the switch 26.

[0030] The hardware structure of the CPU 28 is made up of various processors as follows: The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various functional units, a GPU (Graphics Processing Unit), which is a processor specialized for image processing, a PLD (Programmable Logic Device), which is a processor whose circuit configuration can be changed after manufacture such as an FPGA (Field Programmable Gate Array), and a dedicated electrical circuit, which is a processor having a circuit configuration designed specifically for executing specific processing such as an ASIC (Application Specific Integrated Circuit).

[0031] A single processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types (e.g., multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Multiple functional units may also be configured with a single processor. Examples of multiple functional units configured with a single processor include: a first configuration, as typified by a client or server computer, in which a single processor is configured with a combination of one or more CPUs and software, and this processor operates as multiple functional units; and a second configuration, as typified by a SoC (System on Chip), in which a processor is used to realize the functions of an entire system including multiple functional units on a single IC (Integrated Circuit) chip. In this way, the various functional units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0032] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit made up of a combination of circuit elements such as semiconductor elements.

[0033] The in-camera 20 and the out-camera 22 capture moving images and still images in accordance with instructions from the CPU 28. The in-camera 20 and the out-camera 22 have the same internal configuration. The in-camera 20 and the out-camera 22 each have a photographing lens, an image sensor, and an image processing unit (not shown).

[0034] The in-camera 20 and the out-camera 22 each receive subject light passing through a photographing lens with an image sensor. The image sensor is a photoelectric conversion element such as a CMOS (Complementary Metal-Oxide Semiconductor) or a CCD (Charge-Coupled Device), and has R (red), G (green), and B (blue) color filters (not shown) provided on its light-receiving surface. The subject light forms an image on the light-receiving surface of the image sensor, which then converts the subject light imaged on the light-receiving surface into an electrical signal based on the R, G, and B color signals. The image processing unit performs predetermined processing on the analog image signal output from the image sensor to convert it into a digital image signal.

[0035] The in-camera 20 and the out-camera 22 may convert the image data of the captured moving images and still images into compressed image data using MPEG (Moving Picture Experts Group) or JPEG (Joint Photographic Experts Group) formats.

[0036] The CPU 28 stores the video and still images captured by the in-camera 20 and the out-camera 22 in the memory 34. The CPU 28 may also output the video and still images captured by the in-camera 20 and the out-camera 22 to the outside of the smartphone 10 via the wireless communication unit 30 or the external input / output unit 40.

[0037] Furthermore, the CPU 28 displays the video and still images captured by the in-camera 20 and the out-camera 22 on the touch panel display 14. The CPU 28 may use the video and still images captured by the in-camera 20 and the out-camera 22 in application software.

[0038] The CPU 28 may illuminate the subject with fill light by turning on the light 24 when capturing an image with the outer camera 22. The light 24 may be turned on and off by a touch operation on the touch panel display 14 or by operating the switch 26 by the user.

[0039] The wireless communication unit 30 performs wireless communication with a base station device included in the mobile communication network in accordance with instructions from the CPU 28. Using this wireless communication, the smartphone 10 transmits and receives various file data such as audio data and image data, e-mail data, and the like, and receives Web (abbreviation for World Wide Web) data, streaming data, and the like.

[0040] The speaker 16 and the microphone 18 are connected to the communication unit 32. The communication unit 32 decodes the audio data received by the wireless communication unit 30 and outputs it from the speaker 16. The communication unit 32 converts the user's voice input through the microphone 18 into audio data that can be processed by the CPU 28 and outputs it to the CPU 28.

[0041] The memory 34 stores instructions to be executed by the CPU 28. The memory 34 is composed of an internal storage unit 36 ​​built into the smartphone 10 and an external storage unit 38 that is detachable from the smartphone 10. The internal storage unit 36 ​​and the external storage unit 38 are realized using known storage media. The external storage unit 38 may be a non-transitory computer-readable recording medium.

[0042] The memory 34 stores the control program of the CPU 28, control data, application software, address data associated with names and telephone numbers of communication partners, data of emails sent and received, web data downloaded by web browsing, downloaded content data, etc. The memory 34 may also temporarily store streaming data, etc.

[0043] The external input / output unit 40 serves as an interface with external devices connected to the smartphone 10. The smartphone 10 is directly or indirectly connected to other external devices by communication or the like via the external input / output unit 40. The external input / output unit 40 transmits data received from the external devices to each component within the smartphone 10 and transmits data within the smartphone 10 to the external devices.

[0044] Examples of communication means include a Universal Serial Bus (USB), Institute of Electrical and Electronics Engineers (IEEE) 1394, the Internet, a wireless local area network (LAN), Bluetooth (registered trademark), RFID (Radio Frequency Identification), and infrared communication. Examples of external devices include a headset, an external charger, a data port, an audio device, a video device, a smartphone, a PDA, a personal computer, and earphones.

[0045] The GPS receiver 42 detects the position of the smartphone 10 based on positioning information from GPS satellites ST1, ST2, . . . , STn.

[0046] The power supply unit 44 is a power supply source that supplies power to each component of the smartphone 10 via a power supply circuit (not shown). The power supply unit 44 includes a lithium-ion secondary battery. The power supply unit 44 may also include an A / D converter that generates a DC voltage from an external AC power supply.

[0047] The smartphone 10 configured in this manner is set to a shooting mode by a user's instruction input using the touch panel display 14 or the like, and can take videos and still images using the in-camera 20 and the out-camera 22.

[0048] When the smartphone 10 is set to the shooting mode, it enters a shooting standby state, and a video is shot by the in-camera 20 or the out-camera 22, and the shot video is displayed on the touch panel display 14 as a live view image.

[0049] The user can visually check the live view image displayed on the touch panel display 14 to determine the composition, confirm the subject they want to photograph, and set the photographing conditions.

[0050] When the smartphone 10 is in a standby state for shooting and receives an instruction to take a picture by a user input using the touch panel display 14 or the like, the smartphone 10 performs AF (Autofocus) and (Auto Exposure) control, and takes and stores video and still images.

[0051] <Functional Configuration of Drug Identification Device> Fig. 2 is a block diagram showing the functional configuration of the drug identification device 100 realized by the smartphone 10. As shown in Fig. 2, the drug identification device 100 has an image acquisition unit 102, a drug detection unit 104, a drug type identification unit 106, a drug database 108, a candidate output unit 110, and a determination unit 112. The functions of the image acquisition unit 102, the drug detection unit 104, the drug type identification unit 106, the candidate output unit 110, and the determination unit 112 are realized by the CPU 28 executing a program stored in the memory 34. In addition, the function of the drug database 108 is realized by the memory 34.

[0052] The image acquisition unit 102 acquires a captured image of the identification target surface, which is either the first surface or the second surface of the drug to be identified. The captured image is, for example, an image captured by the in-camera 20 or the out-camera 22. The captured image may be an image acquired from another device via the wireless communication unit 30, the external storage unit 38, or the external input / output unit 40.

[0053] The captured image may be an image of the drug to be identified and a marker. The marker may be multiple, or may be an ArUco marker, a circular marker, or a square marker. The captured image may be an image of the drug to be identified and a reference gray color.

[0054] The captured image may be an image captured at a standard shooting distance and shooting viewpoint. The shooting distance can be expressed by the distance between the drug to be identified and the shooting lens and the focal length of the shooting lens. The shooting viewpoint can be expressed by the angle between the marker printed surface and the optical axis of the shooting lens.

[0055] When the captured image includes a marker, the image acquisition unit 102 may standardize the shooting distance and the shooting viewpoint of the captured image based on the marker to acquire a standardized image. Furthermore, when the captured image includes a reference gray area, the image acquisition unit 102 may perform color correction on the captured image based on the reference gray color.

[0056] The captured image may include the surfaces of a plurality of target drugs to be identified. The target drugs are not limited to the same drug type, but may be different drug types.

[0057] The drug detection unit 104 detects the area of ​​the identification target surface of the drug to be identified from the captured image acquired by the image acquisition unit 102. When a standardized image is acquired by the image acquisition unit 102, the drug detection unit 104 detects the area of ​​the identification target surface from the standardized image. When the captured image contains multiple identification target drugs, the drug detection unit 104 detects the area of ​​the identification target surface of each of the multiple identification target drugs. Furthermore, the drug detection unit 104 cuts out the detected area of ​​the identification target surface from the captured image or the standardized image to generate a cut-out image of the identification target surface (an example of an "image to be identified").

[0058] The drug type identification unit 106 acquires multiple drug type candidates for the drug to be identified based on the cut-out image of the identification target surface. The drug type candidates include drug identification information consisting of drug name, trade name, abbreviation, or a combination thereof. When areas of the identification target surface of multiple drugs to be identified are detected from the captured image, the drug type identification unit 106 acquires drug type candidates corresponding to each of the multiple drugs to be identified.

[0059] The drug type identification unit 106 includes a drug type identification AI 106A. The drug type identification AI 106A is stored in the memory 34. The drug type identification AI 106A is a trained model that, when an image of the first side of a drug or the second side of a drug is given as input, outputs a score representing the likelihood (an example of "accuracy") that each of all drug surfaces to be trained is the drug surface. The score is, for example, a value greater than or equal to 0 and less than or equal to 1, and a larger value indicates a higher likelihood that the drug surface is the drug surface.

[0060] The drug type identification AI 106A is a training data set of a plurality of different drugs, and machine learning has been performed using a training data set consisting of a set of images of drug faces and scores for all drug faces to be trained. The score of the training data set is, for example, 1 for the correct drug face and 0 for other drug faces.

[0061] The drug type identification AI 106A can apply a convolutional neural network (CNN). The drug type identification unit 106 inputs a cut-out image of the identification target surface into the drug type identification AI 106A to obtain scores for all drug surfaces to be learned, and obtains multiple drug surfaces with relatively high scores as multiple drug surface candidates that are correct candidates for the identification target surface. Furthermore, the drug type identification unit 106 obtains multiple drug type candidates, each of which has multiple drug surface candidates.

[0062] The drug database 108 stores drug identification information, master images of both sides, and the classification of the drug side, which will be described later, for all drugs on the drug side that the drug type identification AI 106A has learned, each associated with the other.

[0063] The candidate output unit 110 outputs a plurality of candidate drug types for the drug to be identified acquired by the drug type identification unit 106. The candidate output unit 110, for example, causes the plurality of candidate drug types for the drug to be identified to be selectable on the touch panel display 14. The candidate output unit 110 may acquire drug identification information for the plurality of candidate drug types and master images of both sides from the drug database 108 and display them on the touch panel display 14.

[0064] The confirmation unit 112 confirms a correct drug for the drug to be identified from a plurality of drug type candidates for the drug to be identified. The confirmation unit 112 confirms, as the correct drug, a drug type candidate selected by the user from a plurality of drug type candidates displayed on the touch panel display 14, for example. When there is only one drug type candidate for the drug to be identified output by the candidate output unit 110, the confirmation unit 112 may confirm the drug type candidate as the correct drug.

[0065] <Classification of Drug Surfaces> The user pre-classifies the first and second surfaces of all drugs to be identified into three categories: "identifiable surfaces," "quasi-identifiable surfaces," and "similar multiple surfaces." The definitions of "identifiable surfaces," "quasi-identifiable surfaces," and "similar multiple surfaces" are as follows:

[0066] A "distinguishable surface" is a unique medication surface, i.e., a "distinguishable surface" is a medication surface for which the number of medications having the same or similar first or second surface is zero.

[0067] A "quasi-identifiable surface" is a drug surface with a small number of drug surfaces similar to itself. In other words, a "quasi-identifiable surface" is a drug surface on which the number of drugs having a first surface or a second surface identical to or similar to itself is 1 or more but less than N, where N is an integer equal to or greater than 2. N may be determined based on the number of drug type candidates displayed by the system (e.g., the drug identification device 100). For example, if the system displays five drug type candidates, N may be defined as 5. Furthermore, N does not need to be the same as the number of drug type candidates displayed by the system; if the system displays five drug type candidates, N may be defined as 4 or N = 6.

[0068] A "similar multiple surface" is a drug surface that has many drug surfaces similar to itself. That is, a "similar multiple surface" is a drug surface in which the number of drugs having the same or similar first or second surface as itself is N or more. "Similar multiple surfaces" include plain drug surfaces and drug surfaces with only a secant line.

[0069] Although there is no absolute standard for determining whether any two drugs are identical or similar, possible methods include "the system creator visually checking the master image and defining them as identical or similar," or "template matching any two drug surface master images and defining them as identical or similar based on the score."

[0070] The classification results are stored in the medication database 108 for each medication side.

[0071] First Embodiment [Target of Learning and Flagging] Figure 3 is a diagram showing targets for learning and raising a warning flag for each category of drug surface according to the first embodiment. As shown in Figure 3, the drug type identification AI 106A's learning targets are identifiable faces and quasi-identifiable faces, and similar multiple faces are not the learning targets. That is, the drug type identification AI 106A according to the first embodiment has performed machine learning using images of identifiable drug faces and images of quasi-identifiable drug faces, and images of similar multiple drug faces are not used for learning.

[0072] 3, a warning flag is assigned to the quasi-distinguishable surface of the medicine. In actual data, the value of the warning flag for the quasi-distinguishable surface is set to 1, and the values ​​of the warning flags for the distinguishable surface and the similar multiple surface are set to 0. The warning flag may be stored in the medicine database 108.

[0073] For example, if the first side of a medicine is an identifiable side and the second side is a similar multiple side, learning is performed by the drug type identification AI 106A only on the first side. Also, no warning flag is assigned to either side.

[0074] If both the first and second sides of the medicine are similar multiple sides, neither side is subject to learning by the drug type identification AI 106A. In addition, no warning flag is assigned to either side.

[0075] If the first side of the medicine is an identifiable side and the second side is a semi-identifiable side, both sides are learned by the drug type identification AI 106A, and only the second side is given a warning flag.

[0076] If both the first and second sides of the medicine are identifiable, both sides are learned by the drug type identification AI 106A. In addition, no warning flag is assigned to either side.

[0077] If the first side of the drug is a quasi-identifiable side and the second side is a similar multiple side, only the first side is learned by the drug type identification AI 106A and a warning flag is assigned.

[0078] 4 is a flowchart showing the steps of the drug identification method according to the first embodiment using the drug identification device 100. The drug identification method is realized by the CPU 28 reading and executing a drug identification program from the memory 34. The drug identification program may be provided via the wireless communication unit 30 or the external input / output unit 40.

[0079] In step S1, the user photographs the identification target surface, which is the first or second surface, of one or more tablets (an example of a "medicine") packaged together and whose drug type is to be identified, using the in-camera 20 or the out-camera 22 of the smartphone 10. The tablet may be photographed while packaged in a sachet or while housed in a press-through packaging (PTP) sheet.

[0080] The image acquisition unit 102 acquires the photographed image of the tablet. The image acquisition unit 102 may display the acquired image on the touch panel display 14.

[0081] In step S2, the drug detection unit 104 detects a tablet region from the image captured in step S1 and cuts out the detected tablet region to generate a cut-out image of the identifiable surface. If the image captured in step S1 includes multiple tablets, the drug detection unit 104 generates a cut-out image of the identifiable surface of each of the multiple tablets. The drug detection unit 104 may display the cut-out image on the touch panel display 14.

[0082] The processes in steps S3 to S6 are performed for each of all the tablets detected in step S2. The order in which the processes are performed for the tablets may be arbitrary.

[0083] In step S3, the drug type identification unit 106 inputs the cut-out images of the identifiable surfaces of the tablet to the drug type identification AI 106A, obtains the scores of all drug surfaces to be learned, and sorts all drug surfaces in descending order of score.

[0084] In step S4, the candidate output unit 110 acquires the drug faces with the highest scores acquired in step S3 from 1 to M, where M is a positive integer, as drug face candidates. Furthermore, for M drugs each having M drug face candidates with the highest scores from 1 to M, the candidate output unit 110 acquires drug identification information, a first master image of the drug face candidate, and a second master image of the opposite side of the drug face candidate from the drug database 108. Furthermore, the candidate output unit 110 displays a list of the acquired drug identification information and master images of both sides as drug type candidates for the correct drug in descending order of score on the touch panel display 14.

[0085] The above-mentioned N may be determined based on M. M=N may also be true.

[0086] Furthermore, the candidate output unit 110 displays an alert at the position of the master image of the medication side, which has a warning flag attached, among the master images displayed on the touch panel display 14, i.e., the master image of the quasi-identifiable side. Here, the candidate output unit 110 displays an alert icon as the alert. Note that the alert is not limited to an alert icon, and may be, for example, an alert message or a frame surrounding the master image. Furthermore, the position at which the alert is displayed is not limited to the position of the master image of the quasi-identifiable side. The candidate output unit 110 may display the alert at a position associated with the master image of the quasi-identifiable side. The position associated with the master image includes a position adjacent to the master image, such as a position immediately above the master image.

[0087] In step S5, when the user taps an alert icon displayed on the touch panel display 14 (an example of a "trigger associated with a drug surface candidate"), the candidate output unit 110 causes the touch panel display 14 to display a list of drug surfaces that are identical or similar to the drug surface of the master image on which the alert icon is displayed. The candidate output unit 110 may also cause the touch panel display 14 to display a list of master images of both sides of drugs that have drug surfaces that are identical or similar to the drug surface of the master image on which the alert icon is displayed. Furthermore, when the user taps or long-tap (long press) the master image on which the alert icon is displayed or the drug identification information having that master image, instead of the alert icon, the candidate output unit 110 may cause the touch panel display 14 to display a list of drug surfaces that are identical or similar to the drug surface of the master image.

[0088] In addition, the display screen of the candidate drug type for the correct drug displayed in step S4 and the display screen of the drug surface similar to the quasi-distinguishable surface displayed in step S5 may both be displayed on a single screen, or only one of them may be displayed.

[0089] In step S6, the user selects a correct drug for the tablet to be identified from the drug type candidates displayed on the touch panel display 14. The confirmation unit 112 confirms, as the correct drug, the drug type candidate corresponding to the position tapped by the user among the drug type candidates displayed on the touch panel display 14.

[0090] When the processing of steps S3 to S6 has been completed for all tablets detected in step S2, the process proceeds to step S7. In step S7, the user determines whether or not to also photograph the surface opposite to the surface to be identified. If the correct drug can be determined for all tablets using the image of the surface to be identified photographed in step S1, the process of this flowchart ends. On the other hand, if there is a tablet for which the correct drug cannot be determined using only the image of the surface to be identified photographed in step S1, the process proceeds to step S1, and the same process is performed on the surface opposite to the initial surface to be identified. This allows the correct drug to be determined for all tablets.

[0091] [GUI Example] FIG. 5 is a diagram showing an example of a GUI (Graphical User Interface) of the touch panel display 14 in the medicine identifying method.

[0092] F5A in Fig. 5 is an example of a display screen in step S4. The display screen has display areas A1, A2, and A3.

[0093] The display area A1 is an area where a cut-out image of the surface of the tablet to be identified is displayed. In the example of F5A, a cut-out image IC in which the markings and / or printing are oriented upright is displayed.

[0094] Display area A2 is an area where candidate drug types for the tablet to be identified are displayed in a selectable manner. In the example of F5A, the list of candidate drug types displays candidate drugs CD1, CD2, CD3, and CD4, which have the top 1st, 2nd, 3rd, and 4th most likely candidate drug surface candidates that are identified in the extracted image IC.

[0095] The candidate drug CD1 displays the drug name N1, a first master image IM1A, and a second master image IM1B. The first master image IM1A corresponds to the master image of the candidate drug surface, and the second master image IM1B corresponds to the master image of the opposite side of the candidate drug surface. An alert icon IA1 is superimposed on the first master image IM1A, indicating that a drug surface similar to the first side of the candidate drug CD1 exists.

[0096] That is, the candidate drug CD1 has a quasi-identifiable drug surface candidate, and the candidate output unit 110 displays a first master image IM1A of the candidate drug surface and a second master image IM1B of the opposite side of the candidate drug surface, and also displays an alert icon IA1 on the first master image IM1A.

[0097] In this example, the alert icon IA1 is displayed superimposed on the first master image IM1A, but it is sufficient if the user can recognize that the first master image IM1A is a quasi-distinguishable surface. Therefore, the alert icon IA1 may be displayed at a position associated with the first master image IM1A. The same applies to the alert icons IA2 and IA3 described below.

[0098] For the candidate drug CD2, the drug name N2, the first master image IM2A, and the second master image IM2B are displayed.

[0099] That is, the candidate drug CD2 is an identifiable drug surface candidate, and the candidate output unit 110 displays a first master image IM2A of the drug surface candidate and a second master image IM2B of the opposite side of the drug surface candidate.

[0100] The candidate drug CD3 is displayed with a drug name N3, a first master image IM3A, and a second master image IM3B. An alert icon IA2 indicating that a warning flag has been assigned to the first side of the candidate drug CD3 is superimposed on the first master image IM3A.

[0101] That is, the candidate drug CD3 has a quasi-identifiable drug surface candidate, and the candidate output unit 110 displays a first master image IM3A of the candidate drug surface and a second master image IM3B of the opposite side of the candidate drug surface, and also displays an alert icon IA2 on the first master image IM3A.

[0102] The candidate drug CD4 is displayed with a drug name N4, a first master image IM4A, and a second master image IM4B. An alert icon IA3 indicating that a warning flag has been assigned to the first side of the candidate drug CD4 is superimposed on the first master image IM4A.

[0103] That is, the candidate drug CD4 has a identifiable drug surface candidate, and the candidate output unit 110 displays a first master image IM4A of the candidate drug surface and a second master image IM4B of the opposite side of the candidate drug surface. Also, because the first master image IM4A is an image of a quasi-identifiable surface, the candidate output unit 110 displays an alert icon IA3 on the first master image IM4A.

[0104] Here, the first master image and the second master image are displayed in the same order as the master images printed on the package insert. For example, in the package insert for the candidate drug CD4, "Drug JKL 1 mg," the master image of the side with the "QQ" engraved and / or printed markings is displayed on the left, and the master image of the side with the "#" engraved and / or printed markings is displayed on the right. Therefore, in the same order, the second master image IM4B of the side with the "QQ" engraved and / or printed markings is displayed on the left, and the first master image of the side with the "#" engraved and / or printed markings is displayed on the right.

[0105] In addition, in some cases, the attached document contains a master image of only the side with markings and / or printing, and there is no master image of the plain side, or there are no master images of both sides. In such cases, the display order cannot be determined from the attached document information, so the system creator defines the display order.

[0106] Alternatively, the candidate output unit 110 may display the first master image of the drug surface candidate on the left side of the two master images of each drug type candidate, and the second master image of the opposite side of the drug surface candidate on the right side. The candidate output unit 110 does not display drug surface candidates with similar multiple surfaces. Therefore, in this display method, the master image with similar multiple surfaces is not displayed on the left side of the two master images of each drug type candidate.

[0107] Here, we have described an example in which an alert is displayed on the first master image or at a position associated with the first master image when the first master image is an image of a quasi-identifiable surface, but if the second master image is an image of a quasi-identifiable surface, an alert may also be displayed on the second master image or at a position associated with the second master image.

[0108] A scroll bar SB is displayed at the right end of the display area A2. The scroll bar SB includes a scroll area SA that extends linearly in the vertical direction and a scroll knob SN that can be moved vertically within the scroll area SA. When the user drags the scroll knob SN downward, the candidate drug CD1 is hidden in the upper part of the display area A2, and the candidate drug having the drug surface candidate with the fifth highest score appears in the lower part of the display area A2.

[0109] The display area A3 is an area where GUI components for performing a text search are displayed. The drug identification device 100 is capable of performing a text search using an identification code, etc. In the example of F5A, a text box BT and a search start button BS are displayed.

[0110] On the display screen shown in F5A, when the user taps the alert icon displayed in display area A2, a list of similar drugs that have drug surfaces that are identical or similar to the drug surface of the master image on which the alert icon is displayed is displayed.

[0111] 5B is a diagram showing an example of a display screen when the alert icon IA1 is tapped. In this display screen, similar drugs SD1 and SD2 are displayed as a list of similar drugs in the display area A2.

[0112] The similar drug SD1 displays the drug name N11, a first master image IM11A, and a second master image IM11B. The similar drug SD1 is the same drug as the candidate drug CD1. The similar drug SD2 displays the drug name N12, a first master image IM12A, and a second master image IM12B. The similar drug SD2 is the same drug as the candidate drug CD3.

[0113] Furthermore, on the display screen shown in F5B, a back button BB is displayed in display area A2. The back button BB is a button for returning to the display screen of F5A. When the user taps the back button BB, the display screen of F5B transitions to the display screen of F5A.

[0114] The list of similar drugs shown in F5B is not limited to being displayed in display area A2, but may be displayed in a display area different from display areas A1 to A3 on the display screen of F5A.

[0115] In the display screens shown in F5A and F5B, the user can compare the cut-out image IC displayed in display area A1 with the master image displayed in display area A2. When the user taps the drug name or master image displayed in display area A2 of the display screen of F5A, the confirmation unit 112 confirms that drug as the correct drug. When the user taps the drug name or master image displayed in display area A2 of the display screen of F5B, the confirmation unit 112 may confirm that drug as the correct drug.

[0116] In addition, there may be cases where the correct drug is not included in the drug type candidates displayed in the display area A2. As an example, since the drug type identification AI 106A has not learned about similar multiple faces, if the extracted image IC is an image with similar multiple faces, the correct drug is not displayed among the drug type candidates. In this case, the user can perform a text search. When the user taps the text box BT, a soft keyboard (not shown) is displayed on the touch panel display 14. The user inputs text such as an identification code into the text box BT using the soft keyboard. Then, when the user taps the search start button BS, the candidate output unit 110 searches the drug database 108 to extract drug type candidates corresponding to the text entered in the text box BT. The candidate output unit 110 displays a list of the extracted drug type candidates in the display area A2.

[0117] As another example, even if the surface is identifiable or semi-identifiable, noise in the captured image may cause the score of the correct drug surface to drop relatively, preventing the correct drug surface from appearing on the candidate screen. Even in this case, the user can still identify the correct drug by text search using the same procedure as above.

[0118] In this way, the drug identification device 100 can display drug type candidates even when the correct drug is not included in the drug type candidates acquired by the drug type identification unit 106.

[0119] [Operation] If the extracted image of the drug to be identified is an image of an identifiable surface, it can be identified by the drug type identification AI 106A.

[0120] If the extracted image of the drug to be identified is an image of a quasi-identifiable surface, drug type candidates having the same or similar drug surface are extracted by the drug type identification AI 106A. In this case, an alert is displayed on the quasi-identifiable surface of the drug type candidate to guide the user to check both sides of the drug.

[0121] Even if one side is quasi-identifiable, there are many drugs whose drug type can be uniquely identified by simultaneously considering the other side as information. For this reason, if the user is alerted that "there is a drug with a similar marking" and "the other side should also be checked," and the user visually inspects the back of the drug along with the photographed side and compares it with the master images of the front and back of the candidate drug, the user can more accurately determine the correct drug, reducing the risk of the user selecting the incorrect drug.

[0122] If the extracted image of the drug to be identified is a similar multi-plane image, the correct drug will not exist among the candidates. Therefore, the user can be guided to search by the identification code.

[0123] According to the first embodiment, even when similar drugs exist, the user can accurately identify the correct drug for the drug to be identified. Similar drugs include, for example, drugs with the same ingredients but different amounts of active ingredient, drugs with the same ingredients but different dosage forms, regular tablets and OD (Orally Disintegrating) tablets with the same ingredients, original drugs and generic drugs with the same ingredients, generic drugs and generic drugs with different trade names, and drugs with different ingredients but similar markings and / or printing.

[0124] Second Embodiment [Learning and Flag Targets] Figure 6 is a diagram showing learning and targets of hide flags and warning flags for each category of drug surface according to the second embodiment. As shown in Figure 6, identifiable surfaces, quasi-identifiable surfaces, and similar multiple surfaces are the learning targets of the drug type identification AI 106A. That is, the drug type identification AI 106A according to the second embodiment has undergone machine learning using images of identifiable surfaces, quasi-identifiable surfaces, and similar multiple surfaces of drugs. Note that, among the similar multiple surfaces, images of solid-color similar multiple surfaces and similar multiple surfaces with only secant lines may be excluded from the learning targets.

[0125] 6, a non-display flag is assigned to the similar multiple surfaces of the drug. In actual data, the non-display flag value for the similar multiple surfaces is set to 1, and the non-display flag values ​​for the distinguishable surfaces and quasi-distinguishable surfaces are set to 0. The non-display flag may be stored in the drug database 108.

[0126] Furthermore, as in the first embodiment, a warning flag is attached to the quasi-distinguishable surface of the medicine.

[0127] [Functional Configuration of Drug Identification Device] The functional configuration of the drug identification device 100 according to the second embodiment is the same as that of the drug identification device 100 according to the first embodiment shown in FIG.

[0128] [Medicine Identification Method] Fig. 7 is a flowchart showing the steps of a medicine identification method according to the second embodiment using the medicine identification device 100. Note that parts that are common to the flowchart shown in Fig. 4 are given the same reference numerals, and detailed explanations thereof will be omitted.

[0129] As in the first embodiment, in step S1, the user photographs the identification target surface of the tablet for which the drug type is to be identified. In step S2, the drug detection unit 104 generates a cut-out image of the identifiable surface from the image photographed in step S1.

[0130] The processes in steps S11 to S13 and S4 to S6 are performed for each of all tablets detected in step S2.

[0131] In step S11, the drug type identification unit 106 inputs the cut-out image of the identifiable surface of the tablet to the drug type identification AI 106A, and obtains the scores of all drug surfaces to be learned.

[0132] In step S12, the drug type identification unit 106 deletes drug faces to which a non-display flag has been assigned, that is, similar multiple faces, from the set of all drug faces to be learned.

[0133] In step S13, the drug type identification unit 106 sorts all drug faces in descending order of the scores acquired in step S11. Here, since the similar majority faces have been deleted from the set of all drug faces in step S12, only identifiable faces and quasi-identifiable faces are sorted in descending order of score.

[0134] In step S4, the candidate output unit 110 acquires the drug identification information and master images of both sides for the M drugs having drug faces with the highest scores acquired in step S3 from the drug database 108. The candidate output unit 110 also displays a list of the acquired drug identification information and master images of both sides on the touch panel display 14 in descending order of score as drug type candidates for the correct drug.

[0135] Furthermore, the candidate output unit 110 displays an alert at the position of the master image of the medication side to which the attention flag is attached among the master images displayed on the touch panel display 14 .

[0136] The processing from step S5 onwards is the same as in the first embodiment.

[0137] According to the second embodiment, as in the first embodiment, when the master image of the drug type candidate has a quasi-distinguishable surface, a warning flag can be used to warn the user that similar drugs exist, leading the user to check both sides of the drug.

[0138] In addition, drug surfaces with a hidden flag attached, i.e., similar multiple surfaces, are not displayed as drug type candidates. Therefore, if the surface to be identified is a similar multiple surface, the user can be guided to search for the identification code or to identify the opposite surface.

[0139] <Classification of the drug surface of the drug to be identified and assumed classification flow> Fig. 8 is a diagram showing the classification of the drug surface of the drug to be identified and the assumed classification flow when one side is photographed. In Fig. 8, the vertical direction shows the classification of the first side, and the horizontal direction shows the classification of the second side.

[0140] In the case of a drug to be identified, both the first and second surfaces of which are identifiable, the drug type identification AI 106A can identify whichever drug surface is the identification target surface.

[0141] For drugs to be identified in which the first surface is a distinguishable surface and the second surface is a quasi-distinguishable surface, if the surface to be identified is a distinguishable surface, it can be identified by the drug type identification AI 106A. On the other hand, if the surface to be identified is a quasi-distinguishable surface, identification by the drug type identification AI 106A may reveal drug type candidates with similar drug surfaces, but an alert display can be used to warn the user of the existence of similar drugs. This can lead the user to check the opposite surface. The same applies when the first surface is a quasi-distinguishable surface and the second surface is a distinguishable surface.

[0142] For drugs to be identified in which the first surface is an identifiable surface and the second surface is a similar multiple surface, if the identification target surface is an identifiable surface, it can be identified by the drug type identification AI 106A. On the other hand, if the identification target surface is a similar multiple surface, the correct drug will not appear in the drug type candidates. Therefore, it is possible to guide the user to a search by identification code. The same applies when the first surface is a similar multiple surface and the second surface is an identifiable surface.

[0143] For drugs to be identified where both the first and second surfaces are quasi-identifiable surfaces, there is a possibility that drug candidates with similar drug surfaces may appear regardless of which drug surface is the surface to be identified, but an alert display can be displayed to warn the user that similar drugs exist.

[0144] For drugs to be identified where the first surface is a quasi-distinguishable surface and the second surface is a similar multi-surface, if the surface to be identified is a quasi-distinguishable surface, drug type candidates with similar drug types may appear, but an alert display can be used to warn the user of the existence of similar drugs. On the other hand, if the surface to be identified is a similar multi-surface, the correct drug will not appear among the drug type candidates, so the user can be guided to a search by identification code. The same applies when the first surface is a similar multi-surface and the second surface is a quasi-distinguishable surface.

[0145] For drugs to be identified where both the first and second sides are similar multiple sides, the correct drug will not appear in the drug type candidates regardless of which drug side is the side to be identified, so the user can be guided to search by identification code.

[0146] <Others> The technical scope of the present invention is not limited to the scope described in the above embodiments. The configurations and the like in each embodiment can be appropriately combined with each other within the scope that does not deviate from the spirit of the present invention.

[0147] <Supplementary Notes> As can be understood from the detailed description of the embodiments above, this specification includes disclosure of various technical ideas including the inventions described below.

[0148] [Supplementary Note 1] A drug identification device comprising at least one processor and at least one memory for storing instructions to be executed by the processor, wherein the processor: acquires an identification target image obtained by capturing an identification target surface, which is either a first surface or a second surface, of a drug to be identified among a plurality of drugs, each having a first surface and a second surface opposite to the first surface; displays the identification target image on a display; acquires a plurality of drug surface candidates, which are the first surface or the second surface of any of the plurality of drugs based on the identification target image and are correct candidates for the identification target surface; displays a list of a plurality of drug type candidates, each having a plurality of drug surface candidates, on the display; and determines a drug type candidate selected from the displayed list by an input device as the correct drug for the drug to be identified, where N is an integer of 2 or more, the first and second surfaces of each of the multiple drugs are classified into identifiable surfaces where the number of drugs having a first surface or a second surface that is identical or similar to the drug itself is 0, quasi-identifiable surfaces where the number of drugs having a first surface or a second surface that is identical or similar to the drug itself is 1 or more but less than N, and similar multiple surfaces where the number of drugs having a first surface or a second surface that is identical or similar to the drug itself is N or more, and the processor displays a first master image of the drug surface candidate and a second master image of the surface opposite the drug surface candidate for drug type candidates whose drug surface candidate is an identifiable surface, displays the first master image of the drug surface candidate and a second master image of the surface opposite the drug surface candidate for drug type candidates whose drug surface candidate is a quasi-identifiable surface, and displays an alert at a position associated with the first master image or the first master image, and does not display drug type candidates whose drug surface candidate is a similar multiple surface.

[0149] [Supplementary Note 2] The drug identification device according to Supplementary Note 1, wherein the processor, when the second master image is an image of a quasi-identifiable surface, causes an alert to be displayed on the second master image or at a location associated with the second master image.

[0150] [Supplementary Note 3] A drug identification device as described in Supplementary Note 1 or Supplementary Note 2, wherein the processor, in response to a trigger associated with each drug surface candidate, displays a list of master images of quasi-identifiable surfaces that are the same as or similar to the drug surface candidate and master images of surfaces opposite the quasi-identifiable surfaces.

[0151] [Supplementary Note 4] The drug identification device according to any one of Supplementary Note 1 to Supplementary Note 3, wherein the processor displays a number of drug type candidates determined based on an integer N.

[0152] [Supplementary Note 5] The drug identification device according to any one of Supplementary Note 1 to Supplementary Note 4, wherein the processor displays a plurality of drug type candidates corresponding to the identification code input by the input device.

[0153] [Supplementary Note 6] A drug identification device as described in any of Supplementary Note 1 to Supplementary Note 5, comprising a trained model that, when an image of the first or second side of a drug is input, outputs a score representing the probability that each of all drug sides to be trained is a drug side, and a processor inputs the image to be identified into the trained model and acquires the first or second side with a relatively high score as a drug side candidate.

[0154] [Supplementary Note 7] The drug identification device according to Supplementary Note 6, wherein the trained model does not target similar multiple faces for training.

[0155] [Supplementary Note 8] The drug identification device according to Supplementary Note 6 or Supplementary Note 7, wherein the trained model is a learning target for a large number of similar faces, and the processor assigns a non-display flag to the large number of similar faces.

[0156] [Supplementary Note 9] The drug identification device according to any one of Supplementary Note 6 to Supplementary Note 8, wherein the processor displays the plurality of drug type candidates in descending order of their scores.

[0157] [Supplementary Note 10] The drug identification device according to any one of Supplementary Notes 1 to 9, comprising: a display; and an input device.

[0158] [Supplementary Note 11] A drug identification method, comprising: at least one processor: acquiring a discrimination target image capturing a discrimination target surface, which is either a first surface or a second surface, of a drug to be identified among a plurality of drugs, each having a first surface and a second surface opposite to the first surface; displaying the discrimination target image on a display; acquiring a plurality of drug surface candidates, which are the first surface or the second surface of any of the plurality of drugs based on the discrimination target image and are correct candidates for the discrimination target surface; displaying a list of a plurality of drug type candidates, each having a plurality of drug surface candidates, on the display; and determining a drug type candidate selected from the displayed list by an input device as the correct drug for the drug to be identified, where N is an integer of 2 or greater, the first and second surfaces of each of the multiple drugs are classified into identifiable surfaces where the number of drugs having a first surface or a second surface that is identical or similar to the drug itself is 0, quasi-identifiable surfaces where the number of drugs having a first surface or a second surface that is identical or similar to the drug itself is 1 or more but less than N, and similar multiple surfaces where the number of drugs having a first surface or a second surface that is identical or similar to the drug itself is N or greater, and the processor, for drug type candidates whose drug surface candidates are identifiable surfaces, displays a first master image of the drug surface candidate and a second master image of the surface opposite the drug surface candidate, for drug type candidates whose drug surface candidates are quasi-identifiable surfaces, displays the first master image of the drug surface candidate and a second master image of the surface opposite the drug surface candidate, and displays an alert at a position associated with the first master image or the first master image, and hides drug type candidates whose drug surface candidates are similar multiple surfaces.

[0159] [Supplementary Note 12] A program that causes a computer to execute the drug identification method described in Supplementary Note 11.

[0160] DESCRIPTION OF SYMBOLS 10...Smartphone 14...Touch panel display 16...Speaker 18...Microphone 20...In-camera 22...Out-camera 24...Light 26...Switch 28...CPU 30...Wireless communication unit 32...Call unit 34...Memory 36...Internal storage unit 38...External storage unit 40...External input / output unit 42...GPS receiving unit 44...Power supply unit 100...Drug identification device 102...Image acquisition unit 104...Drug detection unit 106...Drug type identification unit 106A...Drug type identification AI 108...Drug database 110...Candidate output unit 112...Confirmation unit A1...Display area A2...Display area A3...Display area BB...Button BS...Search start button BT...Text box CD1...Candidate drug CD2...Candidate drug CD3...Candidate drug CD4...Candidate drug IA1...Alert icon IA2...Alert icon IA3...Alert icon IC...Cut-out image IM1A...First master image IM1B...Second master image IM2A...First master image IM2B...Second master image IM3A...First master image IM3B...Second master image IM4A...First master image IM4B...Second master image IM11A...First master image IM11B...Second master image IM12A...First master image IM12B...Second master image N1...Drug name N2...Drug name N3...Drug name N4...Drug name N11...Drug name N12...Drug name S1-S7, S11-S13...Steps of drug identification method SA...Scroll area SB...Scroll bar SD1...Similar drug SD2...Similar drug SN...Scroll knob ST1...GPS satellite ST2...GPS satellite

Claims

1. A drug identification device comprising at least one processor and at least one memory for storing instructions to be executed by the processor, wherein the processor: acquires an identification target image obtained by capturing an identification target surface, which is either the first surface or the second surface, of a drug to be identified among a plurality of drugs, each of which has a first surface and a second surface opposite to the first surface; displays the identification target image on a display; acquires a plurality of drug surface candidates, which are the first surface or the second surface of any of the plurality of drugs based on the identification target image and are correct candidates for the identification target surface; displays a list of a plurality of drug type candidates, each of which has the plurality of drug surface candidates, on the display; and determines a drug type candidate selected from the displayed list by an input device as the correct drug for the drug to be identified, where N is an integer of 2 or more, the first surface and the second surface of each of the plurality of drugs are classified into an identifiable surface where the number of drugs having the same or similar first surface or the second surface as the drug surface candidate is 0, a quasi-identifiable surface where the number of drugs having the same or similar first surface or the second surface as the drug surface candidate is 1 or more but less than N, and a similar multiple surface where the number of drugs having the same or similar first surface or the second surface as the drug surface candidate is N or more, and the processor displays a first master image of the drug surface candidate and a second master image of the opposite surface of the drug surface candidate for a drug type candidate whose drug surface candidate is the identifiable surface, displays a first master image of the drug surface candidate and a second master image of the opposite surface of the drug surface candidate for a drug type candidate whose drug surface candidate is the quasi-identifiable surface, and displays an alert at a position associated with the first master image or the first master image, and does not display the drug type candidate whose drug surface candidate is the similar multiple surface.

2. The drug identification device of claim 1, wherein the processor, when the second master image is an image of the quasi-identifiable surface, causes the alert to be displayed on the second master image or at a position associated with the second master image.

3. The drug identification device of claim 1, wherein the processor displays a list of master images of the quasi-identifiable surfaces that are identical or similar to the drug surface candidates and master images of the surfaces opposite the quasi-identifiable surfaces in response to a trigger associated with each of the drug surface candidates.

4. The drug identification device according to claim 1, wherein the processor displays a number of the drug type candidates determined based on the integer N.

5. The drug identification device according to claim 1, wherein the processor displays a plurality of drug type candidates corresponding to the identification code input by the input device.

6. A drug identification device as described in claim 1, comprising a trained model that, when an image of the first side or the second side of a drug is input, outputs a score representing the likelihood that each of all drug surfaces to be trained is a drug surface, and the processor inputs the image to be identified into the trained model and acquires the first side or the second side with a relatively high score as the drug surface candidate.

7. The drug identification device according to claim 6, wherein the trained model does not train the similar multiple faces.

8. The drug identification device according to claim 6, wherein the trained model is a learning target for the similar multiple surfaces, and the processor assigns a hidden flag to the similar multiple surfaces.

9. The drug identification device according to claim 6, wherein the processor displays the plurality of drug type candidates in order of relatively high score.

10. A medication identification device according to any one of claims 1 to 9, comprising: the display; and the input device.

11. A drug identification method, comprising: at least one processor: acquiring an identification target image capturing an identification target surface, which is either the first surface or the second surface, of a drug to be identified among a plurality of drugs, each of which has a first surface and a second surface opposite to the first surface; displaying the identification target image on a display; acquiring a plurality of drug surface candidates, which are the first surface or the second surface of any of the plurality of drugs based on the identification target image, and which are correct candidates for the identification target surface; displaying a list of a plurality of drug type candidates, each of which has the plurality of drug surface candidates, on the display; and determining a drug type candidate selected from the displayed list by an input device as the correct drug for the drug to be identified, wherein N is an integer of 2 or more, the first surface and the second surface of each of the plurality of drugs are classified into identifiable surfaces where the number of drugs having the first surface or the second surface that is identical or similar to the drug surface candidate is 0, quasi-identifiable surfaces where the number of drugs having the first surface or the second surface that is identical or similar to the drug surface candidate is 1 or more but less than N, and similar multiple surfaces where the number of drugs having the first surface or the second surface that is identical or similar to the drug surface candidate is N or more, and the processor displays a first master image of the drug surface candidate and a second master image of the opposite surface of the drug surface candidate for a drug type candidate whose drug surface candidate is the identifiable surface, displays a first master image of the drug surface candidate and a second master image of the opposite surface of the drug surface candidate for a drug type candidate whose drug surface candidate is the quasi-identifiable surface, and displays an alert at a position associated with the first master image or the first master image, and does not display the drug type candidate whose drug surface candidate is the similar multiple surface.

12. A program for causing a computer to execute the drug identification method according to claim 11.

13. A non-transitory computer-readable recording medium having the program according to claim 12 recorded thereon.

Images