Medication support system, medication support method, and medication support program

Abstract

To reduce the operational burden on a medication caregiver.SOLUTION: A portable terminal comprises a photographing unit that acquires information on characters printed on a cartridge, and a display unit that uses a face information database to display a terminal display image corresponding to face data corresponding to the character information. When authentication by visual examination as to whether the terminal display image corresponds to a person scheduled to take medicine is successful, the photographing unit determines whether or not it can photograph the face of the person scheduled to take medicine. When the photographing unit determines that it cannot photograph the face of the person, the display unit displays visual authentication success display indicating that the authentication by visual examination is successful. When the photographing unit determines that it can photograph the face of the person, an authentication unit executes face authentication by using the name and character information corresponding to a face image to be obtained through the face photographing.SELECTED DRAWING: Figure 17

Description

[Technical field]

[0001] The present disclosure relates to a medication support system, a medication support method, and a medication support program. [Background technology]

[0002] In medical institutions or nursing homes, medication assistants such as nurses or caregivers assist users in taking their medication correctly. Systems have been introduced to support medication assistants in preventing mistakes such as taking the wrong medication. For example, there is a system that can confirm whose medicine it is for by reading the QR code (registered trademark) printed on a single medicine package, and can confirm who the medicine is to be given to by reading the QR code (registered trademark) printed on the user's name tag, etc. QR is an abbreviation for Quick Response.

[0003] In addition, Patent Document 1 discloses a technology for supporting the distribution of medicine to residents in a facility, in which the resident who will take the medicine is identified from the medicine package and the name and face image of the identified resident are displayed on a mobile terminal. A feature of Patent Document 1 is that the person in charge of distributing the medicine recognizes the person who takes the medicine by checking the face image displayed on the mobile terminal. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2022-058870 Summary of the Invention [Problem to be solved by the invention]

[0005] Conventional technology uses two-dimensional codes such as QR codes (registered trademark) to identify the person who has taken the medicine inside the medicine package. However, whether or not a QR code (registered trademark) can be printed on a medicine package depends on the packaging machine installed in the dispensing pharmacy, and it is not always possible to print a QR code (registered trademark). In addition, the output format of the QR code (registered trademark) printed on the medicine package is not fixed and differs depending on the system that uses the QR code (registered trademark). Therefore, there are packaging machines that cannot handle the output format of each system, and there is an issue that it takes time and costs to introduce QR codes (registered trademark).

[0006] In addition, Patent Document 1 discloses that the facial image displayed on the mobile device is checked to see who the medication is to be handed to, but the facial image and identity of the resident are ultimately checked visually by the medication assistant. Therefore, it is not possible to prevent mistakes such as judging the same person to be a different person due to changes in hairstyle, or handing the medication to a different person who looks similar. In addition, in order to prevent mistakes such as giving the medicine to a different person who looks similar, it is possible to take a photograph of the face on the spot and perform identity authentication based on whether or not the name corresponding to the face image obtained by taking the photograph on the spot is the same as the name printed on the medicine package. However, in identity authentication using a face image, a face image is always required. However, when taking a photograph of the face, it may not be possible to take a photograph of the face depending on the surrounding conditions, such as poor lighting at night. Alternatively, it may not be possible to take a photograph of the face of the person who is to be photographed, because the person is resistant to the camera, has difficulty lifting his / her face, or is unable to be authenticated due to swelling, etc.

[0007] In this disclosure, while assuming a double check of visual inspection by the medication assistant and face authentication using a facial image obtained by photographing the person to be administered, it automatically determines whether or not face photography is possible, and when face photography is not possible, it enables identity authentication using the visual inspection result, thereby aiming to reduce the workload of the medication assistant. [Means for solving the problem]

[0008] The medication support system according to the present disclosure includes a mobile terminal used by a medication assistant who performs medication assistance, and supports the medication assistant, The mobile terminal includes: an imaging unit that captures an image of a medicine packet containing a medicine to be taken by a person to be administered the medicine, and obtains a name as a printed name from characters printed on the medicine packet; a display unit that uses a face information database in which face data representing a face image of a person who is a subject of medication assistance and the name of the person are set, extracts a face image corresponding to the printed name, and displays the face image on the mobile terminal as a terminal display image; Equipped with The imaging unit includes: a result of visual confirmation by the medication assistant ascertaining whether the terminal display image is the same person as the intended user, and if the authentication is successful by the visual confirmation, determining whether or not it is impossible to photograph the face of the intended user; The display unit is If it is determined that the face image cannot be captured, a visual authentication success display is displayed to indicate that the authentication has been successful through the visual confirmation; The mobile terminal further includes: The face recognition unit includes an authentication unit that, when it is determined that the face image can be photographed, performs face authentication using the name corresponding to the face image obtained by photographing the face and the printed name. Effect of the Invention

[0009] In the medication support system according to the present disclosure, when authentication is successful through visual confirmation by the medication assistant, the photographing unit determines whether or not facial photography of the person scheduled to take the medication is impossible. When it is determined that facial photography is impossible, the display unit displays a visual authentication success display indicating that authentication is successful through visual confirmation. When it is determined that facial photography is possible, the authentication unit performs facial authentication using the name corresponding to the facial image obtained by facial photography and the printed name. In this way, according to the medication support system according to the present disclosure, it is possible to automatically determine whether or not facial photography is impossible, assuming a double check of visual confirmation and facial authentication using a facial image obtained by photographing the person scheduled to take the medication. Therefore, according to the medication support system according to the present disclosure, even if facial photography is impossible, personal authentication can be performed using the result of visual confirmation, and the workload of the medication assistant can be reduced. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of the overall configuration of a medication support system according to a first embodiment. [Diagram 2] FIG. 1 is a diagram showing a configuration example of a medication support device according to a first embodiment. [Diagram 3] FIG. 2 is a diagram showing a configuration example of a mobile terminal according to the first embodiment. [Figure 4] 3 is a flow diagram showing the operation of the medication support system according to the first embodiment. [Diagram 5] 3 is a diagram showing an example of the configuration of a user database and a staff database stored in a storage unit of the medication support device of embodiment 1. FIG. [Figure 6] 3 is a diagram showing an example of the configuration of a face information database and a staff information database stored in a storage unit of the mobile terminal according to the first embodiment. [Figure 7] FIG. 4 is a detailed flow diagram of step S130 according to the first embodiment. [Figure 8] FIG. 2 is a diagram showing how a medicine packet is photographed in the first embodiment. [Figure 9] 4 is a diagram showing an example of the structure of character information and authentication information stored in a storage unit of the mobile terminal according to the first embodiment. [Figure 10]FIG. 4 is a detailed flow diagram of steps S140 and S150 according to the first embodiment. [Figure 11] FIG. 2 is a diagram showing how the face of a person to be administered the medicine is photographed in accordance with the first embodiment. [Figure 12] FIG. 4 is a detailed flow diagram of step S160 according to the first embodiment. [Figure 13] 5A and 5B are diagrams showing examples of an authentication success display and an authentication failure display according to the first embodiment. [Figure 14] 6 is a diagram showing examples of types of authentication results in the authentication process according to the first embodiment. [Figure 15] FIG. 11 is a diagram showing variations of authentication information when authentication according to the first embodiment fails. [Figure 16] FIG. 2 is a diagram showing an example of the configuration of a medication record database according to the first embodiment. [Figure 17] FIG. 11 is a flow diagram showing the operation of the medication support system according to the second embodiment. [Figure 18] FIG. 11 is a diagram showing an example of the configuration of a user database and a staff database stored in the storage unit of the medication support device of embodiment 2. [Figure 19] 13 is a diagram showing an example of the configuration of a face information database and a staff information database stored in a storage unit of a mobile terminal according to the second embodiment. [Figure 20] FIG. 11 is a diagram showing an example of a visual confirmation screen according to the second embodiment. [Figure 21] 13A to 13C are diagrams showing examples of photographing unfeasible conditions in the face photographing determination process according to the second embodiment. [Figure 22] FIG. 13 is a diagram showing an example of a scheduler imaging start screen according to the second embodiment. [Diagram 23] 13 is a diagram showing an example of a skip condition for skipping face photographing according to the second embodiment. [Figure 24] FIG. 11 is a diagram showing an example of the configuration of a medication record database according to the second embodiment. [Diagram 25] FIG. 11 is a diagram showing an example of a visual authentication success display according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the present embodiment will be described with reference to the drawings. In each drawing, the same or corresponding parts are given the same reference numerals. In the description of the embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate. The arrows in the drawing mainly indicate the flow of data or the flow of processing. In addition, the relationship of the size of each component in the following drawings may differ from the actual one. In addition, in the description of the embodiment, directions or positions such as up, down, left, right, front, back, front, and back may be indicated. These notations are for convenience of explanation and do not limit the arrangement, direction, or orientation of devices, instruments, or parts.

[0012] Embodiment 1 ***Configuration Description*** FIG. 1 is a diagram showing an example of the overall configuration of a medication support system 500 according to the present embodiment. The medication support system 500 is a system for supporting medication assistance by a medication assistant (nurse or caregiver) in a facility such as a medical institution or a care facility. The medication assistant provides medication assistance to ensure that facility users take their medicine correctly. The medication support system 500 includes a medication support device 100 and a plurality of mobile terminals 200. The medication support device 100 and the mobile terminals 200 communicate with each other via a network 300. The mobile terminals 200 are terminals carried by a medication assistant. The mobile terminals 200 are computers having a photographing function, such as a smartphone, a tablet, a portable computer, or a personal computer. A plurality of mobile terminals 200 may be present in the same facility, or a plurality of mobile terminals 200 may be present in a plurality of different facilities.

[0013] FIG. 2 is a diagram showing an example of the configuration of the medication support device 100 according to the present embodiment. FIG. 3 is a diagram showing an example of the configuration of the mobile terminal 200 according to the present embodiment.

[0014] The medication support device 100 is a computer. The medication support device 100 includes a processor 910, as well as other hardware such as a memory 921, an auxiliary storage device 922, an input interface 930, an output interface 940, and a communication device 950. The processor 910 is connected to the other hardware via signal lines and controls the other hardware. The medication support device 100 includes, as functional elements, a character extraction unit 110 and an information configuration unit 120. In the storage unit 150, a user database 11, a staff database 12, and a medication record database 13 are stored.

[0015] The mobile terminal 200 is a computer. The mobile terminal 200 includes a processor 910 and other hardware such as a memory 921, an auxiliary storage device 922, an input interface 930, an output interface 940, a communication device 950, and an image capturing device 960. The processor 910 is connected to the other hardware via a signal line and controls the other hardware. The image capturing device 960 is a camera. The mobile terminal 200 includes, as functional elements, a display unit 210, a photographing unit 220, a corresponding name acquisition unit 230, and an authentication unit 240. The storage unit 260 stores a face information database 31, a staff information database 32, character information 33, authentication information 34, and a name identification result 35.

[0016] The hardware components of the processor 910, memory 921, auxiliary storage device 922, input interface 930, output interface 940, display device 941, and communication device 950 are described using the same reference numerals in both the medication support device 100 and the mobile terminal 200. This is done merely to simplify the description, and it is clear that the medication support device 100 and the mobile terminal 200 each have their own hardware.

[0017] The functions of each unit in the medication support device 100 are realized by software. The functions of each unit in the mobile terminal 200 are realized by software. Each of the storage units 150 and 260 is provided in a memory 921. Each of the storage units 150 and 260 may be provided in an auxiliary storage device 922, or may be provided separately in the memory 921 and the auxiliary storage device 922.

[0018] The processor 910 is a device that executes a medication support program. The medication support program is a program that realizes the functions of each unit of the medication support device 100 and the mobile terminal 200. The processor 910 is an IC that performs arithmetic processing. Specific examples of the processor 910 are a CPU, a DSP, and a GPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit. DSP is an abbreviation for Digital Signal Processor. GPU is an abbreviation for Graphics Processing Unit.

[0019] The memory 921 is a storage device that temporarily stores data. Specific examples of the memory 921 are SRAM and DRAM. SRAM is an abbreviation for Static Random Access Memory. DRAM is an abbreviation for Dynamic Random Access Memory. The auxiliary storage device 922 is a storage device that stores data. A specific example of the auxiliary storage device 922 is a HDD. The auxiliary storage device 922 may also be a portable storage medium such as an SD (registered trademark) memory card, CF, NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD. Note that HDD is an abbreviation for Hard Disk Drive. SD (registered trademark) is an abbreviation for Secure Digital. CF is an abbreviation for CompactFlash (registered trademark). DVD is an abbreviation for Digital Versatile Disk.

[0020] The input interface 930 is a port connected to an input device such as a mouse, a keyboard, or a touch panel. Specifically, the input interface 930 is a USB terminal. The input interface 930 may be a port connected to a LAN. USB is an abbreviation for Universal Serial Bus. LAN is an abbreviation for Local Area Network.

[0021] A display device 941 such as a display is connected to the output interface 940. Specifically, the output interface 940 is a USB terminal or an HDMI (registered trademark) terminal. Specifically, the display is an LCD. HDMI (registered trademark) is an abbreviation for High Definition Multimedia Interface. LCD is an abbreviation for Liquid Crystal Display.

[0022] The communication device 950 is connected to a communication network such as a LAN, Wi-Fi (registered trademark), the Internet, or a telephone line.

[0023] The medication support program is executed in each of the medication support device 100 and the mobile terminal 200. The medication support program is loaded into the processor 910 and executed by the processor 910. Memory 921 stores not only the medication support program but also an OS. OS is an abbreviation for Operating System. The processor 910 executes the medication support program while executing the OS. The medication support program and the OS may be stored in an auxiliary storage device 922. The medication support program and the OS stored in the auxiliary storage device 922 are loaded into the memory 921 and executed by the processor 910. Note that a part or all of the medication support program may be incorporated into the OS.

[0024] The "parts" of the parts of the medication support device 100 and the mobile terminal 200 may be read as "circuits," "steps," "procedures," "processing," or "circuitry." The medication support program causes a computer to execute processing in which the parts of the medication support device 100 and the mobile terminal 200 are replaced. The "processing" of the processing in which the parts of the medication support device 100 and the mobile terminal 200 are replaced may be read as "program," "program product," or "computer-readable storage medium storing a program." The medication support program may be provided by being stored in a computer-readable storage medium. The medication support program may also be provided as a program product.

[0025] ***Explanation of Operation*** Next, an operation of the medication support system 500 according to the present embodiment will be described. The operation procedure of the medication support system 500 corresponds to a medication support method. Moreover, a program for realizing the operation of the medication support system 500 corresponds to a medication support program.

[0026] FIG. 4 is a flow diagram showing the operation of medication support system 500 according to this embodiment.

[0027] <Initial Processing: Step S110> When the medication support process is started, in step S110, the display unit 210 of the mobile terminal 200 displays an initial screen on the display device 941. In addition, the authentication unit 240 of the mobile terminal 200 acquires information to be set in the face information database 31 and the staff information database 32 from the medication support device 100 via the communication device 950. Specifically, the authentication unit 240 acquires face information of the user and staff information of the staff at the facility to which the medication assistant belongs from the medication support device 100, and stores them in the memory unit 260 as face information database 31 and staff information database 32.

[0028] The process of step S110 is started, for example, when the power of the mobile terminal 200 is turned on. If the power of the mobile terminal 200 is turned on at a fixed time every morning, the face information database 31 and the staff information database 32 are reset every morning.

[0029] FIG. 5 is a diagram showing an example of the configuration of the user database 11 and the staff database 12 stored in the storage unit 150 of the medication support device 100 according to this embodiment.

[0030] In the user database 11, information on all users in all facilities that use the medication support system 500 is set. Information such as facility ID 111, user ID 112, user name 113, reading 114, and face data 115 is set in the user database 11. Information about the user such as date of birth, gender, special notes, and identifier may also be set. The identifier is information for identifying an individual when, for example, there is a user with the same name. The face data 115 is data generated in advance based on a face image of a person who is a user. The face data 115 is data that represents the face image of the person. Specifically, the face data 115 is data that digitizes the face image of the person.

[0031] In the staff database 12, information on all staff members in all facilities that use the medication support system 500 is set. In the staff database 12, information such as a facility ID 121, a staff ID 122, and a name 123 is set.

[0032] FIG. 6 is a diagram showing an example of the configuration of face information database 31 and staff information database 32 stored in storage unit 260 of mobile terminal 200 according to this embodiment. A user name 311 and face data 312 are set in the face information database 31. The users set in the face information database 31 are users of the facility to which the medication assistant who possesses the mobile terminal 200 belongs. A staff ID 321 and a name 322 are set in the staff information database 32. The staff set in the staff information database 32 is the staff of the facility to which the medication assistant who possesses the mobile terminal 200 belongs.

[0033] For example, the authentication unit 240 transmits an information transmission request for face information and staff information, which includes a facility ID of a facility to which the medication assistant carrying the mobile terminal 200 belongs, to the medication support device 100. Upon receiving the information transmission request, the medication support device 100 searches the user database 11 and the staff database 12 using the facility ID included in the information transmission request, and transmits to the mobile terminal 200 the information on the user and staff included in the facility ID.

[0034] That is, the face information database 31 is obtained by extracting the user name and face data corresponding to the facility ID from the user database 11 of the medication support device 100. The staff information database 32 is obtained by extracting the staff ID and the staff name corresponding to the facility ID from the staff database 12 of the medication support device 100.

[0035] <Staff authentication process: Step S120> In step S120, the authentication unit 240 authenticates staff such as a medication assistant. Any method of staff authentication may be used. For example, the authentication unit 240 performs staff authentication by displaying a staff ID or name, or both, based on the staff information database 32 using the display unit 210 and accepting a selection from the staff member. Alternatively, face data representing a face image of the staff member may be set in advance in the staff information database 32, and the authentication unit 240 may perform staff authentication by performing face authentication using a face image photographed by the staff member using the photographing device 960.

[0036] <Drug Package Image Capture Process and Character Extraction Process: Step S130> In step S130, photographing unit 220 of mobile terminal 200 performs a medicine package image photographing process to photograph the medicine package using photographing device 960. Then, photographing unit 220 transmits the image of the medicine package obtained by photographing to medication support device 100. Character extraction unit 110 of medication support device 100 receives the image of the medicine package from mobile terminal 200 and extracts characters printed on the medicine package. Character extraction unit 110 of medication support device 100 transmits the extracted characters to photographing unit 220 of mobile terminal 200 as character information.

[0037] FIG. 7 is a detailed flow diagram of step S130 according to the present embodiment. FIG. 8 is a diagram showing how medicine packet 51 according to the present embodiment is photographed. In step S131, display unit 210 of mobile terminal 200 displays medicine packet photographing start screen 61 for photographing medicine packet 51.

[0038] In step S132, the photographing section 220 of the mobile terminal 200 photographs an image of the medicine packet 51 by the photographing device 960. Specifically, when the medicine packet photographing start screen 61 is displayed, the medication assistant photographs the medicine packet 51 to be taken by the intended patient via the medicine packet photographing start screen 61. Then, the photographing section 220 transmits the image of the medicine packet 51 to the medication support device 100 via the communication device 950.

[0039] The position where the medicine bag 51 should be set is displayed as a guide on the medicine bag photographing start screen 61, and the medication assistant sets the medicine bag 51 according to the guide and photographs it. Medicine packet 51 is a package containing a combination of medicines to be taken at one time. Medicine packet 51 is packaged at the pharmacy and has information such as the name of the person taking the medicine, the date of taking the medicine, and the timing of taking the medicine (e.g., "before breakfast", "after breakfast") printed on it.

[0040] In step S133, medication support device 100 receives the image of medicine packet 51 from mobile terminal 200. Character extraction unit 110 of medication support device 100 extracts characters printed on medicine packet 51 from the received image of medicine packet 51. In the example of Figure 8, characters such as "Taro Ogawa, Friday, July 22, 2022, after breakfast" are extracted. Character extraction unit 110 transmits the extracted characters to mobile terminal 200 as character information. Note that the meaning of the characters is not determined here. Character extraction unit 110 recognizes character boundaries using existing character recognition technology, and transmits the recognized character strings to mobile terminal 200 in order from the top left of the image. The recognized locations of the recognized character strings are also transmitted as pixel information. This pixel information is preferably the top left and bottom right of the recognized character string to specify the region of the character string as a rectangle. However, it may be only the top left location, or it may include pixel information for four locations: the top left, top right, bottom right, and bottom left.

[0041] In step S134, when the photographing unit 220 of the mobile terminal 200 receives the text information from the medication support device 100, the photographing unit 220 stores the text information in the storage unit 260 as text information 33.

[0042] FIG. 9 is a diagram showing an example of the configuration of character information 33, authentication information 34, and name identification result 35 stored in storage unit 260 of mobile terminal 200 according to this embodiment. Character information 33 in FIG. 9 shows character information extracted from the example image of medicine packet 51 in FIG. 8. The character information transmitted from medication support device 100 corresponds to "date", "name", and "timing of taking medicine", but as described above, the meaning of the items and characters is not acquired. Therefore, character information 33 only stores each character string that is separated and read by medication support device 100. Item 1, item 2, and item 3 in FIG. 9 are described for convenience only and have no meaning. In addition, although not shown in FIG. 9, each character string and position information (pixel information) where the character string was acquired are recorded together. The authentication information 34 and the name identification result 35 will be described later.

[0043] Returning to FIG. 4, the explanation will be continued.

[0044] <Facial image capture process: step S140> In step S140, the photographing section 220 of the mobile terminal 200 photographs, by the photographing device 960, a face image of the intended recipient 52 who is to take the medicine contained in the medicine packet 51. The intended recipient 52 is a person for whom the medication assistant will assist with taking the medicine from the medicine packet 51.

[0045] <Corresponding name acquisition process: Step S150> The corresponding name acquiring section 230 of the mobile terminal 200 acquires, from the face information database 31, a name corresponding to face data representing a face image of the prospective recipient 52 as the image corresponding name 125.

[0046] FIG. 10 is a detailed flow diagram of steps S140 and S150 according to this embodiment. The facial image photographing process for photographing the face of the prospective recipient 52 in step S140 and the corresponding name acquisition process in step S150 will be described with reference to FIG.

[0047] In step S141, the display unit 210 of the mobile terminal 200 displays a scheduled person photographing start screen 62 for photographing the face of the scheduled person 52. At this time, if character information 33, which is characters printed on the medicine packet 51, is stored, the contents of the character information 33 may also be displayed. In step S142, the photographing section 220 of the mobile terminal 200 photographs the face of the intended person 52 by the photographing device 960. Specifically, when the intended person photographing start screen 62 is displayed, the medication assistant photographs the face of the intended person 52 so that the face of the intended person 52 is within an area such as a circle displayed on the intended person photographing start screen 62.

[0048] FIG. 11 is a diagram showing how the face of a prospective recipient 52 is photographed according to this embodiment. 11, the display unit 210 displays a scheduled patient imaging start screen 62. At this time, if character information 33, which is characters printed on the medicine packet 51, is stored in the storage unit 260, the content of the character information 33 may also be displayed. The position of the face of the intended patient 52 is displayed as a guide on the intended patient photographing start screen 62. Then, as shown in the right diagram, the medication assistant photographs the face of the intended patient 52 according to the guide.

[0049] In step S143, the associated name acquiring section 230 digitizes the facial image of the prospective recipient 52 obtained by photographing. Any method may be used to digitize the facial image.

[0050] The corresponding name acquisition unit 230 acquires a name corresponding to the digitized face data from the face information database 31 as an image corresponding name 53 . Specifically, the following applies:

[0051] In step S144, the corresponding name acquisition unit 230 searches the face information database 31 using the digitized face data of the intended recipient 52, and judges whether or not there is face data 312 that can be judged to be the same. The judgment in step S144 does not have to be a judgment of whether or not the numerical values ​​are exactly the same, but may be a judgment that allows for a certain degree of fluctuation. Any technique may be used to judge the range of fluctuation and whether or not the values ​​are the same including the fluctuation.

[0052] If face data 312 that can be determined to be the same exists, the process proceeds to step S145. If there is no face data 312 that can be determined to be the same, the process proceeds to step S146.

[0053] In step S145, the corresponding name acquisition unit 230 acquires, from the face information database 31, a name corresponding to the face data 312 that can be determined to be identical to the face data of the intended recipient 52, as the image corresponding name 53. The corresponding name acquisition unit 230 temporarily stores the acquired image corresponding name 53 in the name identification result 35 of the storage unit 260.

[0054] In step S146, face data 312 that can be determined to be the same as the face data of the intended recipient 52 is not present in the face information database 31. Therefore, the corresponding name acquisition unit 230 sets the name identification result 35 in the storage unit 260 to null. Name identification result 35 is information that temporarily sets the result of whether or not the name could be identified from face information database 31 using the facial data of expected user 52. If image-corresponding name 53 can be identified from the facial data of expected user 52, the identified image-corresponding name 53 is set in name identification result 35. On the other hand, if image-corresponding name 53 cannot be identified from the facial data of expected user 52, name identification result 35 becomes null. If it is known that image-corresponding name 53 cannot be identified, data other than null may be used.

[0055] The order of photographing the medicine packet 51 and photographing the face image of the intended recipient 52 may be reversed. As long as the functions described in steps S130 to S150 can be realized, the order of the processes included in steps S130 to S150 does not matter. Furthermore, in step S144, if there is no face data 312 that can be determined to be the same, the corresponding name acquiring section 230 may prompt the user to retake a face image of the intended recipient 52 on the intended recipient photographing start screen 62.

[0056] Returning to FIG. 4, the explanation will be continued.

[0057] <Authentication process: Step S160> Authentication unit 240 of mobile terminal 200 acquires the name printed on medicine package 51 as printed name 54 from character information 33, which is the characters printed on medicine package 51. Then, authentication unit 240 determines whether printed name 54 and image-corresponding name 53 can be authenticated as being the same. Specifically, the following applies:

[0058] FIG. 12 is a detailed flow diagram of step S160 according to the present embodiment. In step S161, authentication unit 240 acquires the name printed on medicine packet 51 from character information 33 as printed name 54. As described above, all character strings printed on medicine package 51 are set in character information 33. Therefore, authentication unit 240 needs to extract the name therefrom. A method for acquiring the name from character information 33 as printed name 54 is, for example, as follows.

[0059] The authentication unit 240 acquires the position where the name is printed in the image of the medicine package as the name position. The name position is stored in advance in the storage unit 260. The authentication unit 240 acquires the characters printed in the name position among the characters printed on the medicine package 51 as the printed name 54. In the example of the medicine package 51 in FIG. 8, it is assumed that pixel information starting from the upper left corner of the image of the medicine package is stored in the mobile terminal 200 as the name position, which is the position where the name is printed. The authentication unit 240 identifies the printed name 54 by comparing the character information 33 with the pixel information stored as the name position. The position information and pixel information of the character information 33 may not match due to camera shake when the medicine package 51 is photographed, but the authentication unit 240 may be configured to identify the printed name 54 within a certain tolerance range.

[0060] Moreover, the authentication unit 240 may identify the characters “sama” or “dono” and obtain the character string before that as the printed name 54. In addition, to prevent the date and timing information in the character information 33 from being subject to comparison, processing may be performed to eliminate dates containing numbers and words such as "after breakfast" or "before meals," and the remaining character string may be obtained as the printed name 54. According to this identification method, there is no need for the mobile terminal 200 to store name and location information.

[0061] In step S162, the authentication unit 240 determines whether or not it can be authenticated that the printed name 54 and the image corresponding name 53 are the same. First, the authentication unit 240 refers to the name identification result 35 in the storage unit 260. When the name identification result 35 is null, it indicates that the image-corresponding name 53 could not be obtained. Therefore, the printed name 54 and the image-corresponding name 53 cannot be compared, and it is impossible to authenticate whether they are the same. For this reason, when the name identification result 35 is null, the process proceeds to step S164.

[0062] If the image-corresponding name 53 is set in the name identification result 35, the authentication unit 240 compares the printed name 54 and the image-corresponding name 53. When it can be authenticated that the printed name 54 and the image-corresponding name 53 are the same, the process proceeds to step S163. When it cannot be authenticated that the printed name 54 and the image-corresponding name 53 are the same, the process proceeds to step S164. Whether it can be authenticated that the printed name 54 and the image-corresponding name 53 are the same may be based on the condition of exact match, or conditions for being the same may be set in advance. For example, conditions such as authenticating that they are the same in the case of a one-character difference with only similar characters like "大" and "太" may be set in advance. In this embodiment, it is assumed that it can be authenticated that they are the same when the character strings of the printed name 54 and the image-corresponding name 53 exactly match.

[0063] FIG. 13 shows examples of the authentication success display 21 and the authentication failure display 22 according to this embodiment. In step S163, the display unit 210 displays an authentication success display 21 indicating that the scheduled taker 52 is the correct taker. In the authentication success display 21 shown in FIG. 13(a), an authentication success message 211 indicating authentication success, and the name, date, and taking timing of the scheduled taker 52 who is the correct taker are displayed inside a circle representing authentication success.

[0064] In step S164, display unit 210 displays authentication failure display 22 indicating failure in authentication between printed name 54 and image-corresponding name 53. Possible authentication failures include failure caused by inability to acquire image-corresponding name 53 in relation to face authentication (steps S140 to S150, FIG. 13(b)), and failure caused by erroneous recognition of printed name 54 in relation to photographing medicine packet 51 (step S130, FIG. 13(c)).

[0065] 13B shows an authentication failure display 22 when the name identification result 35 is null, that is, when the image-corresponding name 53 cannot be acquired, and therefore the printed name 54 and the image-corresponding name 53 cannot be authenticated as being the same. For example, this is the case when the face authentication itself fails due to factors such as the face of the person to take the medicine 52 being hidden by a mask, the angle of the face when photographed being inappropriate, or insufficient illumination, and therefore the image-corresponding name 53 cannot be acquired. Alternatively, the face data of the person to take the medicine 52 may not be registered in the user database 11, or may have been omitted from registration. In this case, the display unit 210 displays an authentication failure display 22 including a facial authentication failure message 221, character information 33 which is information obtained by character recognition from an image of the medicine package, an image of the medicine package 222, a registration button 223, and a retake button 224. The face authentication failure message 221 is a message indicating that the image-corresponding name 53 could not be obtained. The registration button 223 is a button that the medication assistant uses to forcibly register the recipient who will take the medicine. Here, a list of candidates for the names of the intended recipient 52 is displayed, and the medication assistant can select the name of the intended recipient 52 to be forcibly registered. The registration button 223 may not be a selection type, but may be a type in which the name of the intended recipient 52 to be forcibly registered is input. The retake button 224 is a button that the medication assistant uses to retake the face image of the intended recipient 52. The display unit 210 displays, for example, the printed name 54 first as a candidate for the name of the prospective user 52, and further displays any names similar to the printed name 54 extracted from the names of users in the same facility. Names that differ from the printed name 54 by one or two characters, or all the names of users in the same facility, may be displayed.

[0066] FIG. 13(c) shows an authentication failure display 22 displayed when, as a result of comparing the printed name 54 with the image-corresponding name 53, the printed name 54 and the image-corresponding name 53 do not match, and the printed name 54 and the image-corresponding name 53 cannot be authenticated as being identical. In this case, the display unit 210 displays an authentication failure display 22 including a mismatch message 231, the printed name 54 and the image-corresponding name 53 and explanatory text for identifying each, an image of the medicine package 232, a registration button 233, and a retake button 234. The mismatch message 231 is a message indicating a mismatch between the printed name 54 and the image-corresponding name 53. The registration button 233 is a button that allows the medication assistant to forcibly register the recipient who will take the medication. In FIG. 13(c), when the registration button 233 is selected, the image-corresponding name 53 is registered. Alternatively, a method of inputting the name of the intended recipient 52 to be forcibly registered may be used. Here, when the registration button 233 is selected, the image-corresponding name 53 is registered because this is an operation that assumes that the authentication failure is due to erroneous recognition of the printed name 54. 13(c), it is assumed that the face authentication was OK (registered face), but the printed name 54 does not match the image-corresponding name 53. Therefore, the authentication failure display 22 may suggest a reading error when acquiring the printed name 54. For example, the authentication failure display 22 may suggest an error such as a difference of just one character, a misrecognition of similar characters, or a missing character. 13(c) is not caused by a failure due to misrecognition of the printed name 54 when photographing the medicine packet 51, but may be caused by photographing a face image of a person other than the intended user 52. The retake button 234 is a button for the medication assistant to take a new face image of the intended user 52 when he or she notices this.

[0067] Next, in step S165, the authentication unit 240 generates authentication information 34 based on the authentication result. Then, the authentication unit 240 transmits the generated authentication information 34 to the medication support apparatus 100. 9 includes a facility ID 341, face authentication date and time 342, authentication result 343, image-corresponding name 344, printed name 345, staff ID 346, medicine package read characters 347, and medication assistance memo 348. The medication assistance memo 348 is a memo arbitrarily input by the medication assistant. In addition, the system may be configured to allow information on medication, such as high-risk medication, to be input. The authentication information 34 shown in FIG. 9 is a setting example in which the authentication result is OK, that is, authentication is successful.

[0068] FIG. 14 shows an example of types of authentication results in the authentication process according to the present embodiment. As shown in FIG. 14, the types set for the authentication result in the authentication information 34 include, for example, OK, NG1, and NG2. The type "OK" is when the face authentication in step S144 is successful, and the comparison result between the printed name 54 and the image corresponding name 53 in step S162 also matches. At this time, the final result is authentication success. An authentication success display 21 as shown in FIG. 13(a) is displayed. Type "NG1" is when the face authentication in step S144 is successful, but the comparison result between the printed name 54 and the image-corresponding name 53 in step S162 does not match. In this case, the final result is authentication failure. An authentication failure display 22 as shown in FIG. 13(c) is displayed. The medication assistant may confirm that the correct person is the person who will take the medication by directly asking the person who will take the medication 52 or by visually checking, and may forcibly register the person who will take the medication. Type "NG2" occurs when the face authentication in step S144 fails, and therefore the comparison between the printed name 54 and the image-corresponding name 53 in step S162 cannot be performed. In this case, the final result is authentication failure. An authentication failure display 22 as shown in FIG. 13(b) is displayed. In this case, the medication assistant may confirm the correct medication recipient by directly asking the intended recipient 52 or by visual confirmation, and may forcibly register the medication recipient. It should be noted that the frequency of occurrence of type "NG2" can be reduced by encouraging the recipient to take a photo again when face authentication fails.

[0069] As described above, when the medication assistant confirms that the intended recipient 52 is the correct recipient by directly asking the recipient 52 or by visually checking, and then forcibly registers the recipient, the authentication result type will be “forced OK.”

[0070] Fig. 15 is a diagram showing variations of authentication information 34 when authentication according to the present embodiment has failed. Note that in Fig. 15, the medicine package reading characters are omitted.

[0071] <Variation 1> FIG. 15(a) shows authentication information 34 in a case where printed name 54 and image-corresponding name 53 were acquired, but printed name 54 and image-corresponding name 53 did not match, resulting in "NG1." For example, this is the case when "Oo△Hanako" is read as "Futa△Hanako" due to dirt or the like when acquiring printed name 54. After checking authentication failure display 22 in FIG. 13(c), the medication assistant confirms that intended user 52 is "Oo△Hanako" and clicks registration button 233 to register "Oo△Hanako" as the authentication result "forced OK."

[0072] <Variation 2> Fig. 15(b) shows authentication information 34 in a case where facial authentication failed, image-corresponding name 53 could not be acquired, and the result was "NG2." For example, this is the case when facial authentication was not possible because the patient turned sideways multiple times when acquiring a facial image. After checking the authentication failure display 22 in Fig. 13(b), the medication assistant confirms that the intended user 52 is "Oo△Hanako," and presses the registration button 223 to register "Oo△Hanako" as the authentication result "forced OK."

[0073] <Variation 3> FIG. 15(c) shows authentication information 34 in a case where printed name 54 and image-corresponding name 53 could be acquired, but printed name 54 and image-corresponding name 53 did not match, resulting in "NG1." Here, this is the case where the medication assistant mistakenly takes a facial image of another elderly person who resembles expected medication recipient 52. After checking authentication failure display 22 in FIG. 13(c), the medication assistant presses retake button 234 to take a new photograph of expected medication recipient 52, and authentication is successful, generating authentication information 34 with the authentication result "OK."

[0074] In step S166, the information configuration unit 120 of the medication support apparatus 100 receives the authentication information 34 from the mobile terminal 200, and stores it in the medication record database 13. In step S167, the authentication unit 240 initializes information such as the character information 33, the authentication information 34, and the name identification result 35.

[0075] 16 is a diagram showing an example of the configuration of medication history database 13 according to this embodiment. Note that in FIG. 16, the description of the medicine package read characters is omitted. Medication history database 13 accumulates authentication information in all facilities that use medication support system 500. Items in medication history database 13 are the same as the items in authentication information .

[0076] ***Other configurations*** In this embodiment, the functions of each unit of each device of medication support system 500 are realized by software. As a modification, the functions of each unit of each device of medication support system 500 may be realized by hardware. Specifically, the medication support device 100 and the mobile terminal 200 may include an electronic circuit instead of the processor 910.

[0077] The electronic circuit is a dedicated electronic circuit that realizes the function of each part of each device of the medication support system 500. Specifically, the electronic circuit is a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an ASIC, or an FPGA. GA is an abbreviation for Gate Array. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field-Programmable Gate Array.

[0078] As another modification, some of the functions of each unit of each device of medication support system 500 may be realized by electronic circuits, and the remaining functions may be realized by software. Also, some or all of the functions of each unit of each device of medication support system 500 may be realized by firmware.

[0079] Each of the processor and electronic circuitry is also called a processing circuitry. In other words, the functions of each unit of each device in the medication support system 500 are realized by the processing circuitry.

[0080] ***Explanation of the Effects of the Present Embodiment*** As described above, in the medication support system 500 according to the present embodiment, the name is acquired as the printed name from among the characters printed on the medication package from the image of the medication package. Then, it is determined whether or not the acquired printed name can be authenticated as being the same as the image-corresponding name, which is the name corresponding to the face data representing the face image of the person who is to take the medication. In this way, in the medication support system 500 according to the present embodiment, personal information is identified from the characters printed on the medication package of the medication to be taken, and the name acquired by face authentication is compared with the name printed on the medication package to authenticate the person who takes the medication. Therefore, the medication assistant does not need to authenticate the person by visual inspection. Therefore, according to the medication support system 500 according to the present embodiment, authentication with high reliability can be performed.

[0081] In addition, since the medication support system 500 according to the present embodiment is a system that can be constructed using a mobile terminal equipped with a camera, no new capital investment is required in the facility. Furthermore, it is sufficient that the name of the person who takes the medication is printed on the medicine packet, and printing of additional information such as a two-dimensional code such as a QR code (registered trademark) is not required. Therefore, the medication support system 500 according to the present embodiment does not require coordination with pharmacies or packaging machine manufacturers, making initial introduction easy.

[0082] Furthermore, in the medication support system 500 according to this embodiment, the facial image of the person who is to take the medicine is digitized, and the name corresponding to the digitized facial data is obtained from the face information database as the image-corresponding name. In this way, the medication support system 500 according to this embodiment can compare the facial image with the digitized data, thereby reducing the processing load.

[0083] Furthermore, in the medication support system 500 according to the present embodiment, authentication of the final recipient is performed by character string comparison, such as comparison of the printed name with the image-corresponding name. Therefore, the medication support system 500 according to the present embodiment can perform accurate authentication without relying on visual inspection, by character string comparison with a low processing load.

[0084] In addition, in the medication support system 500 according to the present embodiment, different messages can be displayed for authentication failure when an image-corresponding name cannot be acquired and authentication failure when a printed name and an image-corresponding name do not match. Therefore, the medication support system 500 according to the present embodiment can realize appropriate medication assistance support corresponding to various cases that may occur in a facility.

[0085] Embodiment 2 In this embodiment, differences from and additions to the first embodiment will be mainly described. In this embodiment, components having the same functions as those in the first embodiment are given the same reference numerals, and the description thereof will be omitted.

[0086] ***Configuration Description*** The configurations of the medication support system 500, medication support device 100, and mobile terminal 200 in this embodiment are the same as those in the first embodiment.

[0087] ***Explanation of Operation*** Next, an operation of the medication support system 500 according to the present embodiment will be described. The operation procedure of the medication support system 500 corresponds to a medication support method. Moreover, a program for realizing the operation of the medication support system 500 corresponds to a medication support program.

[0088] FIG. 17 is a flow diagram showing the operation of medication support system 500 according to this embodiment.

[0089] <Initial Processing: Step S110> Step S110, which is the initial process, is the same as in the first embodiment.

[0090] FIG. 18 is a diagram showing an example of the configuration of the user database 11 and the staff database 12 stored in the storage unit 150 of the medication support device 100 according to this embodiment.

[0091] In addition to the information described in the first embodiment, a face photographing flag 116, a photographing prohibited period 117, and a special note 118 regarding the prohibition of photographing are set in the user database 11. In this embodiment, the face data 115 includes the face image as well as the data obtained by converting the face image into digital form. The face photographing flag 116 is a flag indicating whether or not face photographing of a person who is a user is prohibited. For example, if face photographing is permitted, it is set to "1", if it is permitted under certain conditions, it is set to "9", and if face photographing is prohibited, it is set to "0". The photographing prohibited period 117 indicates a period during which photographing of the face is prohibited. Photographing of the face is prohibited during the set period. In the special notes 118 regarding the prohibition of photography, for example, if face photography is permitted under certain conditions, the conditions are set.

[0092] In the staff database 12, in addition to the information explained in the first embodiment, a staff flag 124 is set. The staff flag 124 is a flag indicating whether or not the staff member is a staff member who will photograph the face of the person who is going to take the medicine. For example, "1" is set for staff who will photograph the face, and "0" is set for staff who will not photograph the face. For example, for staff who are not familiar with operating the mobile terminal 200, "0" is set for staff who will not photograph the face.

[0093] FIG. 19 is a diagram showing an example of the configuration of face information database 31 and staff information database 32 stored in storage unit 260 of mobile terminal 200 according to this embodiment.

[0094] In addition to the information described in the first embodiment, a face photographing flag 313, a photographing prohibited period 314, and a special note 315 regarding the photographing prohibited period are set in the face information database 31. In this embodiment, the face data 312 includes a face image as well as digitalized data of the face image. The face photographing flag 313, the photographing prohibited period 314, and the special note 315 regarding photographing prohibited are similar to the face photographing flag 116, the photographing prohibited period 117, and the special note 118 regarding photographing prohibited, respectively.

[0095] In the staff information database 32, in addition to the information explained in the first embodiment, a staff flag 323 is set. The stuff flag 323 is similar to the stuff flag 124 .

[0096] <Staff authentication process: Step S120> <Drug Package Image Capture Process and Character Extraction Process: Step S130> Step S120, which is the staff authentication process, and step S130, which is the medicine package image photographing process and character extraction process, are the same as those in embodiment 1. In step S130, the photographing unit 220 photographs the medicine package containing the medicine to be taken by the intended user 52, and acquires the character information 33 printed on the medicine package.

[0097] <Facial image display process: step S210> In step S210, the display unit 210 uses the face information database 31 to extract face data 312 corresponding to the text information 33, and displays a face image corresponding to the extracted face data 312 as a terminal display image 64 on the mobile terminal 200. Specifically, the display unit 210 extracts face data 312 from the face information database 31 based on the character information 33 acquired in step S130. The display unit 210 searches the face information database 31 using all items included in the character information 33, and extracts face data 312 corresponding to a user name 311 that matches any of the items included in the character information 33. Then, the display unit 210 acquires a face image included in the extracted face data 312 as a terminal display image 64. The display unit 210 displays a visual confirmation screen 63 used by a medication assistant to visually confirm, on the display screen of the mobile terminal 200. The display unit 210 displays a terminal display image 64 acquired from the character information 33 together with the character information 33 as the visual confirmation screen 63.

[0098] FIG. 20 is a diagram showing an example of a visual confirmation screen 63 according to the present embodiment. On visual confirmation screen 63, the contents of character information 33, which is characters read from medicine packet 51, and terminal display image 64 acquired from face information database 31 based on character information 33 are displayed. The visual confirmation screen 63 also displays a visual OK button 641 and a visual NG button 642 for the medication assistant to input the result of the visual confirmation.

[0099] <Visual confirmation process: Step S220> <Visual confirmation determination process: Step S230> <Facial image capture determination process: step S240> The photographing unit 220 acquires the result of visual confirmation by the medication assistant visually confirming whether or not the terminal display image 64 is the same person as the intended medication recipient 52. If the visual confirmation is successful, that is, if the visual authentication is successful, the photographing unit 220 judges whether or not it is impossible to photograph the face of the intended medication recipient 52. Specifically, the following applies:

[0100] In step S220, the medication assistant visually compares the intended user 52 with the terminal display image 64 displayed on the visual confirmation screen 63, and performs authentication by visual confirmation to determine whether they are the same person. If the result of the visual confirmation is OK, the medication assistant presses the visual OK button 641. If the result of the visual confirmation is not OK, the medication assistant presses the visual NG button 642. If the visual confirmation is OK, that is, if the visual OK button 641 is pressed, the process proceeds to step S240. If the visual confirmation result is not OK, that is, if the visual inspection NG button 642 is pressed, the process proceeds to step S270. In step S270, since there is a high possibility that the intended user 52 is a different person, the medication assistant takes measures such as searching for the correct intended user 52, and repeats the process from visual confirmation (step S220).

[0101] In step S240, the photographing section 220 judges whether or not it is possible to photograph the face of the prospective recipient 52 who has been visually confirmed as OK. If it is determined that face photography is not possible (YES in step S240), the process proceeds to step S260. If it is determined that face photographing is not impossible, that is, if it is determined that face photographing is possible (NO in step S240), the process proceeds to step S140a.

[0102] FIG. 21 is a diagram showing an example of face photographing unacceptable conditions 81 in the face photographing determination process according to the present embodiment. The face photographing prohibition condition 81 is a condition for determining that it is not possible to photograph the face of the prospective patient 52. The process of determining whether or not it is possible to photograph the face based on the face photographing prohibition condition 81 is performed automatically. Below are examples of face photography unacceptable conditions 81. Of the face photographing unacceptable conditions 81, those related to users are stored in the face information database 31, and those related to staff are stored in the staff information database 32. Among the face photographing unacceptable conditions 81, those unacceptable conditions not related to users and staff are stored in the storage unit 260 as face photographing unacceptable condition information including unacceptable text information. The face photographing unacceptable condition information includes, for example, text information for comparison with the text information 33 shown below.

[0103] (1) The photographing unit 220 judges whether or not face photographing is possible based on the time condition. The time during which face photographing is not possible as a system can be set in advance. As shown in Fig. 21, for example, the time period "18:00-6:00" is set as a time period during which face photographing is not possible as a system. (2) The photographing unit 220 refers to the character information 33 acquired in step S130, and determines that face photographing is not possible if the medication timing included in the character information 33 is included in the medication timing stored in the face photographing impossible condition information. For example, assume that the medication timing of "dinner" is acquired from the character information 33 included in the character information 33, and "dinner," "before going to bed," and "before falling asleep" are stored as medication timings stored in the face photographing impossible condition information. In this case, since "dinner" acquired from the character information 33 is included in the medication timings stored in the face photographing impossible condition information, it is determined that face photographing is not possible. (3) A facility flag is provided for each facility to set whether or not face photography is permitted. For example, a flag can be set so that face photography is not permitted according to the facility's policy. The photography unit 220 refers to the facility flag to determine whether or not face photography is permitted. (4) A temporary flag is provided to temporarily disable face photography. For example, a flag can be set to temporarily disable face photography due to a shortage of staff or lack of familiarity with the operation. The photographing unit 220 refers to the temporary flag to determine whether face photography is possible. Since a date and time can be specified in the temporary flag, the photographing unit 220 determines that face photography is not possible if the current time is included in the date or time specified in the temporary flag. (5) The photographing unit 220 refers to the staff flag 323 in the staff information database 32 for the medication assistant. If the staff flag 323 of the medication assistant is "0", which indicates that the staff does not photograph the face, the photographing unit 220 determines that the face cannot be photographed. For example, "0", which indicates that the staff does not photograph the face, is set for staff who are unfamiliar with the operation or have little experience. (6) The photographing unit 220 refers to the face photographing flag 313 of the face information database 31 for the intended recipient. If the face photographing flag 313 of the intended recipient is “0”, indicating that face authentication is not possible, the photographing unit 220 determines that face photographing is not possible. (7) The photographing unit 220 refers to the face photographing flag 313 in the face information database 31 for the intended recipient. If the face photographing flag 313 for the intended recipient is “9”, which indicates conditional face authentication, the photographing unit 220 makes a judgment, for example, as follows. When the photographing unavailable period 314 is set, if the current time is included in the set period, the photographing unit 220 determines that face photographing is unavailable. For example, the photographing unavailable period 314 is set due to poor health, injury, or the like. If the special notes 315 regarding the impossibility of photographing include an event that makes face authentication impossible, the photographing unit 220 determines that face photographing is impossible. For example, if there are special notes such as camera phobia, paralysis, facial stiffness, or a high level of dementia, the photographing unit 220 determines that face photographing is impossible. Note that the special notes 315 in Fig. 19 and the face photographing impossible condition 81 in (7) in Fig. 21 describe the conditions under which face photographing is impossible in sentences, but in reality, the conditions are stored as a flag indicating that face photographing is impossible so that the photographing unit 220 can determine that face photographing is impossible. Also, when "9" is specified, indicating that face recognition is conditional, the photographing unit 220 may be configured to display in text the conditions under which face photography is not permitted, and to accept input from the staff as to whether or not to photograph.

[0104] The photographing unit 220 determines that face photographing is not permitted if any one of the face photographing prohibition conditions 81 from (1) to (7) above is met. In addition, the face photographing prohibition conditions 81 from (1) to (7) above may be prioritized. For example, the priority may be set as follows: facility flag ((3), (4)) → time ((1)) → text ((2)) → user ((6), (7)) → staff ((5)). Furthermore, the face photography prohibition condition 81 in FIG. 21 is one example, and other conditions may be added, or conditions may be deleted.

[0105] On the other hand, if it is determined that face photographing is possible, the process proceeds to face image photographing processing (step S140a). However, in the face image photographing processing (step S140a) according to the present embodiment, face photographing may be skipped depending on a predetermined condition.

[0106] <Facial image capture process: step S140a> In step S140a, the photographing section 220 displays, on the mobile terminal 200, a scheduled person photographing start screen 62 for photographing the face of the scheduled person 52. Then, the photographing section 220 determines whether or not to skip photographing the face. In this embodiment, face photography is skipped under a predetermined condition. The skip condition for skipping face photography is a condition for skipping face photography when it is difficult to photograph the face for some reason. The process of determining whether or not to skip face photography is performed automatically. Alternatively, the medication assistant can skip face photography by pressing a skip button.

[0107] FIG. 22 is a diagram showing an example of a scheduled patient imaging start screen 62 according to the present embodiment. The scheduled person image capture start screen 62 according to this embodiment displays a skip button 625 for instructing to skip face image capture, in addition to the configuration described in the embodiment 1. The scheduled person image capture start screen 62 also includes a face authentication button 626 for instructing to start face image capture. The photographing unit 220 determines to skip face photographing when the skip button 625 is pressed. Then, the photographing unit 220 displays a skip notification on the scheduled person photographing start screen 62 to the effect that face photographing will be skipped due to the skip button 625 being pressed.

[0108] FIG. 23 is a diagram showing an example of skip conditions 82 for skipping face photographing according to the present embodiment. (8) When the same medication assistant fails to capture a face image a predetermined number of times in succession, the photographing unit 220 determines to skip photographing the face. (9) When a medication assistant who has failed to capture a face photograph a predetermined number of times in succession is changed and facial photographing is resumed, if multiple medication assistants who have failed to capture a face photograph a predetermined number of times in succession fail consecutively, the photographing unit 220 determines to skip capturing the face photograph. (10) The photographing unit 220 determines to skip face photographing if the face authentication button 626 is not pressed for a certain period of time after the scheduled person photographing start screen 62 is displayed. Then, the display unit 210 displays a skip notification on the scheduled person photographing start screen 62 to the effect that face photographing will be skipped because the face authentication button 626 is not pressed for a certain period of time. Alternatively, the photographing unit 220 does not operate face authentication processing when it recognizes something other than a human or the person does not fit within the frame. Then, the photographing unit 220 determines to skip face photographing when a certain period of time has passed during which the face authentication button 626 cannot be pressed because the face authentication processing is not operating. (11) The photographing unit 220 determines to skip photographing the face when the skip button 625 is pressed. If the medication assistant determines that photographing the face is difficult, the medication assistant presses the skip button 625 to skip photographing the face. For example, when displaying the scheduled patient photographing start screen 62, if a special note is set in the special notes 315 regarding not being able to photograph the scheduled patient 52, the display unit 210 may display the contents of the special note on the scheduled patient photographing start screen 62. The medication assistant checks the displayed special note, and if it is determined that it is not possible to photograph the face image of the scheduled patient 52, it presses the skip button 625. Alternatively, even if no special note is displayed, if it is determined that it is not possible to photograph the face image of the scheduled patient 52 based on the state of the scheduled patient 52, the assistant presses the skip button 625.

[0109] In step S250, photographing section 220 determines whether or not face photographing has been skipped in the face image photographing process (step S140a). If it is determined that face photographing has been skipped (YES in step S250), the process proceeds to step S260. At this time, the photographing section 220 may display a skip notification on the scheduled person photographing start screen 62 to the effect that face photographing has been skipped. If it is determined that face photography has not been skipped (NO in step S250), the process proceeds to step S150.

[0110] The processes from step S150 to step S170 are the same as those in embodiment 1. That is, authentication unit 240 performs face authentication using the name corresponding to the face image obtained by photographing the face in step S140a and character information 33 acquired in step S130.

[0111] If it is determined in step S240 that face photography is not possible or that face photography is to be skipped in step S250, visual confirmation result display processing and registration processing are performed in step S260. If it is determined that face photography is not possible or that face photography is to be skipped, the display unit 210 displays the visual confirmation result, and the authentication unit 240 registers the visual confirmation result in the medication history database 13.

[0112] <Visual inspection result display process and visual inspection result registration process: S260> In step S260, the display unit 210 displays a visual authentication success display indicating that the authentication was successful by visual confirmation. Also, the authentication unit 240 registers in the medication history database 13 the content of "OK2" indicating that face photography was not possible or face photography was skipped but authentication was successful by visual confirmation.

[0113] In this embodiment, the types set in the authentication result include OK2 in addition to OK, NG1, and NG2 described in the first embodiment. OK2 indicates that face capture was not possible or face capture was skipped, but identity authentication was successful through visual confirmation.

[0114] Fig. 24 is a diagram showing an example of the configuration of medication history database 13 according to this embodiment. Note that in Fig. 24, the description of the medicine package read characters is omitted. In the medication history database 13, the reason why face photography was not possible or face photography was skipped is set in the record of OK2. In Figure 24, the authentication result for the printed name "Taro Ogawa" is "OK2." The special notes state, "Although the user does not permit face capture, identity authentication was successful through visual confirmation." This indicates that although "Taro Ogawa" is a "user who does not permit face capture" with the face capture flag set to "0," identity authentication through visual confirmation was successful.

[0115] FIG. 25 is a diagram showing an example of visual authentication success display 24 according to this embodiment. The display unit 210 displays visual authentication success display 24, which indicates that authentication was successful through visual confirmation. This is the screen display in the case of "OK2", which indicates that facial image capture was not possible or facial image capture was skipped, but authentication was successful through visual confirmation. When this visual authentication success display 24 is displayed on the mobile terminal 200, the medication supporter assumes that personal authentication has been completed and starts medication support. Although the visual authentication success display 24 in FIG. 25 shows the OK2 state, the medication assistance memo shown in FIG. 24 may be configured to accept and store special notes input from the staff after the display in FIG. 25 is displayed.

[0116] ***Explanation of the Effects of the Present Embodiment*** The medication support system according to the present embodiment can automatically determine whether or not facial photography is possible, based on a double check of visual confirmation by a medication assistant and face authentication using a face image obtained by photographing the person who is to take a medication. Therefore, the medication support system according to the present embodiment can reduce the workload of the medication assistant in authenticating the identity of the person who is to take a medication.

[0117] In the above first and second embodiments, each unit of each device of the medication support system 500 has been described as an independent functional block. However, the configuration of each device of the medication support system 500 does not have to be the same as that of the above-mentioned embodiments. For example, the character extraction process performed by the medication support device 100 may be performed by the mobile terminal 200. The functional blocks of each device of the medication support system 500 may have any configuration as long as they can realize the functions described in the above-mentioned embodiments. In addition, it is acceptable to combine multiple parts of the first and second embodiments. Alternatively, it is acceptable to implement only one part of these embodiments. In addition, it is acceptable to combine these embodiments in any way, either as a whole or in part. That is, in the first and second embodiments, the respective embodiments can be freely combined, or any of the components in each embodiment can be modified, or any of the components in each embodiment can be omitted.

[0118] The above-described embodiment is essentially a preferred example, and is not intended to limit the scope of the present disclosure, the scope of application of the present disclosure, and the scope of use of the present disclosure. The above-described embodiment can be modified in various ways as necessary. For example, the procedure described using a flow chart or sequence chart may be modified as appropriate.

[0119] Various aspects of the present disclosure are summarized below as appendices.

[0120] (Appendix 1) A medication support system including a mobile terminal used by a medication assistant who performs medication assistance, the system supporting the medication assistant, The mobile terminal includes: an imaging unit that captures an image of a medicine packet containing a medicine to be taken by a person who is to take the medicine, and acquires character information printed on the medicine packet; a display unit that extracts face data corresponding to the character information using a face information database in which face data representing a face image of a person who is a subject of medication assistance and the name of the person are set, and displays the face image corresponding to the extracted face data on a display device as a terminal display image; Equipped with The imaging unit includes: a result of visual confirmation by the medication assistant ascertaining whether the terminal display image is the same person as the intended user, and if the authentication is successful by the visual confirmation, determining whether or not it is impossible to photograph the face of the intended user; The display unit is If it is determined that the face image cannot be captured, a visual authentication success display is displayed to indicate that the authentication has been successful through the visual confirmation; The mobile terminal further includes: A medication support system including an authentication unit that, when it is determined that the face image can be photographed, performs face authentication using the name corresponding to the face image obtained by photographing the face and the text information. (Appendix 2) A face photographing flag indicating whether or not the face photographing of the person is prohibited is set in the face information database, The imaging unit includes: The medication support system according to claim 1, which determines whether or not face photography is possible using the face photography flag set in the face information database. (Appendix 3) The imaging unit includes: A medication support system as described in Appendix 1 or Appendix 2, which uses a staff information database in which a staff ID identifying a staff member providing medication assistance and a staff flag indicating whether the staff member is the staff member who will perform the facial photographing is set, and determines whether the facial photographing is not possible based on the staff flag corresponding to the medication assistant. (Appendix 4) In the face information database, a photographing prohibited period indicating a period during which photographing of the face of the person is prohibited is set, The imaging unit includes: A medication support system according to any one of claims 1 to 3, which determines that face photography is not possible if the current time is within the period during which photography is not possible. (Appendix 5) The imaging unit includes: The medication support system according to any one of Supplementary Note 1 to Supplementary Note 4, wherein face photographing is impossible or not is determined by comparing the character information with the impossible character information using face photographing impossible condition information in which impossible character information is set to indicate that face photographing is impossible. (Appendix 6) The display unit is When it is determined that the facial image can be photographed, a screen for starting photographing of the intended person is displayed on a display device, The imaging unit includes: A medication support system as described in any one of Supplementary Note 1 to Supplementary Note 5, which determines whether or not to skip the face capture, and if it is determined that the face capture is to be skipped, displays a skip notification indicating that the face capture will be skipped on the scheduled patient capture start screen. (Appendix 7) the scheduled person photographing start screen includes a skip button for instructing to skip photographing the face of the scheduled person; The imaging unit includes: A medication support system as described in Appendix 6, which determines that the face capture will be skipped when the skip button is pressed, and displays a skip notification on the scheduled patient capture start screen indicating that the face capture will be skipped due to the skip button being pressed. (Appendix 8) the scheduled person photographing start screen includes a face recognition button for instructing the start of the face photographing, The imaging unit includes: The medication support system according to claim 6 or 7, wherein if the face recognition button is not pressed for a certain period of time after the scheduled patient photographing start screen is displayed, it is determined that the face photographing is to be skipped, and a skip notification indicating that the face photographing is to be skipped because the face recognition button has not been pressed for the certain period of time is displayed on the scheduled patient photographing start screen. (Appendix 9) A medication support method for use in a medication support system that supports a medication assistant, the medication support system comprising: The mobile terminal photographs a medicine package containing a medicine to be taken by the intended recipient and acquires text information printed on the medicine package; the portable terminal extracts face data corresponding to the character information using a face information database in which face data representing a face image of a person who is a subject of medication assistance and the name of the person are set, and displays the face image corresponding to the extracted face data on a display device as a terminal display image; The mobile terminal acquires a visual confirmation result in which the medication assistant visually authenticates whether or not the terminal display image is the same person as the intended user, and if the authentication is successful by the visual confirmation, determines whether or not it is impossible to photograph the face of the intended user; When it is determined that the face image cannot be captured, the portable terminal displays a visual authentication success display indicating that the authentication has been successful through the visual confirmation; A medication support method in which, when it is determined that the mobile terminal is capable of photographing the face, face authentication is performed using the name corresponding to the face image obtained by photographing the face and the text information. (Appendix 10) A medication support program for use in a medication support system that supports a medication assistant, the medication support program comprising: An imaging process of imaging a medicine packet containing a medicine to be taken by a person to take the medicine and acquiring text information printed on the medicine packet; a face image display process for extracting face data corresponding to the character information using a face information database in which face data representing a face image of a person who is a subject of medication assistance and the name of the person are set, and displaying the face image corresponding to the extracted face data on a display device as a terminal display image; a face photographing determination process in which the medication assistant visually determines whether the terminal display image is the same person as the intended user, and if the authentication is successful, determines whether or not the face of the intended user can be photographed; a visual confirmation result display process for displaying a visual authentication success display indicating that authentication has been successful by the visual confirmation when it is determined that the face photographing is not possible; When it is determined that the face image can be photographed, an authentication process is performed to perform face authentication using the name corresponding to the face image obtained by photographing the face and the character information. A medication support program that executes the above on a computer. [Explanation of symbols]

[0121] 11 User database, 111,121,341 Facility ID, 112 User ID, 113,311 User name, 114 Furigana, 115,312 Face data, 116,313 Face photography flag, 117,314 Photography prohibited period, 118,315 Special notes regarding photography prohibited, 12 Staff database, 122,321,346 Staff ID, 123,322 Name, 124,323 Staff flag, 13 Medication history database, 31 Face information database, 32 Staff information database, 33 Character information, 34 Authentication information, 342 Face authentication date and time, 343 Authentication result, 344 Image-compatible name, 345 Printed name, 347 Medicine package read characters, 348 Medication assistance note, 35 Name identification result, 51 Medicine package, 52 Scheduled user, 53 Image-compatible name, 54 Printed name, 61 Medicine package image capture start screen, 62 Scheduled user image capture start screen, 625 Skip button, 626 Face recognition button, 63 Visual confirmation screen, 64 Terminal display image, 641 Visual OK button, 642 Visual NG button, 81 Face image capture not possible condition, 82 Skip condition, 100 Medication support device, 110 Character extraction unit, 120 Information configuration unit, 150, 260 Memory unit, 21 Authentication success display, 211 Authentication success message, 22 Authentication failure display, 221 Face recognition failure message, 222, 232 Medicine package image, 223, 233 Registration button, 224, 234 Retake button, 231 Mismatch message, 24 Visual authentication success display, 200 Portable terminal, 210 Display unit, 220 Image capture unit, 230 Corresponding name acquisition unit, 240 authentication unit, 300 network, 500 medication support system, 910 processor, 921 memory, 922 auxiliary storage device, 930 input interface, 940 output interface, 941 display device, 950 communication device, 960 imaging device.

Claims

1. A medication support system comprising a mobile terminal used by a medication assistant who performs medication assistance, the system supporting the medication assistant, The mobile terminal includes: an imaging unit that captures an image of a medicine packet containing a medicine to be taken by a person who is to take the medicine, and acquires character information printed on the medicine packet; a display unit that extracts face data corresponding to the character information using a face information database in which face data representing a face image of a person who is a subject of medication assistance and the name of the person are set, and displays the face image corresponding to the extracted face data on a display device as a terminal display image; Equipped with The imaging unit includes: a result of visual confirmation by the medication assistant ascertaining whether the terminal display image is the same person as the intended user, and if the authentication is successful by the visual confirmation, determining whether or not it is impossible to photograph the face of the intended user; The display unit is If it is determined that the face image cannot be captured, a visual authentication success display is displayed to indicate that the authentication has been successful through the visual confirmation; The mobile terminal further includes: A medication support system including an authentication unit that, when it is determined that the face image can be photographed, performs face authentication using the name corresponding to the face image obtained by photographing the face and the text information.

2. A face photographing flag indicating whether or not the face photographing of the person is prohibited is set in the face information database, The imaging unit includes: The medication support system according to claim 1, wherein the face photographing flag set in the face information database is used to determine whether or not the face photographing is not possible.

3. The imaging unit includes: The medication support system of claim 1, further comprising a staff information database in which a staff ID identifying the staff member providing medication assistance and a staff flag indicating whether the staff member is the staff member who will be performing the facial photographing is set, and which determines whether or not the facial photographing is not possible based on the staff flag corresponding to the medication assistant.

4. In the face information database, a photographing prohibited period indicating a period during which photographing of the face of the person is prohibited is set, The imaging unit includes: The medication support system according to claim 1 , wherein it is determined that face photography is not possible if the current time is within the period during which photography is not possible.

5. The imaging unit includes: The medication support system of claim 1, further comprising: face photographing unavailable condition information, which sets unavailable character information indicating that face photographing is not possible; and a comparison between the character information and the unavailable character information, which is used to determine whether or not face photographing is not possible.

6. The display unit is When it is determined that the facial image can be photographed, a screen for starting photographing of the intended person is displayed on a display device, The imaging unit includes: The medication support system according to claim 1, further comprising: a determination unit that determines whether or not to skip the face photographing; and, if it determines that the face photographing is to be skipped, a skip notification indicating that the face photographing is to be skipped is displayed on the scheduled patient photographing start screen.

7. the scheduled person photographing start screen includes a skip button for instructing to skip photographing the face of the scheduled person; The imaging unit includes: The medication support system of claim 6, wherein when the skip button is pressed, it is determined that the facial photographing is to be skipped, and a skip notification indicating that the facial photographing is to be skipped due to the pressing of the skip button is displayed on the scheduled patient photographing start screen.

8. the scheduled person photographing start screen includes a face recognition button for instructing the start of the face photographing, The imaging unit includes: The medication support system according to claim 6, wherein if the face recognition button is not pressed for a certain period of time after the scheduled patient photographing start screen is displayed, it is determined that the face photographing is to be skipped, and a skip notification indicating that the face photographing is to be skipped due to the face recognition button not being pressed for the certain period of time is displayed on the scheduled patient photographing start screen.

9. A medication support method for use in a medication support system that supports a medication assistant, the medication support system comprising: The mobile terminal photographs a medicine package containing a medicine to be taken by the intended recipient and acquires text information printed on the medicine package, the portable terminal extracts face data corresponding to the character information using a face information database in which face data representing a face image of a person who is a subject of medication assistance and the name of the person are set, and displays the face image corresponding to the extracted face data on a display device as a terminal display image; The mobile terminal acquires a visual confirmation result in which the medication assistant visually authenticates whether or not the terminal display image is the same person as the intended user, and if the authentication is successful by the visual confirmation, determines whether or not it is impossible to photograph the face of the intended user; When it is determined that the face image cannot be captured, the portable terminal displays a visual authentication success display indicating that the authentication has been successful through the visual confirmation; A medication support method in which, when it is determined that the mobile terminal is capable of photographing the face, face authentication is performed using the name corresponding to the face image obtained by photographing the face and the text information.

10. A medication support program for use in a medication support system that supports a medication assistant, the medication support program comprising: An imaging process of imaging a medicine packet containing a medicine to be taken by a person to take the medicine and acquiring text information printed on the medicine packet; a face image display process for extracting face data corresponding to the character information using a face information database in which face data representing a face image of a person who is a subject of medication assistance and the name of the person are set, and displaying the face image corresponding to the extracted face data on a display device as a terminal display image; a face photographing determination process in which the medication assistant visually determines whether the terminal display image is the same person as the intended user, and if the authentication is successful, determines whether or not the face of the intended user can be photographed; a visual confirmation result display process for displaying a visual authentication success display indicating that authentication has been successful by the visual confirmation when it is determined that the face photographing is not possible; When it is determined that the face photographing is possible, an authentication process is performed to perform face authentication using the name corresponding to the face image obtained by the face photographing and the character information. A medication support program that executes the above on a computer.

Images