A medication guidance system, a medication guidance method, a computer-readable recording medium containing a program, and a medication guidance device.

Abstract

The system provides medication guidance that allows patients to select and partially view videos of medication guidance they received from pharmacists at pharmacies, etc., using a terminal device operated by the patient. These selections should include specific items (e.g., dosage, timing of administration, warning signs, when to seek medical attention, etc.). [Solution] The server computer 105 generates tag information for each classified text data, which is created by classifying the audio text data obtained from the video data of medication guidance according to the content of the medication guidance. This tag information includes time information within the video and indicates the content of the guidance. When a patient obtains access information with the patient terminal device 201, multiple tag information linked to the video data indicated by that information is displayed on the terminal. When the patient taps the desired tag information, the video data for the corresponding section is played.

Description

Technical Field

[0001] The present invention relates to a system for medication guidance in pharmacies and the like, a medication guidance device, a medication guidance method, and a computer-readable recording medium recording a program.

Background Art

[0002] When a patient who has received a medical examination at a hospital or the like receives the medicine prescribed on a prescription at a pharmacy or the like, a dispensing slip and a medicine information provision slip are handed over to the patient by a pharmacist. The medicine information provision slip describes the name of the medicine, a photograph of the medicine, efficacy, etc., and the dispensing slip has the number of doses written on it. When the patient receives the medicine, it is common for the pharmacist to explain each medicine handed over one by one so that the patient can thoroughly confirm the contents shown in the medicine information provision slip and have no uneasiness or unclear points.

[0003] Conventionally, there is known a medication guidance support system having a function of generating a video in which the content displayed on a portable terminal at the time of medication guidance and the conversation between a pharmacist and a patient are displayed for viewing by patients or the like (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, as in the prior art, simply providing a video for viewing by patients or the like has a problem that it is difficult for patients to accurately determine how to take the medicine.

[0006] Therefore, the present invention aims to provide a medication guidance system, a medication guidance method, and a computer-readable recording medium that records a program, which allows patients to select and partially view videos of medication guidance received from a pharmacist at a pharmacy or the like using a terminal device operated by the patient, focusing on desired items (e.g., dosage, timing of administration, warning signs, when to seek medical attention, etc.). [Means for solving the problem]

[0007] An example of a medication guidance system is: A recording device for capturing a video of a pharmacist giving medication guidance to a patient, including audio. 、 Control processor of pharmacy terminal device, The control processor of the server computer, The system comprises a terminal processor for a patient terminal device operated by the patient, The control processor of the aforementioned pharmacy terminal device is A recording process that records video data corresponding to the aforementioned recorded medication guidance video. Put the principles into practice. The control processor of the server computer is By performing speech recognition processing on the aforementioned video data, a speech recognition process outputs speech-to-text data corresponding to the aforementioned speech. Reason and , A tag generation process generates tag information indicating the content of the instruction, which includes time information from the beginning of the medication instruction video, for each classified text data obtained by classifying the aforementioned audio text data according to the content of the medication instruction. Reason and , A video storage unit stores the aforementioned video data and a plurality of tag information corresponding to the video data. Reason and , The system generates access information for accessing the video data in the video storage unit and the tag information corresponding to the video data, and the patient operates the system. The aforementioned Access information generation process that outputs information so that it can be read by the patient terminal device. Based on reason, and in action, The terminal processor of the aforementioned patient terminal device is Access information acquisition process that acquires the aforementioned access information Reason and , Tag display process for reading a plurality of the tag information in the video storage unit according to the obtained access information and displaying it on the patient terminal device Reason and , Video playback process for reading the video data within the time range corresponding to the time information included in the tag information selected by the patient on the patient terminal device from the video storage unit according to the obtained access information and playing it back on the patient terminal device Based on reason, and in action, The tag generation process involves causing the generative artificial intelligence system to input the speech text data and, for each of the multiple tag information items, to prompt the generative artificial intelligence system to perform classification from the speech text data into multiple classification text data items, and to output the time information, tag name, metadata, generation justification statement, and confidence level for each classification text data item as the tag information corresponding to the classification text data item. The access information generation process generates two-dimensional code data containing the access information, a sheet of paper with a two-dimensional code corresponding to the two-dimensional code data is printed by a printer, the access information acquisition process acquires the access information by reading the two-dimensional code written on the sheet of paper based on the patient's instructions, or the access information generation process includes a patient database that stores, in pairs, patient identification information corresponding to the patient's personal information and account information corresponding to the patient's account in the messaging system, the pharmacist acquires the account information corresponding to the patient identification information corresponding to the patient's personal information written on the prescription created by the pharmacist from the patient database, posts the access information to the patient's account in the messaging system corresponding to the acquired account information, and the access information acquisition process acquires the access information posted to the patient's account from the messaging system based on the patient's instructions. . [Effect of the Invention]

[0008] <00> According to the present invention, it is possible to provide a medication guidance system, a medication guidance method, and a computer-readable recording medium recording a program, which enable a patient to select desired items on a terminal device operated by the patient and partially view a video of medication guidance received from a pharmacist at a pharmacy or the like. [Brief Description of the Drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing a video shooting process in an embodiment of a medication guidance system. [Figure 2] FIG. 2 is a block diagram showing a video playback process in an embodiment of a medication guidance system. [Figure 3] FIG. 3 is a functional block diagram of an embodiment of a medication guidance system. [Figure 4] FIG. 4 is a diagram (Part 1) showing an example of a prompt processed by a tag generation processing unit. [Figure 5] FIG. 5 is a diagram (Part 2) showing an example of a prompt processed by a tag generation processing unit. [Figure 6] FIG. 6 is a diagram showing an example of the data configuration of a patient database. [Figure 7] FIG. 7 is a diagram showing an example of the display of a tag confirmation / modification screen. [Figure 8] FIG. 8 is a diagram showing an example of the data configuration of a video database. [Figure 9] Figure 9 shows an example of the tag display screen. [Figure 10] Figure 10 is a block diagram showing an example of a hardware configuration for a server computer, pharmacy terminal device, or patient terminal device. [Figure 11] Figure 11 is a flowchart showing an example of pharmacy terminal control processing in an embodiment. [Figure 12] Figure 12 is a flowchart showing an example of server control processing in an embodiment. [Figure 13] Figure 13 is a flowchart showing an example of patient terminal control processing in an embodiment. [Figure 14] Figure 14 is a diagram and explanatory figure showing an embodiment of a tabletop video recording device. [Modes for carrying out the invention]

[0010] Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. An embodiment of the medication guidance system according to the present invention allows for the recording of a pharmacist's medication guidance to a patient at a pharmacy or the like (for example, the process from when the pharmacist explains the details of the drug, how to take it, how to use it, precautions, etc. to the patient, until the drug is provided), and also allows the patient to easily view videos focused on individual items of medication guidance, such as "dosage and administration," "timing of administration," and "warning signs and when to seek medical attention," from a terminal device operated by the patient.

[0011] The above embodiment will be described in detail below. Figure 1 is a block diagram showing the video recording process in the embodiment of the medication guidance system, and Figure 2 is a block diagram showing the video playback process in the embodiment of the medication guidance system. The embodiment of the medication guidance system is configured as the medication guidance system 100 shown in Figures 1 and 2.

[0012] As shown in Figure 1, this medication guidance system 100 comprises a webcam 103 which is a camera, a pharmacy terminal device 101, a printer 102, a pharmacy LAN 104 which connects these via a wired or wireless local area network (LAN), an internet 107 connected to the pharmacy LAN 104, a server computer 105 equipped with a video database 106 which is a video storage unit accessible from the internet 107, and an external server computer 120. Note that the server computer 105, video database 106, and external server computer 120 may be a cloud server, cloud database, and external cloud server, respectively.

[0013] Furthermore, as shown in Figure 2, the medication guidance system 100 includes a configuration in which a patient terminal device 201 operated by the patient 111 communicates with a server computer 105 and a video database 106 connected to it via the internet 107.

[0014] In the medication guidance system 100 shown in Figure 1, the recording device, a webcam 103, records a video of the pharmacist giving medication guidance to the patient, including the audio obtained from the recording. The pharmacy terminal device 101 is, for example, a tablet device operated by the pharmacist. Furthermore, the patient terminal device 201 is, for example, a smartphone operated by the patient 111. Note that the webcam 103 is not limited to a desktop camera; for example, an in-camera built into the pharmacy terminal device 101 may also be used.

[0015] Figure 3 is a functional block diagram of the medication guidance system 100 for controlling the recording of the above-mentioned medication guidance video.

[0016] The recording processing unit 301 in Figure 3 is executed, for example, by the control processor of the pharmacy terminal device 101 in Figure 1. The recording processing unit 301 records video data corresponding to the medication guidance video captured by the webcam 103 in Figure 1.

[0017] More specifically, as shown in Figure 1, the recording processing unit 301 transmits the video data received by the pharmacy terminal device 101 from the web camera 103 via the pharmacy LAN 104 to the server computer 105 via the pharmacy LAN 104 and the internet 107, along with prescription information, either sequentially or in a batch after recording is complete. Although not specifically shown in Figure 3, some functions of the recording processing unit 301 may be executed by the control processor of the server computer 105. In this case, some functions of the recording processing unit 301 on the server computer 105 side receive the prescription information and video data transmitted from the recording processing unit 301 in the pharmacy terminal device 101 sequentially or in a batch, and then store them in the storage device of the server computer 105.

[0018] The speech recognition processing unit 302 in Figure 3 is executed, for example, by the control processor of the server computer 105. The speech recognition processing unit 302 outputs speech text data corresponding to the speech data by performing speech recognition processing on video data received, for example, from the pharmacy terminal device 101 in Figure 1.

[0019] Specifically, the speech recognition processing unit 302 uses a machine learning model (such as a deep neural network) that outputs speech-to-text data from speech data. This machine learning model is a deep neural network that has been trained using training speech data and training speech-to-text data obtained in advance by manually recognizing that training speech data. It outputs speech-to-text data by inputting speech data contained in video data in a digital multimedia format, such as MP4, received from the pharmacy terminal device 101. In a more specific embodiment, this machine learning model may be used in combination with a large language model (LLM).

[0020] This LLM may be implemented, for example, within a server computer 105 as a software program executed by its control processor, as a dedicated LLM processor, or as a hybrid configuration of a software program and a dedicated LLM processor.

[0021] Alternatively, this LLM may be implemented as a speech recognition service provided on an external server computer 120 connected via the Internet 107 in Figure 1, as a software program executed by its control processor, as a dedicated LLM processor, or as a hybrid configuration of a software program and a dedicated LLM processor. In this case, some functions of the speech recognition processing unit 302 implemented in the server computer 105 use an API (Application Program Interface) provided by the speech recognition service of the external server computer 120 to send speech data contained in the video data received from the pharmacy terminal device 101 to the external server computer 120, and as a result receive and output speech text data returned from the external server computer 120 by the API.

[0022] The speech recognition processing unit 302 may, for example, together with the above-mentioned LLM, include a control vocabulary pack (medical term dictionary) of specialized medical and pharmaceutical terms for the purpose of normalizing variations in notation, ensuring recall of candidate extraction, and referencing boundary / priority judgment terms in the inference of speech-to-text data from speech data. This dictionary may include, for example, the following structure. [Definition] "Tag Information": Classification information (including text, icons, audio, etc.) corresponding to the content of medication guidance, such as safety, adherence, and lifestyle guidance. "Important Tags": Tags that satisfy category priority or confidence level ≥ a specified value. "Viewing History": A log including whether or not a video was played, the time, the playback section, etc. "Token": A unique proxy identifier that allows access to a means of contact without storing a personal identifier.

[0023] The tag generation processing unit 303 in Figure 3 is executed, for example, by the control processor of the server computer 105. The tag generation processing unit 303 generates tag information that indicates the content of the medication guidance, including time information (interval information) from the beginning of the medication guidance video, for each classified text data obtained by classifying the speech text data output by the speech recognition processing unit 302 according to the content of the medication guidance.

[0024] More specifically, the tag generation processing unit 303, for example, causes the generative artificial intelligence (hereinafter referred to as "generative AI") system to input the speech text data output by the speech recognition processing unit 302, and prompts the generative AI system to output at least the definition of the tag information, the criteria for determining the tag information, the metadata to be generated as the tag information, an instruction to output the statement justifying the generation of the tag information, and an instruction to output the confidence level of the generation of the tag information, thereby causing the generative AI system to classify the speech text data into multiple classification text data, and to output the time information, tag name, metadata, statement justifying the generation, and confidence level for each classification text data as tag information corresponding to that classification text data.

[0025] Figure 4 shows examples of the above prompts for each of the multiple tag information items. Figure 4 illustrates three tag information items: T01, T02, and T05.

[0026] In Figure 4, for example, information regarding the tag name, definition, determination, meta, and co-inscription is provided to the generating AI system as a prompt for tag information T01. In this case, the tag name will be given information indicating that it is "Dosage and Administration". By definition, this tag information is given as indicating that it relates to "basic instructions such as frequency, dosage, and duration of administration." For the determination, instructions are given that for each sentence in the given text data, if the sentence contains a numerical value + unit (mg / tablet / mL / dose / day, etc.) + frequency within the same interval, then this tag information T01 should be assigned to that interval. For example, the interval in question may be defined as the start and end times of the classified text data corresponding to that interval, which are given as the elapsed time from the beginning of the speech text data output by the speech recognition processing unit 302. The instructions specify that numerical values ​​related to dose_num, unit, freq (number of doses), and days (duration) should be extracted from the audio-text data and output as metadata. Furthermore, instructions are given to simultaneously output the reason for generating the evidence text (evidence_text) and the confidence level (percentage) of the tag information.

[0027] In Figure 4, for example, information regarding the tag name, definition, determination, priority, meta, and co-inscription is provided to the generating AI system as a prompt for tag information T02. In this case, the tag name will be given the information that it is "timing of administration". By definition, this tag information is given to indicate that it relates to the timing of administration, using tense words such as "before meals / after meals / before bedtime / as needed / upon waking." For the determination, instructions are given to assign the tag information T02 to each sentence segment within the given text data if a tense word appears in that sentence segment. As a priority, for each sentence in the given text data, if a number + unit appears primarily in that sentence, an instruction is given to prioritize tag information T01 in that section. As a meta-instruction, the system is instructed to extract and output numerical values ​​related to timing (enum type) (dosage timing) from the audio-text data. Furthermore, instructions are given to output the same generation reason and generation confidence level as in the case of tag information T01.

[0028] In Figure 4, for example, information regarding the tag name, definition, determination, meta, and co-inscription is provided to the generating AI system as a prompt for tag information T05. In this case, the tag name will be provided with information indicating that it is a "danger sign / recommendation for medical consultation (safety tag)". By definition, this tag information is provided to indicate that it relates to "signs of serious adverse reactions and criteria for seeking medical attention." For the determination, instructions are given to assign the tag information T05 to each section of text within the given text data if the text contains phrases such as "immediately / emergency / severe / difficulty breathing / widespread rash". As a meta instruction, the system is instructed to output information indicating severity=high and urge=true, or numerical values ​​representing each of these, to show that tag verification and correction processing, as described later, is required. Furthermore, instructions are given to output the same generation reason and generation confidence level as in the case of tag information T01.

[0029] Figure 4 illustrates three types of tag information, but in an actual medication guidance system 100, more tag information (for example, 15 types from T01 to T15) will be provided as prompts similar to those shown in Figure 4.

[0030] In addition to the individual prompts for each tag information example shown in Figure 4, the generating AI system may be further given prompts specifying boundary, priority, and exclusion rules. Figure 5 shows an example of prompts specifying such rules. For example, prompts such as "Prioritize tag information T05 for sentences containing medical consultation terms" or "Use tag information T01 for numerical values ​​with units, and tag information T02 for tense words only" may be provided independently of each tag information.

[0031] Here, the above-described generation AI system may be implemented as, for example, a generation AI service provided on an external server computer 120 connected via the Internet 107 in Figure 1, as a software program executed by its control processor, as a dedicated generation AI processor, or as a hybrid configuration of a software program and a dedicated generation AI processor. In this case, some functions of the tag generation processing unit 303 implemented in the server computer 105 use an API provided by the generation AI service of the external server computer 120 to send the speech text data output by the speech recognition processing unit 302 and a set of prompts similar to those shown in the example in Figure 4 or Figure 5 to the external server computer 120. As a result, the tag information returned from the external server computer 120 via the API is received and generated.

[0032] On the other hand, the tag generation processing unit 303 may be an LLM (Long Life Machine) that has been trained using teacher classification text data, which is pre-classified by human intervention according to the content of medication guidance, and teacher tag information pre-assigned by human intervention to the teacher classification text data, and then inputs the speech text data output by the speech recognition processing unit 302 to output tag information.

[0033] In Figure 3, the control processor of the pharmacy terminal device 101 may execute the tag verification and correction processing unit 309. The tag verification and correction processing unit 309 causes the pharmacist 110 in the pharmacy terminal device 101 to verify and correct the tag information generated by the tag generation processing unit 303 executed on the server computer 105.

[0034] Figure 7 shows an example of the display of the tag confirmation / correction screen 701, which is displayed on, for example, the touch panel display of the pharmacy terminal device 101. As exemplified by 701 in Figure 7, the tag verification and correction processing unit 309 displays, for each piece of tag information, classification text data including the statement justifying the creation of the tag information, the confidence level of the creation of the tag information, and the preceding and succeeding audio text data corresponding to the tag information, on, for example, the touch panel display of the pharmacy terminal device 101. As shown in the example display of the tag confirmation / correction screen 701, the pharmacist 110 confirms and corrects the tag information for each tag information T01, T05, etc., based on the generated justification statement, the confidence level of generation, and the classification text data including the preceding and succeeding audio text data corresponding to that tag information, which are displayed on the touch panel display as part of the tag information.

[0035] More specifically, on the tag confirmation / correction screen 701, pharmacist 110 can tap the "□" on the left edge of the tag information whose tag name they want to correct to place a check mark, and then tap the "Change Tag" button at the top to display a pull-down menu of tag name candidates (not specifically illustrated), and change to other tag information. Additionally, pharmacist 110 can tap the "□" on the left edge of one or more tag entries on the tag confirmation / correction screen 701 to place a checkmark, and then tap the "Batch Approval" button at the top to approve all checked tag entries at once (remove any corrections). Furthermore, pharmacist 110 can modify the intervals in the classification text data by highlighting the text of the interval they want to select, for example by tracing it with their finger, on the display of the classification text data corresponding to the tag information they want to modify, and then tapping the "Interval Editing" button at the top.

[0036] The details of the tag information verification and correction performed by the pharmacist 110 are notified by the tag verification and correction processing unit 309 from the pharmacy terminal device 101 to the tag generation processing unit 303 running on the server computer 105. The tag generation processing unit 303 reflects the confirmed and corrected tag information in the new tag information.

[0037] Alternatively, the tag verification and correction processing unit 309 may cause the pharmacist 110 to verify and correct tag information that could mislead the patient 111.

[0038] As described above, pharmacist 110 can generate tag information to select medication guidance videos that appropriately respond to medication guidance for patient 111.

[0039] The video storage processing unit 304 in Figure 3 is executed, for example, by the control processor of the server computer 105. The video storage processing unit 304 stores video data received from the pharmacy terminal device 101 and stored in the storage device of the server computer 105, along with multiple tag information output by the tag generation processing unit 303 corresponding to that video data, in the video database 106, which is the video storage unit.

[0040] Figure 8 shows an example of the data structure of the video database 106. The video database 106 stores multiple sets of video data and multiple tag information, along with a video ID (identification information) that uniquely identifies them.

[0041] The access information generation processing unit 305 in Figure 3 is executed, for example, by the control processor of the server computer 105. The access information generation processing unit 305 generates access information for accessing video data in the video database 106 and multiple tag information corresponding to that video data, and outputs it so that it can be read by the patient terminal device 201 operated by the patient 111. The access information includes, for example, URL (Uniformed Resource Locator) information for accessing the server computer 105 via the Internet 107, and a string of data search commands for searching for records containing video IDs on the video database 106 connected to the server computer 105.

[0042] The access information acquisition processing unit 306 is executed, for example, by a control processor in the patient terminal device 201. The access information acquisition processing unit 306 acquires access information generated by the access information generation processing unit in the server computer 105.

[0043] A first embodiment of the access information generation processing unit 305 and the access information acquisition processing unit 306 will be described. The access information generation processing unit 305 generates two-dimensional code data, such as QR code data, that includes access information. In Figure 1, when pharmacist 110 receives QR code data generated from server computer 105 using pharmacy terminal device 101, when printing a drug information sheet 108 for the drug compounded based on the prescription using printer 102, the pharmacist overlays a QR code 109 corresponding to the received QR code data onto the margin of the drug information sheet 108. In Figure 2, patient 111 reads the QR code 109 printed on the medication information sheet 108 received from a pharmacy, etc., as a QR code image 202 using a QR code reader, which is an access information acquisition processing unit 306 executed on the patient terminal device 201, and acquires access information.

[0044] A second embodiment of the access information generation processing unit 305 and the access information acquisition processing unit 306 will be described. As illustrated in Figure 6, the access information generation processing unit 305 further includes a patient database 601 that stores, as a pair, at least patient identification information corresponding to the personal information of patient 111 and account information corresponding to the account of patient 111 in the messaging system. The access information generation processing unit 305 then obtains account information corresponding to patient identification information corresponding to the personal information of patient 111 written on the prescription created by the pharmacist from the patient database 601, and posts the access information to the account of patient 111 corresponding to the account information obtained by the messaging system, for example via the pharmacy terminal device 101, in accordance with the instructions of the pharmacist 110. In Figure 2, patient 111 receives access information posted to the patient's account from the messaging system via the access information acquisition processing unit 306, which is executed on the patient terminal device 201. For messaging systems, for example, LINE, provided by LINE Corporation, can be used, in which case the account information will be a LINE ID. Other messaging systems that can be used include Facebook Messenger, provided by Meta Platforms Inc. (USA), and Viber, provided by Viber Media, Inc. (Cyprus).

[0045] In Figure 3, the tag display processing unit 307 is executed, for example, by the terminal processor of the patient terminal device 201. The tag display processing unit 307 reads multiple tag information from the video database 106 connected to the server computer 105 according to the access information acquired by the access information acquisition processing unit 306, and displays it as a tag display screen on, for example, the touch panel display of the patient terminal device 201.

[0046] More specifically, the tag display processing unit 307 accesses the video database 106 connected to the server computer 105 via a URL containing the video ID, which is access information, and reads multiple tag information stored in the record corresponding to the specified video ID in the video database 106. As a result, the tag display processing unit 307 displays the tag display screen 901, as illustrated in Figure 9, on, for example, the touch panel display of the patient terminal device 201. Furthermore, the tag display processing unit 307 may highlight important tag information on the tag display screen 901 to indicate its importance. This important tag information may be determined by the tag generation processing unit 303 in Figure 3 based on whether the priority of the classification category, including the safety classification category, or the aforementioned confidence level is above a predetermined threshold when classifying speech text data into classification text data.

[0047] In Figure 3, the video playback processing unit 308 is executed, for example, by the terminal processor of the patient terminal device 201. When patient 111 taps desired tag information on the tag display screen 901, as exemplified in Figure 9, displayed on the touch panel display of the patient terminal device 201, the video playback processing unit 308 reads video data from the video database 106 for the section corresponding to the time information contained in the tag information selected by patient 111 on the tag display screen 901, according to the access information acquired by the access information acquisition processing unit 306. Then, the video playback processing unit 308 plays and displays the medication guidance video 203, which corresponds to the tag information selected by patient 111, from among the medication guidance videos (see Figure 1) taken by the pharmacist 110 at the pharmacy using a web camera 103 (Figure 1).

[0048] As described above, according to this embodiment, the patient 111 can freely select the items they wish to view from among the individual items of medication guidance, such as "Dosage and Administration," "Timing of Administration," and "Warning Signs and When to Seek Medical Attention," on the tag display screen 901, as illustrated in Figure 9, which is displayed on the touch panel display of the patient terminal device 201 that they operate, and easily view videos focused on the selected items.

[0049] The video playback processing unit 308 may play the video data and output the patient 111's viewing history of the medication guidance video. In Figure 3, the viewing history acquisition processing unit 310 is executed, for example, by the control processor of the server computer 105. The viewing history acquisition processing unit 310 acquires the viewing history of video data sent sequentially from the patient terminal device 201 and stores it in a storage device (not shown).

[0050] In Figure 3, the reminder notification processing unit 311 is executed, for example, by the control processor of the server computer 105. Based on the viewing history acquired and stored by the viewing history acquisition processing unit 310, when the reminder notification processing unit 311 detects a reminder notification condition that the patient 111 has not viewed the medication guidance video for a predetermined period of time, it notifies the patient terminal device 201 of a reminder notification prompting the patient 111 to view the medication guidance video. Specifically, the reminder notification processing unit 311 sends reminder notifications based on viewing history, for example, when 72 hours have passed since viewing. On the other hand, in consideration of the patient's well-being, the reminder notification processing unit 311 may suppress the sending of reminder notifications during quiet hours (for example, from 10 PM to 7 AM the following morning), and may also control the number of reminder notifications sent within a certain period (by setting an upper limit). The reminder notification processing unit 311 handles at least SMS (Short Message Service), messaging apps, email, in-app push notifications, or voice calls (IVR). A reminder notification may be sent by one means. Alternatively, the reminder notification may be sent to the patient terminal device 201 only for video data that has been authorized for public release by the public release control unit described later.

[0051] In Figure 3, the reminder notification receiving processing unit 312 is executed, for example, by the terminal processor of the patient terminal device 201. The reminder notification receiving processing unit 312 receives the above-mentioned reminder notification and notifies the patient 111 by voice or display.

[0052] As described above, it becomes possible to prevent patient 111 from forgetting to watch the medication guidance video. In addition, pharmacist 110 can monitor patient 111's viewing status.

[0053] Figure 10 is a block diagram showing an example of the hardware configuration of the pharmacy terminal device 101 in Figure 1, the server computer 105 in Figure 1, or the patient terminal device 201 in Figure 2. Although there are differences in performance depending on the type of terminal or computer, Figure 10 is a typical computer configuration example.

[0054] The CPU (Central Processing Unit) 1001 reads the processing program, which is stored in the ROM (Read-Only Memory) 1002 and is illustrated in the flowcharts shown in Figures 11, 12, or 13 later, into the RAM (Random Access Memory) 1003 and executes it.

[0055] The external storage device 1004 temporarily stores, for example, the aforementioned video database 106, as well as video data, audio-text data, classified audio-text data, tag information, and so on. For the external storage device 1004, if Figure 10 is, for example, the server computer 105 in Figure 1, then a cloud service may be used.

[0056] The input / output device 1005 is, for example, a touch panel display, if Figure 10 is, for example, a tablet terminal like the pharmacy terminal device 101 in Figure 1, or a smartphone like the patient terminal device 201 in Figure 2.

[0057] If Figure 10 is, for example, the pharmacy terminal device 101 in Figure 1 or the patient terminal device 201 in Figure 2, the network interface 1006 is a wireless LAN or Wi-Fi connection device for connecting each device to the pharmacy LAN 104 or the internet 107 in Figure 1. If Figure 10 is, for example, the server computer 105 in Figure 1, then the network interface 1006 is a wired or wireless WAN (Wide Area Network) or LAN connection device for connecting each device to the internet 107 in Figure 1 or the company LAN (not specifically shown).

[0058] The CPU 1001, ROM 1002, RAM 1003, external storage device 1004, input / output device 1005, and network interface 1006 are interconnected by a system bus 1007.

[0059] Figure 11 is a flowchart showing an example of pharmacy terminal control processing performed by the control processor of the pharmacy terminal device 101 in the medication guidance system 100 of Figure 1. This processing involves the CPU 1001, which is the control processor, reading the pharmacy terminal control processing program stored in ROM 1002 into RAM 1003 and executing it in the hardware configuration of Figure 10 corresponding to the pharmacy terminal device 101 of Figure 1. This processing is activated each time the pharmacist 110 receives a prescription from a patient 111 and completes the dispensing of medication, by the pharmacist 110 operating the input / output device 1005, such as the touch panel display, of the pharmacy terminal device 101. Hereafter, unless otherwise specified, the configurations of Figures 1 and 10 will be referred to.

[0060] First, the CPU 1001 prompts the pharmacist 110, who is operating the pharmacy terminal device 101, to input prescription information using an input / output device 1005 such as a touch panel display (step S1101 in Figure 11).

[0061] Next, for example, when pharmacist 110 performs a specific operation on an input / output device 1005 such as a touch panel display of the pharmacy terminal device 101, the CPU 1001 initiates recording of a video of pharmacist 110 giving medication guidance to patient 111 via the pharmacy LAN 104 to the web camera 103 (step S1102 in Figure 11).

[0062] Subsequently, pharmacist 110 provides medication guidance to patient 111.

[0063] During this time, the CPU 1001 continues to determine whether, for example, the pharmacist 110 has instructed the end of the imaging process by performing a specific operation on the input / output device 1005, such as the touch panel display of the pharmacy terminal device 101 (this is a repetitive process where the determination in step S1103 of Figure 11 is NO).

[0064] If the determination in step S1103 is NO, the CPU 1001 sends the video data captured by the webcam 103 from the pharmacy LAN 104 to the server computer 105 via the internet 107 (step S1104 in Figure 11). After that, the CPU 1001 returns to the determination process in step S1103.

[0065] When the pharmacist 110 instructs the end of recording, resulting in a YES judgment in step S1103, the CPU 1001 notifies the server computer 105 that the video data has ended (step S1105 in Figure 11).

[0066] Alternatively, in step S1104, the CPU 1001 may choose to save the video data to the external storage device 1004 within the pharmacy terminal device 101 instead of sequentially transmitting it to the server computer 105. Then, after determining in step S1103 that recording has finished, the CPU 1001 may choose to transmit all the video data stored in the external storage device 1004 to the server computer 105 at once.

[0067] The processes from steps S1101 to S1105 in Figure 11 above realize the functions of the recording processing unit 301 shown in Figure 3.

[0068] After processing in step S1105, the CPU 1001 waits until it receives a tag confirmation / correction notification from the server computer 105 (the determination in step S1106 in Figure 11 is repeatedly NO).

[0069] When the server computer 105 receives a tag verification / correction notification and the determination in step S1106 is YES, the CPU 1001 uses Figure 7 to execute the function of the tag verification / correction processing unit 309 shown in Figure 3 (step S1107 in Figure 11). At the end of step S1107, the CPU 1001 sends a tag correction instruction from the pharmacy terminal device 101 to the server computer 105.

[0070] After the processing in step S1107, the CPU 1001 waits until it receives access information from the access information generation process (steps S1201-6 in Figure 12, described later) executed on the server computer 105 (the determination in step S1108 in Figure 11 is repeatedly NO).

[0071] When access information is received from the server computer 105 and the determination in step S1108 is YES, the CPU 1001 executes the process of outputting the access information (step S1109 in Figure 11). As described above in the explanation of the access information generation processing unit 305 in Figure 3, the access information is information for accessing video data in the video database 106 and multiple tag information corresponding to that video data (see Figure 8).

[0072] When the first embodiment described above of the access information generation processing unit 305 and the access information acquisition processing unit 306 is implemented, the server computer 105 sends two-dimensional code data containing access information, such as QR code data, to the pharmacy terminal device 101. In step S1109 of Figure 11, when the CPU 1001 receives QR code data containing access information from the server computer 105, it prints the drug information sheet 108 on the printer 102 of Figure 1 according to the instructions of the pharmacist 110, and overlays the QR code 109 corresponding to the received QR code data onto the margin of the drug information sheet 108.

[0073] When the second embodiment described above of the access information generation processing unit 305 and the access information acquisition processing unit 306 is implemented, the server computer 105 notifies the user of an instruction to post access information to the patient 111's account in the messaging system. In step S1109 of Figure 11, when CPU 1001 receives a posting instruction to the messaging system, it posts access information to patient 111's account in the messaging system in accordance with the instructions of pharmacist 110.

[0074] After the processing in step S1109 in Figure 11, the CPU 1001 terminates the pharmacy terminal control process as illustrated in the flowchart of Figure 11.

[0075] Figure 12 is a flowchart showing an example of server control processing performed by the control processor of the server computer 105 in the medication guidance system 100 of Figure 1. This processing involves the CPU 1001, which is the control processor, reading the server control processing program stored in ROM 1002 into RAM 1003 and executing it in the hardware configuration of Figure 10 corresponding to the server computer 105 of Figure 1. This processing starts when the server computer 105 is started up. Hereafter, unless otherwise specified, the configurations of Figures 1 and 10 will be referred to.

[0076] The CPU 1001 constantly and repeatedly performs the following checks: whether or not it has received prescription information from the pharmacy terminal device 101 (step S1201 in Figure 12), whether or not it has received viewing history from the patient terminal device 201 (step S1202 in Figure 12), and whether or not it has detected the reminder notification conditions described above in the explanation of the reminder notification processing unit 311 in Figure 3 (step S1203 in Figure 12).

[0077] When prescription information is entered into the pharmacy terminal device 101 (see step S1101 in Figure 11), the CPU 1001 receives notification of the prescription information from the pharmacy terminal device 101 to the server computer 105, and if the determination in step S1201 is YES, it executes a series of processes from step S1201-1 to step S1201-6 in Figure 12.

[0078] In this series of processes, first, the CPU 1001 executes the speech recognition process described above in the explanation of the speech recognition processing unit 302 in Figure 3 (step S1201-1 in Figure 12).

[0079] Next, the CPU 1001 takes the speech text data output in the speech recognition process of step S1201-1 as input and executes the tag generation process described above in the explanation of the tag generation processing unit 303 in Figure 3 (step S1201-2 in Figure 12). Once the CPU 1001 has finished generating the tag information, it sends a tag confirmation / correction notification to the pharmacy terminal device 101. This notification is detected by the pharmacy terminal device 101 in step S1106 of Figure 11 described above.

[0080] After the tag generation process in step S1201-2, the CPU 1001 waits for a tag correction instruction from the pharmacy terminal device 101 by performing the tag verification and correction process described above (see step S1107 in Figure 11) (the determination in step S1201-3 in Figure 12 is repeatedly NO).

[0081] When the pharmacy terminal device 101 receives a tag correction instruction and the determination in step S1201-3 is YES, the CPU 1001 executes the tag information update process instructed by the tag correction instruction (step S1201-4 in Figure 12).

[0082] Next, the CPU 1001 executes the video storage process described in the explanation of the video storage processing unit 304 in Figure 3 (step S1201-5 in Figure 12). As a result, the records of the video database 106 described in the explanation of Figure 8 are stored.

[0083] Furthermore, the CPU 1001 executes the access information generation process described above in the explanation of the access information generation processing unit 305 in Figure 3 (step S1201-6 in Figure 12).

[0084] When the first embodiment described above of the access information generation processing unit 305 and the access information acquisition processing unit 306 is implemented, the CPU 1001 generates two-dimensional code data containing access information, such as QR code data, in step S1201-6 of Figure 12, and transmits the QR code data to the pharmacy terminal device 101 (see step S1109 of Figure 11).

[0085] When the second embodiment described above of the access information generation processing unit 305 and the access information acquisition processing unit 306 is implemented, the CPU 1001 generates a posting instruction for access information to the patient 111's account in the messaging system in steps S1201-6 of Figure 12, and transmits the data to the pharmacy terminal device 101 (see step S1109 of Figure 11).

[0086] If the determination in step S1201 is NO, or after processing in steps S1201-6, the CPU 1001 determines whether or not it has received viewing history from the patient terminal device 201 (step S1202 in Figure 12) (see step S1305 in Figure 13, described later).

[0087] When the viewing history is received from the patient terminal device 201 and the determination in step S1202 is YES, the CPU 1001 executes the viewing history acquisition process described above in the explanation of the viewing history acquisition processing unit 310 in Figure 3 (step S1202-1 in Figure 12).

[0088] If the determination in step S1202 is NO, or after the processing in step S1202-1, the CPU 1001 determines whether the reminder notification conditions described above in the explanation of the reminder notification processing unit 311 in Figure 3 have been met (step S1203 in Figure 12).

[0089] When the reminder notification conditions are met and the determination in step S1203 is YES, the CPU 1001 executes the reminder notification process described above in the explanation of the reminder notification processing unit 311 in Figure 3 (step S1203-1 in Figure 12).

[0090] If the determination in step S1203 is NO, or after the processing in step S1203-1, the CPU 1001 returns to the determination process in step S1201 and continues processing.

[0091] Figure 13 is a flowchart showing an example of patient terminal control processing performed by the terminal processor of the patient terminal device 201 in the medication guidance system 100 of Figure 1. This processing is performed by the CPU 1001, which is the terminal processor, reading the patient terminal control processing program RAM 1003 stored in ROM 1002 and executing it in the hardware configuration of Figure 10 corresponding to the patient terminal device 201 of Figure 1. This processing is started each time patient 111 performs the access information acquisition processing described in Figure 3 using the patient terminal device 201. Hereafter, unless otherwise specified, the configurations of Figures 1 and 10 will be referred to.

[0092] When the first embodiment described above of the access information generation processing unit 305 and the access information acquisition processing unit 306 is implemented, in Figure 2, the patient 111 reads the QR code 109 printed on the medication information sheet 108 received from the pharmacy, etc., as a QR code image 202 by the QR code reader, which is the access information acquisition processing unit 306 executed on the patient terminal device 201, and acquires access information. As a result, the patient terminal control process illustrated in the flowchart of Figure 13 is activated.

[0093] When the second embodiment described above of the access information generation processing unit 305 and the access information acquisition processing unit 306 is implemented, in Figure 2, the patient 111 obtains access information posted to the patient's account from the messaging system in the access information acquisition processing unit 306 executed on the patient terminal device 201. As a result, the patient terminal control process illustrated in the flowchart of Figure 13 is activated.

[0094] First, the CPU 1001 executes the tag display process described above in the explanation of the tag display processing unit 307 in Figure 3 (step S1301 in Figure 13). As a result, the tag display screen 901 exemplified in Figure 9 is displayed on the touch panel display of the smartphone operated by the patient 111.

[0095] Next, the CPU 1001 determines whether or not the tag information has been selected on the tag display screen 901 by the patient 111 (step S1302 in Figure 13).

[0096] If the determination in step S1302 is NO, the CPU 1001 then determines whether or not a termination request has occurred due to the patient 111 performing a specific operation on the touch panel display 110 (step S1303 in Figure 13).

[0097] If the determination in step S1303 is also NO, the CPU 1001 returns to the determination process in step S1302 and continues processing.

[0098] When patient 111 taps to select tag information and the determination in step S1302 is YES, the CPU 1001 executes the video playback processing and viewing history output processing described in the explanation of the video playback processing unit 308 in Figure 3 (step S1304 in Figure 13). The viewing history output here is processed in the determination processing of step S1202 in Figure 12, which is executed by the server computer 105.

[0099] After the processing in step S1304, the CPU 1001 determines whether the patient 111 has instructed the patient terminal device 201 to save the video being played by performing a specific operation on the touch panel display (step S1305 in Figure 13).

[0100] If the determination in step S1305 is YES, the CPU 1001 saves the video data being played to an external storage device 1004 or the like (step S1306 in Figure 13).

[0101] If the determination in step S1305 is NO, the CPU 1001 returns to the determination process in step S1302 and continues processing.

[0102] When the aforementioned termination request occurs and the determination in step S1303 is YES, the CPU 1001 terminates the patient terminal control process as illustrated in the flowchart of Figure 13.

[0103] In the medication guidance system 100 described above, in order to avoid putting a strain on the storage capacity of the server computer 105, records of video IDs, video data, and multiple tag information sets that have been registered in the video database 106 for a predetermined period (e.g., one month) may be deleted. Alternatively, when the patient 111 saves the video data of the medication guidance video 203 to the patient terminal device 201 in step S1306 of Figure 13, the corresponding record in the video database 106 may be deleted. Alternatively, the system may include a configuration of temporary terminal storage → transmission after line restoration → automatic deletion on the terminal side.

[0104] Figure 14 is a diagram and explanatory diagram showing an embodiment of a desktop video recording device. This embodiment is an alternative configuration of the webcam (webcam 103 in Figure 1) and printer (printer 102 in Figure 1) that prints QR codes 109, as shown in the embodiment shown in Figure 1. In this embodiment, the functions corresponding to the webcam 103 and printer 102 in Figure 1 are implemented as a desktop video recording device 1400, as shown in Figure 14.

[0105] In Figure 14(a), the desktop video recording device 1400 is placed on the counter (or desk) where the pharmacist 110 provides medication guidance to the patient 111, and has the function of recording the actions described in Figure 1 and printing the QR code 109.

[0106] In Figure 14(a), the desktop video recording device 1400 comprises a main body 1401, camera arms 1402 (#1 and #2) rotatably mounted to the main body in direction C, a hinge part 1403 (#1) that rotatably connects camera arm 1402 (#2) to camera arm 1402 (#1) in direction A, a camera unit 1404, a hinge part 1403 (#2) that rotatably connects camera unit 1404 to camera arm 1402 (#2) in direction B, an LCD display 1405 and a QR code printing unit 1406 (two-dimensional code printing unit) mounted on the side of the main body 1401, a start button 1407 and a stop button 1408 mounted on the top surface of the main body 1401 to indicate the start and end of recording, respectively, a QR code printing button 1409 for issuing a QR code 109 (see Figure 1), and USB (Universal Serial The system is configured to include a Bus cable 1410 and a USB cable 1411, which are connected to the USB terminal of a pharmacy terminal device 101 similar to that shown in Figure 1, and a USB connector 1411 for connecting the main unit 1401 to the USB terminal of the pharmacy terminal device 101. Furthermore, the desktop video recording device 1400 may be configured to communicate wirelessly with the pharmacy terminal device 101 (for example, via Bluetooth or Wi-Fi) without a USB cable 1410.

[0107] In the embodiment shown in Figure 14, the pharmacist 110 (see Figure 1) places the desktop video recording device 1400, which has the configuration example shown in Figure 14(a), on the counter (or desk) where medication guidance is provided. Next, the pharmacist 110 adjusts the rotation angles of the camera arm 1402 (#1 and #2) and the camera unit 1404 by hand on the liquid crystal display 1405 on the side of the main unit 1401 in Figure 14(a), so that only the medication 1424 (#1 or #2, etc.) placed on the medicine bag 1423 (#1 or #2, etc.) and the hands 1421 and 1422 of the pharmacist 110 and the patient 111 (see Figure 1) are displayed within the black frame 1420 (#1, #2, #3, #4), as shown in Figure 14(b).

[0108] Next, when the pharmacist 110 presses the start button 1407 on the top of the main unit 1401, the recording of the medication guidance session begins as a video. The video is displayed on the LCD display 1405 as an image like Figure 14(b). Note that the video on the LCD display 1405 does not need to be high resolution, as it only needs to give a general idea for confirmation purposes.

[0109] Once the medication guidance is complete, the pharmacist 110 presses the stop button 1408 on the top of the main unit 1401 to stop recording the video of the medication guidance. After that, the pharmacist 110 asks the patient 111 if they would like to review the video later. If the patient 111 requests to review the video later, the pharmacist 110 presses the QR code print button 1409 on the top of the main unit 1401. As a result, two stickers containing the QR code 109, as explained in Figure 1, for accessing the video, are printed from the QR code print section 1406 on the side of the main unit 1401. The pharmacist 110 attaches one sticker with the QR code 109 to the back of a prescription, for example, printed separately from the printer 102 in Figure 1, and to the patient's medication record book, and gives them to the patient 111. The patient 111, as explained in Figure 2, reads the QR code 109 with their patient terminal device 201 after returning home, for example, to display the tag and play the video. If patient 111 does not request to review the video later, pharmacist 110 will end the medication guidance without pressing the QR code print button 1409 on the top of the main unit 1401. In this case, the sticker paper with QR code 109 will not be printed.

[0110] In the embodiments described above, a specific example of the tag generation processing unit 303 in Figure 3 was described using a generation AI or LLM, but it is not limited to this, and the tag generation processing unit 303 may output tag information by inputting the voice text data into, for example, a discrimination model or a rule-based processing unit.

[0111] In the embodiment described above, the input to the tag generation processing unit 303 in Figure 3 was the speech text data output by the speech recognition processing unit 302, but it is not limited to this, and at least one of the following may be used in combination: the OCR (Optical Character Recognition) results of each video frame of the video data, structured data such as prescription records (prescription information), and metadata such as the recording time of the medication guidance video 203.

[0112] In the embodiments described above, the tag display processing unit 307 in Figure 3 may include features such as presenting important tag information as a pop-up on the tag display screen 901 in Figure 9, enabling jumps to the timeline table of contents, a permanent banner at the top of the screen, and a subtitle overlay.

[0113] In the embodiments described above, the acquisition of account information shown in Figure 6 by the access information generation processing unit 305 and the access information acquisition processing unit 306 in Figure 3, and the acquisition of reminder notification destinations in the reminder notification processing unit 311, etc., referencing means of contacting the patient terminal device 201 may be performed by tokenization, and the configuration may include not permanently storing personal identifiers such as telephone numbers. After initial identification, subsequent notifications may be automated using the token.

[0114] In the embodiments described above, the medication guidance system 100 may record the operator, target, time, and reason for tag verification / modification / approval, notification sending, setting changes, and exception operations (such as temporary permission for publication) as audit logs, and may include a mechanism for detecting tampering. Furthermore, tampering detection may be performed by hash chaining.

[0115] In the embodiments described above, the medication guidance system 100 may receive an examination completion event (hereinafter referred to as "external linkage information") from an external system (receipt computer, inspection system, etc.) that includes at least one of the following: GTIN (JAN / GS1), lot number, expiration date, examination implementer identifier, and examination completion time. In this case, the medication guidance system 100 may include a publication control unit that permits / denies the publication of video data to the patient based on the external linkage information. The publication control unit may deny publication if the external linkage information indicates a discrepancy with the prescription instructions or drug identification results, and permit publication when it detects that the discrepancy has been resolved. Furthermore, the publication control unit may temporarily permit publication under predetermined conditions such as emergencies, after recording at least the operator, reason, time, and target in the audit log. In this case, as described above, the reminder notification processing unit 311 in Figure 3 may notify the patient terminal device 201 of a reminder notification only when publication is permitted by the publication control unit.

[0116] In the embodiments described above, the access information generation processing unit 305 may generate QR code data containing access information as described in the first embodiment, and send the QR code data (QR image data) attached to the patient 111 account of the messaging system as described in the second embodiment.

[0117] According to this embodiment, a medication guidance system, a medication guidance method, a program, and a computer-readable recording medium on which the program is recorded can be realized, which allows patient 111 to select and partially view desired items (dosage and administration, timing of administration, warning signs and reasons for seeking medical attention, etc.) from a video of medication guidance received by pharmacist 110 at a pharmacy, etc., using a patient terminal device 201. According to this embodiment, patient 111 can accurately determine how to take their medication. Furthermore, according to this embodiment, patient 111 can effectively prevent forgetting to watch the medication guidance video through the reminder notification mentioned above. Therefore, according to this embodiment, it is possible to improve patient 111's understanding of medication content, increase medication compliance rates, and improve the efficiency of pharmacist 110's work. [Explanation of symbols]

[0118] 100 Medication Guidance Systems 101 Pharmacy terminal device 102 Printing machine 103 Imaging device 104 Pharmacy LAN 105 Server Computers 106 Video Database 107 Internet 108 Drug Information Sheet 109 QR codes 110 Pharmacists 111 patients 120 External Server Computers 201 Patient terminal device 202 QR code image 203 Medication guidance video 301 Recording Processing Unit 302 Speech Recognition Processing Unit 303 Tag generation processing unit 304 Video Memory Processing Unit 305 Access Information Generation Processing Unit 306 Access Information Acquisition Processing Unit 307 Tag display processing unit 308 Video playback processing unit 309 Tag verification / correction processing unit 310 Viewing history acquisition processing unit 311 Reminder Notification Processing Unit 312 Reminder notification receiving processing unit 601 Patient Database 701 Tag Confirmation / Editing Screen 901 Tag display screen 1001 CPU 1002 ROM 1003 RAM 1004 External storage device 1005 Input / Output Device 1006 Network Interface 1007 System Bus 1400 Desktop Video Recording Device 1401 Main Unit 1402 Camera Arm 1403 Hinge 1404 Camera Department 1405 LCD display 1406 QR Code Printing Section 1407 Start button 1408 Stop button 1409 QR code print button 1410 USB cable 1420 Black frame 1421, 1422 Hands 1423 Medicine bag 1424 Pharmaceuticals

Claims

1. A recording device for capturing a video of a pharmacist giving medication guidance to a patient, including audio obtained from the recording, Control processor of pharmacy terminal device, The control processor of the server computer, The system comprises a terminal processor for a patient terminal device operated by the patient, The control processor of the aforementioned pharmacy terminal device is The recording process is performed to record video data corresponding to the aforementioned recorded medication guidance video. The control processor of the server computer is A speech recognition process that outputs speech text data corresponding to the speech by performing speech recognition processing on the aforementioned video data, A tag generation process is performed to generate tag information indicating the content of the instruction, for each classified text data obtained by classifying the aforementioned audio text data into multiple categories according to the content of the medication instruction, and including time information from the beginning in the medication instruction video. A video storage process that stores the aforementioned video data and a plurality of tag information corresponding to the said video data in a video storage unit, The process involves generating access information for accessing the video data in the video storage unit and the tag information corresponding to the video data, and outputting the access information so that it can be read by the patient terminal device operated by the patient. The terminal processor of the aforementioned patient terminal device is The access information acquisition process for acquiring the aforementioned access information, A tag display process that reads multiple tag information in the video storage unit according to the acquired access information and displays it on the patient terminal device, In accordance with the acquired access information, the following video playback process is performed: reading the video data from the video storage unit for a time range corresponding to the time information included in the tag information selected by the patient on the patient terminal device, and playing it on the patient terminal device; The tag generation process involves causing the generative artificial intelligence system to input the speech text data and, for each of the multiple tag information items, to prompt the generative artificial intelligence system to perform classification from the speech text data into multiple classification text data items, and to output the time information, tag name, metadata, generation justification statement, and confidence level for each classification text data item as the tag information corresponding to the classification text data item. The access information generation process generates two-dimensional code data containing the access information, a sheet of paper on which a two-dimensional code corresponding to the two-dimensional code data is printed by a printer, the access information acquisition process acquires the access information by reading the two-dimensional code on the sheet of paper based on the patient's instructions, or the access information generation process comprises a patient database that stores, as a pair, patient identification information corresponding to the patient's personal information and account information corresponding to the patient's account in a messaging system, the pharmacist acquires the account information corresponding to the patient identification information corresponding to the patient's personal information written on the prescription created by the pharmacist from the patient database, posts the access information to the patient's account in the messaging system corresponding to the acquired account information, and the access information acquisition process acquires the access information posted to the patient's account from the messaging system based on the patient's instructions.

2. The medication guidance system according to claim 1, wherein the speech recognition process outputs the speech text data by having a machine learning model, which has been trained using training speech data and training speech text data obtained in advance by manually recognizing the training speech data, input the speech data contained in the video data.

3. The medication guidance system according to claim 1, wherein the speech recognition process uses a large-scale language model and outputs the speech text data by inputting the speech data contained in the video data into a deep neural network that has been trained using training speech data and training speech text data obtained in advance by manually recognizing the training speech data.

4. The generative artificial intelligence system is connected to the control processor of the server computer via a network, The medication guidance system according to claim 1, wherein the control processor of the server computer communicates with the generative artificial intelligence system via an application program interface provided by the generative artificial intelligence system.

5. The medication guidance system according to claim 1, wherein the tag generation process further generates the tag information by using in combination the results of optical character recognition processing of each video frame of the video data, structured data of prescription information, and at least one text data of metadata including the recording time of the medication guidance video.

6. The medication guidance system according to claim 1, wherein the tag display process distinguishes important tag information from the tag information, which is determined based on the priority of a classification category that includes a safety classification category when the tag generation process classifies the speech text data into the classification text data, or whether the confidence level included in the tag information is above a predetermined threshold, and displays such important tag information on the patient terminal device in a manner that indicates its importance.

7. The medication guidance system according to claim 6, wherein the aforementioned important tag information is determined prioritizing the tag information belonging to the safety classification category.

8. The medication guidance system according to claim 6, wherein the important tag information is determined based on the confidence level being above a predetermined threshold.

9. The medication guidance system according to claim 1, wherein the tag display processing includes displaying on the tag display screen of the patient terminal device one or more of the following: presenting important tag information from the tag information as a pop-up; enabling jumps to a timeline table of contents; a permanent banner at the top of the screen; or a subtitle overlay.

10. The medication guidance system according to claim 1, wherein the control processor of the pharmacy terminal device further executes a tag confirmation and correction process in which the pharmacy terminal device causes the pharmacist to confirm and correct the tag information generated in the tag generation process.

11. The aforementioned tag verification and correction process displays, for each piece of tag information, the classification text data, which includes the statement justifying the generation of the tag information, the degree of confidence in the generation of the tag information, and the preceding and succeeding audio text data corresponding to the tag information, on the display of the pharmacy terminal device. The medication guidance system according to claim 10, wherein the pharmacist confirms and corrects the tag information for each piece of tag information based on the classification text data which includes the generation justification statement displayed on the display as part of the tag information, the confidence level of the generation, and the preceding and succeeding audio text data corresponding to the tag information.

12. The medication guidance system according to claim 10, wherein the tag confirmation and correction process causes the pharmacist to confirm and correct the tag information that may cause misidentification to the patient.

13. The aforementioned access information generation process is: Generate two-dimensional code data for displaying a two-dimensional code image that can read the aforementioned access information. The medication guidance system according to claim 12, wherein a message to which the generated two-dimensional code data is attached is posted to the patient's account corresponding to the account information acquired by the messaging system.

14. The aforementioned video playback process plays the video data and outputs the patient's viewing history of the medication guidance video. The control processor of the server computer is The aforementioned viewing history acquisition process acquires and saves the viewing history, When the system detects that the patient has not watched the medication guidance video for a predetermined period based on the saved viewing history, it performs a reminder notification process that notifies the patient terminal device of a reminder notification prompting the patient to watch the medication guidance video. The terminal processor of the aforementioned patient terminal device is The medication guidance system according to claim 1, further comprising a reminder notification reception process that receives the reminder notification and notifies the patient.

15. The medication guidance system according to claim 14, wherein the reminder notification process suppresses the sending of reminder notifications during quiet periods and / or controls the number of times reminder notifications are sent within a certain period.

16. The medication guidance system according to claim 14, wherein the reminder notification process provides the reminder notification by at least one means of short message service, messaging app, email, in-app push notification, or voice call.

17. The medication guidance system further includes an external linkage information that receives external linkage information from an external system, which is an examination completion event including at least one of GTIN (JAN / GS1), lot number, expiration date, examination implementer identifier, and examination completion time, and permits or / or restricts the disclosure of the video data to the patient based on the external linkage information. The medication guidance system according to claim 14, wherein the reminder notification process notifies the patient terminal device of the reminder notification only if the public control unit has authorized the public disclosure of the video data to the patient.

18. The medication guidance system according to claim 1, wherein the means for contacting the patient terminal device is performed using a token that does not permanently store the personal identifier of the patient operating the patient terminal device.

19. The medication guidance system according to claim 1, further comprising means for recording at least the operator, subject, time, and reason of the medication guidance system as an audit log, and for detecting tampering.

20. The medication guidance system according to claim 19, wherein the tampering detection is performed by hash chaining.

21. The medication guidance system according to claim 1, wherein the video data stored in the video storage unit and the plurality of tag information for the video data are deleted after a predetermined period of time has elapsed, deleted when the patient saves the video data of the medication guidance video to the patient terminal device, or includes the modes of temporary storage to the patient terminal device, transmission after line restoration, and automatic deletion on the terminal side.

22. The medication guidance system according to claim 21, wherein the video data temporarily stored in the patient terminal device is automatically deleted after transmission is complete.

23. The medication guidance system according to claim 1, further comprising a disclosure control unit that receives external linkage information from an external system, which is an examination completion event including at least one of GTIN (JAN / GS1), lot number, expiration date, examination implementer identifier, and examination completion time, and permits or / or restricts the disclosure of the video data to the patient based on the external linkage information.

24. The medication guidance system according to claim 23, wherein the disclosure control unit suppresses disclosure when the externally linked information shows a discrepancy with prescription instructions or drug identification results, and permits disclosure when it detects that the discrepancy has been resolved.

25. The medication guidance system according to claim 23, wherein the disclosure control unit, when temporarily permitting disclosure under predetermined conditions, records at least the operator, reason, time, and subject in the audit log.

26. At least one control processor, A recording process involves recording a video of a pharmacist giving medication guidance to a patient, including audio, using a recording device, and then recording the video data corresponding to the recorded medication guidance video. A speech recognition process that outputs speech text data corresponding to the speech by performing speech recognition on the video data output in the aforementioned recording process, A tag generation process is performed to generate tag information indicating the content of the instruction, for each classified text data obtained by classifying the aforementioned audio text data into multiple categories according to the content of the medication instruction, and including time information from the beginning in the medication instruction video. A video storage process that stores the aforementioned video data and a plurality of tag information corresponding to the said video data in a video storage unit, Access information generation process generates access information for accessing the video data in the video storage unit and the tag information corresponding to the video data, and outputs the access information so that it can be read by the patient terminal device operated by the patient. Make it run, The terminal processor of the aforementioned patient terminal device The access information acquisition process for acquiring the aforementioned access information, Tag display processing that reads multiple tag information in the video storage unit and displays it on the patient terminal device according to the acquired access information, A video playback process that reads video data from the video storage unit for a time range corresponding to the time information included in the tag information selected by the patient on the patient terminal device, according to the acquired access information, and plays it on the patient terminal device; Make it run, The tag generation process involves causing the generative artificial intelligence system to input the speech text data and, for each of the multiple tag information items, to prompt the generative artificial intelligence system to perform classification from the speech text data into multiple classification text data items, and to output the time information, tag name, metadata, generation justification statement, and confidence level for each classification text data item as the tag information corresponding to the classification text data item. A method of medication guidance comprising: the access information generation process generates two-dimensional code data containing the access information, a sheet of paper on which a two-dimensional code corresponding to the two-dimensional code data is printed by a printer, the access information acquisition process acquires the access information by reading the two-dimensional code on the sheet of paper based on the patient's instructions; or the access information generation process comprises a patient database that stores, as a pair, patient identification information corresponding to the patient's personal information and account information corresponding to the patient's account in a messaging system, the pharmacist acquires the account information corresponding to the patient identification information corresponding to the patient's personal information written on the prescription created by the pharmacist from the patient database, posts the access information to the patient's account in the messaging system corresponding to the acquired account information, and the access information acquisition process acquires the access information posted to the patient's account from the messaging system based on the patient's instructions.

27. At least one control processor, A recording process involves recording a video of a pharmacist giving medication guidance to a patient, including audio, using a recording device, and then recording the video data corresponding to the recorded medication guidance video. A speech recognition process that outputs speech text data corresponding to the speech by performing speech recognition on the video data output in the aforementioned recording process, A tag generation process is performed to generate tag information indicating the content of the instruction, for each classified text data obtained by classifying the aforementioned audio text data into multiple categories according to the content of the medication instruction, and including time information from the beginning in the medication instruction video. A video storage process that stores the aforementioned video data and a plurality of tag information corresponding to the said video data in a video storage unit, Access information generation process generates access information for accessing the video data in the video storage unit and the tag information corresponding to the video data, and outputs the access information so that it can be read by the patient terminal device operated by the patient. Make it run, The terminal processor of the aforementioned patient terminal device The access information acquisition process for acquiring the aforementioned access information, Tag display processing that reads multiple tag information in the video storage unit and displays it on the patient terminal device according to the acquired access information, A video playback process that reads video data from the video storage unit for a time range corresponding to the time information included in the tag information selected by the patient on the patient terminal device, according to the acquired access information, and plays it on the patient terminal device; Make it run, The tag generation process involves causing the generative artificial intelligence system to input the speech text data and, for each of the multiple tag information items, to prompt the generative artificial intelligence system to perform classification from the speech text data into multiple classification text data items, and to output the time information, tag name, metadata, generation justification statement, and confidence level for each classification text data item as the tag information corresponding to the classification text data item. The access information generation process generates two-dimensional code data containing the access information, a sheet of paper on which a two-dimensional code corresponding to the two-dimensional code data is printed by a printer, the access information acquisition process acquires the access information by reading the two-dimensional code on the sheet of paper based on the patient's instructions, or the access information generation process includes a patient database that stores, in pairs, patient identification information corresponding to the patient's personal information and account information corresponding to the patient's account in a messaging system, acquires the account information corresponding to the patient identification information corresponding to the patient's personal information written on a prescription created by the pharmacist from the patient database, posts the access information to the patient's account in the messaging system corresponding to the acquired account information, and the access information acquisition process acquires the access information posted to the patient's account from the messaging system based on the patient's instructions.

28. A control processor for a pharmacy terminal device, A server computer comprising a control processor, The control processor of the aforementioned pharmacy terminal device is The recording process is performed to record video data corresponding to a medication guidance video, which includes audio obtained by recording a pharmacist giving medication guidance to a patient using a recording device. The control processor of the server computer is A speech recognition process that outputs speech text data corresponding to the speech by performing speech recognition processing on the aforementioned video data, A tag generation process is performed to generate tag information indicating the content of the instruction, for each classified text data obtained by classifying the aforementioned audio text data into multiple categories according to the content of the medication instruction, and including time information from the beginning in the medication instruction video. A video storage process that stores the aforementioned video data and a plurality of tag information corresponding to the said video data in a video storage unit, The process involves generating access information for accessing the video data in the video storage unit and the tag information corresponding to the video data, and outputting the access information so that it can be read by the patient terminal device operated by the patient. The tag generation process involves causing the generative artificial intelligence system to input the speech text data and, for each of the multiple tag information items, to prompt the generative artificial intelligence system to perform classification from the speech text data into multiple classification text data items, and to output the time information, tag name, metadata, generation justification statement, and confidence level for each classification text data item as the tag information corresponding to the classification text data item. The access information generation process generates two-dimensional code data containing the access information, and a sheet of paper on which a two-dimensional code corresponding to the two-dimensional code data is printed by a printer, or the access information generation process comprises a patient database that stores, in pairs, patient identification information corresponding to the patient's personal information and account information corresponding to the patient's account in a messaging system, and retrieves the account information corresponding to the patient identification information corresponding to the patient's personal information written on a prescription created by the pharmacist from the patient database, and posts the access information to the patient's account in the messaging system corresponding to the retrieved account information.

Images