Teaching support device, teaching support system, and program

Abstract

Provide better support for medical guidance. [Solution] The training support server 30 includes a video call control unit 352, a voice recognition processing unit 353, and a recommendation information generation unit 354. The video call control unit 352 controls a video call between a terminal used by a medical instructor and a terminal used by a patient in order to remotely provide medical guidance to the patient. The voice recognition processing unit 353 performs voice recognition processing on voices transmitted and received during the video call. The recommendation information generation unit 354 generates recommendation information, which is information indicating guidance content recommended for pharmaceutical guidance. Furthermore, in parallel with the progress of the video call, the video call control unit 352 causes the terminal used by the pharmacist to output training support information based on the results of voice recognition by the voice recognition processing unit 353 and the recommendation information generated by the recommendation information generation unit 354.

Description

[Technical Field]

[0001] The present invention relates to a training support device, a training support system, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, information processing techniques for supporting medical guidance, such as medical guidance and pharmaceutical guidance, have been known. For example, Patent Document 1 discloses a technology for managing patient medication history data in a database. According to the technology disclosed in Patent Document 1, when a pharmacist dispenses medication to a patient, the pharmacist can refer to the database and provide medication instructions based on the patient's medication history, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-122253 Summary of the Invention [Problem to be solved by the invention]

[0004] In general techniques such as that disclosed in Patent Document 1, when medical guidance is provided, it is assumed that the instructor providing the guidance and the patient receiving the guidance will meet face-to-face. In other words, general techniques do not take into account situations where the instructor and the patient do not (or cannot) meet face-to-face, for example, in consideration of contact infection of viruses or bacteria. However, it is desirable to provide more appropriate support for medical guidance even in such situations where the instructor and the patient do not (or cannot) meet face-to-face.

[0005] An object of the present invention is to provide more appropriate support for medical guidance. [Means for solving the problem]

[0006] In order to solve the above problem, a training support device according to one aspect of the present invention includes: a video call control means for controlling a video call between a terminal used by a medical instructor providing medical guidance to a patient and a terminal used by the patient, in order to remotely provide medical guidance to the patient; a voice recognition processing means for performing voice recognition processing on the voice transmitted and received during the video call; a recommendation information generating means for generating recommendation information that indicates the content of guidance recommended in the medical guidance; Equipped with The video call control means causes the terminal used by the instructor to output, in parallel with the progress of the video call, training support information based on the results of voice recognition by the voice recognition processing means and the recommendation information generated by the recommendation information generation means. [Effects of the Invention]

[0007] According to the present invention, it is possible to more appropriately support medical guidance. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram showing the system configuration of a training support system 1 according to an embodiment of the present invention. [Figure 2] FIG. 8 is a diagram showing the hardware configuration of an information processing device 800 that constitutes each device. [Figure 3] 2 is a block diagram showing the functional configuration of an instructor's terminal 10 and a patient's terminal 20. FIG. [Figure 4] FIG. 2 is a block diagram showing the functional configuration of the training support server 30. [Figure 5] FIG. 10 is a schematic diagram illustrating an example of a training support screen involving a video call. [Figure 6] 10A and 10B are schematic diagrams showing an example of transition of a training support screen. [Figure 7] 10 is a flowchart showing the flow of a training support process executed by the instructor's terminal 10. [Figure 8]10 is a flowchart showing the flow of an instruction support process executed by the patient terminal 20. [Figure 9] 10 is a flowchart showing the flow of training support processing executed by the training support server 30. [Figure 10] FIG. 10 is a schematic diagram illustrating an example of an approval reception screen. [Figure 11] 10 is a flowchart showing the flow of a data management process executed by the training support server 30. [Figure 12] 10 is a flowchart showing the flow of a reading process executed by the instructor-side terminal 10 or the patient-side terminal 20 and the training support server 30. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] [System Configuration] FIG. 1 is a diagram showing the system configuration of a training support system 1 according to one embodiment of the present invention. The training support system 1 in this embodiment is a system that supports medical training. For example, the training support system 1 is used to support medical training that is conducted remotely via video call, without the need for a medical professional to meet face-to-face with a patient receiving the training. Hereinafter, as an example for explanation, it is assumed that the medical professional is a pharmacist working at an insurance pharmacy. Also, as an example for explanation, it is assumed that the medical training is medication guidance provided at the time of dispensing. However, these are merely examples for explanation and are not intended to limit the scope of application of this embodiment. This embodiment can support various medical training, such as medical and pharmaceutical guidance, provided by various medical professionals, such as pharmacists, doctors, and nurses, at various medical facilities, such as insurance pharmacies, hospitals, and clinics.

[0011] 1, the training support system 1 includes an instructor terminal 10, a patient terminal 20, and a training support server 30. The instructor terminal 10, the patient terminal 20, the training support server 30, the pharmacy computer, and the external server are configured to be able to communicate with each other via a network 40. This network 40 is realized by, for example, the Internet or a LAN (Local Area Network) installed in an insurance pharmacy or the like.

[0012] The pharmacy computer is an information processing device such as a server computer that executes processes related to the operations of an insurance pharmacy, and has the function of a receipt computer that creates medical fee statements (receipt computer function) and the function of managing patients' medication histories (medication history management function).The external server is an information processing device such as a server computer that works in conjunction with the pharmacy computer, and has the function of a database server that stores electronic medication notebooks and drug information.

[0013] Although the diagram shows one of each device and one of each terminal, this is not limiting. For example, there may be multiple training support servers 30 and pharmacy computers, each corresponding to a plurality of insurance pharmacies. Additionally, there may be multiple instructor terminals 10, each corresponding to a medical professional who uses the terminal to provide medical training. Similarly, there may be multiple patient terminals 20, each corresponding to a patient who uses the terminal to receive medical training.

[0014] The instructor-side terminal 10 is used by a medical professional (here, a pharmacist) who provides medical guidance, and is configured as an information processing device such as a tablet terminal or a PC (Personal Computer). The instructor-side terminal 10 displays various information to support medical guidance when the pharmacist provides medication guidance to a patient, and accepts various information input by the pharmacist. For example, in the medical guidance support process described below, the instructor-side terminal 10 displays videos of the pharmacist and the patient and outputs the voices of the pharmacist and the patient via the medical guidance support server 30 to realize medication guidance between the pharmacist and the patient via a video call. In this case, the instructor-side terminal 10 further displays medical guidance support information based on information such as the results of voice recognition of the pharmacist and the patient, which indicates the progress of medication guidance, and recommended guidance content.

[0015] The patient terminal 20 is used by a patient receiving medical guidance, and, like the instructor terminal 10, is configured as an information processing device such as a tablet terminal or PC, or a smartphone carried by the patient. The patient terminal 20 displays various information to support medical guidance when a pharmacist provides medication guidance to a patient, and accepts various information input by the patient. For example, in the medical guidance support process described below, the patient terminal 20 displays videos of the pharmacist and patient and outputs audio of the pharmacist and patient via the medical guidance support server 30 in order to realize medication guidance between the pharmacist and patient via a video call. In this embodiment, it is assumed that the training support information is provided to the pharmacist, but the training support information may also be provided to the patient. In this case, the patient terminal 20 displays the training support information in the same way as the instructor terminal 10.

[0016] The training support server 30 cooperates with the instructor terminal 10 and the patient terminal 20 to support medical training, and is composed of an information processing device such as a server computer. To support medical training, the training support server 30 receives various data managed by a pharmacy computer installed in the insurance pharmacy (or by each pharmacy computer if there are multiple computers). The training support server 30 also manages the received various data in a group of databases (including a medication history database) similar to those of the pharmacy computers. In other words, the training support server 30 acquires some or all of the information managed by the pharmacy computer installed in the insurance pharmacy (or by each pharmacy computer if there are multiple computers) to realize its business support function.

[0017] The training support server 30 also executes "training support processing" using this business support function. In this training support processing, the training support server 30 controls a video call between an instructor-side terminal 10 used by a pharmacist and a patient-side terminal 20 used by a patient in order to remotely provide medical guidance to a patient. The training support server 30 also performs voice recognition processing on voices transmitted and received during the video call. Furthermore, the training support server 30 generates recommendation information that indicates guidance content recommended in pharmaceutical guidance. Furthermore, in parallel with the progress of the video call, the training support server 30 causes the instructor-side terminal 10 used by the pharmacist to output training support information based on the result of the voice recognition and the generated recommendation information. In this way, the training support server 30 remotely provides pharmaceutical guidance via video calls. This allows medical guidance to be provided even in situations where the instructor and the patient do not (or cannot) meet face-to-face, for example, in consideration of contact infections of viruses and bacteria. In this case, the training support server 30 not only provides medical guidance, but also provides the pharmacist with training support information based on information such as the results of voice recognition of the pharmacist and patient that indicates the progress of the guidance and recommended guidance content. Therefore, the training support server 30 can provide more appropriate support for medical training.

[0018] [Hardware configuration] Next, the hardware configuration of each device and each terminal included in the training support system 1 will be described. As described above, each device and each terminal included in the training support system 1 is configured by an information processing device such as a PC, a server computer, a tablet terminal, or a smartphone, and each device and terminal has the same basic configuration.

[0019] FIG. 2 is a diagram showing the hardware configuration of an information processing device 800 that constitutes each device and each terminal. As shown in Figure 2, the information processing device 800 that constitutes each device includes a CPU (Central Processing Unit) 811, a ROM (Read Only Memory) 812, a RAM (Random Access Memory) 813, a bus 814, an input unit 815, an output unit 816, a memory unit 817, a communication unit 818, a drive 819, and an imaging unit 820.

[0020] The CPU 811 executes various processes according to a program recorded in the ROM 812 or a program loaded from the storage unit 817 into the RAM 813 . The RAM 813 also stores data and the like necessary for the CPU 811 to execute various processes.

[0021] The CPU 811, ROM 812, and RAM 813 are connected to one another via a bus 814. To the bus 814, an input unit 815, an output unit 816, a storage unit 817, a communication unit 818, a drive 819, and an imaging unit 820 are connected.

[0022] The input unit 815 is configured with various buttons and the like, and inputs various information in response to instruction operations. The input unit 815 also includes a microphone for collecting voice. The output unit 816 is composed of a display, a speaker, etc., and outputs images and sounds. The storage unit 817 is configured with a hard disk or a DRAM (Dynamic Random Access Memory), etc., and stores various data managed by each server. The communication unit 818 controls communication with other devices via the network 40 .

[0023] Removable media 831, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, is appropriately loaded into the drive 819. A program read from the removable media 831 by the drive 819 is installed in the storage unit 817 as needed.

[0024] The imaging unit 820 is configured by an imaging device equipped with a lens, an imaging element, etc., and captures a digital image of a subject.

[0025] When the information processing device 800 is configured as the training support server 30, it is also possible to omit the imaging unit 820. When the information processing device 800 is configured as a tablet terminal, it is also possible to configure the input unit 815 using a touch sensor and place it over the display of the output unit 816, thereby providing a touch panel.

[0026] [Functional configuration] Next, the functional configuration of each device and each terminal in the training support system 1 will be described.

[0027] [Functional Configuration of Instructor's Terminal 10] Fig. 3(A) is a block diagram showing the functional configuration of the instructor's terminal 10. Fig. 3(B) is a block diagram showing the functional configuration of the patient's terminal 20, which will be described later. As shown in FIG. 3(A), in the CPU 811 of the instructor's terminal 10, a user interface display control section (UI display control section) 151, a video call control section 152, and a recommended information acquisition section 153 function.

[0028] The UI display control unit 151 controls the display of various input / output screens (hereinafter referred to as "UI screens") in various processes described below, based on information for displaying a user interface screen (hereinafter referred to as "UI information") received from the training support server 30. For example, in the training support process, the UI display control unit 151 outputs and displays, on a display included in the output unit 810, a screen including a display area for displaying video of a video call to realize remote medication guidance, and a display area for displaying training support information to support pharmacists. In addition to these displays, the UI display control unit 151 outputs audio included in the video from a speaker included in the output unit 810. Furthermore, the UI display control unit 151 transmits to the training support server 30 various pieces of information input on the UI screen and the contents of the input operation instructions.

[0029] The video call control unit 152 cooperates with video call control units in other devices (here, the video call control unit 252 provided in the patient's terminal 20 and the video call control unit 352 provided in the training support server 30) to realize a video call between the instructor's terminal 10 and the patient's terminal 20 (i.e., a video call between the pharmacist and the patient).

[0030] To this end, the video call control unit 152 collects the pharmacist's voice using a microphone included in the input unit 815, and captures an image of the pharmacist using the imaging unit 820. The video call control unit 152 also converts the pharmacist's voice and image into a video with audio that conforms to a predetermined data format (hereinafter referred to as "pharmacist's video"). The video call control unit 152 then transmits the pharmacist's video to the training support server 30. Meanwhile, the video call control unit 252 included in the patient-side terminal 20 performs similar processing on the patient, thereby converting the patient's voice and image into a video that conforms to a predetermined data format (hereinafter referred to as "patient's video"). The video call control unit 252 then transmits the patient's video to the training support server 30.

[0031] Then, the video call control unit 352 included in the training support server 30 generates a video with audio for realizing a video call (hereinafter referred to as "video call video") based on the video of the pharmacist and the video of the patient, and transmits it to each of the instructor-side terminal 10 and the patient-side terminal 20. The video call video is a video that, for example, displays an image of one of the pharmacist and the patient in a part of the display area when displaying the video call video, and displays an image of the other in another part of the display area, and outputs the audio of the pharmacist and the patient. The UI display control unit 151 of the instructor's terminal 10 and the UI display control unit 251 of the patient's terminal 20 receive this video call video and output the video call video from the output unit 816 of their own terminal (i.e., displaying images and outputting audio included in the video call video, etc.).

[0032] The video call is realized by such control, and in parallel with the progress of the video call, the training support server 30 performs voice recognition processing on the pharmacist's voice and the patient's voice, and transmits the voice recognition result to the instructor-side terminal 10. Furthermore, in parallel with the progress of the video call, the training support server 30 generates recommendation information, which is information indicating recommended guidance content for medication guidance, and transmits this recommendation information to the instructor-side terminal 10. The UI display control unit 151 receives the voice recognition result and the recommendation information, and outputs them from the output unit 816 (i.e., displays the voice recognition result and the recommended information, etc.) in the same way as the video call video. The output voice recognition result and the recommendation information are used by the pharmacist as training support information, which is information for supporting the pharmacist's guidance.

[0033] The recommended information acquiring unit 153 determines whether recommended information has been received from the training support server 30. Videos in video calls transmitted by the training support server 30 and their voice recognition results change over time. On the other hand, the recommended information transmitted by the training support server 30 is updated at a predetermined timing and transmitted periodically. Therefore, the recommended information acquiring unit 153 determines whether recommended information has been received from the training support server 30, and when it determines that updated recommended information has been newly received, controls the UI display control unit 151 to newly output the recommended information.

[0034] [Functional Configuration of Patient Terminal 20] FIG. 3B is a block diagram showing the functional configuration of the patient terminal 20. As shown in FIG. As shown in FIG. 3(B), in the CPU 811 of the patient-side terminal 20, a user interface display control unit (UI display control unit) 251 and a video call control unit 252 function. Here, the UI display control unit 251 has the same function as the UI display control unit 151 of the instructor-side terminal 10 described above. Moreover, the video call control unit 252 has the same function as the video call control unit 152 of the instructor-side terminal 10 described above. Moreover, the method of cooperation between the video call control units (here, the video call control unit 152, the video call control unit 252, and the video call control unit 352) is as described above in the explanation of the video call control unit 152. Therefore, redundant explanation will be omitted here.

[0035] [Functional configuration of the training support server 30] FIG. 4 is a block diagram showing the functional configuration of the training support server 30. As shown in FIG. As shown in FIG. 4, the CPU 811 of the training support server 30 functions as a user interface information generating unit (UI information generating unit) 351, a video call control unit 352, a voice recognition processing unit 353, a recommended information generating unit 354, a medication history management unit 355, and a database management unit (DB management unit) 356.

[0036] Also, as shown in FIG. 4, in the memory unit 817 of the medical assistance server 30, a patient attribute database (patient attribute DB) 371, a prescription database (prescription DB) 372, a medication history database (medication history DB) 373, a voice recognition dictionary database (voice recognition dictionary DB) 374, a voice recognition result database (voice recognition result DB) 375, and a medication instruction database (medication instruction DB) 376 are formed.

[0037] Patient attribute data such as patient address, name, date of birth, age, sex, insurance card number, and emergency contact information is stored in the patient attribute DB 371 in association with information identifying each patient. These patient attributes are composed of information acquired by pharmacists in conversations with patients and information provided by patients in response to questionnaires, etc., and include, for example, the patient's hobbies, job, family composition, favorite foods, etc.

[0038] Prescription DB372 stores data on prescriptions issued to patients (such as the name of the insured medical institution and physician that prescribed the prescription, the prescription date, and prescription contents) in association with information identifying each patient and the date the prescription was brought in.

[0039] The medication history DB 373 stores data on the history (medication history) of drugs prescribed to patients (prescription history, information on medical history including complications, whether or not the patient has visited other departments, etc.) in association with information identifying each patient and information identifying each pharmacist who prescribed the drug. In addition, the medication history also stores data on the history of medication instructions given to patients by pharmacists (medication instruction statements, etc.).

[0040] The voice recognition dictionary DB 374 stores various dictionaries used to perform voice recognition processing on the voice of a pharmacist included in a video of the pharmacist and the voice of a patient included in a video of the patient. This dictionary includes, for example, an acoustic model for extracting voice characteristics, a pronunciation dictionary for identifying spoken words, and a language model. This dictionary may be a general-purpose dictionary, or may be a dictionary specialized for the medical field or the like that is suited to the application of this embodiment. Furthermore, the dictionary may be manually created based on statistical information or the like, or may be created by machine learning using a technique such as deep learning.

[0041] The voice recognition result DB 375 stores the voice recognition results obtained by the voice recognition processing unit 353 in association with information identifying each patient and information identifying each pharmacist who prescribed the medicine.

[0042] The medication instruction DB 376 generally stores medication instruction contents when a pharmacist provides medication instruction in association with the prescribed drug and characteristics of the patient to whom the drug is prescribed (e.g., the patient's age, sex, chronic illness, medical history, etc.). The medication instruction contents include, for example, the effects and side effects of the dispensed drug and drug interactions (contraindications for combined use) that the pharmacist (especially an experienced pharmacist) should explain to the patient to whom the pharmacist provides medication instruction, and interview contents (listening items) that the pharmacist should ask the patient as part of the medication instruction. These medication instruction contents may be expressed in the form of medication instruction statements using technical terms that the pharmacist can understand, or in a form that is easier for the patient to understand (e.g., medication instruction statements using simple language without technical terms, or questions using colloquial language). Furthermore, the medication instruction contents may be manually created based on statistical information, etc., like the dictionary stored in the voice recognition dictionary DB 374, or may be created by machine learning using a technique such as deep learning.

[0043] The UI information generating unit 351 generates UI information for the instructor-side terminal 10 and the patient-side terminal 20 to display a UI screen, and transmits the generated UI information to the instructor-side terminal 10 and the patient-side terminal 20. In this case, the UI information generating unit 351 generates, as UI information, a frame format for displaying the UI screen and substantial content to be inserted into the format. In this embodiment, the substantial content to be inserted into the format includes, for example, in the training support processing, training support information such as a video call video for realizing remote medication guidance, and speech recognition results and recommended information for supporting pharmacists.

[0044] In addition, in response to transmitting UI information to the instructor-side terminal 10 and the patient-side terminal 20, the UI information generation unit 351 receives various information transmitted from the instructor-side terminal 10 and the patient-side terminal 20 and the contents of input operation instructions.

[0045] The video call control unit 352 cooperates with video call control units in other devices (here, the video call control unit 152 provided in the instructor-side terminal 10 and the video call control unit 252 provided in the patient-side terminal 20) to realize a video call between the instructor-side terminal 10 and the patient-side terminal 20 (i.e., a video call between the pharmacist and the patient). Specifically, as described above in the explanation of the video call control unit 152, the video call control unit 352 generates a video call video based on the video of the pharmacist received from the instructor-side terminal 10 and the video of the patient received from the patient-side terminal 20, and transmits the generated video call video to each of the instructor-side terminal 10 and the patient-side terminal 20.

[0046] The voice recognition processing unit 353 performs voice recognition processing on the voice of the pharmacist included in the video of the pharmacist received from the instructor-side terminal 10 and the voice of the patient included in the video of the patient received from the patient-side terminal 20. In this case, the voice recognition processing unit 353 converts the voices of the pharmacist and the patient into text by referring to, for example, various dictionaries stored in the voice recognition dictionary DB 374. Then, the voice recognition processing unit 353 stores the voice recognition result (here, the textualized voices of the pharmacist and the patient) in the voice recognition result DB 375. In addition, the UI information generation unit 351 transmits this voice recognition result to the instructor-side terminal 10.

[0047] The recommended information generating unit 354 generates recommended information, which is information indicating guidance content recommended in medication instruction, at a predetermined generation timing in parallel with the progress of the video call realized by the video call control unit 352. Here, the predetermined generation timing includes both the timing when the video call starts and the timing according to the progress status of the video call (i.e., the progress status of medication instruction).

[0048] First, when generating recommended information at the start of a video call, the recommended information generation unit 354 identifies the patient with whom the video call will be made this time based on information for identifying each patient included in the video call start request received from the patient-side terminal 20. Next, the recommended information generation unit 354 identifies the prescription prescribed for the identified patient, the patient's medical history, and the attributes of the patient by referring to each database, such as the patient attribute DB 371, prescription DB 372, and medication history DB 373. Furthermore, the recommended information generation unit 354 extracts medication instruction content (e.g., medication instruction statement) corresponding to the identified various information from the medication instruction DB 376. Then, the recommended information generation unit 354 generates recommended information by including the extracted medication instruction content in the recommended information as instruction content recommended for medication instruction. As described above in the explanation of the medication instruction DB376, the content of the medication instructions here includes, for example, the effects and side effects of the dispensed medication and drug interactions (contraindications for combined use) that a pharmacist (especially an experienced pharmacist) should explain to the patient when providing medication instructions, as well as the content of interviews (listening items) that should be conducted with the patient.

[0049] Furthermore, the recommended information generating unit 354 changes and generates the content of the recommended information at a timing corresponding to the progress of the video call (i.e., the progress of the medication instruction). That is, the recommended information generating unit 354 updates the recommended information in parallel with the progress of the video call. This allows the recommended information generating unit 354 to appropriately generate recommended information corresponding to the progress of the video call. To this end, the recommended information generating unit 354 compares the speech recognition result of the speech recognition processing unit 353 with the medication instruction content included in the generated recommended information to determine whether medication instruction corresponding to any of the medication instruction content has been provided by the pharmacist. For example, the recommended information generating unit 354 makes this determination based on criteria such as the similarity between the speech recognition result and the medication instruction content as text, or whether the speech recognition result contains words related to the medication instruction content.

[0050] Then, the recommendation information generating unit 354 changes and generates the content of the recommendation information when it detects, through this determination, that medication instruction corresponding to any of the medication instruction contents has been provided by the pharmacist. In this case, the recommendation information generating unit 354 adds, for example, information indicating that this medication instruction has been provided to the recommendation information. Alternatively, the recommendation information generating unit 354 may, for example, delete this medication instruction content from the recommendation information. Alternatively, the recommendation information generating unit 354 may, for example, add new medication instruction content to the recommendation information. Alternatively, the recommendation information generating unit 354 may combine these deletions, additions, and other changes, or make stepwise changes. Specific examples of these changes will be described later with reference to FIGS. 5 and 6. Each time the recommendation information generated in this way is generated, it is transmitted to the instructor's terminal 10 by the UI information generating unit 351. Then, the transmitted recommendation information is output (for example, displayed) in the instructor's terminal 10 together with the video call video and the voice recognition result.

[0051] Specific examples of such displays and transitions on the instructor's terminal 10 will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a schematic diagram showing an example of an instruction support screen accompanying a video call, which is displayed on the instructor's terminal 10 during the instruction support process. Fig. 6 is a schematic diagram showing an example of transitions of the instruction support screen, which is updated at a timing according to the progress of the video call (i.e., the progress of medication instruction).

[0052] As shown in FIG. 5, the training support screen 50 includes a pharmacist image display area 51, a patient image display area 52, a patient attribute display area 53, a recommended information display area 54, and a voice recognition result display area 55 as display areas.

[0053] The pharmacist image display area 51 displays an image of the pharmacist (i.e., a video) included in the video call video generated by the video call control unit 352. The patient image display area 52 displays an image of the patient (i.e., a video) included in the video call video generated by the video call control unit 352. During the medical assistance process, the patient-side terminal 20 also provides display areas similar to the pharmacist image display area 51 and the patient image display area 52 for display. The patient-side terminal 20 may display the pharmacist image display area 51 in a larger size while the patient image display area 52 in a smaller size, for example, so that the pharmacist's facial expression and the like can be easily seen. The instructor-side terminal 10 and the patient-side terminal 20 also output the voices of the pharmacist and the patient included in the video call video together with such displays. This allows the pharmacist and the patient to provide remote medication instruction even when they do not (or cannot) meet face-to-face.

[0054] In the patient attribute display area 53, information indicating the attributes of the patient stored in the patient attribute DB 371 is displayed. In the recommended information display area 54, the recommended information generated by the recommended information generating unit 354 is displayed. In the voice recognition result display area 55, the voice recognition result by the voice recognition processing unit 353 is displayed.

[0055] The displayed information on patient attributes, recommended information, and voice recognition results are used by pharmacists as instruction support information to assist pharmacists in providing instruction. This enables remote medication instruction and more appropriate support for medical instruction.

[0056] 6, the recommended information and the voice recognition result are updated at a timing corresponding to the progress of the video call (i.e., the progress of the medication instruction). In FIG. 6, the recommended information display area 54 and the voice recognition result display area 55 on the instruction support screen 50 are particularly illustrated.

[0057] As an example, assume that at the start of a video call, medication instruction content 54-1 such as "(1) Please confirm whether you have taken all of the medication A prescribed last time" and medication instruction content such as "(2) Please explain the effects of medication B being prescribed this time" are displayed as recommended information, as shown in FIG. 6(A). Then, assume that, as shown as speech recognition result 55-1, a pharmacist who references the recommended information provides medication instruction such as "Have you taken all of the medication A prescribed last time?" In this case, the recommended information generation unit 354 compares medication instruction content 54-1 with speech recognition result 55-1 to determine that medication instruction corresponding to medication instruction content 54-1 has been provided by the pharmacist. Upon this detection, the recommended information generation unit 354 then changes and generates the recommended information.

[0058] Then, as shown in FIG. 6(B), the text "Instruction Completed" is added to the medication instruction content 54-1, or the medication instruction content 54-1 is underlined, as information indicating that the medication instruction has been completed. In this case, the voice recognition processing unit 353 may also change the manner in which the voice recognition result 55-1 is displayed. For example, the manner in which the voice recognition result 55-1 is displayed may be changed to be displayed in a different color, or hatching may be added. Based on such changes to the recommended information, the pharmacist can confirm that he or she has provided the recommended medication instruction content. Furthermore, after such display, the recommended information generation unit 354 further changes and generates the content of the recommended information.

[0059] Then, as shown in FIG. 6(C), medication instruction content 54-1 is deleted, and instead, new medication instruction content 54-2, "(3) Please check whether any side effects of drug A have occurred," is added to the recommendation information. Then, as shown as voice recognition result 55-2, it is assumed that a pharmacist who references the recommendation information provides guidance such as, "Have you ever felt drowsy or fatigued after taking drug A?" In this case, the recommendation information generation unit 354 detects that medication instruction corresponding to medication instruction content 54-2 has been provided by the pharmacist by comparing medication instruction content 54-2 with voice recognition result 55-2. Upon this detection, the recommendation information generation unit 354 changes the content of the recommendation information and generates it. For example, the same changes are made to medication instruction content 54-2 and voice recognition result 55-2 as were made to medication instruction content 54-1 and voice recognition result 55-1 in FIG. 6(B).

[0060] In this way, by changing the content of recommended information, etc. at a timing appropriate to the progress of the video call (i.e., the progress of the medication instructions) until all medication instructions have been given, it becomes possible to provide more appropriate support for medical instructions.

[0061] Returning to FIG. 4 , the medication history management unit 355 creates a draft medication instruction statement to be registered in the medication history of the patient for whom medication instruction has been given, based on the voice recognition results obtained by the voice recognition processing unit 353. As described above, the voice recognition results of the pharmacist's and patient's voices during medication instruction are stored in the voice recognition result DB 375. Based on the stored voice recognition results, the medication history management unit 355 identifies the medication instruction content actually given during medication instruction via video call. Then, for example, the medication history management unit 355 extracts a medication instruction statement corresponding to the identified medication instruction content from the medication instruction DB 376, and creates the extracted medication instruction statement as a draft medication instruction statement. The medication history management unit 355 then registers the created draft medication instruction statement in the medication history DB 373. This allows the draft medication instruction statement to be created without requiring the pharmacist to perform cumbersome character input operations, etc.

[0062] The DB management unit 356 manages the synchronization of data in various databases managed by the pharmacy computer or the external server with data in various databases provided in the training support server 30. For example, the DB management unit 356 transmits updated data in the various databases managed by the training support server 30 to the pharmacy computer or the external server at a preset time (for example, 3:00 a.m., when the servers are not frequently used) or when the various databases are updated, and also receives data in the various databases updated in the pharmacy computer or the external server from the pharmacy computer or the external server, and updates the various databases provided in the training support server 30. As a result, the contents of the data in the various databases provided in the training support server 30 are updated to the latest contents as appropriate.

[0063] [Operation] Next, the operation of each device and each terminal included in the training support system 1 during training support processing will be described.

[0064] [Operation of instructor's terminal 10] FIG. 7 is a flowchart showing the flow of the training support process executed by the instructor's terminal 10. The training support process in the instructor's terminal 10 is started in response to the instructor's terminal 10 accepting an instruction to start a video call.

[0065] In step S11, the video call control unit 152 transmits a request to start a video call with the patient-side terminal 20 to the training support server 30. This call start request includes information for identifying the pharmacist who is using the instructor-side terminal 10. The video call control unit 152 also starts transmitting a video of the pharmacist to the training support server 30.

[0066] In step S12, the video call control unit 152 determines whether a video call has started. If a video call has not been started, the determination in step S12 is NO, and the process repeats the determination in step S12. On the other hand, if a video call has been started, the determination in step S12 is YES, and the process proceeds to step S13.

[0067] In step S13, the video call control unit 152 receives the video call moving image from the training support server 30.

[0068] In step S14, the UI display control unit 151 displays the video received in step S13.

[0069] In step S15, the video call control unit 152 receives the voice recognition result from the training support server 30. In step S16, the UI display control unit 151 displays the voice recognition result received in step S15.

[0070] In step S17, the recommended information acquisition unit 153 determines whether or not recommended information has been received. If no recommendation information has been received, the determination in step S17 is NO, and the process proceeds to step S16. On the other hand, if recommendation information has been received, the determination in step S17 is YES, and the process proceeds to step S18.

[0071] In step S18, the UI display control unit 151 displays the recommended information whose reception was confirmed in step S17.

[0072] In step S19, the video call control unit 152 determines whether the video call has ended. Here, the video call ends when, for example, an instruction to end the video call is received at the instructor-side terminal 10 or the patient-side terminal 20. The terminal that has received the instruction to end the video call transmits a request to end the video call to the training support server 30. If the video call has not ended, the determination in step S19 is NO, and the process returns to step S13 and is repeated. On the other hand, if the video call has ended, the determination in step S19 is YES, and the training support process in the instructor's terminal 10 ends.

[0073] [Operation of patient side terminal 20] FIG. 8 is a flowchart showing the flow of the training support process executed by the patient terminal 20. The training support process in the patient-side terminal 20 is started in response to the patient-side terminal 20 receiving an instruction to start a video call.

[0074] In step S21, the video call control unit 252 transmits a request to start a video call with the instructor-side terminal 10 to the training support server 30. This call start request includes information for identifying the patient who is using the patient-side terminal 20. The video call control unit 252 also starts transmitting a video of the patient to the training support server 30.

[0075] In step S22, the video call control unit 252 determines whether or not a video call has started. If a video call has not been started, the determination in step S22 is NO, and the process repeats the determination in step S22. On the other hand, if a video call has been started, the result of determination in step S22 is YES, and the process proceeds to step S23.

[0076] In step S23, the video call control unit 252 receives the video call moving image from the training support server 30. In step S24, the UI display control unit 251 displays the video received in step S23.

[0077] In step S25, the video call control unit 252 determines whether the video call has ended. Here, the video call ends when, for example, an instruction to end the video call is received at the instructor-side terminal 10 or the patient-side terminal 20. The terminal that has received the instruction to end the video call transmits a request to end the video call to the training support server 30. If the video call has not ended, the determination in step S25 is NO, and the process returns to step S23 and is repeated. On the other hand, if the video call has ended, the determination in step S25 is YES, and the training support process in the patient-side terminal 20 ends.

[0078] [Operation of the training support server 30] FIG. 9 is a flowchart showing the flow of the training support process executed by the training support server 30. The training support process in the training support server 30 starts when the power of the training support server 30 is turned on.

[0079] In step S31, the video call control unit 352 determines whether a video call has been started. Here, a video call is started, for example, when a request to start a video call is received from each of the instructor-side terminal 10 and the patient-side terminal 20. Note that these video call start requests contain information for identifying the pharmacist and information for identifying the patient. Therefore, the training support server 30 can identify the pharmacist and the patient based on this identifying information and then perform subsequent processing. If a video call has not been started, the determination in step S31 is NO, and the process proceeds to step S31. On the other hand, if a video call has been started, the determination in step S31 is YES, and the process proceeds to step S32.

[0080] In step S32, the video call control unit 352 receives the video of the pharmacist from the instructor-side terminal 10 and also receives the video of the patient from the patient-side terminal 20. Then, the video call control unit 352 generates a video call video based on the received videos of the pharmacist and the patient.

[0081] In step S33, the UI information generating unit 351 transmits the video call moving image generated in step S32 to the instructor-side terminal 10 and the patient-side terminal 20, respectively.

[0082] In step S34, the voice recognition processing unit 353 performs voice recognition processing on the voices of the pharmacist and the patient received in step S32. In step S35, the video call control unit 352 transmits the voice recognition result obtained in step S32 to the instructor's terminal 10.

[0083] In step S36, the recommended information generation unit 354 determines whether or not it is the timing to generate recommended information (as described above, this includes both the timing when the video call starts and the timing based on the progress of the video call (i.e., the progress of medication instructions)). If it is not time to generate recommendation information, the determination in step S36 is NO, and the process proceeds to step S39. On the other hand, if it is time to generate recommendation information, the determination in step S36 is YES, and the process proceeds to step S37. In step S37, the recommendation information generating unit 354 generates recommendation information.

[0084] In step S38, the recommendation information generating section 354 transmits the recommendation information generated in step S37 to the instructor's terminal 10. In step S39, the medication history management unit 355 creates a draft of a medication instruction message based on the speech recognition result obtained in step S32.

[0085] In step S40, the video call control unit 352 determines whether the video call has ended. Here, the video call ends when, for example, a request to end the video call is received from either the instructor-side terminal 10 or the patient-side terminal 20. If the video call has not ended, the determination in step S40 is NO, and the process returns to step S32 and is repeated. On the other hand, if the video call has ended, the determination in step S40 is YES, and the process proceeds to step S41.

[0086] In step S41, the medication history management section 355 registers the proposed medication instruction in the medication history, whereby the training support process in the training support server 30 ends.

[0087] According to the above-described teaching support process, pharmaceutical teaching is provided remotely via video calls. This allows medical teaching to be provided even in situations where the instructor and the patient do not (or cannot) meet face-to-face, for example, in consideration of contact infections such as viruses and bacteria. In this case, the teaching support process not only provides medical teaching, but also provides the pharmacist with teaching support information based on information such as the results of speech recognition of the pharmacist and patient that indicates the progress of the teaching, and recommended teaching content. Therefore, the training support process makes it possible to provide more appropriate support for medical training.

[0088] [Variations] Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take on various other embodiments and can undergo various modifications such as omissions and substitutions without departing from the spirit of the present invention. In such cases, these embodiments and their modifications are included in the scope and spirit of the invention described in this specification, etc., and are also included in the scope of the invention described in the claims and their equivalents. As an example, the embodiment of the present invention described above may be modified as follows: In addition, the following modifications may be combined. [Variation 1] In the above-described embodiment, as an example for explanation, it is assumed that in the training support system 1, a pharmacist uses the instructor-side terminal 10 to provide medication instructions, etc., to a patient using the patient-side terminal 20 at the time of dispensing. However, the training support system 1 can also support various medical guidance, such as medical guidance and pharmaceutical guidance, provided by various medical professionals, such as pharmacists, doctors, and nurses, in various medical facilities, such as insurance pharmacies, hospitals, and clinics. That is, in the training support system 1, the instructor who provides medical guidance using the instructor-side terminal 10 is not limited to a pharmacist, but may be another medical professional.

[0089] For example, when a doctor working at a hospital provides online medical care, the doctor may use the instructor terminal 10 to provide medical guidance. In this case, a doctor's guidance database is provided instead of the medication instruction DB 376. This doctor's guidance database stores the medical guidance content that a doctor typically provides when providing online medical care and the content to be heard from patients as part of this medical guidance (interview content) in association with the medical department of the instructing doctor and the characteristics of the patient receiving the guidance (e.g., the patient's age, gender, chronic illness, medical history, etc.). In addition, in cases where the initial consultation is conducted face-to-face and the follow-up consultation is conducted online, the results of the initial face-to-face consultation may also be stored in association with the medical guidance content.

[0090] The medical guidance content may be, for example, information that a doctor (especially a veteran doctor in charge of that department) should ask a patient about when providing medical guidance (e.g., information necessary for diagnosis, such as the patient's symptoms and chronic illnesses) or information that should be explained to the patient (e.g., treatment methods for a suspected illness and the advantages and disadvantages of each treatment method). The medical guidance content may be expressed in a format using technical terms that the doctor can understand, or in a format that is easier for the patient to understand (e.g., a format using simple language without technical terms or questions written in colloquial language). Similarly to the medication guidance DB 376, the medical guidance content may be created manually based on statistical information or may be created by machine learning using techniques such as deep learning.

[0091] When the video call begins, the recommendation information generator 354 extracts medical instruction content from the doctor's instruction database in a manner similar to the method for extracting medication instruction content from the medication instruction DB 376 in the embodiment described above. The recommendation information generator 354 then generates recommendation information by including the extracted medical instruction content in the recommendation information as instruction content recommended for medical instruction. This recommendation information is utilized in a manner similar to the embodiment described above, such as by displaying it as support information to the doctor.

[0092] In this case, the medication history management unit 355 registers a draft of the medical instruction content in the medication history (or an electronic medical record, electronic questionnaire, or the like that replaces the medication history) in the same manner as the method for registering a draft of the medication instruction content in the medication history in the above-described embodiment, based on the voice recognition result by the voice recognition processing unit 353. This makes it possible to create a draft of the medical instruction content without requiring a doctor to perform cumbersome character input operations, etc.

[0093] In this way, by creating an instruction database as appropriate for each medical professional who will be the instructor, it is possible to realize support for medical instruction for other medical professionals, similar to the above-mentioned support for medication instruction for pharmacists. For example, in addition to online medical consultations by doctors, other medical professionals such as nurses and physical therapists can be supported in situations where they remotely provide some kind of medical instruction to patients. In other words, the instruction support system 1 is capable of supporting various medical instructions, such as medical instruction and pharmaceutical instruction, provided by various medical professionals such as pharmacists, doctors, and nurses, regardless of the medical professional who is the instructor.

[0094] [Variation 2] In the above-described embodiment, the medication history management unit 355 prepared a draft medication instruction statement. This draft medication instruction statement automatically prepared by a computer is merely a draft, and it may not be desirable to register this as a formal medication instruction statement in the patient's medication history. Therefore, a process may be added in which, after the medication instruction statement is completed, a pharmacist checks the draft medication instruction statement prepared by the medication history management unit 355, and only those statements approved by the pharmacist as a result of the check are registered as formal medication instructions in the patient's medication history.

[0095] In this case, for example, an approval acceptance screen is generated in the training support server 30 and displayed on the instructor's terminal 10. Fig. 10 is a schematic diagram showing an example of the approval acceptance screen. As shown in FIG. 10, the approval reception screen 60 includes a bibliographical information display area 61, a medication instruction draft display area 62, a voice result display area 63, and an approval button display area 64 as display areas.

[0096] Bibliographic items related to the medication instruction for which the medication history is to be registered are displayed in the bibliographic item display area 61. For example, the patient's name, the date and time of the medication instruction, the prescription number, the pharmacist's name, etc. are displayed.

[0097] A list of medication instruction statements indicating the medication instruction content when the pharmacist provides medication instruction and check boxes for the pharmacist to select medication instruction statements to be approved are displayed in the medication instruction statement draft display area 62. The medication instruction statements included in this list correspond to the list of medication instruction statement drafts created by the medication history management unit 355 in the instruction support process described above.

[0098] The voice result display area 63 displays the voice recognition results of the voices of the pharmacist and the patient during medication instruction. In the approval button display area 64, an operation button for approving all of the medication instruction drafts selected by the check boxes displayed in the medication instruction draft display area 62 is displayed.

[0099] The pharmacist selects one of the proposed medication instructions displayed in the medication instruction suggestion display area 62 to be registered in the patient's medical history as the official medication instruction, and presses the operation button displayed in the approval button display area 64 to approve the registration. In addition, the pharmacist may refer to the voice recognition results of the pharmacist's and patient's voices during medication instruction, which are displayed in the voice result display area 63, as needed, for reference when selecting a medication instruction suggestion. In this way, being able to refer to the voice recognition results, etc., can help the pharmacist approve an appropriate medication instruction suggestion.

[0100] Then, the medication history management unit 355 registers, as official medication instruction sentences, only draft medication instruction sentences for which the pharmacist's approval operation has been received at the instructor-side terminal 10, in the medication history DB 373. On the other hand, the medication history management unit 355 does not register, as official medication instruction sentences, draft medication instruction sentences for which the pharmacist's approval operation has not been received at the instructor-side terminal 10. This makes it possible to register, in the medication history, only appropriate medication instruction sentences that have been confirmed to have no problems by the pharmacist.

[0101] [Variation 3] In the above-described embodiment, a training support system 1 is constructed using a plurality of clients, namely, an instructor's terminal 10 and a patient's terminal 20, and a server, namely, a training support server 30, and medication instructions are given via video calls between the instructor's terminal 10 and the patient's terminal 20 via the training support server 30.

[0102] Without being limited to this, for example, the functions of the training support server 30 may be distributed and implemented on more servers (e.g., cloud servers), or the functions of the training support server 30 and the pharmacy computer may be implemented together in a single information processing device 800.

[0103] Alternatively, for example, a function such as a video call by the instructor's terminal 10 and a function to support medication instruction by the medical instruction support server 30 may be implemented together in a single information processing device 800. In this case, a function such as a video call by the patient's terminal 20 and a function to support medication instruction by the medical instruction support server 30 may be realized in another single information processing device 800, and these information processing devices 800 may communicate with each other in a peer-to-peer manner.

[0104] [Variation 4] In the above-described embodiment, the training support server 30 performs the training support process for training support involving video calls in cooperation with the instructor-side terminal 10 and the patient-side terminal 20. However, the present invention is not limited to this, and the training support process may be performed in further cooperation with a function highly related to training support involving video calls.

[0105] For example, the medication support process may be performed in cooperation with functions provided in a general pharmacy computer, such as a medication history management function and a medication history support function. In this case, the medication history management function is a function for managing various information, such as not only medication instruction statements but also prescriptions, patient questionnaire results, patient interview content, patient medical interview results, pharmacist impressions of patients, dispensed medications, medical fee statements, and medication notebooks, for pharmacists and pharmacy staff, and providing a user interface for this purpose. Furthermore, the medication support function in cooperation with the medication history is a function for suggesting to pharmacists the content of interviews and medical interviews and medication instruction content, outputting medication histories in a way that is easy for patients to understand, and providing a user interface for these purposes.

[0106] When linked with these functions, the video call control unit 352 starts a video call, for example, when an operation from a pharmacist is received on a display screen related to one or both of these functions.

[0107] This allows the instruction support server 30 to perform processing in conjunction with functions that are highly relevant to the pharmaceutical instruction that is the target of instruction support (i.e., functions for managing medication history and functions for supporting medication instruction in conjunction with medication history).

[0108] [Variation 5] In the above-described embodiment, the training support server 30 managed, through the DB management unit 356, synchronizing the data of various databases managed by the pharmacy computer or external server with the data of various databases provided in the training support server 30.

[0109] The training support server 30 may also manage data in various databases in other ways. For example, data showing the results of speech recognition by the speech recognition processing unit 353 (here, text data obtained by converting the speech of a pharmacist or patient in a video call into text) may be classified, and text data showing the results of speech recognition may be output to an output destination corresponding to the classified results. The management process of text data showing the results of speech recognition accompanied by such classification will be described with reference to FIG. 11.

[0110] [Operation of the training support server 30 in Modification 5] FIG. 11 is a flowchart showing the flow of the data management process executed by the training support server 30. The data management process in the training support server 30 is started in response to reception of an instruction to start the data management process in the training support server 30. Alternatively, the data management process may be started as a batch process at a preset time (for example, 3:00 a.m., when the frequency of use of each server is low). In this way, by distributing the time periods during which the data management process is performed at a preset time as a batch process rather than simultaneously with the video call, the load of the calculation processes occurring simultaneously on the training support server 30 can be reduced.

[0111] In step S71, the DB management unit 356 acquires text data indicating the result of speech recognition by the speech recognition processing unit 353 from the speech recognition result DB 375. This text data is unclassified text data that has not yet been subjected to this process.

[0112] In step S72, the voice recognition processing unit 353 extracts keywords from the text data acquired in step S71. These keywords are, for example, keywords that are considered important in medication instructions, such as "shortness of breath" or "coughing" spoken by a patient or a pharmacist. Such keyword extraction can be achieved by registering the words to be used as keywords in the voice recognition dictionary DB 374.

[0113] In step S73, the voice recognition processing unit 353 classifies each of the keywords extracted in step S72 into a category corresponding to that keyword. For example, if a first keyword is extracted, the first keyword is classified into one of categories such as "medication status," "side effects," "effects and effects," "precautions," and "messages to contact a doctor." This classification can be realized by registering the keyword and the category in association with each other in the voice recognition dictionary DB 374.

[0114] In step S74, DB management unit 356 outputs each keyword classified in step S73 to an output destination corresponding to the classification result. For example, if the classified category is related to a patient's attributes, such as special notes of the patient, the keyword is output to patient attribute DB 371 and stored in patient attribute DB 371. Similarly, if the classified category is related to a medication history, such as medication status, the keyword is output to medication history DB 373 and stored in medication history DB 373.

[0115] In this case, the output destination is not limited to each database formed in the training support server 30. For example, the output destination may be another server computer, etc. In this case, for example, if the classified category is a category related to treatment and diagnosis such as side effects, the keyword is output by transmitting it to an external server that stores electronic medical records, and is stored in the external server. Alternatively, the output destination may be a printer or a facsimile. In this case, if the classified category is related to matters to be communicated to other pharmacists or doctors, the keywords may be printed as a document.

[0116] In step S75, the DB management unit 356 determines whether or not the speech recognition result DB 375 contains unclassified text data that has not yet been subjected to this process. If unclassified text data exists, the determination in step S75 is YES, and the data management process ends. If the video call has not ended, the determination in step S75 is NO, and the process returns to step S71 and is repeated.

[0117] According to the data management process described above, it is possible to classify text data indicating speech recognition results and store the classified results in a database, etc. In other words, the training support server 30 can appropriately manage text data indicating speech recognition results.

[0118] In the data management process described above, extracted keywords are classified as units, and these keywords are output as units to a database for storage, etc. However, the present invention is not limited to this, and sentences before and after an extracted keyword may be classified as units, and these sentences may be output as units to a database for storage, etc.

[0119] [Variation 6] In the above-mentioned variant example 5, the training support server 30 classifies the text data indicating the voice recognition results (here, text data in units of keywords or sentences before and after the keyword) and stores the classification results in a database corresponding to the classification results.

[0120] However, the classified text data may be used for other purposes. For example, the classified text data may be converted into voice data using voice synthesis technology. The voice data obtained by this conversion may then be played back to the pharmacist or patient. In other words, the classified text data may be read aloud to the pharmacist or patient.

[0121] Here, the text data classified in variant example 5 is text data that includes the extracted keywords and is classified into appropriate categories, and corresponds to particularly important content related to medication instructions extracted from the content of the conversation exchanged between the pharmacist and the patient via video call. Therefore, by reading this text data to the pharmacist or patient, the pharmacist or patient can efficiently review the contents of the video call, which is more efficient than simply recording all the audio sent and received during the video call and playing it back.

[0122] The conversion of the classified text data into voice data can be realized, for example, by adding a voice synthesis engine function to the voice recognition processing unit 353 in addition to the voice recognition engine function, and by storing a voice synthesis library in the voice recognition dictionary DB 374. Furthermore, the reproduction (i.e., reading) of the voice data obtained by the conversion can be realized, for example, by outputting the voice of the reproduced voice data from a speaker included in the output unit 816 of the instructor's terminal 10 or the patient's terminal 20.

[0123] The reading can be performed at any timing. For example, when providing new medication instructions, by reading the text data of the previous medication instructions in advance, the pharmacist and patient can review the content exchanged in the previous medication instructions before providing the new medication instructions. This is particularly useful when a long time has passed since the previous medication instructions. Furthermore, even if the pharmacist who provided the previous medication instructions and the pharmacist providing the new medication instructions are different people, it is useful in that the content of the previous medication instructions can be properly inherited.

[0124] In this case, the UI display control unit 151 of the instructor-side terminal 10 displays a user interface for selecting a patient for whom new medication instruction is to be given (for example, a patient who has brought a current prescription). When the pharmacist selects a patient using this user interface, the UI display control unit 151 further displays a user interface for selecting medication instructions given to this patient in the past (for example, including medication instructions given last time and before). When the pharmacist selects any of the medication instructions given in the past, the pharmacist transmits a request to the training support server 30 to acquire voice data corresponding to this medication instruction together with information for identifying the patient. In response to this acquisition request, the training support server 30 transmits the corresponding voice data to the instructor-side terminal 10. As a result, the UI display control unit 151 can acquire the voice data that the pharmacist desires to be read aloud from the training support server 30 and realize reading the voice data to the pharmacist.

[0125] In addition, for example, when a patient is at home looking back on the contents of the previous medication instruction, the information may be read out loud. In this case, the UI display control unit 251 of the patient-side terminal 20 displays a user interface for selecting medication instructions given in the past to the patient using this patient-side terminal 20 (for example, including medication instructions given last time and before). When the patient selects any of the medication instructions given in the past, the UI display control unit 251 transmits a request to acquire voice data corresponding to this medication instruction, together with information for identifying the patient, to the training support server 30. In response to this acquisition request, the training support server 30 transmits the corresponding voice data to the instructor-side terminal 10. As a result, the UI display control unit 251 can acquire the voice data that the patient wishes to have read out from the training support server 30 and realize reading out to the patient.

[0126] The transmission of the voice data from the training support server 30 may be performed by sending the voice data corresponding to the acquisition request to the instructor's terminal 10 or the patient's terminal 20 by a communication means such as e-mail. Alternatively, the training support server 30 may issue a predetermined URL, and the instructor's terminal 10 or the patient's terminal 20 that accesses this URL may download the voice data corresponding to the acquisition request. Next, the operation of the reading process for realizing such reading will be described.

[0127] [Operations of the instructor's terminal 10 and the training support server 30 in Modification 6] 12 is a flowchart showing the flow of the reading-out process. In the following, an example will be described in which the reading-out process is executed by the instructor-side terminal 10 and the training support server 30. The reading-out process in the instructor's terminal 10 and the training support server 30 is started in response to the instructor's terminal 10 accepting an instruction to start the reading-out process.

[0128] In step S81, the UI display control unit 151 of the instructor-side terminal 10 transmits a request to acquire voice data to the training support server 30. This acquisition request includes information for identifying the pharmacist, information for identifying the patient, and information for identifying the medication instruction corresponding to the requested voice data.

[0129] In step S82, the UI display control section 151 of the instructor's terminal 10 determines whether or not voice data has been received from the training support server 30 as a response to the acquisition request sent in step S81. If audio data has been received, the determination in step S82 is YES, and the process proceeds to step S83. On the other hand, if no audio data has been received, the determination in step S82 is NO, and the process repeats the determination in step S82.

[0130] In step S83, the UI display control unit 151 of the instructor's terminal 10 reproduces the audio data received from the training support server 30 and outputs the reproduced audio from the speaker included in the output unit 816. That is, the audio data is read aloud. This completes the read aloud process in the instructor's terminal 10.

[0131] In step S91, the voice recognition processing unit 353 of the training support server 30 determines whether or not a request to acquire voice data has been received from the instructor's terminal 10. If a request to acquire voice data is received, the determination in step S91 is YES, and the process proceeds to step S92. Note that this request to acquire voice data includes information for identifying the pharmacist, the patient, and the medication instruction corresponding to the voice data. Therefore, the training support server 30 can identify the pharmacist, the patient, and the medication instruction corresponding to the voice data based on this identifying information, and then perform subsequent processing. On the other hand, if a request to acquire audio data has not been received, the determination in step S92 is NO, and the process repeats the determination in step S91.

[0132] In step S92, the voice recognition processing unit 353 of the medical instruction support server 30 acquires the classified text data on the medication instruction corresponding to the received acquisition request.

[0133] In step S93, the voice recognition processing unit 353 of the training support server 30 converts the text data acquired in step S92 into voice data.

[0134] In step S94, the UI information generating section 351 of the training support server 30 transmits the voice data converted in step S93 to the instructor-side terminal 10. This completes the reading process in the training support server 30.

[0135] The reading-out process described above is an example in which the reading-out process is performed by the instructor-side terminal 10 and the training support server 30. To explain the reading-out process as being performed by the patient-side terminal 20 and the training support server 30, the "instructor-side terminal 10" in the above description should be read as the "patient-side terminal 20" and the "UI display control unit 151" should be read as the "UI display control unit 251."

[0136] According to the reading process described above, the text data including the extracted keywords is read out to the pharmacist or patient, allowing the pharmacist or patient to efficiently review the contents of the video call.

[0137] [Configuration example] As described above, the training support server 30 according to this embodiment includes the video call control unit 352, the voice recognition processing unit 353, and the recommendation information generation unit 354. The video call control unit 352 controls the video call between the instructor-side terminal 10 used by the instructor who provides medical instruction and the patient-side terminal 20 used by the patient, in order to remotely provide medical instruction to the patient. The voice recognition processing unit 353 performs voice recognition processing on the voices transmitted and received during the video call. The recommendation information generating unit 354 generates recommendation information that indicates the contents of guidance recommended in medical guidance. Furthermore, in parallel with the progress of the video call, the video call control unit 352 causes the instructor's terminal 10 used by the instructor to output training support information based on the results of voice recognition by the voice recognition processing unit 353 and the recommended information generated by the recommended information generation unit 354. In this way, the training support server 30 can not only provide medical training remotely via video calls, but also provide the instructor with training support information based on information such as the results of voice recognition of the instructor and patient's speech indicating the progress of the training, and recommended training content. Therefore, the training support server 30 according to the embodiment of the present invention can provide more appropriate support for medical training given to patients.

[0138] The recommended information generating unit 354 generates recommended information based on at least information relating to the patient, and also changes the content of the recommended information based on the result of voice recognition by the voice recognition processing unit 353. This allows the training support server 30 to change the recommended information to more appropriate content (for example, content that corresponds to the progress of training) based on the recognition results of the voices spoken by the instructor and the patient that indicate the progress of training.

[0139] The recommended information generating unit 354 changes the recommended information at a timing determined based on the result of the voice recognition by the voice recognition processing unit 353. This allows the training support server 30 to change the recommended information at more appropriate timing (for example, timing according to the progress of the training) based on the recognition results of the voices spoken by the instructor and the patient that indicate the progress of the training.

[0140] The training support server 30 further includes a medication history management unit 355 . The medication history management unit 355 creates a draft of an instruction message corresponding to the instruction on the patient's medical care based on the result of the voice recognition by the voice recognition processing unit 353. This allows the training support server 30 to create a draft of a teaching message without requiring the trainer to perform cumbersome character input operations or the like.

[0141] The medication history management unit 355 registers the proposed medication instruction in the patient's medication history as an instruction corresponding to medical guidance, provided that the instructor approves the registration. This allows the training support server 30 to register only appropriate training statements that have been confirmed by the trainer to have no problems in the medication history.

[0142] The education support server 30 is linked to either or both of a function for managing a medication history and a function for supporting medical education in cooperation with the medication history, The video call control unit 352 starts a video call when it receives an operation from an instructor on a display screen related to either or both of a function for managing medication history and a function for supporting medical guidance in cooperation with medication history. This allows the training support server 30 to perform processing in conjunction with functions that are highly relevant to the medical training that is the target of training support (i.e., functions for managing medication history and functions for supporting medical training in conjunction with medication history).

[0143] The training support server 30 further includes a DB management unit 356 . The DB management unit 356 classifies data indicating the results of voice recognition by the voice recognition processing unit 353 based on the contents of the voice recognition, and outputs the data indicating the results of voice recognition to an output destination corresponding to the classification result. This allows the training support server 30 to classify data indicating the voice recognition results (e.g., text data) and store the data in a database corresponding to the classification results, etc. That is, the training support server 30 can appropriately manage data indicating the voice recognition results (e.g., text data).

[0144] The voice recognition processing unit 353 converts data indicating the result of voice recognition output by the DB management unit 356 into voice data for reproducing voice to the pharmacist or patient. This allows the pharmacist or patient to efficiently review the contents of the video call by reading the text data containing the extracted keywords as audio data to the pharmacist or patient.

[0145] As described above, the training support system 1 according to this embodiment includes the instructor terminal 10, the patient terminal 20, and the training support server 30, which are configured to be able to communicate with each other via a network. The instructor's terminal 10 includes a video call control unit 152 and an output unit 816. The video call control unit 152 controls a video call between the instructor-side terminal 10 used by the instructor who provides medical instruction and the patient-side terminal 20 used by the patient, in order to remotely provide medical instruction to the patient. The output unit 816 outputs the training support information. The patient-side terminal 20 includes a video call control unit 252 . The video call control unit 252 controls the video call. The training support server 30 includes a video call control unit 352 , a voice recognition processing unit 353 , and a recommendation information generation unit 354 . The video call control unit 352 controls the video call. The voice recognition processing unit 353 performs voice recognition processing on the voices transmitted and received during the video call. The recommendation information generating unit 354 generates recommendation information that indicates the contents of guidance recommended in medical guidance. Furthermore, in parallel with the progress of the video call, the video call control unit 352 outputs, from the output unit 816 provided in the patient side terminal 20, training support information based on the results of voice recognition by the voice recognition processing unit 353 and the recommended information generated by the recommended information generation unit 354. As a result, the training support system 1, which is configured with multiple terminals and a server device, not only realizes remote medical training via video calls, but also provides the instructor with training support information based on information such as the recognition results of the voices spoken by the instructor and patient indicating the progress of the training, and recommended training content. Therefore, the training support system 1 according to the embodiment of the present invention can provide more appropriate support for medical training.

[0146] The present invention is not limited to the above-described embodiment, and any modifications and improvements that can achieve the object of the present invention are included in the present invention. For example, in the above embodiment, the case where medication instruction by a pharmacist is supported is described as an example, but the present invention is not limited to this. That is, the objects to be supported in the present invention include various types of medical information.

[0147] The above-described series of processes can be executed by hardware or software. In other words, the functional configurations in the above-described embodiments are merely examples and are not particularly limited. That is, it is sufficient that any of the computers constituting the training support system 1 has a function capable of executing the above-described series of processes as a whole, and the functional blocks used to realize this function are not particularly limited to the examples shown. Furthermore, one functional block may be configured as a single piece of hardware, a single piece of software, or a combination thereof.

[0148] Furthermore, the recording medium containing the program for executing the above-mentioned series of processes may be configured not only as a removable medium distributed separately from the device main body in order to provide the program to the user, but also as a recording medium provided to the user in a state where it is pre-installed in the device main body.

[0149] Furthermore, the effects described in the above-mentioned embodiments are merely a list of the most favorable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the present embodiments. [Explanation of symbols]

[0150] 1 Instructor support system, 10 Instructor side terminal, 20 Patient side terminal, 30 Instructor support server, 40 Network, 151, 251 User interface display control unit (UI display control unit), 152, 252, 352 Video call control unit, 153 Recommended information acquisition unit, 351 User interface information generation unit (UI information generation unit), 353 Speech recognition processing unit, 354 Recommended information generation unit, 355 Medication history management unit, 356 Database management unit (DB management unit), 371 Patient attribute database (Patient attribute DB), 372 Prescription database (Prescription DB), 373 Medication history database (Medication history DB), 374 Speech recognition dictionary database (Speech recognition dictionary DB), 375 Speech recognition result database (Speech recognition result DB), 376 Medication instruction database (Medication instruction DB), 800 Information processing device, 811 CPU, 812 ROM, 813 RAM, 814 Bus, 815 Input unit, 816 output unit, 817 storage unit, 818 communication unit, 819 drive, 820 imaging unit, 831 removable media

Claims

1. In order to enable remote medical guidance to be provided to patients, a video call control means controls the video call between the terminal used by the instructor providing the medical guidance and the terminal used by the patient. A speech recognition processing means that performs speech recognition processing on the audio transmitted and received in the aforementioned video call, Equipped with, The video call control means, in parallel with the progress of the video call, outputs the content of the instruction regarding medical guidance from the terminal used by the instructor, and, when it detects that the instruction corresponding to the instruction content has been given by the instructor based on the results of the speech recognition processing, changes the content of the instruction output from the terminal used by the instructor. A teaching support device characterized by the following features.

2. The video call control means detects that instruction has been given by the instructor corresponding to the instruction content when the result of the speech recognition processing includes words related to the instruction content. The instructional support device according to feature 1.

3. The video call control means detects that instruction has been given by the instructor corresponding to the instruction content when the similarity between the result of the speech recognition processing and the instruction content as text is high. The instructional support device according to feature 1.

4. The video call control means is When it is detected that instruction corresponding to the instruction content has been given by the instructor based on the results of the speech recognition process, The instruction content is changed by outputting information from the terminal used by the instructor indicating that the instruction content has been completed. A teaching support device according to any one of items 1 to 3.

5. The video call control means is When it is detected that instruction corresponding to the instruction content has been given by the instructor based on the results of the speech recognition process, The output from the terminal used by the instructor deletes the instructional content and adds new instructional content that is different from the aforementioned instructional content. A teaching support device according to any one of items 1 to 3.

6. The video call control means is In the aforementioned medical guidance, multiple pieces of guidance regarding the same drug are output from the terminal used by the instructor. The instructional support device according to feature 5.

7. A method of providing instructional support performed by a computer, In order to enable remote medical guidance to be provided to a patient, a video call control step controls the video call between the terminal used by the instructor providing the medical guidance and the terminal used by the patient. A speech recognition processing step which performs speech recognition processing on the audio transmitted and received in the video call, Includes, In the video call control step, in parallel with the progress of the video call, the content of the instruction regarding medical guidance is output from the terminal used by the instructor, and when it is detected that instruction corresponding to the instruction content has been given by the instructor based on the results of the speech recognition processing, the content of the instruction output from the terminal used by the instructor is changed. A method of instructional support characterized by the following features.

8. On the computer, In order to enable remote medical guidance to be provided to patients, a video call control function is provided to control video calls between the terminal used by the instructor providing the medical guidance and the terminal used by the patient. A speech recognition processing function that performs speech recognition processing on the audio transmitted and received in the aforementioned video call, To achieve this, The video call control function, in parallel with the progress of the video call, outputs the content of the instruction regarding medical guidance from the terminal used by the instructor, and, when it detects that the instruction corresponding to the instruction content has been given by the instructor based on the results of the speech recognition processing, changes the content of the instruction output from the terminal used by the instructor. A program characterized by the following features.

Images