Guidance support device, guidance support system, and program

Abstract

To more appropriately support guidance related to medical care.SOLUTION: A guidance support server 30 comprises a video call control unit 352, a speech recognition processing unit 353, and a recommendation information generation unit 354. The video call control unit 352 controls a video call between a terminal used by an instructor who performs guidance related to medical care and a terminal used by a patient so as to remotely achieve the guidance related to medical care performed for the patient. The speech recognition processing unit 353 performs speech recognition processing on a speech transmitted and received in the video call. The recommendation information generation unit 354 generates recommendation information that is information indicating the details of guidance recommended in pharmaceutical guidance. Further, in parallel with the advance of the video call, the video call control unit 352 causes a terminal used by a pharmacist to output guidance support information based on a result of the speech recognition performed by the speech recognition processing unit 353 and the recommendation information generated by the recommendation information generation unit 354.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a training support device, a training support system, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, information processing techniques are known that support medical guidance, such as medical guidance or pharmaceutical guidance. For example, Patent Document 1 discloses a technology for managing data on a patient's medical history using a database. According to the technology disclosed in Patent Document 1, when a pharmacist performs dispensing work for a patient, the pharmacist can refer to the database and provide medication instructions based on the medical history, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-122253 A Summary of the Invention [Problem to be solved by the invention]

[0004] In general techniques such as that disclosed in Patent Document 1, it is assumed that when providing medical guidance, the instructor providing the guidance and the patient receiving the guidance meet face to face. In other words, in general techniques, consideration is not given to situations in which the instructor and the patient do not (or cannot) meet face to face, for example, in consideration of contact infection with viruses or bacteria. However, even in such situations in which the instructor and the patient do not (or cannot) meet face to face, it is desirable to more appropriately support medical guidance.

[0005] An object of the present invention is to provide more appropriate support for medical guidance. [Means for solving the problem]

[0006] In order to solve the above problems, a training support device according to one aspect of the present invention comprises: a video call control means for controlling a video call between a terminal used by a medical instructor providing medical guidance to a patient and a terminal used by the patient, in order to remotely provide medical guidance to the patient; a voice recognition processing means for performing voice recognition processing on the voice transmitted and received during the video call; a recommendation information generating means for generating recommendation information which is information indicating recommended guidance content in the medical guidance; Equipped with The video call control means, in parallel with the progress of the video call, causes the terminal used by the instructor to output training support information based on the results of voice recognition by the voice recognition processing means and the recommendation information generated by the recommendation information generation means. Effect of the Invention

[0007] According to the present invention, it is possible to more appropriately support medical guidance. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing a system configuration of a training support system 1 according to an embodiment of the present invention. [Diagram 2] FIG. 8 is a diagram showing a hardware configuration of an information processing device 800 constituting each device. [Diagram 3] 2 is a block diagram showing the functional configuration of an instructor's terminal 10 and a patient's terminal 20.

[0023] FIG. [Figure 4] 2 is a block diagram showing a functional configuration of the training support server 30. FIG. [Diagram 5] FIG. 13 is a schematic diagram showing an example of a training support screen involving a video call. [Figure 6] 13 is a schematic diagram showing an example of a transition of a training support screen. FIG. [Figure 7] 13 is a flowchart showing a flow of a training support process executed by the instructor's terminal 10. [Figure 8]13 is a flowchart showing the flow of a training support process executed by the patient terminal 20. [Figure 9] 13 is a flowchart showing a flow of a training support process executed by the training support server 30. [Figure 10] FIG. 13 is a schematic diagram showing an example of an approval receiving screen. [Figure 11] 13 is a flowchart showing the flow of a data management process executed by the training support server 30. [Figure 12] 13 is a flowchart showing the flow of a reading process executed by the instructor's terminal 10 or the patient's terminal 20, and the training support server 30. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] [System Configuration] FIG. 1 is a diagram showing a system configuration of a training support system 1 according to an embodiment of the present invention. The training support system 1 in this embodiment is a system that supports medical training. For example, the training support system 1 is used to support medical training that is performed remotely by video calling without a direct face-to-face meeting between a medical professional who is a trainer and a patient who is receiving training. In the following, as an example for explanation, a case is assumed in which the medical professional who is a trainer is a pharmacist working at an insurance pharmacy. Also, as an example for explanation, a case is assumed in which medical training is medication instruction performed at the time of dispensing. However, these are merely examples for explanation and are not intended to limit the scope of application of this embodiment. This embodiment can support various medical training such as medical instruction and pharmaceutical instruction performed by various medical professionals such as pharmacists, doctors, and nurses in various medical facilities such as insurance pharmacies, hospitals, and clinics.

[0011] 1, the training support system 1 includes an instructor terminal 10, a patient terminal 20, and a training support server 30. The instructor terminal 10, the patient terminal 20, and the training support server 30, as well as the pharmacy computer and the external server, are configured to be able to communicate with each other via a network 40. This network 40 is realized by, for example, the Internet or a LAN (Local Area Network) installed in an insurance pharmacy or the like.

[0012] The pharmacy computer is an information processing device such as a server computer that executes processes related to the insurance pharmacy's business, and has a function as a receipt computer that creates medical fee statements (receipt computer function) and a function that manages patients' medical histories (medication history management function).The external server is an information processing device such as a server computer that cooperates with the pharmacy computer, and has a function of a database server that stores an electronic medicine notebook and a function of a database server that stores drug information.

[0013] In the figure, each device and each terminal are illustrated as one each, but the present invention is not limited to this. For example, there may be a plurality of training support servers 30 and pharmacy computers corresponding to a plurality of insurance pharmacies. In addition, for example, there may be a plurality of instructor-side terminals 10 corresponding to a plurality of medical staff who use the terminals to provide medical training. Similarly, there may be a plurality of patient-side terminals 20 corresponding to a plurality of patients who use the terminals to receive medical training.

[0014] The instructor-side terminal 10 is used by a medical professional (here, a pharmacist) who provides medical guidance, and is configured with an information processing device such as a tablet terminal or a PC (Personal Computer). The instructor-side terminal 10 displays various information to support medical guidance when the pharmacist provides medication guidance to a patient, and accepts input of various information by the pharmacist. For example, in a later-described training support process, the instructor-side terminal 10 displays videos of the pharmacist and the patient and outputs the voices of the pharmacist and the patient via the training support server 30 in order to realize medication guidance between the pharmacist and the patient by video call. In this case, the instructor-side terminal 10 further displays training support information based on information such as the recognition results of the voices spoken by the pharmacist and the patient indicating the progress of medication guidance and the recommended training content.

[0015] The patient terminal 20 is used by a patient receiving medical guidance, and is configured by an information processing device such as a tablet terminal or PC, or a smartphone carried by the patient, like the instructor terminal 10. The patient terminal 20 displays various information to support medical guidance when a pharmacist provides medication guidance to a patient, and accepts input of various information by the patient. For example, in the later-described training support process, the patient terminal 20 displays videos of the pharmacist and the patient and outputs the voices of the pharmacist and the patient via the training support server 30 in order to realize medication guidance between the pharmacist and the patient by video call. In this embodiment, it is assumed that the training support information is provided to the pharmacist, but the training support information may also be provided to the patient. In this case, the patient terminal 20 displays the training support information in the same manner as the instructor terminal 10.

[0016] The training support server 30 cooperates with the instructor terminal 10 and the patient terminal 20 to support medical training, and is composed of an information processing device such as a server computer. In order to support medical training, the training support server 30 receives various data managed by a pharmacy computer (if there are multiple computers, each pharmacy computer). The training support server 30 also manages the received various data in a database group (including a medical history database) similar to that of the pharmacy computer. In other words, the training support server 30 acquires some or all of the information managed by a pharmacy computer (if there are multiple computers, each pharmacy computer) installed in the insurance pharmacy, and realizes the business support function.

[0017] The training support server 30 also executes "training support processing" by using this task support function. In this training support processing, the training support server 30 controls a video call between an instructor side terminal 10 used by a pharmacist and a patient side terminal 20 used by a patient in order to remotely realize medical guidance given to a patient. The training support server 30 also performs a voice recognition process on the voice transmitted and received during the video call. Furthermore, the training support server 30 generates recommendation information that is information indicating the guidance content recommended in pharmaceutical guidance. Furthermore, in parallel with the progress of the video call, the training support server 30 causes the instructor-side terminal 10 used by the pharmacist to output training support information based on the result of the voice recognition and the generated recommendation information. In this way, the training support server 30 remotely realizes pharmaceutical training by video calling. This allows for medical training to be realized even in a situation where the instructor and the patient do not meet face to face (or cannot meet face to face), for example, in consideration of contact infection of viruses and bacteria. In this case, the training support server 30 not only realizes medical training, but also provides the pharmacist with training support information based on information such as the recognition results of the voices spoken by the pharmacist and the patient indicating the progress of the training, and the recommended training contents. Therefore, the training support server 30 can provide more appropriate support for medical training.

[0018] [Hardware configuration] Next, the hardware configuration of each device and each terminal included in the training support system 1 will be described. As described above, each device and each terminal included in the training support system 1 is configured by an information processing device such as a PC, a server computer, a tablet terminal, or a smartphone, and each of them has the same basic configuration.

[0019] FIG. 2 is a diagram showing a hardware configuration of an information processing device 800 constituting each device and each terminal. As shown in FIG. 2, the information processing device 800 constituting each device includes a CPU (Central Processing Unit) 811, a ROM (Read Only Memory) 812, a RAM (Random Access Memory) 813, a bus 814, an input unit 815, an output unit 816, a memory unit 817, a communication unit 818, a drive 819, and an imaging unit 820.

[0020] The CPU 811 executes various processes according to a program recorded in a ROM 812 or a program loaded from a storage unit 817 to a RAM 813 . The RAM 813 also stores data and the like necessary for the CPU 811 to execute various processes.

[0021] The CPU 811, the ROM 812, and the RAM 813 are connected to one another via a bus 814. To the bus 814, an input unit 815, an output unit 816, a storage unit 817, a communication unit 818, a drive 819, and an imaging unit 820 are connected.

[0022] The input unit 815 is configured with various buttons and the like, and inputs various information in response to instruction operations. The input unit 815 further includes a microphone for collecting voice. The output unit 816 is composed of a display, a speaker, etc., and outputs images and sounds. The storage unit 817 is composed of a hard disk or a dynamic random access memory (DRAM) or the like, and stores various data managed by each server. The communication unit 818 controls communications with other devices via the network 40 .

[0023] Removable media 831, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is appropriately loaded into the drive 819. A program read from the removable media 831 by the drive 819 is installed in the storage unit 817 as necessary.

[0024] The imaging section 820 is configured with an imaging device including a lens, an imaging element, etc., and captures a digital image of a subject.

[0025] When the information processing device 800 is configured as the training support server 30, it is also possible to omit the imaging unit 820. When the information processing device 800 is configured as a tablet terminal, it is also possible to configure the input unit 815 with a touch sensor and arrange it on top of the display of the output unit 816 to provide a touch panel.

[0026] [Functional configuration] Next, the functional configuration of each device and each terminal in the training support system 1 will be described.

[0027] [Functional configuration of the instructor's terminal 10] Fig. 3(A) is a block diagram showing the functional configuration of the instructor's terminal 10. Fig. 3(B) is a block diagram showing the functional configuration of the patient's terminal 20, which will be described later. As shown in FIG. 3(A), in the CPU 811 of the instructor's terminal 10, a user interface display control section (UI display control section) 151, a video call control section 152, and a recommended information acquisition section 153 function.

[0028] The UI display control unit 151 controls the display of various input / output screens (hereinafter referred to as "UI screens") in various processes described below, based on information for displaying a user interface screen (hereinafter referred to as "UI information") received from the training support server 30. For example, in the training support process, the UI display control unit 151 outputs and displays a screen including a display area for displaying video of a video call to realize remote medication guidance and a display area for displaying training support information for supporting a pharmacist, on a display included in the output unit 810. In addition to these displays, the UI display control unit 151 outputs the sound included in the video from a speaker included in the output unit 810. In addition, the UI display control unit 151 transmits to the training support server 30 various pieces of information input on the UI screen and the contents of the input operation instructions.

[0029] The video call control unit 152 cooperates with video call control units in other devices (here, the video call control unit 252 provided in the patient's terminal 20 and the video call control unit 352 provided in the training support server 30) to realize a video call between the instructor's terminal 10 and the patient's terminal 20 (i.e., a video call between the pharmacist and the patient).

[0030] For this purpose, the video call control unit 152 collects the pharmacist's voice using a microphone included in the input unit 815, and captures an image of the pharmacist using the imaging unit 820. The video call control unit 152 also converts the pharmacist's voice and image into a video with voice that complies with a predetermined data format (hereinafter referred to as "pharmacist's video"). Then, the video call control unit 152 transmits the pharmacist's video to the training support server 30. Meanwhile, the video call control unit 252 included in the patient-side terminal 20 performs the same process on the patient, thereby converting the patient's voice and image into a video that complies with a predetermined data format (hereinafter referred to as "patient's video"). Then, the video call control unit 252 transmits the patient's video to the training support server 30.

[0031] Then, the video call control unit 352 included in the training support server 30 generates a video with audio for realizing a video call (hereinafter referred to as "video call video") based on the pharmacist's video and the patient's video, and transmits it to each of the instructor's terminal 10 and the patient's terminal 20. The video call video is a video for displaying, for example, one image of the pharmacist and the patient in a part of the display area when displaying the video call video, and the other image in another part of the display area, and outputting the audio of the pharmacist and the patient. The UI display control unit 151 of the instructor's terminal 10 and the UI display control unit 251 of the patient's terminal 20 receive this video call video and output the video call video from the output unit 816 of their own terminal (i.e., displaying images and outputting audio contained in the video call video, etc.).

[0032] The video call is realized by such control, and in parallel with the progress of the video call, the training support server 30 performs voice recognition processing on the pharmacist's voice and the patient's voice, and transmits the voice recognition result to the instructor's terminal 10. Furthermore, in parallel with the progress of the video call, the training support server 30 generates recommendation information, which is information indicating guidance contents recommended in medication guidance, and transmits the recommendation information to the instructor's terminal 10. The UI display control unit 151 receives the voice recognition result and the recommendation information, and outputs them from the output unit 816 (i.e., displays the voice recognition result and the recommendation information, etc.) in the same way as the video call video. The output voice recognition result and the recommendation information are used by the pharmacist as training support information, which is information for supporting the pharmacist's guidance.

[0033] The recommended information acquiring unit 153 determines whether or not recommended information has been received from the training support server 30. Video in a video call transmitted by the training support server 30 and the voice recognition result thereof change over time. Meanwhile, the recommended information transmitted by the training support server 30 is updated at a predetermined timing and transmitted periodically. Therefore, the recommended information acquiring unit 153 determines whether or not recommended information has been received from the training support server 30, and when it is determined that updated recommended information has been newly received, controls the UI display control unit 151 to newly output the recommended information.

[0034] [Functional configuration of patient side terminal 20] FIG. 3B is a block diagram showing the functional configuration of the patient terminal 20. As shown in FIG. As shown in FIG. 3(B), in the CPU 811 of the patient-side terminal 20, a user interface display control unit (UI display control unit) 251 and a video call control unit 252 function. Here, the UI display control section 251 has a function equivalent to the UI display control section 151 of the instructor's terminal 10 described above. Moreover, the video call control section 252 has a function equivalent to the video call control section 152 of the instructor's terminal 10 described above. Moreover, the method of cooperation between the video call control sections (here, the video call control section 152, the video call control section 252, and the video call control section 352) is as described above in the explanation of the video call control section 152. Therefore, redundant explanation will be omitted here.

[0035] [Functional configuration of the training support server 30] FIG. 4 is a block diagram showing the functional configuration of the training support server 30. As shown in FIG. As shown in FIG. 4, the CPU 811 of the training support server 30 functions as a user interface information generating unit (UI information generating unit) 351, a video call control unit 352, a voice recognition processing unit 353, a recommended information generating unit 354, a medication history management unit 355, and a database management unit (DB management unit) 356.

[0036] Also, as shown in FIG. 4, in the memory unit 817 of the medical support server 30, a patient attribute database (patient attribute DB) 371, a prescription database (prescription DB) 372, a medication history database (medication history DB) 373, a voice recognition dictionary database (voice recognition dictionary DB) 374, a voice recognition result database (voice recognition result DB) 375, and a medication instruction database (medication instruction DB) 376 are formed.

[0037] In the patient attribute DB 371, data on patient attributes such as the address, name, date of birth, age, sex, insured person certificate code number, and emergency contact information are stored in association with information for identifying each patient. These patient attributes are composed of information acquired by the pharmacist in conversation with the patient, information provided by the patient in response to a questionnaire, etc., and include, for example, the patient's hobbies, work, family structure, favorite foods, etc.

[0038] Prescription DB372 stores data on prescriptions issued to patients (such as the name of the insurance medical institution and name of the insured physician who prescribed the medicine, the prescription date, and the prescription contents) in correspondence with information that identifies each patient and the date the prescription was brought in.

[0039] The medical history DB 373 stores data on the history (medicinal history) of drugs prescribed to patients (prescription history, information on medical history including complications, whether or not the patient has visited other departments, etc.) in association with information identifying each patient and information identifying each pharmacist who prescribed the drug. In addition, the medical history also stores data on the history of the drug use instructions given to the patient by the pharmacist (medicinal use instructions, etc.).

[0040] The voice recognition dictionary DB374 stores various dictionaries used for performing voice recognition processing on the voice of a pharmacist included in a video of the pharmacist and the voice of a patient included in a video of the patient. This dictionary includes, for example, an acoustic model for extracting voice features, a pronunciation dictionary for identifying spoken words, and a language model. This dictionary may be a general-purpose dictionary, or a dictionary specialized for the medical field or the like that is suited to the application of this embodiment. In addition, the dictionary may be one that is manually created based on statistical information or the like, or one that is created by machine learning using a method such as deep learning.

[0041] The voice recognition result DB 375 stores the voice recognition results by the voice recognition processing unit 353 in association with information identifying each patient and information identifying each pharmacist who prescribed the medicine.

[0042] In the medication instruction DB376, the medication instruction contents when a pharmacist generally provides medication instruction are stored in association with the characteristics of the prescribed drug and the patient to whom the drug is prescribed (for example, the patient's age, sex, chronic disease, medical history, etc.). The medication instruction contents are, for example, the actions and side effects of the dispensed drug and drug interactions (contraindications for combined use) that the pharmacist (especially an experienced pharmacist) should explain to the patient to whom the pharmacist provides medication instruction, and the contents of interviews (listening items) that should be conducted with the patient as part of the medication instruction. These medication instruction contents may be expressed in the form of medication instruction sentences using technical terms that the pharmacist can understand, or may be expressed in a form that is easier for the patient to understand (for example, medication instruction sentences in simple terms without technical terms, or questions using colloquial language). In addition, the medication instruction contents may be manually created based on statistical information, etc., like the dictionary stored in the voice recognition dictionary DB374, or may be created by machine learning using a method such as deep learning.

[0043] The UI information generating unit 351 generates UI information for the instructor-side terminal 10 and the patient-side terminal 20 to display a UI screen, and transmits the generated UI information to the instructor-side terminal 10 and the patient-side terminal 20. In this case, the UI information generating unit 351 generates a frame format for displaying the UI screen and substantial content to be inserted into the format as UI information. In this embodiment, the substantial content to be inserted into the format includes, for example, training support information such as a video call video for realizing remote medication guidance in the training support process, and a voice recognition result and recommended information for supporting a pharmacist.

[0044] In addition, in response to transmitting UI information to the instructor's terminal 10 and the patient's terminal 20, the UI information generating unit 351 receives various information transmitted from the instructor's terminal 10 and the patient's terminal 20 and the contents of input operation instructions.

[0045] The video call control unit 352 cooperates with video call control units in other devices (here, the video call control unit 152 provided in the instructor-side terminal 10 and the video call control unit 252 provided in the patient-side terminal 20) to realize a video call between the instructor-side terminal 10 and the patient-side terminal 20 (i.e., a video call between the pharmacist and the patient). Specifically, as described above in the explanation of the video call control unit 152, the video call control unit 352 generates a video call video based on the video of the pharmacist received from the instructor-side terminal 10 and the video of the patient received from the patient-side terminal 20, and transmits the generated video call video to each of the instructor-side terminal 10 and the patient-side terminal 20.

[0046] The voice recognition processing unit 353 performs voice recognition processing on the voice of the pharmacist included in the video of the pharmacist received from the instructor-side terminal 10, and the voice of the patient included in the video of the patient received from the patient-side terminal 20. In this case, the voice recognition processing unit 353, for example, refers to various dictionaries stored in the voice recognition dictionary DB 374, and converts the voices of the pharmacist and the patient into text. Then, the voice recognition processing unit 353 stores the voice recognition result (here, the text-converted voices of the pharmacist and the patient) in the voice recognition result DB 375. In addition, this voice recognition result is transmitted to the instructor-side terminal 10 by the UI information generating unit 351.

[0047] The recommended information generating unit 354 generates recommended information, which is information indicating guidance contents recommended in medication instruction, at a predetermined generation timing in parallel with the progress of the video call realized by the video call control unit 352. Here, the predetermined generation timing includes both the timing when the video call starts and the timing according to the progress status of the video call (i.e., the progress status of medication instruction).

[0048] First, when generating recommended information at the timing of starting a video call, the recommended information generating unit 354 identifies a patient to be video-called this time based on information for identifying each patient included in a request to start a video call received from the patient-side terminal 20. Next, the recommended information generating unit 354 identifies the prescription prescribed for the identified patient, the patient's medical history, and the attributes of the patient by referring to each database such as the patient attribute DB 371, the prescription DB 372, and the medical history DB 373. Furthermore, the recommended information generating unit 354 extracts medication instruction contents (e.g., medication instruction text) corresponding to the identified various information from the medication instruction DB 376. Then, the recommended information generating unit 354 generates recommended information by including the extracted medication instruction contents in the recommended information as instructions recommended in medication instruction. As described above in the explanation of the medication instruction DB376, the contents of the medication instructions here include, for example, the actions and side effects of the dispensed medication and drug interactions (contraindications for combined use) that a pharmacist (especially an experienced pharmacist) should explain to the patient to whom he or she is providing medication instructions, as well as the contents of questions that should be asked of the patient (listening items), etc.

[0049] Furthermore, the recommended information generating unit 354 changes and generates the content of the recommended information at a timing according to the progress of the video call (i.e., the progress of the medication instruction). That is, the recommended information generating unit 354 updates the recommended information in parallel with the progress of the video call. This allows the recommended information generating unit 354 to appropriately generate recommended information corresponding to the progress of the video call. For this purpose, the recommended information generating unit 354 compares the voice recognition result of the voice recognition processing unit 353 with the medication instruction content included in the generated recommended information to determine whether or not the pharmacist has provided medication instruction corresponding to any of the medication instruction contents. For example, the recommended information generating unit 354 makes this determination based on criteria such as the similarity as text between the voice recognition result and the medication instruction content, and whether or not the voice recognition result contains words related to the medication instruction content.

[0050] Then, the recommendation information generating unit 354 changes and generates the content of the recommendation information at the timing when it is detected by this determination that the pharmacist has given medication instruction corresponding to any of the medication instruction contents. In this case, the recommendation information generating unit 354 adds, for example, information indicating that this medication instruction has been given to the recommendation information. In addition, the recommendation information generating unit 354 deletes, for example, this medication instruction content from the recommendation information. In addition, the recommendation information generating unit 354 adds, for example, new medication instruction content to the recommendation information. Alternatively, the recommendation information generating unit 354 combines these deletions, additions, and other changes, or changes in stages. Specific examples of these changes will be described later with reference to FIG. 5 and FIG. 6. The recommended information thus generated is transmitted to the instructor's terminal 10 by the UI information generating unit 351 every time it is generated. Then, the transmitted recommended information is output (for example, displayed) in the instructor's terminal 10 together with the video call video and the voice recognition result.

[0051] Specific examples of such displays and transitions on the instructor's terminal 10 will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a schematic diagram showing an example of an instruction support screen accompanying a video call, which is displayed on the instructor's terminal 10 during instruction support processing. Fig. 6 is a schematic diagram showing an example of transitions of the instruction support screen, which is updated at a timing according to the progress of the video call (i.e., the progress of medication instruction).

[0052] As shown in FIG. 5, the training support screen 50 includes a pharmacist image display area 51, a patient image display area 52, a patient attribute display area 53, a recommended information display area 54, and a voice recognition result display area 55 as display areas.

[0053] The pharmacist image display area 51 displays an image (i.e., a video) of the pharmacist included in the video call video generated by the video call control unit 352. The patient image display area 52 displays an image (i.e., a video) of the patient included in the video call video generated by the video call control unit 352. Note that, during the instruction support process, the patient side terminal 20 also provides display areas similar to the pharmacist image display area 51 and the patient image display area 52 and displays them. Note that, for example, in the patient side terminal 20, the pharmacist image display area 51 may be enlarged while the patient image display area 52 may be reduced so that the pharmacist's facial expression and the like can be easily seen. In addition, the instructor side terminal 10 and the patient side terminal 20 output the voices of the pharmacist and the patient included in the video call video together with such a display. This allows the pharmacist and the patient to realize remote medication instruction even in a situation where they do not meet face to face (or cannot meet face to face).

[0054] In the patient attribute display area 53, information indicating the attributes of the patient stored in the patient attribute DB 371 is displayed. In the recommended information display area 54, the recommended information generated by the recommended information generating unit 354 is displayed. In the voice recognition result display area 55, the voice recognition result by the voice recognition processing unit 353 is displayed.

[0055] The displayed information on the patient's attributes, the recommended information, and the voice recognition results are used by the pharmacist as instruction support information to support the pharmacist's instruction. This makes it possible to realize remote medication instruction and to more appropriately support medical instruction.

[0056] In addition, as shown in Fig. 6, the recommended information and the voice recognition result are updated at a timing corresponding to the progress of the video call (i.e., the progress of the medication instruction). In Fig. 6, the recommended information display area 54 and the voice recognition result display area 55 on the medication instruction support screen 50 are particularly illustrated.

[0057] As an example, assume that at the start of a video call, as shown in FIG. 6(A), medication instruction content 54-1 such as "(1) Please confirm that you have taken all of the medication A prescribed last time" and medication instruction content such as "(2) Please explain the effect of medication B prescribed this time" are displayed as recommended information. Then, as shown as a voice recognition result 55-1, assume that a pharmacist who has referred to the recommended information provides medication instruction such as "Have you taken all of the medication A prescribed last time?". In this case, the recommended information generating unit 354 detects that the pharmacist has provided medication instruction corresponding to the medication instruction content 54-1 by comparing the medication instruction content 54-1 with the voice recognition result 55-1. Then, the recommended information generating unit 354 changes and generates the recommended information at the timing of this detection.

[0058] Then, as shown in FIG. 6(B), the text "Instruction Completed" is added to the medication instruction content 54-1 as information indicating that the medication instruction has been completed, or the medication instruction content 54-1 is underlined. In this case, the voice recognition processing unit 353 may also change the manner of displaying the voice recognition result. For example, the manner of displaying the voice recognition result 55-1 may be changed to be displayed in a different color, or hatching or the like may be added. Based on such changes in the recommended information, the pharmacist can confirm that he or she has been able to provide guidance on the recommended medication instruction content. After such display, the recommended information generating unit 354 further changes and generates the content of the recommended information.

[0059] Then, as shown in FIG. 6(C), the medication instruction content 54-1 is deleted, and instead, a new medication instruction content 54-2, "(3) Please check whether side effects of drug A have occurred," is added to the recommendation information. Then, as shown as the voice recognition result 55-2, it is assumed that the pharmacist who referred to the recommendation information gave the instruction, "Have you ever felt drowsy or fatigued after taking drug A?" based on the recommendation instruction content 54-2. In this case, the recommendation information generating unit 354 detects that the pharmacist has given medication instruction corresponding to the medication instruction content 54-2 by comparing the medication instruction content 54-2 with the voice recognition result 55-2. Then, at the timing of this detection, the recommendation information generating unit 354 changes the content of the recommendation information and generates it. For example, the medication instruction content 54-2 and the voice recognition result 55-2 are changed in the same way as the medication instruction content 54-1 and the voice recognition result 55-1 in FIG. 6(B) are changed.

[0060] In this way, by changing the content of the recommended information, etc. at a timing appropriate to the progress of the video call (i.e., the progress of the medication instructions) until all medication instructions have been provided, it is possible to provide more appropriate support for medical instructions.

[0061] Returning to FIG. 4, the medication history management unit 355 creates a proposal for a medication instruction statement to be registered in the medication history of the patient who has received medication instruction, based on the voice recognition result by the voice recognition processing unit 353. As described above, the voice recognition results of the voices of the pharmacist and the patient in the medication instruction are stored in the voice recognition result DB 375. Then, the medication history management unit 355 identifies the medication instruction content that was actually given in the medication instruction involving a video call, based on the stored voice recognition result. Then, for example, the medication history management unit 355 extracts a medication instruction statement corresponding to the identified medication instruction content from the medication instruction DB 376, and creates the extracted medication instruction statement as a proposal for a medication instruction statement. Then, the medication history management unit 355 registers the created proposal for a medication instruction statement in the medication history DB 373. This makes it possible to create a proposal for a medication instruction statement without requiring the pharmacist to perform cumbersome character input operations, etc.

[0062] The DB management unit 356 manages the synchronization of data in various databases managed by the pharmacy computer or the external server with data in various databases provided in the training support server 30. For example, the DB management unit 356 transmits updated data in various databases managed by the training support server 30 to the pharmacy computer or the external server at a preset time (for example, 3:00 a.m., when the servers are not used frequently) or when various databases are updated, and receives data in various databases updated in the pharmacy computer or the external server from the pharmacy computer or the external server to update various databases provided in the training support server 30. As a result, the contents of data in various databases provided in the training support server 30 are appropriately updated to the latest contents.

[0063] [Operation] Next, the operation of each device and each terminal included in the training support system 1 during training support processing will be described.

[0064] [Operation of instructor's terminal 10] FIG. 7 is a flowchart showing the flow of the training support process executed by the instructor's terminal 10. The training support process in the instructor's terminal 10 is started in response to the instructor's terminal 10 accepting an instruction to start a video call.

[0065] In step S11, the video call control unit 152 transmits a request to start a video call with the patient-side terminal 20 to the training support server 30. This request to start a call includes information for identifying the pharmacist who is using the instructor-side terminal 10. The video call control unit 152 also starts transmitting a video of the pharmacist to the training support server 30.

[0066] In step S12, the video call control unit 152 determines whether or not a video call has been started. If a video call has not been started, the result of the determination in step S12 is NO, and the process repeats the determination in step S12. On the other hand, if a video call has been started, the result of determination in step S12 is YES, and the process proceeds to step S13.

[0067] In step S13, the video call control unit 152 receives the video call video from the training support server 30.

[0068] In step S14, the UI display control unit 151 displays the video received in step S13.

[0069] In step S15, the video call control unit 152 receives the voice recognition result from the training support server 30. In step S16, the UI display control unit 151 displays the voice recognition result received in step S15.

[0070] In step S17, the recommended information acquisition unit 153 determines whether or not recommended information has been received. If no recommendation information has been received, the result of the determination in step S17 is NO, and the process proceeds to step S16. On the other hand, if recommended information has been received, the determination in step S17 is YES, and the process proceeds to step S18.

[0071] In step S18, the UI display control unit 151 displays the recommended information confirmed to have been received in step S17.

[0072] In step S19, the video call control unit 152 determines whether the video call has ended. Here, the video call ends when, for example, an instruction to end the video call is received in the instructor's terminal 10 or the patient's terminal 20. The terminal that has received the instruction to end the video call transmits a request to end the video call to the training support server 30. If the video call has not ended, the determination in step S19 is NO, and the process returns to step S13 and is repeated. On the other hand, if the video call has ended, a YES determination is made in step S19, and the training support process in the instructor's terminal 10 ends.

[0073] [Operation of the patient terminal 20] FIG. 8 is a flowchart showing the flow of the training support process executed by the patient terminal 20. The training support process in the patient-side terminal 20 is started in response to the acceptance of an instruction to start a video call in the patient-side terminal 20.

[0074] In step S21, the video call control unit 252 transmits a request to start a video call with the instructor's terminal 10 to the training support server 30. This call start request includes information for identifying the patient who is using the patient's terminal 20. The video call control unit 252 also starts transmitting a video of the patient to the training support server 30.

[0075] In step S22, the video call control unit 252 determines whether or not a video call has been started. If a video call has not been started, the result of the determination in step S22 is NO, and the process repeats the determination in step S22. On the other hand, if a video call has been started, the result of determination in step S22 is YES, and the process proceeds to step S23.

[0076] In step S23, the video call control unit 252 receives the video call video from the training support server 30. In step S24, the UI display control unit 251 displays the video received in step S23.

[0077] In step S25, the video call control unit 252 judges whether the video call has ended. Here, the video call ends when, for example, an instruction to end the video call is received in the instructor's terminal 10 or the patient's terminal 20. The terminal that has received the instruction to end the video call transmits a request to end the video call to the training support server 30. If the video call has not ended, the result of the determination in step S25 is NO, and the process returns to step S23 and is repeated. On the other hand, if the video call has ended, a YES determination is made in step S25, and the training support process in the patient's terminal 20 ends.

[0078] [Operation of the training support server 30] FIG. 9 is a flowchart showing the flow of the training support process executed by the training support server 30. As shown in FIG. The training support process in the training support server 30 is started when the power of the training support server 30 is turned on.

[0079] In step S31, the video call control unit 352 judges whether a video call has been started. Here, the video call is started, for example, when a request to start a video call is received from each of the instructor's terminal 10 and the patient's terminal 20. Note that these video call start requests contain information for identifying the pharmacist and information for identifying the patient. Therefore, the training support server 30 can identify the pharmacist and the patient based on this identifying information and then perform the subsequent processing. If a video call has not been started, the result of the determination in step S31 is NO, and the process proceeds to step S31. On the other hand, if a video call has been started, the result of determination in step S31 is YES, and the process proceeds to step S32.

[0080] In step S32, the video call control unit 352 receives the video of the pharmacist from the instructor-side terminal 10 and receives the video of the patient from the patient-side terminal 20. Then, the video call control unit 352 generates a video call video based on the received videos of the pharmacist and the patient.

[0081] In step S33, the UI information generating unit 351 transmits the video call moving image generated in step S32 to the instructor's terminal 10 and the patient's terminal 20, respectively.

[0082] In step S34, the voice recognition processing unit 353 performs voice recognition processing on the voices of the pharmacist and the patient received in step S32. In step S35, the video call control unit 352 transmits the voice recognition result obtained in step S32 to the instructor's terminal 10.

[0083] In step S36, the recommendation information generation unit 354 determines whether or not it is the timing to generate recommendation information (which, as described above, includes both the timing when the video call starts and the timing based on the progress of the video call (i.e., the progress of medication instructions)). If it is not time to generate recommended information, a negative determination is made in step S36, and the process proceeds to step S39. On the other hand, if it is time to generate recommended information, the determination in step S36 is YES, and the process proceeds to step S37. In step S37, the recommendation information generating unit 354 generates recommendation information.

[0084] In step S 38 , the recommendation information generating section 354 transmits the recommendation information generated in step S 37 to the instructor's terminal 10 . In step S39, the medication history management unit 355 creates a draft of medication instructions based on the results of the voice recognition performed in step S32.

[0085] In step S40, the video call control unit 352 determines whether the video call has ended. Here, the video call ends when a request to end the video call is received from either the instructor's terminal 10 or the patient's terminal 20, for example. If the video call has not ended, the determination in step S40 is NO, and the process returns to step S32 and is repeated. On the other hand, if the video call has ended, the result of the determination in step S40 is YES, and the process proceeds to step S41.

[0086] In step S41, the medical history management unit 355 registers the proposed medication instruction in the medical history. This completes the training support process in the training support server 30.

[0087] According to the above-described training support process, pharmaceutical training is remotely realized by video calling. This makes it possible to realize medical training even in a situation where the instructor and the patient do not meet face-to-face (or cannot meet face-to-face), for example, in consideration of contact infection of viruses and bacteria. In this case, the training support process not only realizes medical training, but also provides the pharmacist with training support information based on information such as the recognition results of the voices spoken by the pharmacist and the patient indicating the progress of the training, and the recommended training contents. Therefore, the training support process makes it possible to provide more appropriate support for medical training.

[0088] [Variations] Although the embodiment of the present invention has been described above, this embodiment is merely an example and does not limit the technical scope of the present invention. The present invention can take various other embodiments and can be modified in various ways, such as omissions and substitutions, without departing from the gist of the present invention. In this case, these embodiments and their modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention described in the claims and their equivalents. As an example, the embodiment of the present invention described above may be modified as follows. In addition, the following modifications may be combined. [Variation 1] In the above embodiment, as an example for explanation, it is assumed that in the training support system 1, a pharmacist uses the instructor-side terminal 10 to provide medication instruction at the time of dispensing to a patient using the patient-side terminal 20. However, the training support system 1 can support various medical guidance such as medical guidance and pharmaceutical guidance provided by various medical professionals such as pharmacists, doctors, and nurses in various medical facilities such as insurance pharmacies, hospitals, and clinics. That is, in the training support system 1, the instructor who provides medical guidance using the instructor-side terminal 10 is not limited to a pharmacist, and may be another medical professional.

[0089] For example, when a doctor working in a hospital performs online medical care, the doctor may use the instructor's terminal 10 to provide medical guidance. In this case, a doctor's guidance database is provided instead of the medication instruction DB 376. In this doctor's guidance database, the medical guidance content that a doctor generally provides when performing online medical care and the content to be heard from a patient as part of this medical guidance (interview content) are stored in association with the medical department in charge of the instructor doctor and the characteristics of the patient receiving the guidance (for example, the patient's age, sex, chronic illness, medical history, etc.). In addition, in a case where the first consultation is a face-to-face consultation and the second consultation is an online consultation, the medical results of the first face-to-face consultation may be further stored in association with the medical guidance content.

[0090] The medical guidance contents are, for example, the contents that a doctor (especially a veteran doctor in charge of the medical department) should ask the patient about when providing medical guidance (e.g., the symptoms the patient is aware of and information necessary for diagnosis, such as the patient's chronic illness) and the contents that should be explained to the patient (e.g., treatment methods for a disease suspected to be afflicted, and the advantages and disadvantages of each of multiple treatment methods). These medical guidance contents may be expressed in a format using technical terms that the doctor can understand, or in a format that is easier for the patient to understand (e.g., a format using simple expressions without technical terms, or questions using colloquial language). In addition, the medical guidance contents may be manually created based on statistical information, as with the medication guidance DB 376, or may be created by machine learning using a method such as deep learning.

[0091] Then, when the video call is started, the recommended information generating unit 354 extracts medical instruction contents from the doctor's instruction database in a manner similar to the method of extracting medication instruction contents from the medication instruction DB 376 in the above-mentioned embodiment.The recommended information generating unit 354 then generates recommended information by including the extracted medical instruction contents in the recommended information as instruction contents recommended in medical instruction.This recommended information is utilized in a manner such as being displayed as support information to the doctor in the same manner as in the above-mentioned embodiment.

[0092] In this case, the medical history management unit 355 registers a draft of medical instructions in the medical history (or an electronic medical record, an electronic questionnaire, or the like instead of the medical history) in a manner similar to the manner in which a draft of medication instructions is registered in the medical history in the above-mentioned embodiment, based on the voice recognition result by the voice recognition processing unit 353. This makes it possible to create a draft of medical instructions without requiring a doctor to perform cumbersome character input operations, etc.

[0093] In this way, by creating an instruction database as appropriate for each medical professional who will be the instructor, it is possible to realize support for medical instruction for other medical professionals, similar to the above-mentioned support for medication instruction for pharmacists. For example, in addition to online medical examinations by doctors, nurses and other medical professionals such as physical therapists can be supported in situations where they remotely provide some kind of medical instruction to patients. In other words, the training support system 1 is capable of supporting various medical training such as medical and pharmaceutical training provided by various medical professionals such as pharmacists, doctors, and nurses, regardless of the medical professional who serves as the instructor.

[0094] [Variation 2] In the above embodiment, the medical history management unit 355 creates a draft of the medication instruction. The draft of the medication instruction created automatically by the computer in this way is merely a draft, and it may not be desirable to register this as a formal medication instruction in the patient's medical history as it is. Therefore, after the medication instruction is completed, a process may be added in which a pharmacist checks the draft of the medication instruction created by the medical history management unit 355, and only those approved by the pharmacist as a result of the check are registered as formal medication instructions in the patient's medical history.

[0095] In this case, for example, an approval acceptance screen is generated in the training support server 30 and displayed on the instructor's terminal 10. Fig. 10 is a schematic diagram showing an example of the approval acceptance screen. As shown in FIG. 10, the approval reception screen 60 includes a bibliographical information display area 61, a medication instruction proposal display area 62, an audio result display area 63, and an approval button display area 64 as display areas.

[0096] Bibliographic items related to the medication instruction for which the medical history is to be registered are displayed in the bibliographic item display area 61. For example, the patient's name, the date and time of the medication instruction, the prescription number, the pharmacist's name, etc. are displayed.

[0097] In the proposed medication instruction display area 62, a list of medication instruction statements showing the medication instruction contents when the pharmacist provides medication instruction and check boxes for the pharmacist to select the medication instruction statements to be approved are displayed. The medication instruction statements included in this list correspond to the list of proposed medication instruction statements created by the medication history management unit 355 in the above-mentioned instruction support process.

[0098] In the voice result display area 63, the voice recognition results of the voices of the pharmacist and the patient during medication instruction are displayed. In the approval button display area 64, an operation button for approving all of the proposed medication instruction statements selected by the check boxes displayed in the medication instruction statement proposal display area 62 is displayed.

[0099] The pharmacist selects one to be registered in the patient's medical history as a formal medication instruction sentence from among the proposed medication instruction sentences displayed in the medication instruction sentence proposal display area 62, and presses an operation button for approving the registration displayed in the approval button display area 64. In addition, the pharmacist refers to the voice recognition results of the voices of the pharmacist and the patient during medication instruction, which are displayed in the voice result display area 63 as appropriate, for reference when selecting a medication instruction sentence proposal. In this way, by being able to refer to the voice recognition results, etc., it is possible to support the pharmacist in approving an appropriate medication instruction sentence proposal.

[0100] Then, the medication history management unit 355 registers, as a formal medication instruction sentence, only a draft of the medication instruction sentence that has received the pharmacist's approval operation at the instructor's terminal 10 in the medication history DB 373. On the other hand, the medication history management unit 355 does not register, as a formal medication instruction sentence, a draft of the medication instruction sentence that has not received the pharmacist's approval operation at the instructor's terminal 10 in the medication history DB 373. This makes it possible to register, in the medication history, only appropriate medication instruction sentences that have been confirmed by the pharmacist to have no problems.

[0101] [Variation 3] In the above-described embodiment, a training support system 1 is constructed using a plurality of clients, namely, an instructor's terminal 10 and a patient's terminal 20, and a server, namely, a training support server 30, and medication instructions involving video calls are provided between the instructor's terminal 10 and the patient's terminal 20 via the training support server 30.

[0102] Without being limited to this, for example, the functions of the training support server 30 may be distributed and implemented on more servers (e.g., cloud servers), or the functions of the training support server 30 and the pharmacy computer may be implemented together in a single information processing device 800.

[0103] In addition, for example, a function such as a video call by the instructor's terminal 10 and a function of supporting medication instruction by the medical instruction support server 30 may be implemented together in a single information processing device 800. In this case, a function such as a video call by the patient's terminal 20 and a function of supporting medication instruction by the medical instruction support server 30 may be realized in another single information processing device 800, and these information processing devices 800 may communicate with each other in a peer-to-peer manner.

[0104] [Variation 4] In the above embodiment, the training support server 30 performs the training support process for training support involving video calls in cooperation with the instructor's terminal 10 and the patient's terminal 20. However, the present invention is not limited to this, and may perform the training support process in cooperation with a function highly related to training support involving video calls.

[0105] For example, the instruction support process may be performed in cooperation with functions of a general pharmacy computer, such as a function for managing a medical history and a function for supporting medication instruction in cooperation with the medical history. In this case, the function for managing a medical history is, for example, a function for managing various information such as not only medication instruction texts but also prescriptions, questionnaire results for patients, contents of interviews with patients, results of interviews with patients, pharmacist's impressions of patients, dispensed medicines, medical fee statements, and medicine notebooks for pharmacists and pharmacy staff, and providing a user interface for this purpose. In addition, the function for supporting medication instruction in cooperation with the medical history is, for example, a function for suggesting to pharmacists the contents of interviews and interviews and the contents of medication instruction, outputting medication history so that patients can easily understand it, and providing a user interface for these purposes.

[0106] When linked with these functions, the video call control unit 352 starts a video call, for example, when an operation from a pharmacist is received on a display screen related to one or both of these functions.

[0107] This enables the medical support server 30 to process in cooperation with functions that are highly relevant to the pharmaceutical medical guidance that is the subject of the medical support (i.e., functions for managing medical history and functions for supporting medication guidance in cooperation with medical history).

[0108] [Variation 5] In the above-described embodiment, the training support server 30 managed, through the DB management unit 356, synchronizing the data of various databases managed by the pharmacy computer or the external server with the data of various databases provided in the training support server 30.

[0109] Without being limited to this, the training support server 30 may also manage data of various databases by other methods. For example, data showing the results of voice recognition by the voice recognition processing unit 353 (here, text data obtained by converting the voices of pharmacists and patients in a video call into text) may be classified, and text data showing the results of voice recognition may be output to an output destination corresponding to the classified results. The management process of text data showing the results of voice recognition accompanied by such classification will be described with reference to FIG. 11.

[0110] [Operation of the training support server 30 in Modification 5] FIG. 11 is a flowchart showing the flow of the data management process executed by the training support server 30. As shown in FIG. The data management process in the training support server 30 is started in response to the reception of an instruction to start the data management process in the training support server 30. Alternatively, the data management process may be started as a batch process at a preset time (for example, 3:00 a.m., when the frequency of use of each server is low). In this way, instead of performing the data management process simultaneously with a video call, the data management process is performed at a preset time as a batch process to distribute the time periods during which the data management process is performed, thereby reducing the load of the calculation process that occurs simultaneously in the training support server 30.

[0111] In step S71, the DB management unit 356 acquires text data indicating the result of the voice recognition by the voice recognition processing unit 353 from the voice recognition result DB 375. This text data is unclassified text data that has not yet been subjected to this process.

[0112] In step S72, the voice recognition processing unit 353 extracts keywords from the text data acquired in step S71. These keywords are, for example, keywords that are considered important in medication instructions, such as "shortness of breath" or "coughing" spoken by a patient or a pharmacist. Such keyword extraction can be realized by registering the words to be used as keywords in the voice recognition dictionary DB 374.

[0113] In step S73, the voice recognition processing unit 353 classifies each of the keywords extracted in step S72 into a category according to the keyword. For example, when a first keyword is extracted, the first keyword is classified into one of categories such as "medication status," "side effects," "effects and effects," "notes," and "messages to contact a doctor." This classification can be realized by registering the keyword and the category in association with each other in the voice recognition dictionary DB 374.

[0114] In step S74, DB management unit 356 outputs each keyword classified in step S73 to an output destination corresponding to the classification result. For example, if the classified category is a category related to a patient's attribute such as a patient's special notes, the keyword is output to patient attribute DB 371 and stored in patient attribute DB 371. Similarly, if the classified category is a category related to a medical history such as a medication status, the keyword is output to medical history DB 373 and stored in medical history DB 373.

[0115] In this case, the output destination is not limited to each database formed in the training support server 30. For example, the output destination may be another server computer, etc. In this case, for example, if the classified category is a category related to treatment and diagnosis such as side effects, the keyword is output by transmitting it to an external server that stores electronic medical records, and is stored in the external server. Alternatively, for example, the output destination may be a printer or a facsimile. In this case, for example, if the classified category is related to matters to be communicated to other pharmacists or doctors, the keywords may be printed as a document.

[0116] In step S75, the DB management unit 356 determines whether or not the voice recognition result DB 375 contains unclassified text data that has not yet been subjected to this process. If unclassified text data exists, a YES determination is made in step S75, and the data management process ends. If the video call has not ended, the determination in step S75 is NO, and the process returns to step S71 and is repeated.

[0117] According to the data management process described above, it is possible to classify text data indicating the voice recognition result and store the classified result in a database, etc. In other words, the training support server 30 can appropriately manage the text data indicating the voice recognition result.

[0118] In the data management process described above, the extracted keywords are classified as units, and the keywords are output as units to the database and stored, etc. However, the present invention is not limited to this, and sentences before and after the extracted keyword may be classified as units, and the sentences may be output as units to the database and stored, etc.

[0119] [Variation 6] In the above-mentioned variant example 5, the training support server 30 classifies text data indicating the voice recognition results (here, text data in units of keywords or sentences before and after a keyword) and stores the classification results in a database corresponding to the classification results.

[0120] However, the classified text data may be used for other purposes. For example, the classified text data may be converted into voice data using a voice synthesis technique. The voice data obtained by this conversion may be played back for a pharmacist or a patient. That is, the classified text data may be read out to a pharmacist or a patient.

[0121] Here, the text data classified in variant example 5 is text data that includes the extracted keywords and is classified into appropriate categories, and corresponds to particularly important content related to medication instructions extracted from the content of the conversation exchanged between the pharmacist and the patient via video call. Therefore, by reading this text data to the pharmacist or patient, the pharmacist or patient can efficiently review the contents of the video call, which is more efficient than, for example, simply recording all the audio sent and received in the video call and playing it back.

[0122] The conversion of the classified text data into voice data can be realized, for example, by adding a voice synthesis engine function to the voice recognition processing unit 353 in addition to the voice recognition engine function, and by storing a voice synthesis library in the voice recognition dictionary DB 374. Furthermore, the reproduction (i.e., reading out) of the voice data obtained by the conversion can be realized, for example, by outputting the voice of the reproduced voice data from a speaker included in the output unit 816 of the instructor's terminal 10 or the patient's terminal 20.

[0123] The reading can be performed at any timing, but for example, when a new medication instruction is to be given, by reading out the text data of the previous medication instruction in advance, the pharmacist and the patient can give the new medication instruction after reviewing the contents exchanged in the previous medication instruction. This is particularly useful when a long time has passed since the previous medication instruction. It is also useful in that the contents of the previous medication instruction can be properly taken over even if the pharmacist who gave the previous medication instruction and the pharmacist who will give the new medication instruction are different people.

[0124] In this case, the UI display control unit 151 of the instructor-side terminal 10 displays a user interface for selecting a patient for whom new medication instruction is to be given (for example, a patient who has brought a current prescription). When the pharmacist selects a patient using this user interface, the UI display control unit 151 further displays a user interface for selecting a medication instruction given in the past to this patient (for example, including a medication instruction given last time or before). When the pharmacist selects any of the medication instructions given in the past, the pharmacist transmits to the training support server 30 a request for acquiring voice data corresponding to this medication instruction together with information for identifying the patient. In response to this acquisition request, the training support server 30 transmits the corresponding voice data to the instructor-side terminal 10. As a result, the UI display control unit 151 can acquire the voice data that the pharmacist wishes to read out from the training support server 30 and realize reading out to the pharmacist.

[0125] In addition, for example, when a patient is at home looking back on the contents of the previous medication instruction, the information may be read out. In this case, the UI display control unit 251 of the patient-side terminal 20 displays a user interface for selecting medication instructions given in the past to the patient using the patient-side terminal 20 (including, for example, medication instructions given last time or before). When the patient selects any of the medication instructions given in the past, the patient transmits a request for acquiring voice data corresponding to the medication instruction to the training support server 30 together with information for identifying the patient. In response to the acquisition request, the training support server 30 transmits the corresponding voice data to the instructor-side terminal 10. This allows the UI display control unit 251 to acquire the voice data that the patient wishes to be read out from the training support server 30, and realize reading out to the patient.

[0126] The transmission of the voice data from the training support server 30 may be a method of transmitting the voice data corresponding to the acquisition request to the instructor's terminal 10 or the patient's terminal 20 by a communication means such as e-mail. Alternatively, the training support server 30 may issue a predetermined URL, and the instructor's terminal 10 or the patient's terminal 20 that accesses this URL may download the voice data corresponding to the acquisition request. Next, the operation of the read-aloud process for realizing such read-aloud will be described.

[0127] [Operations of the instructor's terminal 10 and the training support server 30 in Modification 6] 12 is a flowchart showing the flow of the reading out process. In the following, an example will be described in which the reading out process is executed by the instructor's terminal 10 and the training support server 30. The reading out process in the instructor's terminal 10 and the training support server 30 is started in response to the instructor's terminal 10 accepting an instruction to start the reading out process.

[0128] In step S81, the UI display control unit 151 of the instructor-side terminal 10 transmits a request to acquire voice data to the training support server 30. This request includes information for identifying the pharmacist, information for identifying the patient, and information for identifying the medication instruction corresponding to the requested voice data.

[0129] In step S82, the UI display control section 151 of the instructor's terminal 10 determines whether or not voice data has been received from the training support server 30 as a response to the acquisition request transmitted in step S81. If voice data has been received, the determination in step S82 is YES, and the process proceeds to step S83. On the other hand, if no voice data has been received, the result of the determination in step S82 is NO, and the process repeats the determination in step S82.

[0130] In step S83, the UI display control unit 151 of the instructor's terminal 10 reproduces the audio data received from the training support server 30 and outputs the reproduced audio from the speaker included in the output unit 816. That is, the audio data is read aloud. This ends the read aloud process in the instructor's terminal 10.

[0131] In step S91, the voice recognition processing unit 353 of the training support server 30 determines whether or not a request for acquiring voice data has been received from the instructor's terminal 10. When a request for voice data is received, the result of the determination in step S91 is YES, and the process proceeds to step S92. The request for voice data includes information for identifying the pharmacist, the patient, and the medication instruction corresponding to the voice data. Therefore, the training support server 30 can identify the pharmacist, the patient, and the medication instruction corresponding to the voice data based on the identifying information, and then perform the subsequent processes. On the other hand, if a request to obtain voice data has not been received, the result of the determination in step S92 is NO, and the process repeats the determination in step S91.

[0132] In step S92, the voice recognition processing unit 353 of the medical instruction support server 30 acquires classified text data regarding the medication instruction corresponding to the received acquisition request.

[0133] In step S93, the voice recognition processing unit 353 of the training support server 30 converts the text data acquired in step S92 into voice data.

[0134] In step S94, the UI information generating section 351 of the training support server 30 transmits the voice data converted in step S93 to the instructor's terminal 10. This ends the reading process in the training support server 30.

[0135] In the above-described reading out process, the reading out process is performed by the instructor's terminal 10 and the training support server 30. To explain the reading out process by the patient's terminal 20 and the training support server 30, the "instructor's terminal 10" in the above description should be read as the "patient's terminal 20" and the "UI display control unit 151" should be read as the "UI display control unit 251."

[0136] According to the reading out process described above, the text data including the extracted keywords is read out to the pharmacist or patient, allowing the pharmacist or patient to efficiently review the contents of the video call.

[0137] [Configuration example] As described above, the training support server 30 according to this embodiment includes the video call control unit 352, the voice recognition processing unit 353, and the recommendation information generating unit 354. The video call control unit 352 controls a video call between the instructor-side terminal 10 used by an instructor providing medical guidance and the patient-side terminal 20 used by a patient, in order to remotely provide medical guidance to a patient. The voice recognition processor 353 performs voice recognition processing on the voice transmitted and received during the video call. The recommendation information generating unit 354 generates recommendation information which is information indicating guidance contents recommended in guidance on medical care. Furthermore, in parallel with the progress of the video call, the video call control unit 352 causes the instructor's terminal 10 used by the instructor to output training support information based on the results of voice recognition by the voice recognition processing unit 353 and the recommendation information generated by the recommendation information generation unit 354. In this way, the training support server 30 can not only provide remote medical training via video calls, but also provide the instructor with training support information based on information such as the results of voice recognition of the instructor and patient indicating the progress of the training, and recommended training content. Therefore, the training support server 30 according to the embodiment of the present invention can provide more appropriate support for medical training given to patients.

[0138] The recommended information generating unit 354 generates recommended information based on at least information relating to the patient, and changes the content of the recommended information based on the result of voice recognition by the voice recognition processing unit 353. This allows the training support server 30 to change the recommended information to more appropriate content (e.g., content according to the progress of training) based on the recognition results of the voices spoken by the instructor and the patient indicating the progress of training.

[0139] The recommended information generating unit 354 changes the recommended information at a timing determined based on the result of the voice recognition by the voice recognition processing unit 353 . This enables the training support server 30 to change the recommended information at more appropriate timing (e.g., timing according to the progress of training) based on the recognition results of the voices spoken by the instructor and the patient indicating the progress of training.

[0140] The training support server 30 further includes a medication history management unit 355 . The medical history management unit 355 creates a draft of an instruction message corresponding to the instruction on the medical care of the patient based on the result of the voice recognition by the voice recognition processing unit 353. This allows the training support server 30 to create a draft of a teaching message without requiring the instructor to perform cumbersome character input operations or the like.

[0141] The medical history management unit 355 registers the proposed medication instruction text in the patient's medical history as an instruction text corresponding to medical instructions, provided that the instructor has approved the registration. This allows the training support server 30 to register in the medical history only the proper training sentences that have been confirmed by the trainer to have no problems.

[0142] The medical education support server 30 is linked to either or both of a function for managing a medical history and a function for supporting medical education in cooperation with the medical history, The video call control unit 352 starts a video call when it receives an operation from an instructor on a display screen related to either or both of a function for managing a medical history and a function for supporting medical guidance in cooperation with the medical history. This allows the training support server 30 to process in conjunction with functions that are highly relevant to the medical training that is the subject of training support (i.e., a function for managing medical history and a function for supporting medical training in conjunction with medical history).

[0143] The training support server 30 further includes a DB management unit 356 . The DB management unit 356 classifies data indicating the results of voice recognition by the voice recognition processing unit 353 based on the contents of the voice recognition, and outputs the data indicating the results of voice recognition to an output destination corresponding to the classification result. This allows the training support server 30 to classify data (e.g., text data) indicating the voice recognition result and store the data in a database corresponding to the classification result, etc. In other words, the training support server 30 can appropriately manage data (e.g., text data) indicating the voice recognition result.

[0144] The voice recognition processing unit 353 converts data indicating the result of the voice recognition output by the DB management unit 356 into voice data for playing back a voice to a pharmacist or a patient. This allows the text data containing the extracted keywords to be read aloud to the pharmacist or patient as audio data, allowing the pharmacist or patient to efficiently review the contents of the video call.

[0145] As described above, the training support system 1 according to this embodiment includes the instructor's terminal 10, the patient's terminal 20, and the training support server 30 that are configured to be able to communicate with each other via a network. The instructor's terminal 10 includes a video call control unit 152 and an output unit 816 . The video call control unit 152 controls a video call between the instructor-side terminal 10 used by an instructor providing medical guidance and the patient-side terminal 20 used by a patient, in order to remotely provide medical guidance to a patient. The output unit 816 outputs the training support information. The patient terminal 20 includes a video call control unit 252 . The video call control unit 252 controls the video call. The training support server 30 includes a video call control unit 352 , a voice recognition processing unit 353 , and a recommendation information generating unit 354 . The video call control unit 352 controls the video call. The voice recognition processor 353 performs voice recognition processing on the voice transmitted and received during the video call. The recommendation information generating unit 354 generates recommendation information which is information indicating guidance contents recommended in guidance on medical care. Furthermore, in parallel with the progress of the video call, the video call control unit 352 causes the output unit 816 of the patient side terminal 20 to output training support information based on the results of voice recognition by the voice recognition processing unit 353 and the recommended information generated by the recommended information generation unit 354. As a result, the training support system 1, which is composed of multiple terminals and a server device, can not only provide medical training remotely via video calls, but can also provide the instructor with training support information based on information such as the results of voice recognition of the instructor and patient indicating the progress of the training, and recommended training content. Therefore, according to the training support system 1 according to the embodiment of the present invention, it is possible to more appropriately support training on medical care.

[0146] The present invention is not limited to the above-described embodiment, and modifications and improvements within the scope of the present invention that can achieve the object of the present invention are included in the present invention. For example, in the above embodiment, the case where the medication instruction by a pharmacist is supported is described as an example, but the present invention is not limited to this. That is, the subject of support in the present invention includes various types of medical information.

[0147] The above-described series of processes can be executed by hardware or software. In other words, the functional configurations in the above-mentioned embodiments are merely examples and are not particularly limited. That is, it is sufficient that any of the computers constituting the training support system 1 has a function capable of executing the above-mentioned series of processes as a whole, and the type of functional block used to realize this function is not particularly limited to the examples shown. Furthermore, one functional block may be configured as a single piece of hardware, a single piece of software, or a combination of both.

[0148] In addition, the recording medium containing the program for executing the series of processes described above may be composed not only of a removable medium distributed separately from the device body in order to provide the program to the user, but also of a recording medium etc. provided to the user in a state in which it is pre-installed in the device body.

[0149] Furthermore, the effects described in the above-mentioned embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the present embodiments. [Explanation of symbols]

[0150] 1 Training support system, 10 instructor side terminal, 20 patient side terminal, 30 training support server, 40 network, 151, 251 user interface display control unit (UI display control unit), 152, 252, 352 video call control unit, 153 recommendation information acquisition unit, 351 user interface information generation unit (UI information generation unit), 353 voice recognition processing unit, 354 recommendation information generation unit, 355 medication history management unit, 356 database management unit (DB management unit), 371 patient attribute database (patient attribute DB), 372 prescription database (prescription DB), 373 medication history database (medication history DB), 374 voice recognition dictionary database (voice recognition dictionary DB), 375 voice recognition result database (voice recognition result DB), 376 medication instruction database (medication instruction DB), 800 information processing device, 811 CPU, 812 ROM, 813 RAM, 814 bus, 815 An input unit, 816 an output unit, 817 a storage unit, 818 a communication unit, 819 a drive, 820 an imaging unit, and 831 a removable media

Claims

1. a video call control means for controlling a video call between a terminal used by a medical instructor providing medical guidance to a patient and a terminal used by the patient, in order to remotely provide medical guidance to the patient; a voice recognition processing means for performing voice recognition processing on the voice transmitted and received during the video call; a recommendation information generating means for generating recommendation information which is information indicating recommended guidance content in the medical guidance; Equipped with The video call control means controls the terminal used by the instructor to output, in parallel with the progress of the video call, training support information based on the results of voice recognition by the voice recognition processing means and the recommendation information generated by the recommendation information generation means.

2. the recommendation information generating means generates the recommendation information based on at least information about the patient, and modifies the content of the recommendation information based on a result of voice recognition by the voice recognition processing means.

2. The training support device according to claim 1.

3. the recommendation information generating means changes the recommendation information at a timing determined based on a result of the voice recognition by the voice recognition processing means.

3. The training support device according to claim 1 or 2.

4. and a medical history management means for creating a draft of an instruction message corresponding to the medical instruction based on a result of the voice recognition by the voice recognition processing means.

4. The training support device according to claim 1, wherein the training support device is a training support device for performing a training operation.

5. the medical history management means registers the draft of the instruction text in the medical history of the patient as an instruction text corresponding to the medical instruction text, on condition that the instructor approves the registration; 5. The training support device according to claim 4.

6. The medical training support device is linked to one or both of a function for managing a medical history and a function for supporting medical training in cooperation with the medical history, the video call control means starts the video call in response to receipt of an operation from the instructor on a display screen related to either or both of a function for managing the medical history and a function for supporting the medical guidance in cooperation with the medical history.

6. The training support device according to claim 1, wherein the training support device is a training support device for performing a training operation.

7. a data management means for classifying data representing the results of the voice recognition by the voice recognition processing means based on the contents of the voice recognition, and for outputting the data representing the results of the voice recognition to an output destination corresponding to the classification result, 7. The training support device according to claim 1, wherein the training support device is a training support device for performing a training operation.

8. The voice recognition processing means converts the data indicating the result of the voice recognition output by the data management means into voice data for playing back the voice to the instructor or the patient.

8. The training support device according to claim 7.

9. A training support system including a first terminal, a second terminal, and a server configured to be able to communicate with each other via a network, The first terminal comprises: a first video call control means for controlling a video call between a first terminal used by a medical instructor and a second terminal used by the patient in order to remotely realize medical instruction to be given to the patient; and an output means for outputting training support information, The second terminal comprises: a second video call control means for controlling the video call; The server, a third video call control means for controlling the video call; a voice recognition processing means for performing voice recognition processing on the voice transmitted and received during the video call; a recommendation information generating means for generating recommendation information which is information indicating recommended guidance content in the medical guidance; Equipped with The third video call control means outputs, in parallel with the progress of the video call, training support information based on the results of voice recognition by the voice recognition processing means and the recommendation information generated by the recommendation information generation means from the output means provided in the second terminal.

10. On the computer, a video call control function for controlling a video call between a terminal used by a medical instructor providing medical guidance to a patient and a terminal used by the patient in order to remotely provide medical guidance to the patient; a voice recognition processing function for performing voice recognition processing on the voice transmitted and received during the video call; a recommendation information generating function for generating recommendation information which is information indicating recommended guidance content in the medical guidance; While realizing this, The video call control function is a program characterized by outputting, in parallel with the progress of the video call, from a terminal used by the instructor, teaching support information based on the results of voice recognition by the voice recognition processing function and the recommendation information generated by the recommendation information generation function.

Images