Information processing system, information processing method, information processing device, and program

Abstract

To allow a medical practitioner to confirm target information rapidly and easily.SOLUTION: An information processing system 1 interprets a user's intension by allowing a medical practitioner who is a user to hold a conversation (instruction input) by voice or operation of a screen and hence performing voice recognition processing. Then, data are matched with information in medical support information DB 21 storing medical information, and medical information that matches a user's intension is specified. Further, the specified medical information is transmitted to a terminal device 10 that a medical practitioner who is a user uses.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, an information processing apparatus, and a program.

Background Art

[0002] Conventionally, various information processing technologies have been used to support the work of medical personnel. For example, in the input and viewing (information collection) of records in the current medical system, when confirming the target information, after displaying a predetermined screen, the process of confirming the target information on the screen is performed. Note that a technology for supporting the work of medical personnel is described in, for example, Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology for supporting the work of medical personnel, it is necessary to collect information from a predetermined screen associated with the target information, even for viewing existing description contents (past records) or for various information of patients. When going through such procedures, it is difficult for medical staff to quickly and easily confirm the target information.

[0005] An object of the present invention is to enable medical staff to quickly and easily confirm the target information.

Means for Solving the Problems

[0006] To achieve the above object, an information processing system according to an aspect of the present invention is An information processing system configured such that a terminal device and a server can communicate with each other, wherein the terminal device includes: dialog reception means for receiving voice input or screen operations of dialogs related to the work of medical staff; dialog transmission means for transmitting the data of the received dialog to the server that provides support for the medical staff; support information reception means for receiving support information for medical staff transmitted from the server according to the content of the dialog; support information display means for displaying the received support information; and is provided with: wherein the server includes: intention interpretation means for interpreting the intention of the dialog represented by the data of the dialog based on the data of the dialog transmitted from the terminal device; data matching means for performing data matching between the intention of the dialog represented by the data of the dialog and the medical information managed by the information processing device installed in the medical institution; response specifying means for specifying the response content for the dialog represented by the data of the dialog based on the result of the data matching; response transmission means for transmitting the specified response content to the terminal device as the support information; and is characterized by being provided with the above.

Effect of the Invention

[0007] According to the present invention, medical staff can more quickly and easily confirm the target information.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. [First Embodiment] In the information processing system according to this embodiment, when a medical staff member makes a voice-based interaction (input of an instruction) with a terminal device, the server receives the voice data. The server holds, as a database, data of patients who are the targets for whom the medical staff member provides medical services. The server receives voice data from the terminal device and interprets the intention of the instruction from the medical staff member by performing voice recognition processing. Then, according to the intention of the medical staff member, the server refers to the database and acquires information that matches the intention of the medical staff member. Then, the server transmits, to the terminal device of the medical staff member, information that matches the intention of the medical staff member as a response to the voice-based interaction (input of an instruction). With such a configuration, in the information processing system according to this embodiment, when a medical staff member makes a voice-based interaction with a terminal device, the intention of the medical staff member is interpreted, and information that matches the intention is provided to the medical staff member. Therefore, it is possible to realize a system in which a medical staff member can confirm the target information more quickly and easily. Note that, in the information processing system according to this embodiment, it is also possible to receive an interaction by a screen operation (input of text or selection of options, etc.) from a user who uses the terminal device.

[0010] [Configuration] FIG. 1 is a schematic diagram showing the system configuration of the information processing system 1 according to this embodiment. As shown in FIG. 1, the information processing system 1 according to this embodiment includes a plurality of terminal devices 10, an assist server 20 (first server), a voice recognition / intention interpretation server 30 (second server), and a hospital internal server 40. The terminal devices 10, the assist server 20, and the voice recognition / intention interpretation server 30 are configured to be communicable via a network 50 such as the Internet. Also, the assist server 20 and the hospital internal server 40 are configured to be communicable via the network 50 or other communication means (communication via a dedicated line, etc.).

[0011] The terminal device 10 is constituted by an information processing device such as a PC (Personal Computer) or a tablet terminal, and is used by a medical worker (such as a nurse, a doctor, a pharmacist, or a radiological technologist) who is a user. The terminal device 10 displays a user interface screen (UI screen) in a process where the user (medical worker) checks medical information regarding a patient. Further, the terminal device 10 transmits voice data of an interaction (input of an instruction) by the user (medical worker) to the assist server 20. However, the terminal device 10 may transmit the voice data of the interaction (input of an instruction) by the user (medical worker) to the voice recognition and intention interpretation server 30. Also, the terminal device 10 receives medical information transmitted from the voice recognition and intention interpretation server 30 in response to an interaction by the user (medical worker) and displays it on the UI screen. Note that the terminal device 10 can receive an interaction by the user (medical worker) not only by voice but also by an operation on the UI screen. When receiving an interaction by the user (medical worker) by an operation on the UI screen, the content of the interaction can be received by text input or by selecting an option displayed on the screen. When a screen operation is performed as an interaction (input of an instruction) by the user (medical worker), the terminal device 10 transmits data of the screen operation (such as text data or option data) to the assist server 20. However, the terminal device 10 may transmit the data of the screen operation of the interaction (input of an instruction) by the user (medical worker) to the voice recognition and intention interpretation server 30.

[0012] The assist server 20 is configured by an information processing device such as a server computer, and includes a medical support information database (medical support information DB) 21 that stores predetermined medical information and the like acquired from the in-hospital server 40. The assist server 20 accesses the in-hospital server 40 every predetermined time (for example, 2 minutes). When the data of the medical information stored in the in-hospital server 40 is updated, the assist server 20 acquires the updated medical information data and reflects it in the medical support information DB 21. In addition, when the assist server 20 receives voice data of a dialogue by a user (medical staff), the assist server 20 transmits the received voice data to the voice recognition and intention interpretation server 30 and receives the data of the voice recognition result transmitted from the voice recognition and intention interpretation server 30. Note that the assist server 20 can also receive the data of the voice recognition result for the voice data of the dialogue by the user (medical staff) received by the voice recognition and intention interpretation server 30 from the terminal device 10. Further, the assist server 20 stores the voice data of the dialogue by the user (medical staff) and the data of the voice recognition result in the medical support information DB 21. Note that the voice data and the data of the voice recognition result stored in the medical support information DB 21 are used for emotion analysis and the like of medical staff.

[0013] In addition, when a request for support such as schedule management is made by the user (medical staff) from the terminal device 10, the assist server 20 sets a task such as schedule management requested by the medical staff. For example, the assist server 20 can accept a request for schedule management arbitrarily set by the user (medical staff), and can also accept a request for schedule management related to medical information acquired as a response to a voice dialogue by the user. In addition, the assist server 20 has a function of referring to and analyzing the history of voice data transmitted by a plurality of users (medical staff). For example, the assist server 20 executes a process of analyzing the emotions of medical staff. By analyzing the history, it is possible to analyze the health status and work load of individual users (medical staff), or to analyze the business situation of the entire organization.

[0014] The voice recognition and intention interpretation server 30 is configured by an information processing device such as a server computer, for example. It receives voice data of a conversation by a user (medical staff) from the assist server 20 and performs voice recognition processing on the voice data. However, the voice recognition and intention interpretation server 30 can also receive voice data of a conversation by a user (medical staff) from the terminal device 10 and perform voice recognition processing on the voice data. At this time, after converting the voice data into text data, the voice recognition and intention interpretation server 30 performs morphological analysis and further interprets the intention of the sentence represented by the voice. For example, when the voice data of a conversation by a user (medical staff) represents a sentence "Tell me the taboo allergy information of Mr. Y in Ward X", the voice recognition and intention interpretation server 30 interprets that the information of "taboo allergy" of "Patient Y" admitted to "Ward X" is requested. Note that since there are variations in expressions such as "show", "search", and "please give" corresponding to "tell", possible words can be associated with regular expressions and uniquely associated with commands such as "provide information".

[0015] Also, the voice recognition and intention interpretation server 30 performs data matching on the data stored in the medical support information DB 21 according to the content of the instruction of the voice data after voice recognition and intention interpretation, and acquires information that matches the intention of the user (medical staff). In data matching, the medical support information DB 21 is searched using the content of the interpreted instruction as a key, and a process of matching (comparing) the searched data with the content of the instruction of the user (medical staff) is performed. Then, as a result of the matching, the data determined to match the content of the user's instruction is specified as the information that matches the intention of the user (medical staff). Then, the voice recognition and intention interpretation server 30 transmits the information that matches the intention of the user (medical staff) specified by data matching to the terminal device 10 of the user (medical staff) as a response to the voice conversation (instruction input).

[0016] The in-hospital server 40 is constituted by an information processing device such as a server computer, etc., and stores medical information such as a patient's electronic medical record and examination result data. For example, the in-hospital server 40 stores data of patients receiving treatment in the hospital (name, age, gender, inpatient ward, type of disease, allergy information, examination schedule information, examination value information, etc.). Also, the in-hospital server 40 transmits predetermined medical information to the assist server 20 in response to a request from the assist server 20. Note that, among the medical information stored in the in-hospital server 40, the information that can be presented to the assist server 20 is preset by the administrator of the in-hospital server 40. In the present embodiment, the patient data stored in the in-hospital server 40 includes information for identifying whether the patient is an inpatient or an outpatient, and the assist server 20 is permitted to acquire only the data of inpatients. Note that, in the schematic diagram shown in FIG. 1, the in-hospital server 40 is shown as constituting the information processing system 1, but the in-hospital server 40 may be an external system of the information processing system 1, and various implementation forms can be adopted as long as it can cooperate with the information processing system 1 to provide medical information.

[0017] [Hardware Configuration] FIG. 2 is a schematic diagram showing the hardware configuration of the information processing device 800 that constitutes the terminal device 10, the assist server 20, the voice recognition / intention analysis server 30, or the in-hospital server 40. As shown in FIG. 2, the information processing device 800 includes a CPU (Central Processing Unit) 811, a ROM (Read Only Memory) 812, a RAM (Random Access Memory) 813, a bus 814, an input unit 815, an output unit 816, a storage unit 817, a communication unit 818, a drive 819, and an imaging unit 820.

[0018] The CPU 811 executes various processes according to the programs recorded in the ROM 812 or the programs loaded from the storage unit 817 to the RAM 813. In the RAM 813, data and the like necessary for the CPU 811 to execute various processes are appropriately stored.

[0019] The CPU 811, ROM 812, and RAM 813 are interconnected via a bus 814. Connected to the bus 814 are an input unit 815, an output unit 816, a storage unit 817, a communication unit 818, a drive 819, and an imaging unit 820.

[0020] The input unit 815 receives input of various types of information to the information processing apparatus 800. In the present embodiment, the input unit 815 includes an operation input unit 815a that receives operation input via a display screen using a mouse, keyboard, or the like, and an audio input unit 815b that receives audio input using a microphone or the like. The output unit 816 is composed of a display, a speaker, or the like, and outputs images and audio. The storage unit 817 is composed of a hard disk or DRAM (Dynamic Random Access Memory), etc., and stores various data managed by each server. The communication unit 818 controls communication with other devices via the network 50.

[0021] In the drive 819, a removable medium 831 made of a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, etc., is appropriately mounted. The program read from the removable medium 831 by the drive 819 is installed in the storage unit 817 as necessary. The imaging unit 820 is composed of an imaging device including a lens and an imaging element, etc., and captures a digital image of a subject. Note that the above hardware configuration is the basic configuration of the information processing apparatus 800, and it is possible to have a configuration without some hardware, a configuration with additional hardware, or a configuration with a changed hardware implementation form. For example, the information processing apparatus 800 can have a configuration without the imaging unit 820, or can have a DSP (Digital Signal Processor) for executing voice recognition processing at high speed. Further, the information processing apparatus 800 can be configured with a touch panel by configuring the input unit 815 with a touch sensor and arranging it over the display of the output unit 816.

[0022] [Functional Configuration] Next, the functional configuration of the information processing system 1 will be described. FIG. 3 is a block diagram showing the functional configuration of the terminal device 10. As shown in FIG. 3, in the CPU 811 of the terminal device 10, a user interface control unit (UI control unit) 51, a dialogue data transmission unit 52, and a support information reception unit 53 function.

[0023] The UI control unit 51 controls the display of various input / output screens (hereinafter referred to as "UI screens") and the input / output of voice when a user (medical staff) of the terminal device 10 performs various operations. For example, the UI control unit 51 controls the display of the UI screen and the input / output of voice in the medical staff support process and the support task execution process described later. In the present embodiment, the content of the user interface screen displayed by the UI control unit 51 (what kind of input / output information display screen it is) is notified to the assist server 20 or the voice recognition / intention interpretation server 30, and when the voice data is subjected to voice recognition processing, the voice recognition processing is performed in a state where the regular expression set corresponding to the interface screen is specified. Thereby, it is possible to more accurately recognize or interpret the content of the utterance represented by the voice data. Further, when receiving a dialogue by the user (medical staff) through the operation of the UI screen, the UI control unit 51 can receive the content of the dialogue by text input or by selecting an option displayed on the screen.

[0024] When the UI screen is being displayed, the dialogue data transmission unit 52 transmits voice data (dialogue data) of the dialogue by the user (medical staff) to the assist server 20. Also, when the dialogue by the user (medical staff) is received by operating the UI screen, the dialogue data transmission unit 52 transmits text data or option data (dialogue data) representing the content of the dialogue to the assist server 20. However, the dialogue data transmission unit 52 may transmit the dialogue data by the user (medical staff) to the speech recognition and intention interpretation server 30. Note that when the dialogue data transmission unit 52 transmits voice data to the assist server 20 or the speech recognition and intention interpretation server 30, it also transmits information indicating the content of the displayed user interface screen. The support information reception unit 53 receives a response transmitted from the speech recognition and intention interpretation server 30 corresponding to the transmitted voice data. The content of the received response is displayed on the display screen by the UI control unit 51.

[0025] Figure 4 is a block diagram showing the functional configuration of the assist server 20. As shown in Figure 4, in the CPU 811 of the assist server 20, a medical information acquisition unit 151, a DB management unit 152, a speech recognition request unit 153, a request reception unit 154, a support task execution unit 155, a system management unit 156, and a history analysis unit 157 function. Also, in the storage unit 817 of the assist server 20, a user information storage unit 171 and a medical support information database (medical support information DB) 172 are formed.

[0026] In the user information storage unit 171, information regarding the attributes of the user (medical staff) who uses the terminal device 10 in the information processing system 1 is stored. For example, in the user information storage unit 171, information such as the user's name, age, gender, affiliated ward, affiliated department, staff ID, contact information (e-mail address, mobile phone number, etc.), and terminal ID of the terminal device 10 used is stored. The medical support information DB 172 stores medical information obtained from the in-hospital server 40, voice data transmitted from the terminal device 10, the voice recognition result of the voice data, and the response result (transmitted medical information, etc.) for the dialogue content represented by the voice data.

[0027] The medical information acquisition unit 151 sets the range (attributes of medical information) of medical information to be acquired from the in-hospital server 40. The range of medical information set at this time is, for example, among the medical information permitted by the operator of the in-hospital server 40, such as medical information of inpatients, medical information related to a specific ward, medical information related to dispensing, information related to the schedule of the examination department, etc., those used in the information processing system 1 are set. As an example, the medical information acquisition unit 151 acquires, among the data stored in the in-hospital server 40, staff data (staff name, occupation, affiliation, etc.) related to the staff of the medical institution, patient data (basic information such as name, age, gender, address, insurance card number, etc.) of patients admitted to or visiting the medical institution, order system information such as examinations and treatments (radiological examinations, physiological examinations, endoscopic examinations, specimen examinations, prescriptions, injections, transfusions, surgeries, other schedules (hyperbaric oxygen therapy, etc.)), and information related to the necessary nursing for patients (nursing instructions, nursing implementation, etc.). In addition, the medical information acquisition unit 151 determines whether it is the timing to acquire medical information from the in-hospital server 40, and when it is the acquisition timing, accesses the in-hospital server 40 to acquire predetermined medical information. As the timing for acquiring medical information, for example, a timing every predetermined time such as every two minutes can be set. Note that when the medical information acquisition unit 151 acquires medical information from the in-hospital server 40, it can acquire only the updated portion from the medical information acquired at the previous acquisition timing.

[0028] The DB management unit 152 manages the medical support information DB 21. For example, the DB management unit 152 stores the medical information acquired by the medical information acquisition unit 151 in the medical support information DB 21, or searches for information in response to a request to search for information in the medical support information DB 21 from the voice recognition / intention interpretation server 30. When the assist server 20 receives voice data from the terminal device 10, the voice recognition request unit 153 transmits the voice data to the voice recognition and intention interpretation server 30 and requests the execution of voice recognition processing. Further, the voice recognition request unit 153 receives the result of the requested voice recognition processing from the voice recognition and intention interpretation server 30 and stores it in the medical support information DB 21 via the DB management unit 152.

[0029] The request reception unit 154 determines whether the voice recognition result received by the voice recognition request unit 153 from the voice recognition and intention interpretation server 30 is a request for setting a support task by the user (medical staff) of the terminal device 10. When the request reception unit 154 determines that the voice recognition result is a request for setting a support task, it associates the request with the user who made the request and transfers information representing the content of the request to the support task execution unit 155. Note that the support task represents the processing executed by the information processing system 1 to support the work of the user (medical staff), and can be set individually by the user or preset to support the work of multiple users. For example, as a support task set individually by the user, a reminder (prior notice) is sent 30 minutes before the reservation time of an examination (such as an X-ray examination) of an in-patient for whom the user provides care. Also, as a support task for supporting the work of multiple users, a reminder is sent 30 minutes before the closing time of the examination department.

[0030] The request reception unit 154 can accept not only the reminder arbitrarily set by the user (medical staff), but also the reminder related to the medical information obtained as a response to the voice dialogue by the user. For example, when the user instructs to teach information about a patient's examination in a voice dialogue, and the schedule of the patient's examination is transmitted as a response, the schedule of the patient's examination is displayed on the terminal device 10. In a state where this display is being performed, when the user requests the setting of a reminder in a voice dialogue, the reminder is set in association with the schedule of the patient's examination. Therefore, an appropriate reminder can be easily set based on the actually scheduled examination information.

[0031] When the support task execution unit 155 receives information representing the content of the request to set a support task from the request reception unit 154, it sets the support task in association with the user (terminal device 10) that made the request. In addition, the support task execution unit 155 performs schedule management of the set support task, and executes the set support task when the execution timing of the support task (for example, a predetermined time or the end of a related support task, etc.) arrives. The system management unit 156 manages the operation of the entire information processing system 1. For example, it manages the reception of voice data from the terminal device 10 and the provision of medical support information, the acquisition of medical information from the in-hospital server 40, the access to the medical support information DB 21 from the voice recognition / intention interpretation server 30, etc.

[0032] The history analysis unit 157 analyzes history data such as voice data accumulated in the medical support information DB 21, the voice recognition results of the voice data, and the response results for the dialogue content represented by the voice data. For example, the history analysis unit 157 executes a process of measuring emotions based on the voice data, and records the emotions of the user who input the voice data in association with the staff ID of the user and the input date and time of the voice data. The measurement results of the emotions of the user measured based on the voice data can be used to analyze the health status and work load of the individual user, or to analyze the business situation of the entire organization.

[0033] Figure 5 is a block diagram showing the functional configuration of the voice recognition / intention interpretation server 30. As shown in Figure 5, in the CPU 811 of the voice recognition / intention interpretation server 30, a voice recognition processing unit 251, an intention interpretation unit 252, a support information acquisition unit 253, and a response transmission unit 254 function. In addition, in the storage unit 817 of the voice recognition / intention interpretation server 30, a voice recognition dictionary database (voice recognition dictionary DB) 271, a word dictionary database (word dictionary DB) 272, and a voice recognition result database (voice recognition result DB) 273 are formed.

[0034] The voice recognition dictionary DB271 stores data of various dictionaries and models (acoustic model, language model, pronunciation dictionary, etc.) used by the voice recognition engine. For example, the voice recognition dictionary DB271 stores an acoustic model in which acoustic feature amounts of voices are associated and stored with phoneme models, a language model storing the result of statistically processing patterns in which character strings or word sequences are used as languages, and data of a pronunciation dictionary in which words of the language model are associated and stored with phonemes of the acoustic model.

[0035] The word dictionary DB272 stores a dictionary for word extraction in which words to be extracted are registered in advance. In the present embodiment, the word dictionary DB272 stores a dictionary including words (words constituting a regular expression) constituting a sentence for a normalized instruction and similar words aggregated by the words. Each word in the word dictionary DB272 is attached with a reading (here, katakana representing pronunciation). When voice recognition processing is executed, registered words can be extracted based on the reading (pronunciation) information. The voice recognition result DB273 stores voice recognition result data in association with the input voice data.

[0036] The voice recognition processing unit 251 executes voice recognition processing on the voice data transmitted from the terminal device 10 or the assist server 20 by the voice recognition engine. As a result of the voice recognition processing, pronunciation (here, katakana) of the voice data and data of a sentence expressed as a natural language are acquired.

[0037] The intention interpretation unit 252 morphologically analyzes the speech recognition result (sentence in natural language) obtained by the speech recognition processing unit 251. Further, the intention interpretation unit 252 extracts, based on a preset regular expression, among the words constituting the sentence obtained by morphological analysis, the words that are the types of instructions by the user and the purpose of the instructions. The regular expression referred to at this time corresponds to the screen displayed on the terminal device 10 when the user inputs voice. For example, when voice is input on a screen for displaying an inspection order, the intention interpretation unit 252 refers to the regular expression associated with the screen related to the inspection order. Also, when a similar word similar to the word constituting the regular expression is spoken instead of the word constituting the regular expression, the similar word is extracted, replaced (converted) with the word constituting the regular expression, and the words that are the types of instructions by the user and the purpose of the instructions are extracted.

[0038] In this way, in the speech recognition processing unit 251, intention interpretation based on the voice data is performed so that the speech recognition result appropriately reflects the intention of the user. The support information acquisition unit 253 searches the data stored in the medical support information DB 21 based on the speech recognition result and the intention interpretation result by the speech recognition processing unit 251, and performs data matching between the instruction by the user and the data stored in the medical support information DB 21.

[0039] In the present embodiment, when searching for the data stored in the medical support information DB 21, the support information acquisition unit 253 uses a search term representing the word to be searched in katakana. For example, the patient is searched for by a search term representing the patient's name in katakana. This is because it is not easy to confirm whether the recognition result represented by Chinese characters is correct when voice data is speech-recognized, and if confirmation is performed, the processing time and processing load increase, making it difficult to perform a quick response.

[0040] In data matching, the medical support information DB21 is searched using the types of instructions and the purposes of the instructions obtained by intention interpretation as keys, and a process of matching (comparing) the retrieved data with the types of instructions and the purposes of the instructions of the user (medical staff) is performed. Then, as a result of the matching, the data determined to match the content of the user's instruction is specified as information that conforms to the intention of the user (medical staff).

[0041] The response transmission unit 254 transmits the data specified by the support information acquisition unit 253 to the terminal device 10 as a response (support information) to the content of the instruction (dialogue) by the user.

[0042] [Operation] Next, the operation of the information processing system 1 will be described. [Medical Information Acquisition Processing] FIG. 6 is a flowchart showing the flow of the medical information acquisition processing executed by the information processing system 1. The medical information acquisition processing is started in the assist server 20 in response to an operation instructing the execution of the medical information acquisition processing.

[0043] When the medical information acquisition processing is started, in step S1, the medical information acquisition unit 151 sets the range of medical information to be acquired from the in-hospital server 40. In step S2, the medical information acquisition unit 151 determines whether it is the timing to acquire medical information from the in-hospital server 40. If it is not the timing to acquire medical information, it is determined NO in step S2, and the process proceeds to step S6. On the other hand, if it is the timing to acquire medical information, it is determined YES in step S2, and the process proceeds to step S3.

[0044] In step S3, the medical information acquisition unit 151 accesses the in-hospital server 40. In step S4, the medical information acquisition unit 151 acquires predetermined medical information from the in-hospital server 40. In step S5, the DB management unit 152 stores the medical information acquired by the medical information acquisition unit 151 in the medical support information DB 21.

[0045] In step S6, the medical information acquisition unit 151 determines whether an instruction to end the medical information acquisition process has been given. If an instruction to end the medical information acquisition process has not been given, it is determined as NO in step S6, and the process proceeds to step S2. On the other hand, if an instruction to end the medical information acquisition process has been given, it is determined as YES in step S6, and the medical information acquisition process ends.

[0046] [Medical staff support process] FIG. 7 is a flowchart showing the flow of the medical staff support process executed by the information processing system 1. The medical staff support process is started in response to an operation instructing the execution of the medical staff support process being performed in the assist server 20.

[0047] When the medical staff support process is started, in step S11, the UI control unit 51 of the terminal device 10 accepts the input of user information such as login information. In step S12, the UI control unit 51 accepts a screen operation by the user. In step S13, the dialogue data transmission unit 52 determines whether there is an input of a dialogue (such as voice, text, or options) by the user. If there is no input of a dialogue by the user, it is determined as NO in step S13, and the process proceeds to step S12. On the other hand, if there is an input of a dialogue by the user, it is determined as YES in step S13, and the process proceeds to step S14.

[0048] In step S14, the dialogue data transmission unit 52 transmits the dialogue data to the assist server 20 (or the voice recognition / intent interpretation server 30). In step S15, the speech recognition request unit 153 of the assist server 20 transmits the dialogue data (voice data) received from the terminal device 10 to the speech recognition and intention interpretation server 30, and requests the execution of speech recognition processing. Note that the process of step S15 is executed only when the assist server 20 receives voice data. In step S16, the speech recognition processing unit 251 of the speech recognition and intention interpretation server 30 executes speech recognition processing on the voice data received from the terminal device 10 or the assist server 20.

[0049] In step S17, the intention interpretation unit 252 morphologically analyzes the speech recognition result (natural language) obtained by the speech recognition processing unit 251, and extracts, from among the words constituting the sentence obtained by the morphological analysis, the words that are the types of instructions and the purpose of the instructions by the user based on a preset regular expression. That is, in step S17, the intention of the instruction content represented by the speech recognition result is interpreted.

[0050] In step S18, the support information acquisition unit 253 searches for the data of the medical information stored in the medical support information DB 21 based on the speech recognition result and the intention interpretation result by the speech recognition processing unit 251, and performs data matching between the instruction by the user and the data stored in the medical support information DB 21. In step S19, the support information acquisition unit 253 determines a response to the instruction of the user.

[0051] In step S20, the response transmission unit 254 transmits the response determined by the support information acquisition unit 253 as support information. In step S21, the system management unit 156 determines whether or not the end of the medical staff support process has been instructed. If the end of the medical staff support process has not been instructed, it is determined as NO in step S21, and the process proceeds to step S12. On the other hand, if the end of the medical staff support process has been instructed, it is determined as YES in step S21, and the medical staff support process ends.

[0052] [Support Task Execution Process] FIG. 8 is a flowchart showing the flow of the support task execution process executed by the information processing system 1. The support task execution process is started in the assist server 20 in response to an operation instructing the execution of the support task execution process.

[0053] When the support task execution process is started, in step S31, the support task execution unit 155 sets a default support task. In step S32, the request reception unit 154 determines whether or not the voice recognition result received by the voice recognition request unit 153 from the voice recognition / intention interpretation server 30 is a request for setting a support task by the user (medical staff) of the terminal device 10. If it is not a request for setting a support task by the user (medical staff) of the terminal device 10, it is determined as NO in step S32, and the process proceeds to step S34. On the other hand, if it is a request for setting a support task by the user (medical staff) of the terminal device 10, it is determined as YES in step S32, and the process proceeds to step S33.

[0054] In step S33, the support task execution unit 155 sets the support task in association with the user (terminal device 10) that made the request. In step S34, the support task execution unit 155 determines whether or not it is the execution timing of the set support task. If it is not the execution timing of the set support task, it is determined as NO in step S34, and the process proceeds to step S32. On the other hand, if it is the execution timing of the set support task, it is determined as YES in step S34, and the process proceeds to step S35.

[0055] In step S35, the support task execution unit 155 executes the set support task. After step S35, the process proceeds to step S32. Note that the support task execution process ends along with the end of the medical staff support process.

[0056] [History data analysis process] FIG. 9 is a flowchart showing the flow of the history data analysis process executed by the information processing system 1. The history data analysis process is started in the assist server 20 in response to an operation instructing the execution of the history data analysis process.

[0057] When the history data analysis process is started, in step S41, the history analysis unit 157 acquires history data from the medical support information DB 21. In step S42, the history analysis unit 157 executes analysis of the history data. For example, the history analysis unit 157 analyzes the emotion of the voice data transmitted from the terminal device 10. In step S43, the history analysis unit 157 outputs analysis results from the viewpoints of attributes such as the user, time, and ward. After step S43, the history data analysis process ends.

[0058] As a result of performing the above-described processes, the information processing system 1 provides various functions for supporting medical staff by giving instructions through voice or screen operations. FIG. 10 is a schematic diagram showing an example of a state in which medical support information is presented in response to a dialogue by a medical staff member's voice. Note that FIG. 10 shows an example in which medical support information by the patient-related information dialogue function is presented among the following functions. Hereinafter, examples of functions realized by the information processing system 1 will be described. (1) Patient-related information dialogue function The patient-related information dialogue function is a function that selects a patient and acquires information about the patient. In the patient-related information dialogue function, for example, basic information of the patient, allergy information, blood glucose level information, etc. can be displayed. In the example shown in FIG. 10, when a medical staff inputs a voice saying "Tell me the contraindicated allergy information of Mr. Shiraishi Yu in Ward 5 West.", the "information on contraindicated allergies" of the patient name "Yuu Shiraishi (Shiraishi Yu)" who is hospitalized in the ward name "Ward 5 West" is displayed. (2) Inspection order information dialogue function The inspection order information dialogue function is a function that confirms and browses the inspection information (for example, radiation (physiological) inspection information, etc.) on the day. In the inspection order information dialogue function, settings such as displaying the content of the inspection information and sending a notice that the inspection will be performed a predetermined time before the inspection start time can be made. (3) Remaining inspection order notification function The remaining inspection order notification function is a function that acquires information on inspections (remaining inspections) that have not been completed despite being scheduled at a fixed time every day (for example, 15:00, etc.). In the remaining inspection order notification function, it is possible to receive and display a list of notifications of remaining inspection information for the corresponding ward. (4) General reminder function The general reminder function is a function that registers / confirms reminders for medical staff themselves. In the general reminder function, reminder information for the purpose of memo and task management (to prevent forgetting) can be registered, and a reminder notification can be received at the corresponding time. (5) Patient / order unit reminder function The patient / order unit reminder function is a function that registers / confirms reminders related to patients, inspections, or prescriptions. In the patient / order unit reminder function, by specifying a patient or an order, reminder information related to that patient and order can be registered, and a reminder notification can be received at the corresponding time.

[0059] As described above, in the information processing system 1 according to the present embodiment, when a medical worker who is a user makes an interaction (input of an instruction) by voice or screen operation, voice recognition processing and the like are performed, and the intention of the user is interpreted. Then, data matching is performed with the information in the medical support information DB 21 that stores medical information, and medical information that matches the intention of the user is specified. Then, the specified medical information is transmitted to the terminal device 10 used by the medical worker who is the user. Therefore, it is possible to realize a system that enables a medical worker to more quickly and easily confirm the target information. In addition, since the content of the interaction by the user's voice is interpreted using a regular expression, it is possible to more accurately perform voice recognition and intention interpretation of the content of the utterance. In addition, the user can request support such as schedule management through voice interaction, and the information processing system 1 executes support tasks such as sending a reminder according to the content of the requested support. Therefore, the support required by the user can be flexibly executed. Furthermore, the information processing system 1 accumulates and analyzes the voice data of the user. Therefore, it is possible to accurately grasp the situation of the individual user or the situation of the entire organization.

[0060] [Second Embodiment] Next, a second embodiment of the present invention will be described. In the information processing system 1 according to the first embodiment, when an interaction (input of an instruction by voice) by a user (medical worker) is performed via the terminal device 10, the assist server 20 accepts the interaction, and the voice recognition / intention interpretation server 30 performs voice recognition processing and interprets the voice recognition result based on a regular expression, thereby specifying information that matches the intention of the user among the data stored in the medical support information DB 21. An example has been described. Also, it has been assumed that the information that matches the intention of the user thus specified is transmitted to the terminal device 10 by the voice recognition / intention interpretation server 30. On the other hand, in the information processing system 1 according to the present embodiment, a generation AI server equipped with a language model is provided. When a user (medical staff) makes a dialogue (input of voice instructions), in addition to presenting information (data) that matches the user's intention by the voice recognition and intention interpretation server 30, the generation AI server analyzes the content of the dialogue by the user (medical staff) and makes a proposal (display of a message) corresponding to the information that matches the user's intention. Therefore, according to the information processing system 1 according to the present embodiment, it is possible to promptly present highly reliable and appropriate proposal content in response to a dialogue by a user (medical staff). Hereinafter, the information processing system 1 according to the present embodiment will be specifically described.

[0061] [Configuration] FIG. 11 is a schematic diagram showing the system configuration of the information processing system 1 according to the second embodiment. As shown in FIG. 11, the information processing system 1 according to the present embodiment includes a plurality of terminal devices 10, an assist server 20 (first server), a voice recognition and intention interpretation server 30 (second server), a hospital internal server 40, and a generation AI server 60. The terminal device 10, the assist server 20, the voice recognition and intention interpretation server 30, and the generation AI server 60 are configured to be communicable via a network 50 such as the Internet. Further, the assist server 20 and the generation AI server 60 and the hospital internal server 40 are configured to be communicable via the network 50 or other communication means (communication by a dedicated line, etc.).

[0062] The information processing system 1 shown in FIG. 11 is different from the information processing system 1 of the first embodiment shown in FIG. 1 mainly in that a generation AI server 60 is added. Therefore, hereinafter, the configuration of the generation AI server 60, which is a part different from the information processing system 1 of the first embodiment, will be mainly described, and for other configurations, reference will be made to the description of the information processing system 1 shown in FIG. 1. Note that, in the present embodiment, the speech recognition and intention interpretation server 30 identifies the content of the dialogue (such as the text data input by voice recognition results or screen operations) by the user (medical staff) through natural language processing, and transmits the natural language processing result (information representing the intention of the user's dialogue) to the generation AI server 60. At this time, for example, through natural language processing, the content of the dialogue is elementarily divided into "whose", "which time's", "which content", etc. In addition, processing for absorbing the fluctuations in expressions, fillers ("um", etc.), or hesitation in the dialogue is performed. Further, in the present embodiment, the speech recognition and intention interpretation server 30 performs matching between each element elementarily divided by natural language processing and the data stored in the medical support information DB 21, and identifies information that matches the intention of the user (medical staff).

[0063] The generation AI server 60 is configured by an information processing device such as a server computer, etc., and stores predetermined medical information obtained from the in-hospital server 40. The generation AI server 60 accesses the in-hospital server 40 every predetermined time (for example, 2 minutes), and when the data of the medical information stored in the in-hospital server 40 is updated, it acquires the updated data of the medical information. Note that, in the present embodiment, it is assumed that the data of the medical information acquired by the generation AI server 60 is synchronized with the data of the medical information acquired by the assist server 20, and the data of the medical information acquired from the in-hospital server 40 is distributed to and stored in the assist server 20 and the generation AI server 60.

[0064] In addition, the generative AI server 60 uses the medical information data obtained from the in-hospital server 40 as teacher data, takes the content of the conversation by the user (medical staff) as input (prompt), and generates a language model that outputs proposals (sentences in natural language) for the user (medical staff). For example, in the medical information data obtained from the in-hospital server 40, the generative AI server 60 uses the content filled in by medical staff (such as the history of "tendency to forget to measure blood sugar levels" or "forgetting to take △△ medicine", and notes of various incidents and near misses information), the information obtained by analyzing the data (continuity of the patient's measurement records, fluctuations in measurement values, relationship between lifestyle patterns and symptoms, etc.), and the information obtained in relation to the schedule (preparations required based on the examination items and examination times in the hospital, timing of follow-up confirmation after surgery / treatment, etc.) as teacher data to perform machine learning and generate a language model. In this embodiment, the "proposals" output by the generative AI server 60 include various medically useful information related to the content of the conversation by the user (medical staff), such as recommended information, precautions, advice, reference information, etc.

[0065] In this embodiment, the language model generated by the generative AI server 60 is a small-scale language model specialized in fields and applications compared to the so-called large language model (LLM). That is, in this embodiment, the language model is generated by machine learning targeting the medical information (medical information in a specific medical institution) stored in the in-hospital server 40. As a result, it is possible to generate a language model that conforms to the conversations (sentences in natural language) of medical staff within a specific medical institution, and it is also possible to provide a high-speed response to the input (conversation). In particular, even when specific abbreviations or jargon are used within a specific medical institution, it is possible to accurately interpret them and provide support information that conforms to the conversation.

[0066] However, it is also possible to generate a language model that expands a small language model through machine learning targeting medical information in multiple medical institutions, or to generate a language model (large language model) through machine learning with an expanded target (machine learning expanded to the entire medical field or general natural language). It is also possible to generate a composite language model that combines a small language model, a language model that expands the small language model, and a large language model. For example, a small language model, a language model that expands the small language model, and a large language model are implemented, and high priority is set in the order of the small language model, the language model that expands the small language model, and the large language model to obtain an inference result. As a result, when an inference result cannot be obtained with the small language model, it is possible to appropriately select and use different types and applications of language models, such as obtaining the inference result of the language model that expands the small language model.

[0067] In addition, the generation AI server 60 receives the natural language processing result representing the content of the dialogue (such as the voice recognition result or text data input by screen operation) by the user (medical staff) from the voice recognition and intention interpretation server 30, and performs analysis using the language model on the dialogue by the user (medical staff). Specifically, the generation AI server 60 uses the content of the dialogue represented in natural language as a prompt to perform inference by the language model, and outputs the inference result as a proposal (sentence in natural language) to the user (medical staff). That is, the generation AI server 60 makes a proposal based on the medical information in the medical institution where the in-hospital server 40 is installed for the dialogue by the user (medical staff). Then, the generation AI server 60 transmits the analysis result (the content of the proposal obtained by inference) to the voice recognition and intention interpretation server 30.

[0068] In the present embodiment, the voice recognition and intention interpretation server 30 (response transmission unit 254) integrates the data identified by data matching and the content of the proposal by the generation AI server 60 (the message of the proposal in natural language), and transmits it to the terminal device 10 as support information. FIG. 12 is a schematic diagram showing an example of a display screen of support information in the terminal device 10. As shown in FIG. 12, in the terminal device 10, as support information in the present embodiment, data identified by data matching is displayed in the center of the screen, and the content of the proposal (message of the proposal in natural language) by the generation AI server 60 is displayed at the lower part of the screen.

[0069] Furthermore, in the upper part of the screen of the terminal device 10, a display area for displaying an important proposal (important proposal in natural language) by the generation AI server 60 is formed. As the important proposal by the generation AI server 60, among the proposals obtained as analysis results using a language model, the content of a category with a high degree of importance (for example, information related to medical safety, information related to treatment, etc.) can be set. Whether the proposal obtained as an analysis result using a language model corresponds to the content of a category with a high degree of importance can be determined, for example, by semantic content analysis by natural language processing, classification using machine learning, or the like.

[0070] As an example, in a specific diabetic patient, when it is analyzed by using a language model that there is a tendency that the timing of insulin injection is inappropriate, the proposal "Please pay attention to the timing of insulin injection!" obtained for the diabetic patient is information related to treatment, so it can be displayed in the area for displaying an important proposal in natural language.

[0071] [Functional Configuration] Next, the functional configuration of the generation AI server 60 will be described. FIG. 13 is a block diagram showing the functional configuration of the generation AI server 60. As shown in FIG. 13, in the CPU 811 of the generation AI server 60, a medical information acquisition unit 351, a machine learning processing unit 352, an analysis processing unit 353, and a history management unit 354 function. In addition, in the storage unit 817 of the generation AI server 60, a medical information storage unit 371, a language model storage unit 372, and a history database (history DB) 373 are formed.

[0072] The medical information storage unit 371 stores the medical information acquired by the medical information acquisition unit 351 from the in-hospital server 40. The language model storage unit 372 stores the data of the language model generated by the generation AI server 60. The data of the language model stored in the language model storage unit 372 is sequentially updated when the generation AI server 60 performs additional machine learning. The history DB 373 stores the dialogues by the user (medical staff) analyzed by the generation AI server 60 in association with the analysis results (proposal contents).

[0073] The medical information acquisition unit 351 acquires predetermined medical information from the in-hospital server 40. Further, the medical information acquisition unit 351 accesses the in-hospital server 40 every predetermined time (for example, 2 minutes), and when the data of the medical information stored in the in-hospital server 40 is updated, it acquires the updated data of the medical information. In this embodiment, the data of the medical information acquired by the generation AI server 60 is synchronized with the data of the medical information acquired by the assist server 20.

[0074] Using the data of the medical information acquired from the in-hospital server 40 as teacher data, the machine learning processing unit 352 generates a language model that takes the content of the dialogue by the user (medical staff) as input (prompt) and outputs a proposal (sentence in natural language) for the user (medical staff). In this embodiment, when the machine learning processing unit 352 acquires the updated data of the medical information by the medical information acquisition unit 351, it updates the language model by performing additional machine learning.

[0075] The analysis processing unit 353 receives, as a prompt, the natural language processing result representing the content of the dialogue by the user (medical staff) (such as the voice recognition result or text data input by screen operation) from the voice recognition / intention interpretation server 30, and performs analysis using the language model on the dialogue by the user (medical staff). Further, the analysis processing unit 353 transmits the analysis result (the content of the proposal obtained by inference) to the voice recognition / intention interpretation server 30. The history management unit 354 associates the conversation by the user (medical staff) analyzed in the generative AI server 60 with the analysis result and stores it in the history DB 373. The history data stored in the history DB 373 can be used for fine-tuning of the language model, reinforcement learning, or the like.

[0076] [Operation] Next, the operation of the information processing system 1 will be described. [Language model generation process] FIG. 14 is a flowchart showing the flow of the language model generation process executed by the information processing system 1. The language model generation process is started in response to an operation for instructing the execution of the language model generation process in the generative AI server 60.

[0077] When the language model generation process is started, in step S51, the medical information acquisition unit 351 acquires medical information from the in-hospital server 40. In step S52, the machine learning processing unit 352 executes machine learning for associating the content of the conversation in natural language (prompt) with the proposed content (analysis result). In step S53, the machine learning processing unit 352 generates a language model with the content of the conversation by the user (medical staff) as input (prompt) and the proposal to the user (medical staff) (sentence in natural language) as output. When the machine learning processing unit 352 performs additional machine learning, the language model is updated.

[0078] In step S54, the machine learning processing unit 352 determines whether it is the update timing of the language model. Whether it is the update timing of the language model can be determined by whether it is the timing of accessing the in-hospital server 40 (for example, the timing every two minutes). If it is not the update timing of the language model, in step S54, it is determined as NO, and the process proceeds to step S55. On the other hand, when it is the update timing of the language model, if it is determined YES in step S54, the process proceeds to step S51. Note that if the updated medical information is not acquired from the in-hospital server 40, the update of the language model can be skipped.

[0079] In step S55, the machine learning processing unit 352 determines whether an instruction to end the language model generation process has been given. If an instruction to end the language model generation process has not been given, it is determined NO in step S55, and the process proceeds to step S54. On the other hand, if an instruction to end the language model generation process has been given, it is determined YES in step S55, and the language model generation process ends.

[0080] [Proposed content analysis process] FIG. 15 is a flowchart showing the flow of the proposed content analysis process executed by the information processing system 1. The proposed content analysis process is started in response to an operation instructing the execution of the proposed content analysis process in the generation AI server 60.

[0081] When the proposed content analysis process is started, in step S61, the analysis processing unit 353 receives, as a prompt, the natural language processing result representing the content of the dialogue by the user (medical staff) (such as text data input by voice recognition result or screen operation). In step S62, the analysis processing unit 353 performs an analysis using the language model on the natural language processing result representing the content of the dialogue by the user (medical staff). In step S63, the analysis processing unit 353 transmits, as an analysis result, the proposed content represented in natural language for the content of the dialogue by the user (medical staff). In step S64, the history management unit 354 associates the dialogue by the user (medical staff) analyzed in the generation AI server 60 with the analysis result and stores it in the history DB 373. After step S64, the proposed content analysis process ends.

[0082] As a result of performing the above-described processing, the information processing system 1 according to the present embodiment can realize a proposal function by the generation AI server 60 in addition to the functions provided by the information processing apparatus 1 of the first embodiment. For example, in the information processing system 1 according to the present embodiment, support information for the user (medical staff) is provided in the following procedure. (1) The user conducts a dialogue on the client app on the terminal device 10 (e.g., "Tell me the information of Mr. Shiraishi in Ward 5 West."). (2) The uttered voice in the dialogue is converted into text by the speech recognition and intention interpretation server 30. (3) The texturized dialogue content (speech recognition result) is elementarily divided into "who's", "which time's", "which content", etc. by natural language processing in the speech recognition and intention interpretation server 30. At this time, processing for absorbing fluctuations in expressions, fillers ("um...", etc.), or hesitation in speech in the dialogue is performed (e.g., "admitted in Ward 5 West", "patient name is Shiraishi Yuu", "information to be known is patient information"). (4) Based on the elements elementarily divided by the natural language processing by the speech recognition and intention interpretation server 30, data matching is performed with the data stored in the medical support information DB 21. (5) The speech recognition and intention interpretation server 30 transmits the data obtained by the data matching to the client app of the terminal device 10 as information (support information) that matches the intention of the user (medical staff) (e.g., "height 143.0 cm", "weight 50.0 kg"... allergies are "shrimp, crab, crustaceans", "asthma present", "endoscopy examination at 11:00 today"). (6) Also, a prompt is created from the elements divided by the natural language processing, and an inquiry is made to the generation AI server 60 (e.g., "I want to know the tendency of forgetting to measure blood glucose levels for one month for patient number XXXXXX"). (7) The voice recognition and intention interpretation server 30 transmits the inference result (proposal) from the generative AI server 60 to the client app of the terminal device 10 as support information (e.g., "Mr. xxxx has forgotten to take measurements at dinner X times in one month. It seems advisable to pay attention to taking measurements at dinner."). Note that this inference result can be transmitted together with the support information transmitted in step (5), or it can also be transmitted at different timings. (8) In the terminal device 10, support information suitable for the dialogue of the user (medical staff) is displayed.

[0083] As described above, in the information processing system 1 in this embodiment, a generative AI server 60 equipped with a language model is provided. When a dialogue (input of voice instructions) by the user (medical staff) is performed, in addition to presenting information that matches the user's intention by the voice recognition and intention interpretation server 30, the generative AI server 60 analyzes the content of the dialogue by the user (medical staff) using the language model, and outputs a proposal (message of the proposal in natural language) corresponding to the information that matches the user's intention. Then, the voice recognition and intention interpretation server 30 (response transmission unit 254) integrates the data identified by data matching and the content of the proposal by the generative AI server 60 (message of the proposal in natural language), and transmits it to the terminal device 10 as support information. Therefore, it is possible to promptly present highly reliable and appropriate proposal content for the dialogue by the user (medical staff).

[0084] [Modification Example 1] In the first and second embodiments, the terminal device 10 carried by medical staff such as doctors is equipped with a positioning system such as GPS (Global Positioning System), a positioning function by reading RFID with an RFID reader installed in a room or the like, or a positioning function using the reading of a barcode installed in a room or the like. According to the input of dialogue by the user (medical staff), or automatically by the information processing system 1, processing for assisting contact with a specific medical staff can be performed, such as detecting the position of a specific medical staff or calling a specific medical staff. For example, when an emergency situation requiring immediate response occurs in a patient, the information processing system 1 can detect the position of the attending doctor of the patient and notify the nurse by inputting through dialogue, or call the attending doctor. Also, when the position of the attending doctor is unknown, the information processing system 1 may present a place where the attending doctor may be located by estimating from the action pattern of the attending doctor. This enables necessary responses such as grasping the position or making a call to be promptly made to medical staff who need to be contacted.

[0085] [Modification Example 2] In the first and second embodiments, when the history analysis unit 157 of the assist server 20 executes a process of measuring emotions based on voice data, emotions can be measured and analyzed for a specific situation. For example, when an important incident occurs, emotions can be measured for the voice data of the dialogue input by the user (medical staff), and the mental state of the user (medical staff) can be analyzed. Whether it is an important incident or not can be determined, for example, based on the result of the voice recognition and intention interpretation server 30 interpreting the intention of the content of the dialogue by the user (medical staff), or based on the result of analyzing the content of the dialogue in the generation AI server 60. In addition, depending on the emotion analysis result, the voice recognition / intention interpretation server 30 can provide the user (medical worker) who has had a dialogue with support information for supporting the task corresponding to the analyzed emotion factor (the task that is the factor, or the task that is assumed to be the factor) For example, if there is a possibility that the user (medical worker) is feeling anxious or impatient, it is possible to suggest that the user seeks support from others (for example, coworkers such as colleagues or leaders) based on the content of the dialogue, or to automatically request support from a designated supporter. This makes it possible to analyze the emotions of users (medical professionals) in specific situations, helping to manage their mental health and ensure the smooth execution of their work.

[0086] [Variation 3] In the second embodiment, when the voice recognition / intention interpretation server 30 identifies the contents of the dialogue (voice recognition result or text data input by screen operation, etc.) by the user (medical worker) by natural language processing, the natural language processing may be performed by reflecting the contents of the user interface screen (the display screen of input / output information) displayed by the UI control unit 51. For example, the natural language processing may be performed after weighting the relevance between the displayed contents and words for the contents displayed on the user interface screen. This makes it possible to specify, through natural language processing, content that is more suitable for the content displayed on the user interface screen.

[0087] The information processing system 1 configured as above includes a terminal device 10, an assist server 20, and a voice recognition and intention interpretation server 30. The terminal device 10 includes a UI control unit 51, a dialogue data transmission unit 52, and an assistance information receiving unit 53. The assist server 20 includes a medical information acquisition unit 151, a DB management unit 152, and a voice recognition request unit 153. The voice recognition and intention interpretation server 30 includes a voice recognition processing unit 251, an intention interpretation unit 252, an assistance information acquisition unit 253, and a response transmission unit 254. The UI control unit 51 of the terminal device 10 receives voice input or screen operations of dialogues related to the work of medical staff. The dialogue data transmission unit 52 transmits the received dialogue data to the assist server 20 for assisting medical staff or the voice recognition / intention interpretation server 30. The support information reception unit 53 receives support information for medical staff transmitted from the voice recognition / intention interpretation server 30 according to the content of the dialogue. The UI control unit 51 displays the received support information. The intention interpretation unit 252 of the voice recognition / intention interpretation server 30 performs intention interpretation of the dialogue represented by the dialogue data transmitted from the terminal device 10 based on the dialogue data. The support information acquisition unit 253 performs data matching between the intention of the dialogue represented by the dialogue data and the medical information managed by the information processing device installed in the medical institution. Based on the result of the data matching, the support information acquisition unit 253 specifies the response content for the dialogue represented by the dialogue data. The response transmission unit 254 transmits the specified response content to the terminal device 10 as support information. Thereby, a system can be realized in which medical staff can confirm the target information more quickly and easily.

[0088] As a dialogue related to the work of medical staff, the UI control unit 51 receives dialogues related to patients, examinations, or reminders related to the work of the medical staff. The response transmission unit 254 transmits information related to patients, information related to examinations, the setting result of reminders, or reminder notifications as medical support information. Thereby, the support required by the user can be executed flexibly.

[0089] The assist server 20 includes a request reception unit 154. The request reception unit 154 receives a request for setting a support task for assisting medical staff in association with the medical information indicated in the response transmitted in response to the dialogue from the terminal device 10. Accordingly, appropriate support tasks can be easily set based on the actually existing medical information.

[0090] The assist server 20 includes a medical information acquisition unit 151 and a DB management unit 152. The medical information acquisition unit 151 acquires medical information from the in-hospital server 40 installed in a medical institution. The DB management unit 152 manages support information for assisting the medical staff including medical information by means of a medical support information DB 172. The support information acquisition unit 253 performs data matching between the intention of the dialogue represented by the dialogue data and the medical information stored in the medical support information DB 172. Accordingly, the medical information of a patient can be used without directly referring to the medical information managed by the in-hospital server 40.

[0091] The medical information acquisition unit 151 sets the attributes of the medical information acquired by the assist server 20 from the in-hospital server 40 installed in a medical institution. Accordingly, medical information can be acquired within the scope permitted by the medical institution.

[0092] The attributes of the medical information set by the medical information acquisition unit 151 include that it is the medical information of inpatients. The terminal device 10 is used by medical staff related to inpatients. Accordingly, it becomes possible to more quickly and easily present information regarding inpatients to the medical staff caring for the inpatients.

[0093] The assist server 20 manages support information for medical staff. The voice recognition / intention interpretation server 30 presents support information for medical staff. When the dialogue data transmitted from the terminal device is voice data, the voice recognition / intention interpretation server 30 performs voice recognition processing on the voice data, and performs intention interpretation and data matching based on the content of the dialogue represented by the result of the voice recognition processing or the content of the dialogue input by screen operation. As a result, the management of support information and the processes of speech recognition, intention interpretation, and data matching can be executed by different servers in a shared manner.

[0094] The content of the dialogue regarding the work of medical staff received by the UI control unit 51 is set corresponding to the display screen of the terminal device 10. As a result, it becomes possible to more accurately perform speech recognition and intention interpretation of the content of the utterance represented by the speech data.

[0095] The intention interpretation unit 252 converts the content of the dialogue regarding the work of medical staff into a normalized expression and performs data matching with medical information. As a result, it becomes possible to acquire information that is more suitable for the content of the dialogue.

[0096] The DB management unit 152 stores in the medical support information DB 21 the speech data of the dialogue regarding the work of medical staff, the speech recognition processing result of the speech data, the content of the dialogue input by screen operation, and the response content. The assist server 20 includes a history analysis unit 157. The history analysis unit 157 performs an analysis regarding the work of medical staff based on the speech data of the dialogue regarding the work of medical staff stored in the medical support information DB 21, the speech recognition processing result of the speech data, the content of the dialogue input by screen operation, and the response content. As a result, it becomes possible to analyze the health status and work load of an individual user or analyze the work situation of the entire organization.

[0097] Furthermore, the information processing system 1 includes a generation AI server 60. The generation AI server 60 includes a language model generated with natural language based on dialogue data as input and proposal information based on medical information as output. The generation AI server 60 takes as input the content expressed in natural language based on the dialogue data transmitted from the terminal device 10, performs analysis using a language model, and outputs proposed content expressed in natural language corresponding to the dialogue data. This enables the prompt presentation of highly reliable and appropriate proposed content in response to the dialogue by the user (medical staff).

[0098] The language model is generated by performing machine learning that targets medical information in a specific medical institution and outputs proposed content included in the medical information for the content of the dialogue by medical staff in that medical institution. This makes it possible to generate a language model that conforms to the dialogue (sentences in natural language) of medical staff within a specific medical institution and enables a fast response to the input (dialogue). In particular, even when specific abbreviations or jargon are used within a specific medical institution, it is possible to accurately interpret them and provide support information that conforms to the dialogue.

[0099] The generation AI server 60 is equipped with a plurality of language models (small-scale language models, language models obtained by expanding small-scale language models, large-scale language models, etc.) with different target medical information. The generation AI server 60 takes as input the content expressed in natural language based on the dialogue data transmitted from the terminal device 10 and performs analysis using a plurality of language models. This makes it possible to appropriately select and differentiate the types and uses of language models.

[0100] The assist server 20 includes a history analysis unit 157. The history analysis unit 157 analyzes the emotions of the medical staff who had the dialogue based on the dialogue data. Based on the analysis result by the history analysis unit 157, the response transmission unit 254 transmits support information for supporting the work corresponding to the analyzed emotional factors to the terminal device for the medical staff who had the dialogue. This makes it possible to analyze the emotions of a user (medical staff) in a specific situation, manage their mental health, and support the smooth execution of their work.

[0101] The information processing system 1 is equipped with a location information acquisition function for acquiring the location information of medical staff. In response to a request from the terminal device 10, the assist server 20 executes processing for assisting contact with a specific medical staff based on the location information of the specific medical staff acquired by the location information acquisition function. This makes it possible to promptly take necessary actions such as grasping the location or making a call for medical staff who need to be contacted.

[0102] Note that the present invention is not limited to the above-described embodiments, and modifications, improvements, etc. within the scope that can achieve the object of the present invention are included in the present invention. For example, in the above-described embodiment, when performing speech recognition processing and intention interpretation on the dialogue of a medical staff who is a user, it was described as using a regular expression, but it is not limited to this. That is, if the content of the dialogue of a medical staff who is a user can be accurately interpreted, a configuration that does not use a regular expression is also possible, and it is also possible to interpret the content of the dialogue using AI, etc. Also, in the above-described embodiment, the assist server 20 acquires predetermined medical information from the in-hospital server 40 and stores and manages it in the medical support information DB 21, but it is not limited to this. That is, if the voice recognition / intention interpretation server 30 or the like can directly access the medical information managed by the in-hospital server 40, it may be possible to refer to the medical information managed by the in-hospital server 40.

[0103] Also, the system configuration of the information processing system 1 in the above-described embodiment is an example. As long as the functions of the information processing system 1 are realized as a whole, it is also possible to distribute and implement the functions among more information processing devices, or to combine the functions of multiple information processing devices and implement them in a smaller number of information processing devices. For example, the assist server 20 and the speech recognition / intention interpretation server 30 can be implemented as one server, or the functions of these servers can be distributed and implemented as three or more servers. Also, it is possible to assign the functions of the multiple information processing devices in the information processing system 1 differently from the above-described embodiment. In addition, the information processing system 1 in the above-described embodiment can be realized as a cloud system using a cloud server, or can be realized as an on-premises system using an on-premises server, or can also be realized by a combination of these.

[0104] Also, the present invention can be implemented by appropriately combining the examples described in the above-described embodiment. The above-described series of processes can be executed by hardware or by software. In other words, the functional configurations in FIGS. 3 to 5 are merely examples and are not particularly limited. That is, it is sufficient that the information processing system 1 is provided with a function capable of executing the above-described series of processes as a whole, and the functional blocks used to realize this function are not particularly limited to the examples in FIGS. 3 to 5. Also, one functional block may be configured by hardware alone, by software alone, or by a combination thereof.

[0105] When the series of processes are executed by software, the program constituting the software is installed in a computer or the like from a network or a recording medium. The computer may be a computer incorporated into dedicated hardware. Also, the computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

[0106] The storage medium for storing the program is composed of a removable medium distributed separately from the device body, or a storage medium pre-installed in the device body, etc. The removable medium is composed of, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a flash memory, etc. The optical disk is composed of, for example, a CD-ROM (Compact Disk - Read Only Memory), a DVD (Digital Versatile Disk), a Blu-ray Disc (registered trademark), etc. The magneto-optical disk is composed of an MD (Mini-Disk), etc. The flash memory is composed of, for example, a USB (Universal Serial Bus) memory or an SD card. Also, the storage medium pre-installed in the device body is composed of, for example, a ROM or a hard disk in which the program is stored.

[0107] Note that in this specification, the step of describing the program recorded on the recording medium includes not only the processing performed in chronological order along with the order, but also the processing that may be executed in parallel or individually without necessarily being processed in chronological order. Also, in this specification, the term "system" shall mean the entire device composed of a plurality of devices, a plurality of means, etc.

Description of Reference Numerals

[0108] 1 Information Processing System, 10 Terminal Device, 20 Assist Server, 21 Medical Information Database (Medical Information DB), 30 Speech Recognition and Intention Interpretation Server, 40 In-Hospital Server, 50 Network, 60 Generation AI Server, 800 Information Processing Device, 811 CPU, 812 ROM, 813 RAM, 814 Bus, 815 Input Unit, 815a Operation Input Unit, 815b Voice Input Unit, 816 Output Unit, 817 Memory Unit, 818 Communication Unit, 819 Drive, 820 Imaging Unit, 831 Removable Media, 51 User Interface Control Unit (UI Control Unit), 52 Dialogue Data Transmission Unit, 53 Support Information Reception Unit, 151 Medical Information Acquisition Unit, 152 DB Management Unit, 153 Speech Recognition Request Unit, 154 Request Reception Unit, 155 Support Task Execution Unit, 156 System Management Unit, 157 History Analysis Unit, 171 User Information Memory Unit, 172 Medical Support Information Database (Medical Support Information DB), 251 Speech Recognition Processing Unit, 252 Intention Interpretation Unit, 253 Support Information Acquisition Unit, 254 Response Transmission Unit, 271 Speech Recognition Dictionary Database (Speech Recognition Dictionary DB), 272 Word Dictionary Database (Word Dictionary DB), 273 Speech Recognition Result Database (Speech Recognition Result DB), 351 Medical Information Acquisition Unit, 352 Machine Learning Processing Unit, 353 Analysis Processing Unit, 354 History Management Unit, 371 Medical Information Memory Unit, 372 Language Model Memory Unit, 373 History Database (History DB)

Claims

1. An information processing system in which a terminal device and a server are configured to be able to communicate with each other, The terminal device A dialogue receiving means for receiving voice input or screen operation of dialogue related to the work of medical personnel; A dialogue transmission means for transmitting data of the received dialogue to the server that provides support for the medical staff; a support information receiving means for receiving support information for a medical worker transmitted from the server in response to the content of the dialogue; a support information display means for displaying the received support information; Equipped with The server, an intention interpretation means for interpreting an intention of a dialogue represented by the dialogue data transmitted from the terminal device, based on the dialogue data; a data matching means for performing data matching between the intent of the dialogue represented by the data of the dialogue and medical information managed by an information processing device installed in a medical institution; a response identification means for identifying a response content to the dialogue represented by the dialogue data based on a result of the data matching; a response transmission means for transmitting the specified response content to the terminal device as the support information; An information processing system comprising:

2. The dialogue receiving means receives, as the dialogue related to the medical worker's work, a dialogue about a patient, an examination, or a reminder related to the medical worker's work; 2. The information processing system according to claim 1, wherein the response sending means sends, as the support information, information about the patient, information about the examination, a setting result of the reminder, or a notification of the reminder.

3. The server, The information processing system according to claim 1 or 2, further comprising a request receiving means for receiving from the terminal device a request to set an assistance task for assisting the medical worker in association with the medical information indicated in the response transmitted in response to the dialogue.

4. The server, a medical information acquiring means for acquiring medical information from an information processing device installed in the medical institution; a database management means for managing support information for supporting the medical staff, including the medical information, in a database; Equipped with 2. The information processing system according to claim 1, wherein the data matching means performs data matching between the intent of the dialogue represented by the dialogue data and the medical information stored in the database.

5. 5. The information processing system according to claim 4, wherein the medical information acquisition means sets attributes of the medical information acquired by the server from an information processing device installed in the medical institution.

6. The attributes of the medical information set by the medical information acquisition means include that the medical information is medical information of an inpatient; 6. The information processing system according to claim 5, wherein the terminal device is used by a medical professional who is involved with a hospitalized patient.

7. The server, A first server that manages support information for the medical staff; a second server that presents support information for the medical staff; The information processing system according to claim 1 or 2, characterized in that, when the dialogue data transmitted from the terminal device is voice data, the second server performs voice recognition processing on the voice data, and performs the intention interpretation and the data matching based on the content of the dialogue represented by the result of the voice recognition processing or the content of the dialogue input by the screen operation.

8. 3. The information processing system according to claim 1, wherein the contents of the dialogue related to the work of the medical staff accepted by the dialogue accepting means are set in accordance with the display screen of the terminal device.

9. 3. The information processing system according to claim 1, wherein the intention interpretation means converts the content of the dialogue related to the medical staff's work into a normalized expression and performs data matching with the medical information.

10. the database management means stores in the database at least one of voice data of a dialogue related to the medical staff's work, a voice recognition processing result of the voice data, the content of the dialogue input by operating the screen, and the content of the response; The server, The information processing system according to claim 4, further comprising a history analysis means for performing an analysis of the work of the medical staff based on at least one of voice data of dialogues related to the work of the medical staff stored in the database, the results of voice recognition processing of the voice data, the content of the dialogue input by operating the screen, and the content of the response.

11. The server, a language model generated using natural language based on the dialogue data as an input and suggested information based on medical information as an output, The information processing system according to claim 1 or 2, characterized in that the server receives as input content expressed in natural language based on the dialogue data transmitted from the terminal device, analyzes the content using the language model, and outputs proposal content expressed in natural language corresponding to the dialogue data.

12. The information processing system according to claim 11, characterized in that the language model is generated by performing machine learning on the medical information of a specific medical institution, in which the language model outputs the suggested content contained in the medical information in response to the content of a dialogue between medical professionals at the medical institution.

13. The server includes a plurality of language models each having different target medical information; The information processing system according to claim 11, wherein the content expressed in natural language based on the dialogue data transmitted from the terminal device is used as an input, and analysis is performed using a plurality of the language models.

14. The server, a sentiment analysis means for analyzing sentiment of the medical staff who has engaged in the dialogue based on the dialogue data; 3. The information processing system according to claim 1, further comprising: transmitting, to the terminal device, the support information for supporting the medical professional who conducted the dialogue in performing work corresponding to the analyzed emotional cause based on the analysis result by the emotion analysis means.

15. A location information acquisition means for acquiring location information of the medical staff, The information processing system according to claim 1 or 2, characterized in that, in response to a request from the terminal device, the server executes processing to assist in contact with a specific medical worker based on the location information of the specific medical worker acquired by the location information acquisition means.

16. An information processing method executed by an information processing system configured to enable communication between a terminal device and a server, comprising: The terminal device, A dialogue acceptance step of accepting voice input or screen operation of dialogue related to the work of a medical professional; a dialogue sending step of sending data of the received dialogue to the server that provides support for the medical staff; a support information receiving step of receiving support information for a medical worker transmitted from the server in response to the content of the dialogue; a support information display step of displaying the received support information; Run The server, an intention interpretation step of interpreting an intention of a dialogue represented by the dialogue data transmitted from the terminal device based on the dialogue data; a data matching step of performing data matching between the intent of the dialogue represented by the data of the dialogue and medical information managed by an information processing device installed in a medical institution; a response identifying step of identifying a response content to the dialogue represented by the dialogue data based on a result of the data matching; a response transmission step of transmitting the specified response content to the terminal device as the support information; 2. An information processing method comprising:

17. An information processing device constituting an information processing system in which a terminal device and a server are configured to be able to communicate with each other, The terminal device A dialogue receiving means for receiving voice input or screen operation of dialogue related to the work of medical personnel; A dialogue transmission means for transmitting data of the received dialogue to the server that provides support for the medical staff; a support information receiving means for receiving support information for a medical worker transmitted from the server in response to the content of the dialogue; a support information display means for displaying the received support information; Equipped with As the server, an intention interpretation means for interpreting an intention of a dialogue represented by the dialogue data transmitted from the terminal device, based on the dialogue data; a data matching means for performing data matching between the intent of the dialogue represented by the data of the dialogue and medical information managed by an information processing device installed in a medical institution; a response identification means for identifying a response content to the dialogue represented by the dialogue data based on a result of the data matching; a response transmission means for transmitting the specified response content to the terminal device as the support information; An information processing device comprising:

18. A program for controlling a computer constituting an information processing system in which a terminal device and a server are configured to be able to communicate with each other, The terminal device A dialogue receiving means for receiving voice input or screen operation of dialogue related to the work of medical personnel; A dialogue transmission means for transmitting data of the received dialogue to the server that provides support for the medical staff; a support information receiving means for receiving support information for a medical worker transmitted from the server in response to the content of the dialogue; a support information display means for displaying the received support information; Equipped with As the server, an intent interpretation function that performs, based on the data of the dialogue transmitted from the terminal device, an intent interpretation of the dialogue represented by the data of the dialogue; a data matching function that performs data matching between the intent of the dialogue represented by the dialogue data and medical information managed by an information processing device installed in a medical institution; a response identification function that identifies a response content to the dialogue represented by the dialogue data based on a result of the data matching; a response transmission function of transmitting the specified response content to the terminal device as the support information; A program for realizing the above.

Images