Medical information processing system

Abstract

To reduce a load in confirming a moving image during a medical examination related to a mental disorder.SOLUTION: According to an embodiment, a medical information processing system includes an acquisition part, a position specification part, a moving image generation part, an information generation part, a storage part, a retrieval part, and an output part. The acquisition part acquires moving image data obtained by photographing medical examination content to a patient related to a mental disorder. The position specification part specifies a diagnostic position usable to a diagnosis in the moving image data. The moving image generation part generates a moving image for a diagnosis including the diagnostic position. The information generation part generates specific information related to a specification method of the diagnostic position, and gives the moving image for a diagnosis. The storage part stores a plurality of moving images for a diagnosis. The retrieve part retrieves a moving image for a diagnosis with specific information satisfying a request attached from the storage part on the basis of the request from an operator. The output part outputs the retrieved moving image for a diagnosis.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to a medical information processing system.

Background Art

[0002] When diagnosing a patient with a mental disorder, a doctor not only performs biological tests such as blood tests and imaging tests, but also comprehensively diagnoses using various information such as the patient's psychology, behavioral characteristics, symptoms of distress, changes in life, the patient's speech and expressions. In addition, objective information obtained from the patient's family or people close to the patient may also be used in the diagnosis. In the diagnosis, since it is necessary to make a comprehensive judgment using various information, the amount of information is enormous.

[0003] When creating a medical record after a medical examination, it is necessary to remember and record the content of the patient's conversation during the medical examination. To address this issue, a method of taking a video of the situation during the medical examination and reviewing the recorded video to fill out the medical record is known. However, with this method, since it is necessary to check the video over the same amount of time as the medical examination, the burden on the doctor is great.

[0004] Also, a method of converting the voice data of a video of the medical examination content into text and recording it in the medical record is known. However, for diagnosis, it is necessary to use important information other than voice, such as the tone of the patient's voice, facial expressions, reaction to questions, and response speed. Therefore, by converting voice data into text, there is a risk that important information other than voice will be left out.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] One of the problems that the embodiments disclosed in this specification and drawings aim to solve is to reduce the burden of reviewing videos during consultations related to mental illness. However, the problems that the embodiments disclosed in this specification and drawings aim to solve are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0007] The medical information processing system according to the embodiment comprises an acquisition unit, a location identification unit, a video generation unit, an information generation unit, a storage unit, a search unit, and an output unit. The acquisition unit acquires video data of the content of a medical examination of a patient with a mental illness. The location identification unit identifies diagnostic locations in the video data that can be used for diagnosis. The video generation unit generates a diagnostic video including the diagnostic locations. The information generation unit generates identification information regarding the method of identifying the diagnostic locations and attaches it to the diagnostic video. The storage unit stores a plurality of the diagnostic videos. The search unit searches the storage unit for a diagnostic video that has been attached with identification information that satisfies the request from the operator. The output unit outputs the retrieved diagnostic video. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows an example of the configuration of a medical information processing system according to an embodiment. [Figure 2] Figure 2 is a flowchart illustrating the processing procedure for recording medical examination videos by the medical information processing system according to the embodiment. [Figure 3] Figure 3 is a flowchart illustrating the processing procedure for diagnostic support processing by the medical information processing system according to the embodiment. [Figure 4] Figure 4 shows an example of a diagnostic support screen displayed by the medical information processing system according to this embodiment. [Figure 5] Figure 5 is a flowchart illustrating the processing steps for step S207 in Figure 4. [Figure 6]Figure 6 shows an example of a diagnostic support screen displayed by the process of step S303 in Figure 5, which relates to a modified example. [Figure 7] Figure 7 shows an example of a diagnostic support screen displayed after the processing in step S306 of Figure 5. [Figure 8] Figure 8 shows an example of a diagnostic support screen displayed by the process of step S306 in Figure 5, which relates to a modified example. [Modes for carrying out the invention]

[0009] The embodiments of the medical information processing system will be described in detail below with reference to the drawings. In the following description, components having substantially the same function and configuration will be denoted by the same reference numeral, and redundant explanations will be given only when necessary.

[0010] (Embodiment) Figure 1 shows the configuration of the medical information processing system 1. The medical information processing system 1 identifies positions that contribute to diagnosis from video footage of interviews with patients with mental illness, and extracts and displays the video footage of the identified positions, thereby providing physicians with information to help them make decisions when filling out medical records and during diagnosis. The medical information processing system 1 comprises a client terminal 10 and a server 20. The client terminal 10 is connected to the server 20 via a network 30.

[0011] Network 30 is, for example, a LAN (Local Area Network). Connection to Network 30 can be wired or wireless. Furthermore, the connection is not limited to a LAN, as long as security is ensured by a VPN (Virtual Private Network), etc. Connecting to a public communication line such as the Internet is also acceptable.

[0012] The client terminal 10 is, for example, a terminal device used by a doctor when diagnosing a patient with a mental illness. The client terminal 10 is, for example, a PC terminal or a tablet terminal. The client terminal 10 can send and receive various information to and from the server 20 via the network 30. The client terminal 10 acquires video data of the examination of the patient with a mental illness and sends it to the server 20.

[0013] Server 20 manages, for example, patient information and information related to medical facilities such as hospitals. Electronic medical records (EMRs) and personal health records (PHRs) are stored in the server's memory. Server 20 also stores video data extracted from client terminals 10 that contribute to diagnosis (hereinafter referred to as "diagnostic videos"). Furthermore, Server 20 stores video data taken during past consultations and diagnostic videos generated using past video data.

[0014] The client terminal 10 acquires video data and diagnostic videos from the server 20 in accordance with the operator's instructions, and displays this data on a display screen (hereinafter referred to as the diagnostic screen) that supports the doctor's diagnosis.

[0015] Next, we will describe the configuration of client terminal 10. The client terminal 10 includes memory 11, a communication interface 12, a display 13, an input interface 14, and a processing circuit 15. Although the client terminal 10 is described below as a single device performing multiple functions, these functions may be performed by separate devices. For example, each function performed by the client terminal 10 may be distributed and installed on different console or workstation devices.

[0016] The memory 11 is a storage device such as a HDD (Hard Disk Drive), SSD (Solid State Drive), or integrated circuit that stores various types of information. In addition to HDDs, SSDs, etc., the memory 11 may also be a portable storage medium such as a CD (Compact Disc), DVD (Digital Versatile Disc), or flash memory. Note that the memory 11 may be a drive device that reads and writes various types of information to and from semiconductor memory elements such as flash memory and RAM (Random Access Memory). Also, the storage area of the memory 11 may be within the client terminal 10 or within an external storage device connected via a network.

[0017] The memory 11 stores programs executed by the processing circuit 15, various data used in the processing of the processing circuit 15, and the like. As the program, for example, a program that is installed in a computer in advance from a network or a non-transitory computer-readable storage medium and causes the computer to realize each function of the processing circuit 15 is used. Note that the various types of data handled in this specification are typically digital data. The memory 11 is an example of a storage unit.

[0018] The communication interface 12 is a network interface that controls the transmission of communication with the server 20, external systems, external devices, etc. via the network 30.

[0019] The display 13 displays various types of information. For example, the display 13 outputs medical information generated by the processing circuit 15, a GUI (Graphical User Interface) for receiving various operations from an operator, and the like. For example, the display 13 is a liquid crystal display or a CRT (Cathode Ray Tube) display. The display 13 is an example of a display unit.

[0020] The input interface 14 receives various input operations from the operator, converts the received input operations into electrical signals, and outputs them to the processing circuit 15. For example, the input interface 14 receives input of medical information, various command signals, etc., from the operator. The input interface 14 is realized by a mouse, keyboard, trackball, switch buttons, a touchscreen integrating a display screen and touchpad, a non-contact input circuit using an optical sensor, and an audio input circuit, etc., for performing various processes of the processing circuit 15. The input interface 14 is connected to the processing circuit 15 and converts the input operations received from the operator into electrical signals and outputs them to the control circuit. In this specification, the input interface is not limited to those equipped with physical operating components such as a mouse or keyboard. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the device and outputs these electrical signals to the processing circuit 15 is also included as an example of an input interface. The input interface 14 is an example of an input unit.

[0021] The processing circuit 15 controls the overall operation of the client terminal 10. The processing circuit 15 is a processor that performs acquisition function 151, transmission function 152, reception function 153, and display control function 154 by calling and executing programs in memory 11.

[0022] In Figure 1, the acquisition function 151, transmission function 152, reception function 153, and display control function 154 are described as being implemented by a single processing circuit 15. However, it is also possible to configure a processing circuit by combining multiple independent processors, with each processor executing a program to implement each function. Furthermore, the acquisition function 151, transmission function 152, reception function 153, and display control function 154 may each be implemented as separate hardware circuits. The above description of each function performed by the processing circuit 15 is the same in the following embodiments and modifications.

[0023] Furthermore, although the client terminal 10 is described as executing multiple functions on a single console, it is also acceptable for multiple functions to be executed by separate devices. For example, the functions of the processing circuit 15 may be distributed and installed on different devices.

[0024] In the above description, the term "processor" refers to circuits such as a CPU (central processing unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD)), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). When the processor is a CPU, for example, it performs its functions by reading and executing a program stored in a memory circuit. On the other hand, when the processor is an ASIC, for example, instead of the program being stored in a memory circuit, the function is directly incorporated as a logic circuit within the processor's circuit. In this embodiment, each processor is not limited to being configured as a single circuit; multiple independent circuits may be combined to form a single processor and perform its functions. Furthermore, multiple components shown in Figure 1 may be integrated into a single processor to perform its functions. The above description of "processor" is the same in the following embodiments and modifications.

[0025] The processing circuit 15 acquires video data of the examination of a patient with a mental illness using the acquisition function 151. For example, the processing circuit 15 acquires video data recording the examination of a patient with a mental illness from a recording device such as a camera connected to the client terminal 10. The acquired video data is stored in memory 11, for example.

[0026] Furthermore, if the patient's vital signs were measured during the examination, the processing circuit 15, using the acquisition function 151, acquires vital sign measurement data (hereinafter referred to as vital data) from the vital sign measuring device. The acquired vital data is stored in the memory 11, linked to the corresponding video data. Vital signs include, for example, heart rate and blood pressure.

[0027] The processing circuit 15 transmits the acquired video data to the server 20 via the communication interface 12 using the transmission function 152. If vital data associated with the video data is stored, both the video data and the vital data are transmitted to the server 20. The processing circuit 15 that implements the transmission function 152 is an example of a transmission unit.

[0028] The processing circuit 15, via its receiving function 153, receives data such as video data and diagnostic videos stored in the server 20 via the communication interface 12 in response to instructions from the operator. The operator is, for example, a doctor. Instructions from the operator are input, for example, via the input interface 14. The received data is stored in the memory 11. The processing circuit 15 that implements the receiving function 153 is an example of a receiving unit.

[0029] The processing circuit 15 displays various information on the display 13 using the display control function 154. For example, the processing circuit 15 displays a diagnostic screen on the display 13 to assist a doctor's diagnosis. The diagnostic video displays a diagnostic video in which the portion contributing to the diagnosis has been extracted from the video data. The processing circuit 15 that implements the display control function 154 is an example of a display control unit.

[0030] Next, we will describe the configuration of server 20. Server 20 includes memory 21, a communication interface 22, and a processing circuit 23. Hereafter, Server 20 is described as a single device that performs multiple functions, but the multiple functions may be performed by separate devices. For example, each function performed by Server 20 may be distributed and installed on different console devices or workstation devices.

[0031] Memory 21 is a storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or integrated circuit that stores various types of information. Memory 21 may also be a portable storage medium other than an HDD or SSD, such as a CD (Compact Disc), DVD (Digital Versatile Disc), or flash memory. Furthermore, memory 21 may be a drive device that reads and writes various types of information to and from semiconductor memory elements such as flash memory or RAM (Random Access Memory). The storage area of ​​memory 21 may be located within the server 20 or in an external storage device connected via a network.

[0032] Memory 21 stores programs executed by the processing circuit 23, various data used in the processing of the processing circuit 23, etc. As a program, for example, a program is used that is pre-installed on the computer from a network or a non-transient computer-readable storage medium, and that enables the computer to implement each function of the processing circuit 23. The various data dealt with in this specification are typically digital data. Memory 21 is an example of a storage unit. Memory 21 also stores video data and diagnostic videos transmitted from the client terminal 10.

[0033] The communication interface 22 is a network interface that transmits and controls communication with the client terminal 10, external systems, external devices, etc., via the network 30.

[0034] The processing circuit 23 controls the operation of the entire server 20. The processing circuit 23 is a processor that performs acquisition function 231, location identification function 232, video generation function 233, information generation function 234, search function 235, summarization function 236, and output function 237 by calling and executing programs in memory 21.

[0035] In Figure 1, the acquisition function 231, location identification function 232, video generation function 233, information generation function 234, search function 235, summarization function 236, and output function 237 are described as being implemented by a single processing circuit 23. However, it is also possible to configure a processing circuit by combining multiple independent processors, with each processor executing a program to implement each function. Furthermore, the acquisition function 231, location identification function 232, video generation function 233, information generation function 234, search function 235, summarization function 236, and output function 237 may each be implemented as separate hardware circuits. The above description of each function performed by the processing circuit 23 is the same in the following embodiments and modifications.

[0036] Furthermore, although the server 20 is described as performing multiple functions on a single console, it is also acceptable for multiple functions to be performed by separate devices. For example, the functions of the processing circuit 23 may be distributed and installed on different devices.

[0037] The processing circuit 23, using the acquisition function 231, acquires video data of the patient's medical examination related to mental illness from the client terminal 10 via the communication interface 22. The acquired video data is sequentially stored in the memory 21. The processing circuit 23 that implements the acquisition function 231 is an example of an acquisition unit.

[0038] Next, we will describe the position identification function 232 performed by the processing circuit 23. The processing circuit 23 uses a position identification function 232 to identify positions usable for diagnosis (hereinafter referred to as diagnostic positions) in the video data acquired by the acquisition function 231 (hereinafter also referred to as the video data to be processed). Diagnostic positions are, for example, positions that affect the results. For example, parts of the video data to be processed where physical behavior has changed, parts where the content of the answer to a specific question has changed compared to the previous examination, parts where the conversation contains content related to the change in behavior, or parts where vital data has changed are identified as diagnostic positions. Information regarding the results of the identification of diagnostic positions is stored in memory 21, linked to the video data to be processed. The processing circuit 23 that realizes the position identification function 232 is an example of a position identification unit.

[0039] Here, we will explain in detail the types of diagnostic locations and methods for identifying them. For example, a diagnostic location may be identified as a position where the patient's physical behavior changes. In this case, the processing circuit 23 in the position identification function 232 identifies the diagnostic location based on the change in the patient's physical behavior. Changes in physical behavior include, for example, changes in facial expression, changes in voice tone, changes in behavior, etc. For example, if the part where the facial expression changes is used as the diagnostic location, the processing circuit 23 first analyzes the patient's facial expression in the video data to be processed and quantifies specific emotions. For example, known facial expression analysis software can be used to analyze the patient's facial expression. Specific emotions include, for example, joy, interest, sadness, emptiness, despair, etc. The processing circuit 23 identifies the part in the video data where the numerical value related to the specific emotion is largest or smallest as the diagnostic location. For example, the part in the video data where the numerical value related to joy is largest or the numerical value related to interest is smallest may be identified as a diagnostic location that contributes to the diagnosis. Alternatively, the part where the numerical value related to the specific emotion exceeds a preset standard value may be identified as the diagnostic location.

[0040] When using a section where the tone of voice changes as a diagnostic location, the processing circuit 23 first quantifies the pitch of the patient's voice and the speed of their speech in the video data being processed. For example, known language analysis software can be used to quantify the pitch of the voice and the speed of speech. The processing circuit 23 identifies the section in the video data where the pitch of the voice or the speed of speech is highest or lowest as the diagnostic location. Alternatively, the section where the pitch of the voice or the speed of speech exceeds a preset reference value may be identified as the diagnostic location. The reference value may be set by the user, or the average of the values ​​from the previous examination of the same patient may be used.

[0041] When using a section of the video data to be processed where the patient's behavior has changed as a diagnostic location, the processing circuit 23 first detects specific patient behaviors in response to specific questions in the video data to be processed. Examples of specific behaviors include leaning forward, nodding, or avoiding eye contact with the doctor. For behavior detection, for example, known behavior detection software can be used. Memory 21 also stores the detection results of behaviors in response to specific questions during past consultations with the same patient. The processing circuit 23 reads the detection results of behaviors in response to the same questions from the previous consultation from memory 21 and compares them with the behavior detected in the video data to be processed. The processing circuit 23 identifies the section of the video data to be processed related to the question in which the patient's behavior has changed as a diagnostic location.

[0042] Furthermore, when the portion of the answer to a specific question that has changed compared to the previous consultation is used as the diagnostic position, the processing circuit 23 in the position identification function 232 identifies the diagnostic position based on the change in the answer to the specific question. In this case, the processing circuit 23 first analyzes the answer to the specific question in the video data to be processed. The specific question may be, for example, a standardized question that is confirmed each time during a consultation. Standardized questions may include, for example, questions based on the judgment criteria of the DSM (Diagnostic and Statistical Manual of Mental Disorders). For the analysis of the answer content, for example, known language analysis software may be used. For example, the software may analyze whether the answer content is positive or negative. In addition, the memory 21 stores the analysis results of the answer content to the specific question from past consultations for the same patient. The processing circuit 23 reads the answer content to the same question from the previous consultation from the memory 21 and compares it with the analysis results of the answer content in the video data to be processed. The processing circuit 23 identifies the portion of the video data being processed where the patient's responses have changed compared to the responses given during the previous consultation as the diagnostic position. Alternatively, instead of comparing the patient's responses to specific questions (hereinafter referred to as reaction speed), the processing circuit 23 may compare the patient's response speed to specific questions. In this case, the processing circuit 23 identifies the portion of the video data where the change in reaction speed is greater than a predetermined value as the diagnostic position.

[0043] Furthermore, when using a portion of the conversation containing information about a change in behavior as a diagnostic location, the processing circuit 23 in the location identification function 232 identifies the diagnostic location based on the fact that the conversation includes information about a change in behavior. In this case, the processing circuit 23 first analyzes the content of the conversation in the video data to be processed. For example, known language analysis software can be used to analyze the content of the conversation. Based on the results of the analysis of the content of the conversation, the processing circuit 23 identifies the portion of the video data to be processed in which the patient talks about a change in behavior since the last consultation as the diagnostic location.

[0044] When a patient's emotional state changes, their vital data may change. Therefore, changes in vital data are important information for diagnosis. When identifying a diagnostic location where vital data has changed, the processing circuit 23 in the acquisition function 231 acquires the patient's vital data measured at the time the video data was captured, in addition to the video data to be processed. Then, in the location identification function 232, the processing circuit 23 identifies the diagnostic location based on the changes in vital data. At this time, the processing circuit 23 detects changes in the acquired vital data and identifies the portion of the video data to be processed that corresponds to a period in which the change in vital data is greater than the reference value as the diagnostic location. Examples of vital data used for detecting changes include heart rate and blood pressure.

[0045] Next, we will describe the video generation function 233 performed by the processing circuit 23. The processing circuit 23 generates a diagnostic video including the diagnostic location using the video generation function 233. Specifically, the processing circuit 23 generates a diagnostic video by extracting the portion of the video data to be processed that includes the diagnostic location, based on the diagnostic location information associated with the video data to be processed. The generated diagnostic video is stored in memory 21 in association with the video data to be processed. The processing circuit 23 that implements the video generation function 233 is an example of a video generation unit.

[0046] For example, one diagnostic video is generated for each diagnostic location. In this case, if multiple diagnostic locations are associated with the video data being processed, the same number of diagnostic videos as diagnostic locations will be generated for each video data. Alternatively, one diagnostic video may be generated for multiple diagnostic locations. In this case, for example, a video created by combining and stitching together videos of the same type of diagnostic locations identified using the same identification method may be generated as the diagnostic video.

[0047] For example, if the location with the highest numerical value calculated through analysis of the video data to be processed is identified as the diagnostic location, then, for instance, a video clip from a predetermined period centered around the diagnostic location will be extracted as the diagnostic video. Alternatively, if the location of a change since the last examination is identified as the diagnostic location, then, for example, a video clip from the point where the change began to the point where the change ended will be extracted as the diagnostic video. Furthermore, if the location of a diagnostic location is identified as a portion related to a specific question or conversation, then, for example, a video clip from the period in which the patient's responses to that specific question or conversation are taking place will be extracted as the diagnostic video.

[0048] Next, the information generation function 234 performed by the processing circuit 23 will be described. The processing circuit 23 generates information regarding the method of identifying the diagnostic location (hereinafter referred to as "identification information") using the information generation function 234 and attaches it to the diagnostic video. In the information generation function 234, the processing circuit 23 generates identification information indicating the method of identifying the diagnostic location contained in the diagnostic video for each of the one or more diagnostic videos generated from the video data to be processed. The generated identification information is stored in the memory 21 in association with the diagnostic video. The processing circuit 23 that implements the information generation function 234 is an example of an information generation unit.

[0049] Next, we will describe the search function 235 performed by the processing circuit 23. On the client terminal 10, a request from the operator regarding a video to be reviewed during diagnosis is input via the input interface 14. The operator is, for example, a psychosomatic medicine or psychiatry physician. The processing circuit 15 of the client terminal 10 transmits the input request from the operator to the server 20 via the communication interface 12 and the network 30. The processing circuit 23 of the server 20 receives the operator's request transmitted from the client terminal 10 via the communication interface 22 and the network 30. The processing circuit 23 uses a search function 235 to search the memory 21 for a diagnostic video that satisfies the request based on the operator's request and specific information. Specifically, in the search function 235, the processing circuit 23 searches the memory 21 for a diagnostic video that has specific information that satisfies the request based on the operator's request. The processing circuit 23 that implements the search function 235 is an example of a search unit.

[0050] The operator's request may include, for example, search criteria for diagnostic videos. The search criteria may specify, for example, a method for identifying diagnostic locations contained in the diagnostic videos. In this case, the processing circuit 23 in the search function 235 searches for a diagnostic video containing a diagnostic location identified by the specified identification method from among multiple diagnostic videos stored in the memory 21, based on the received search criteria and the identification information attached to each diagnostic video. The processing circuit 23 then saves the search result in the memory 21 as a diagnostic video that satisfies the search criteria.

[0051] Next, we will describe the summarization function 236 performed by the processing circuit 23. The processing circuit 23, using the summarization function 236, performs a process (hereinafter also referred to as summarization) to summarize text data that is a transcription of the patient's conversation for a diagnostic video containing pre-specified specific information, thereby generating text data that shows the summarization result (hereinafter referred to as summary data). The processing circuit 23 that implements the summarization function 236 is an example of a summarization unit. The generated summary data is linked to the diagnostic video and stored in the memory 21.

[0052] For example, for diagnostic videos that include sections showing changes in physical behavior or vital data, summary data is generated by transcribing and summarizing the content of the conversation immediately preceding the diagnostic point. Similarly, for diagnostic videos that include sections showing changes in responses since the previous consultation or sections where the patient talks about changes in behavior since the previous consultation, summary data is generated by transcribing and summarizing the content of the conversation included in the diagnostic video.

[0053] The processing circuit 23 outputs the diagnostic video retrieved by the search function 235 via the output function 237. In the output function 237, the processing circuit 23 reads the diagnostic video included in the search results of the search function 235 from the memory 21 and transmits the retrieved diagnostic video to the client terminal 10 via the communication interface 22 and the network 30. The transmitted diagnostic video is displayed on the diagnostic screen of the display 13 on the client terminal 10. The processing circuit 23 that implements the output function 237 is an example of an output unit.

[0054] Furthermore, in the output function 237, if the summary data generated by the summarization function 236 is linked to the diagnostic video to be output, the processing circuit 23 outputs the summary data together with the diagnostic video to the client terminal 10. The transmitted summary data is displayed on the diagnostic screen of the client terminal 10 together with the diagnostic video.

[0055] Next, the operation of the medical examination video recording process performed by the client terminal 10 and server 20 of the medical information processing system 1 will be described. The medical examination video recording process is the process of generating a diagnostic video by cutting out the parts that contribute to the diagnosis from the video data acquired when the patient's condition was filmed during the examination (hereinafter referred to as the medical examination video), and recording the generated diagnostic video. Figure 2 is a flowchart of an example of the procedure for the medical examination video recording process. Note that the processing procedures in each process described below are merely examples, and each process can be modified as appropriate as possible. In addition, steps in the processing procedures described below can be omitted, replaced, and added as appropriate, depending on the embodiment.

[0056] (Video recording and processing of medical examinations) (Step S101) The processing circuit 15 of the client terminal 10 acquires a video of the patient's condition during the examination using the acquisition function 151. In this process, the processing circuit 15 controls the video camera to record the examination. Once recording is complete, it automatically acquires the examination video from the camera. If a measuring device for measuring vital data is used, the processing circuit 15 controls the measuring device to simultaneously acquire the patient's vital data during the examination.

[0057] (Step S102) The processing circuit 15 of the client terminal 10 transmits the medical examination video acquired in step S101 to the server 20 via the network 30 using the transmission function 152. If the patient's vital data was acquired in step S101, the vital data is linked to the medical examination video and transmitted to the server 20.

[0058] (Step S103) The processing circuit 23 of the server 20 receives the medical examination video transmitted from the client terminal 10 using the acquisition function 231. The received medical examination video is stored in the memory 21.

[0059] (Step S104) The processing circuit 23 of the server 20 uses a position identification function 232 to identify the diagnostic positions of the medical examination video acquired in step S103 as the medical examination video to be processed. In this process, the diagnostic positions are identified for each medical examination video using the multiple methods described above. Therefore, typically, multiple diagnostic positions are identified for a single medical examination video. The identified diagnostic positions are associated with the medical examination video to be processed and stored in the memory 21.

[0060] (Step S105) The processing circuit 23 of the server 20 generates a diagnostic video including the diagnostic location based on the medical examination video to be processed and the diagnostic location identified in step S104, using the video generation function 233. The diagnostic video is generated in a manner corresponding to how the diagnostic location is identified. For example, if the location where a specific numerical value is maximum or minimum is identified as the diagnostic location, a video of a predetermined duration centered on the diagnostic location is extracted from the medical examination video as the diagnostic video. The predetermined duration is, for example, several tens of seconds to several minutes. Also, if the location where a specific conversation takes place is identified as the diagnostic location, a video of the portion identified as the diagnostic location is extracted from the medical examination video as the diagnostic video. Furthermore, the diagnostic video may include the portion of the conversation immediately preceding the specific conversation.

[0061] Furthermore, the processing circuit 23 of the server 20 generates still images (hereinafter referred to as display still images) to be used as thumbnails for the diagnostic video, based on the generated diagnostic video and the diagnostic position, using the video generation function 233. For example, if the part where a specific numerical value is maximum or minimum is identified as the diagnostic position, a still image of the position in the diagnostic video where the specific numerical value is maximum or minimum is extracted as a display still image. Also, if the part where a specific conversation takes place is identified as the diagnostic position, for example, a still image of the time in the middle of the part identified as the diagnostic position is extracted from the examination video as a display still image. The generated display still images are linked to the diagnostic video and stored in memory 21.

[0062] In this way, at least one diagnostic video is generated for each diagnostic location. Therefore, if multiple diagnostic locations are stored for the examination video being processed, multiple diagnostic videos are generated for a single examination video. The generated diagnostic videos are stored in memory 21, linked to the examination video.

[0063] (Step S106) The processing circuit 23 of the server 20 generates information (hereinafter referred to as "identification information") regarding the method of identifying the diagnostic location contained in the diagnostic video for each diagnostic video generated in the processing of step S105, using the information generation function 234, and attaches it to the diagnostic video. The identification information is stored in memory 21, linked to the diagnostic video.

[0064] Through the processing from step S101 to step S106, an examination video recording of the examination is generated, and a diagnostic video including diagnostic locations that contribute to the diagnosis in the examination video is also generated and stored in association with the examination video. Each time an examination is performed, the examination video and the diagnostic video are accumulated in memory 21.

[0065] Next, the operation of the diagnostic support processing performed by the client terminal 10 and server 20 of the medical information processing system 1 will be described. Diagnostic support processing is the process of displaying a screen to assist the doctor in making a diagnosis using a medical examination video that records the contents of the examination. Figure 3 is a flowchart of an example of the diagnostic support processing procedure. Note that the processing procedures in each process described below are merely examples, and each process can be modified as appropriate as possible. In addition, steps in the processing procedures described below can be omitted, replaced, and added as appropriate, depending on the embodiment.

[0066] (Diagnostic support processing) (Step S201) The processing circuit 15 of the client terminal 10 accepts the selection of a medical examination video to be used for diagnosis via the input interface 14. The processing circuit 15 uses the display control function 154 to display a display screen (hereinafter referred to as the diagnostic support screen) on the display 13 to assist the doctor's diagnosis.

[0067] Figure 4 shows an example of a diagnostic support screen 100 displayed on the display 13. The diagnostic support screen 100 shown in Figure 4 includes a search condition input unit 110, a candidate video display unit 120, and a selected video display unit 130.

[0068] (Step S202) The processing circuit 15 of the client terminal 10 receives search criteria input from the doctor via the input interface 14. The search criteria are entered into the search criteria input section 110 of the diagnostic support screen 100. For example, search criteria can include words related to the diagnostic location or words related to the content of the questions. For example, conditions such as "smile," "change in behavior," "pulse rate," "change in actions," and "video of anger" can be entered. The processing circuit 15 of the client terminal 10 transmits the entered search criteria to the server 20 using the transmission function 152.

[0069] (Step S203) The server 20's processing circuit 23 uses a search function 235 to search for a diagnostic video that matches the search criteria received from the client terminal 10, from among the diagnostic videos associated with the selected examination video. At this time, specific information associated with the diagnostic video is used to determine whether or not it matches the search criteria. For example, if the phrase "change in behavior" is entered into the search condition input unit 110 as a search criterion, the memory 21 searches for a diagnostic video that includes a diagnostic location identified by analyzing the patient's change in behavior from among multiple diagnostic videos stored in the memory 21. The server 20's processing circuit 23 uses an output function 237 to send the searched diagnostic video to the client terminal 10. At this time, in addition to the diagnostic video, a display still image associated with the diagnostic video is also sent to the client terminal 10 as a thumbnail of the diagnostic video.

[0070] (Step S204) The processing circuit 15 of the client terminal 10, using the display control function 154, displays diagnostic videos that match the search criteria on the diagnostic support screen 100 based on the search results received from the server 20. At this time, a still image associated with the diagnostic video is displayed as a thumbnail for the diagnostic video. In the example shown in Figure 4, the candidate video display unit 120 displays diagnostic videos that match the search criteria entered in the search criteria input unit 110. For example, if multiple diagnostic videos match the search criteria, multiple diagnostic videos are displayed in the candidate video display unit 120.

[0071] (Step S205) The processing circuit 15 of the client terminal 10 accepts the selection of a diagnostic video to be played via the input interface 14. For example, a doctor selects the diagnostic video to display by selecting one of the thumbnails displayed on the candidate video display unit 120.

[0072] (Step S206) The processing circuit 15 of the client terminal 10 displays the selected diagnostic video (hereinafter referred to as the selected video) on the diagnostic support screen 100 using the display control function 154. In the example shown in Figure 4, the selected video is displayed and played on the selected video display unit 130. The diagnostic support screen 100 also includes a timeline display unit 140. The timeline display unit 140 shows the position of the selected video within the entire examination video.

[0073] (Step S207) The processing circuit 15 of the client terminal 10 and the processing circuit 23 of the server 20 acquire specific information associated with the selected video and change the display method of the selected video according to the specific information.

[0074] Figure 5 is a flowchart showing an example of the procedure for the process performed in step S207 of Figure 4. Note that the procedure for each process described below is merely an example, and each process can be modified as appropriate. Furthermore, depending on the embodiment, steps in the procedure described below can be omitted, replaced, or added as appropriate.

[0075] (Step S301) The processing circuit 15 of the client terminal 10 determines, based on specific information associated with the selected video, whether or not there is a change in physical behavior or vital data in the selected video. In other words, the processing circuit 15 determines whether or not the diagnostic location included in the diagnostic video displayed on the selected video display unit 130 was identified by detecting a change in physical behavior or a change in vital data.

[0076] (Step S302) If the selected video contains changes in physical behavior or vital data (step S301-Yes), the processing circuit 15 of the client terminal 10 sends an instruction to the server 20 to summarize the video content of the selected video. The processing circuit 23 of the server 20 uses a summarization function 236 to analyze the conversation content immediately preceding the diagnostic position in the selected video and generates summary data that summarizes the conversation content. The processing circuit 23 of the server 20 sends the generated summary data to the client terminal 10.

[0077] (Step S303) The processing circuit 15 of the client terminal 10 displays the summary data acquired from the server 20 on the same screen as the diagnostic video using the display control function 154. In the example shown in Figure 4, the diagnostic support screen 100 includes a summary display unit 150. The summary display unit 150 displays the summary results of the selected video. The summary display unit 150 is displayed overlaid on the selected video display unit 130. For example, if the diagnostic position included in the selected video is the position where the value of "joy" is the highest, the summary display unit 150 displays text data summarizing the conversation content immediately preceding the diagnostic position.

[0078] Figure 6 shows a modified example of the diagnostic support screen 100 displayed by the processing in step S303. As shown in Figure 6, the summary display unit 150 may be located in a different area from the selected video display unit 130.

[0079] (Step S304) If the selected video does not contain any changes in physical behavior or vital data (step S301-No), the processing circuit 15 of the client terminal 10 determines whether or not there are changes in the conversation content in the selected video, based on specific information associated with the selected video. In other words, the processing circuit 15 determines whether or not the diagnostic position included in the diagnostic video displayed on the selected video display unit 130 was identified by detecting a change in specific conversation content.

[0080] (Step S305) If the selected video contains changes in conversation content (step S304-Yes), the processing circuit 15 of the client terminal 10 sends an instruction to the server 20 to search for the diagnostic video from the previous consultation. The processing circuit 23 of the server 20 uses the video generation function 233 to generate a video (hereinafter referred to as the "past diagnostic video") that includes the responses to the specific questions based on past consultation videos. Specifically, the processing circuit 23 searches the consultation videos stored in memory 21 for consultation videos created during the previous consultation of the same patient, and based on the searched past consultation videos, generates a past diagnostic video that extracts the responses to the same questions asked in the selected video. The processing circuit 23 of the server 20 sends the generated past video to the client terminal 10.

[0081] Furthermore, the processing circuit 23 of the server 20 generates first text data, which is a text transcription of the conversation content related to standardized questions in the medical examination video, and second text data, which is a text transcription of the conversation content related to standardized questions in past medical examination videos, using the summarization function 236. The processing circuit 23 of the server 20 sends the generated first text data and second text data to the client terminal 10.

[0082] (Step S306) The processing circuit 15 of the client terminal 10 displays the selected video, the past diagnostic video, the first text data, and the second text data acquired from the server 20 on the same screen using the display control function 154. Figure 7 shows an example of the diagnostic support screen 100 displayed by the processing in step S306. In the example shown in Figure 7, the diagnostic support screen 100 includes, in addition to the selected video display unit 130, a previous video display unit 160, a first text display unit 131, and a second text display unit 161.

[0083] The selected video display unit 130 displays and plays the selected video. The previous video display unit 160 displays and plays a past diagnostic video containing a conversation with the same content as the selected video. By comparing the selected video displayed on the selected video display unit 130 with the past diagnostic video displayed on the previous video display unit 160, the doctor can easily grasp how the patient's responses to questions with altered conversation content have changed.

[0084] Furthermore, the first text display unit 131 displays a list of text data containing standardized questions and answers from the medical examination video linked to the selected video. The second text display unit 161 displays a list of text data containing standardized questions and answers from the medical examination video of the previous examination. In this way, the first text display unit 131 and the second text display unit 161 display tables listing the answers from the previous and current examinations. By using the tables listing the answers from the previous and current examinations, the doctor can easily grasp the changes in the answers to standardized questions by comparing the answers.

[0085] Figure 8 shows a modified version of the diagnostic support screen 100 displayed by the processing in step S306. In this modified version, the processing circuit 15 of the client terminal 10 sends an instruction to the server 20 to summarize the contents of the medical examination video associated with the selected video and the medical examination video from the previous examination. The processing circuit 23 of the server 20 generates first summary data, which is a text summary of the conversation content regarding non-standard questions in the medical examination video associated with the selected video, and second summary data, which is a text summary of the conversation content regarding non-standard questions in the medical examination video from the previous examination. The processing circuit 23 of the server 20 sends the generated first summary data and second summary data, along with the first text data and second text data, to the client terminal 10. The processing circuit 23 that executes the summarization function 236 in this modified version is an example of a text generation unit.

[0086] In this modified example, the processing circuit 15 in the display control function 154 displays the selected video, past diagnostic videos, first text data, second text data, first summary data, and second summary data acquired from the server 20 on the same screen. As shown in Figure 8, the diagnostic support screen 100 further comprises a first summary display unit 132 and a second summary display unit 162. The first summary display unit 132 displays first summary data, which summarizes the non-standard questions and answers to the non-standard questions in the examination video linked to the selected video. The second summary display unit 162 displays second summary data, which summarizes the non-standard questions and answers to the non-standard questions in the examination video linked to the previous video. By comparing the first summary data and the second summary data, the physician can easily compare the answers to the non-standard questions.

[0087] (Step S307) If the selected video does not contain a change in conversation content (step S304-No), the processing circuit 15 of the client terminal 10 determines whether or not the selected video contains conversation related to a change in behavior, based on specific information associated with the selected video. In other words, the processing circuit 15 determines whether or not the diagnostic position included in the diagnostic video displayed on the selected video display unit 130 was identified by detecting conversation related to a change in behavior.

[0088] (Step S308) If the selected video contains a conversation about a change in behavior (step S307-Yes), the processing circuit 15 of the client terminal 10 sends an instruction to the server 20 to summarize the content of the selected video. The processing circuit 23 of the server 20 analyzes the conversation content at the diagnostic position in the selected video using the summarization function 236 and generates summary data that summarizes the conversation content. The processing circuit 23 of the server 20 sends the generated summary data to the client terminal 10.

[0089] (Step S309) The processing circuit 15 of the client terminal 10 displays the summary data acquired from the server 20 on the same screen as the diagnostic video using the display control function 154. For example, as shown in the example in Figures 4 and 6, the summary result of the selected video is displayed on the summary display unit 150 of the diagnostic support screen 100. If the diagnostic position included in the selected video is a position where a conversation about behavioral changes is being spoken, the summary display unit 150 displays text data summarizing the content of the conversation about behavioral changes.

[0090] (Step S310) If the selected video does not contain a conversation about a change in behavior (step S307-No), that is, if the method for identifying the diagnostic location does not fall under any of the above methods, the processing circuit 15 in the display control function 154 displays the diagnostic support screen 100 using a pre-set default display method. For example, the diagnostic support screen 100, which includes only the search condition input unit 110, the candidate video display unit 120, the selected video display unit 130, and the timeline display unit 140, is displayed on the display 13.

[0091] The effects of the medical information processing system 1 according to this embodiment will be described below.

[0092] The medical information processing system 1 according to this embodiment can acquire video data of the examination of a patient with a mental illness, identify diagnostic positions in the video data that can be used for diagnostic results, generate a diagnostic video including the diagnostic positions, generate specific information regarding the method of identifying the diagnostic positions, attach it to the diagnostic video, and store multiple diagnostic videos. Furthermore, the medical information processing system 1 according to this embodiment can search the memory 21 for a diagnostic video that has been attached with specific information that satisfies the request, based on a request from the operator, and output the retrieved diagnostic video.

[0093] With the above configuration, the medical information processing system 1 according to this embodiment can automatically identify diagnostic locations that contribute to the diagnosis from the video during the medical interview, generate a diagnostic video including the diagnostic locations, and display it at the operator's request, thereby providing physicians with information to help them make decisions when filling out medical records or during diagnosis. Unlike conventional methods, physicians can efficiently review the video during the medical interview by checking the diagnostic video from which the parts contributing to the diagnosis have been extracted. This supports physicians in making diagnoses and reduces their workload. Furthermore, with the above configuration, unlike conventional methods, it is possible to output a diagnostic video that includes important information other than transcribed audio.

[0094] Furthermore, subjective information regarding the patient's psychological state is crucial in diagnosing mental illness. By presenting physicians with diagnostic videos that include diagnostic locations identified considering the patient's psychological state, physicians can efficiently acquire subjective information important for diagnosis. In addition, because information regarding diagnostic locations can be reviewed as a video, physicians can make diagnoses using objective information as well as memory.

[0095] When a patient's emotional state changes, their physical behavior and vital data may also change. Therefore, changes in physical behavior and vital data are important information for diagnosis. As a result, the diagnostic location can be identified based on changes in the patient's physical behavior. Furthermore, the patient's vital data can be acquired during the recording of the video data, and the diagnostic location can be identified based on changes in the vital data. The generated diagnostic video can also be displayed on the display 13. In such cases, the medical information processing system 1 generates summary data by transcribing the content of the conversation immediately preceding the diagnostic location in the video data into text, and can display the summary data on the same screen as the diagnostic video, as shown in Figures 4 and 6. By reviewing diagnostic videos of changes in physical behavior and then examining a text summary of the conversation immediately preceding the change in physical behavior, physicians can determine the cause of the change without any special manipulation.

[0096] Furthermore, changes in the patient's behavior are important information in diagnosis. Therefore, the diagnostic location can be identified based on whether the conversation includes information about changes in behavior. In this case, the medical information processing system 1 generates summary data by transcribing the content of the conversation in the diagnostic video into text, and can display the summary data on the same screen as the diagnostic video, as shown in Figures 4 and 6.

[0097] Doctors can confirm changes in a patient's behavior without any special procedures by reviewing diagnostic videos of patients describing changes in their behavior and by reviewing a text summary of the conversation.

[0098] Furthermore, changes in the patient's responses are important information for diagnosis. Therefore, it is possible to identify the diagnostic position based on changes in the answers to specific questions. In this case, the medical information processing system 1 generates a diagnostic past video that includes the patient's responses to specific questions based on past video data, and can display the selected diagnostic video and the diagnostic past video on the same screen, as shown in Figure 7.

[0099] For example, by using a set of standard questions that are asked at every consultation as specific questions, if the answer to that standard question changes, the doctor can review the current and past videos of the answer without any special operation.

[0100] Furthermore, it is possible to generate first text data, which is a text transcription of the conversation content in the video data, and second text data, which is a text transcription of the conversation content in past video data, and display the selected diagnostic video, past diagnostic video, first text data, and second text data on the same screen, as shown in Figure 8. It is also possible to generate first summary data, which is a text transcription and summary of the conversation content in the video data, and second summary data, which is a text transcription and summary of the conversation content in past video data, and display the selected diagnostic video, past diagnostic video, first summary data, and second summary data on the same screen. For example, it is possible to display the answers to standard questions in text format, and display the answers to non-standard questions in a summarized format.

[0101] Without requiring any special procedures, doctors can review diagnostic videos to see how patients have responded to questions with varying answers, and easily compare the responses from the previous consultation with those from the current consultation.

[0102] Furthermore, the medical information processing system 1 can generate still images based on diagnostic videos and output the generated still images as thumbnails for the diagnostic videos. The generated still images are used as thumbnails when displaying multiple diagnostic videos. By checking the thumbnails, doctors can easily understand the content of the diagnostic videos.

[0103] According to at least one embodiment described above, the burden of reviewing videos during consultations related to mental illness can be reduced.

[0104] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0105] 1…Medical information processing system 10…Client terminals 11…Memory 12…Communication Interface 13…Display 14…Input Interface 15…Processing circuit 151... Acquisition function 152...Transmission function 153... Receiving function 154…Display control function 20... Server 21…Memory 22…Communication Interface 23… Processing circuit 231... Acquisition function 232...Positioning function 233...Video generation function 234…Information generation function 235... Search function 236…Summarization function 237…Output function 30…Network 100...Diagnostic support screen 110... Search criteria input section 120... Candidate video display section 130...Selected video display section 140...Timeline display section 150, 132, 162... Summary display section 160...Previous video display section 131, 161... Text display section

Claims

1. An acquisition unit that acquires video data of the content of medical examinations for patients with mental illnesses, A position identification unit identifies a diagnostic position in the aforementioned video data that can be used for diagnosis, Multiple diagnostic videos are generated, each of which includes a video generation unit that includes a diagnostic location identified using a different identification method. An information generation unit generates specific information that can identify the method for identifying the diagnostic location and adds it to the diagnostic video, A storage unit that stores multiple diagnostic videos, A search unit retrieves a diagnostic video from the storage unit that includes a diagnostic location identified by the specified method, based on a request from an operator to specify a method for identifying the diagnostic location. An output unit that outputs the searched diagnostic video, A medical information processing system equipped with [the following features].

2. The location identification unit identifies the diagnostic location based on the changes in the patient's physical behavior. The medical information processing system according to claim 1.

3. The acquisition unit further acquires the patient's vital data at the time of recording the video data. The location identification unit identifies the diagnostic location based on the changes in the vital data. The medical information processing system according to claim 1.

4. A display control unit that displays the aforementioned diagnostic video on a display, A summarization unit generates summary data by transcribing the content of the conversation immediately preceding the diagnostic position in the video data into text and summarizing it. Furthermore, The display control unit displays the summary data on the same screen as the diagnostic video. The medical information processing system according to claim 2 or 3.

5. The location identification unit identifies the diagnostic location based on the fact that the conversation includes a change in behavior. The medical information processing system according to claim 1.

6. A display control unit that displays the aforementioned diagnostic video on a display, A summarization unit generates summary data by transcribing the content of the conversation in the aforementioned diagnostic video into text and summarizing it. Furthermore, The display control unit displays the summary data on the same screen as the diagnostic video. The medical information processing system according to claim 5.

7. The location identification unit identifies the diagnostic location based on changes in the content of the answers to specific questions. The medical information processing system according to claim 1.

8. The system further includes a display control unit that displays the aforementioned diagnostic video on a display, The video generation unit generates a diagnostic past video, including the video of the response to the specific question, based on past video data. The display control unit displays the selected diagnostic video and the past diagnostic video on the same screen. The medical information processing system according to claim 7.

9. The system further includes a text generation unit that generates first text data obtained by transcribing the content of the conversation in the aforementioned video data into text, and second text data obtained by transcribing the content of the conversation in the aforementioned past video data into text. The display control unit displays the selected diagnostic video, the past diagnostic video, the first text data, and the second text data on the same screen. The medical information processing system according to claim 8.

10. The system further includes a summarization unit that generates a first summary data obtained by transcribing and summarizing the content of the conversation in the aforementioned video data into text, and a second summary data obtained by transcribing and summarizing the content of the conversation in the aforementioned past video data into text. The display control unit displays the selected diagnostic video, the past diagnostic video, the first summary data, and the second summary data on the same screen. The medical information processing system according to claim 8.

11. The aforementioned video generation unit generates still images based on the diagnostic video, The output unit outputs the still image as a thumbnail for the diagnostic video. The medical information processing system according to claim 1.

Images