System

Abstract

To provide a system that can accurately record the contents of medical examination and easily review them later.SOLUTION: A system according to the embodiment comprises a recording unit, a transcription unit, a summary creation unit, and an answer unit. The recording unit records audio. The transcription unit transcribes the audio recorded by the recording unit. The summary creation unit creates a summary of diagnosis results based on a text generated by the transcription unit. The answer unit provides an answer to the user's question based on the summary created by the summary creation unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional techniques have the problem that it is difficult to accurately record the contents of a medical examination and check them later.

[0005] The system according to the embodiment aims to accurately record the contents of a medical examination so that the contents can be easily checked later. [Means for solving the problem]

[0006] The system according to the embodiment includes a recording unit, a transcription unit, a summary creation unit, and an answering unit. The recording unit records audio. The transcription unit transcribes the audio recorded by the recording unit. The summary creation unit creates a summary of the diagnosis result based on the text created by the transcription unit. The answering unit provides an answer to the user's question based on the summary created by the summary creation unit. Effect of the Invention

[0007] The system according to the embodiment can accurately record the contents of a medical examination, allowing the contents to be easily checked later. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The medical examination content recording system according to the embodiment of the present invention is a system that allows elderly people to record the contents of their medical examination without forgetting them and to accurately understand the diagnosis results and advice. In this system, when a user visits a hospital, the user starts an app and uses the smartphone's microphone to record the doctor's diagnosis by voice. Next, the generation AI analyzes the recorded voice and transcribes it. The generation AI creates a summary of the diagnosis results based on the transcribed text. In addition, information such as the doctor's advice and prescribed medicine is also recorded. When a user forgets the diagnosis, advice, or prescribed medicine, the user can ask the app a question and the generation AI will provide an answer from the recorded text. For example, when the user asks "Tell me what the doctor said," the generation AI will display a summary of the diagnosis results. Also, when the user asks "Tell me what medicine was prescribed," the generation AI will display information about the prescribed medicine. This app prevents elderly people from forgetting the contents of their medical examination and allows them to accurately understand the diagnosis results and advice. In addition, by using the generation AI, the transcription of voice and the creation of summaries are automated, reducing the burden on the user. This allows the medical examination details recording system to ensure that elderly people do not forget to record the details of their medical examinations, and to accurately understand diagnosis results and advice.

[0029] The medical examination content recording system according to the embodiment includes a recording unit, a transcription unit, a summary creation unit, and an answering unit. The recording unit records the doctor's diagnosis by voice using a microphone of a smartphone when a user is examined at a hospital. The recording unit can record the voice using, for example, a smartphone built-in microphone or an external microphone. The transcription unit uses a generation AI to transcribe the voice recorded by the recording unit. The generation AI analyzes the voice data using, for example, deep learning or machine learning algorithms and performs transcription. The summary creation unit uses a generation AI to create a summary of the diagnosis result based on the text generated by the transcription unit. The generation AI creates a summary of the diagnosis result using, for example, a text generation AI (for example, LLM). The answering unit analyzes the user's question and provides an answer from the recorded text. The answering unit analyzes the user's question using, for example, natural language processing technology and provides an appropriate answer. As a result, the medical examination content recording system according to the embodiment allows the elderly to record the medical examination content without forgetting it and to accurately understand the diagnosis result and advice. Some or all of the above-mentioned processing in the answering unit may be performed using, for example, AI, or may be performed without using AI. For example, the answering unit may provide an answer using an AI model that takes a user's question as input and outputs an answer from recorded text.

[0030] The recording unit records the doctor's diagnosis by voice using the smartphone's microphone when the user is examined at a hospital. The recording unit can record voice using, for example, the smartphone's built-in microphone or an external microphone. Specifically, the smartphone application is started and recording begins before the examination begins. During recording, the smartphone's microphone picks up surrounding sounds and records the conversation in the examination room in high quality. When using an external microphone, by selecting a microphone with a noise canceling function, noise in the examination room can be reduced and clearer voice can be recorded. The recorded voice data is stored in the smartphone's storage and used for subsequent processing. In addition, the recording unit can be equipped with a function to automate the start and stop of recording. For example, it can be set so that recording starts automatically when entering the examination room and stops when the examination ends. This allows the user to concentrate on the examination without being conscious of the recording operation. In addition, the recording unit can also be equipped with a function to automatically upload the recorded data to the cloud. This can save the storage capacity of the smartphone and ensure a backup of the data. The data stored in the cloud is encrypted to protect the user's privacy with security measures in place. The recorder thereby provides a reliable means for the user to reliably record the contents of a medical examination for later review.

[0031] The transcription unit transcribes the voice recorded by the recording unit using the generation AI. The generation AI, for example, analyzes voice data using deep learning or machine learning algorithms and performs transcription. Specifically, it receives voice data as input and converts the voice signal into text using a voice recognition model. The voice recognition model is pre-trained using a large amount of voice data and corresponding text data, and can convert voice into text with high accuracy. For example, the voice recognition model learns the pronunciation patterns of phonemes and words and detects these patterns from the voice data. Furthermore, the generation AI can analyze the voice data taking into account the context and insert appropriate punctuation and line breaks. This makes the generated text in a format that is easy to read and understand. The transcription unit can also convert voice into text in real time, and can generate text sequentially as the consultation progresses. This allows the user to check the transcribed text immediately after the consultation is completed. In addition, the transcription unit can support multiple languages ​​and can accurately transcribe consultation contents in different languages. This allows the consultation content recording system to be used for users and doctors who speak foreign languages. In addition, the transcription unit can automatically save the generated text and store it in a database for subsequent processing, allowing the transcription unit to convert audio data into text quickly and accurately, and efficiently record medical consultations.

[0032] The summary creation unit uses a generation AI to create a summary of the diagnosis result based on the text generated by the transcription unit. The generation AI creates a summary of the diagnosis result using, for example, a text generation AI (e.g., LLM). Specifically, it receives the transcribed text as input, extracts important information, and generates a concise and easy-to-understand summary. The text generation AI has learned a large amount of medical data and diagnostic reports and has knowledge of medical terminology and diagnostic processes. This allows it to accurately understand the diagnosis content and create an appropriate summary. For example, the text generation AI extracts important information such as the diagnosis result, treatment plan, and doctor's advice, and summarizes them concisely. In addition, the summary creation unit can customize the format and content of the summary according to the user's needs. For example, if a detailed summary is required, a summary containing more information can be generated, and if a concise summary is required, a summary containing only the most important points can be generated. In addition, the summary creation unit can automatically save the generated summary so that the user can check it later. This allows the user to easily understand the diagnosis result and the doctor's advice and quickly check the necessary information. Furthermore, the summary creation unit can link the generated summaries with other medical systems and electronic medical records. This allows diagnostic results and treatment plans to be shared across medical institutions, enabling consistent care for patients. This allows the summary creation unit to efficiently create summaries of diagnostic results, playing an important role in enabling users to accurately understand the contents of medical examinations.

[0033] The answering unit analyzes the user's question and provides an answer from the recorded text. The answering unit, for example, uses natural language processing technology to analyze the user's question and provide an appropriate answer. Specifically, the answering unit uses natural language processing technology to analyze the question entered by the user and understand the intent and content of the question. For example, the answering unit analyzes the keywords and context of the question and extracts related information from the recorded text. The answering unit can generate an optimal answer to the user's question using a generation AI. The generation AI has learned a large amount of medical data and diagnostic reports and has knowledge about medicine. This makes it possible to provide accurate and appropriate answers to the user's questions. For example, if a user asks, "What are the side effects of this medicine?", the answering unit extracts information about the corresponding medicine from the recorded text and provides information about the side effects. In addition, the answering unit can provide answers to the user's questions in real time. This allows the user to quickly obtain answers even if they have questions after the examination. In addition, the answering unit can collect user feedback and continuously improve the accuracy and quality of the answers. For example, the user can evaluate the answers provided and the generative AI model can be updated based on the evaluation. This allows the answering unit to continue to provide high-quality answers that meet the user's needs. The answering unit can also respond to multiple questions, and provide appropriate answers even if the user asks multiple questions in succession. This allows the answering unit to play an important role in helping the user accurately understand the contents of the examination and resolve any doubts.

[0034] The recording unit can record the sound using a microphone. The recording unit records the sound using, for example, a smartphone built-in microphone. For example, the smartphone built-in microphone can record conversations in the examination room with high quality. The recording unit can also record the sound using an external microphone. For example, the external microphone has a noise canceling function and can remove environmental sounds in the examination room to record the sound clearly. Furthermore, the recording unit can also record the sound using a Bluetooth (registered trademark) microphone. For example, the Bluetooth microphone is connected wirelessly and can record the sound while the user is moving around freely. This makes it possible to easily record the sound by using the smartphone microphone. Part or all of the above-mentioned processing in the recording unit may be performed using, for example, AI, or may be performed without using AI. For example, the recording unit can automatically adjust the timing to start recording the sound using AI.

[0035] The transcription unit can analyze the voice data using AI and transcribe it. The transcription unit uses the generation AI to analyze the voice data recorded by the recording unit and transcribe it. The generation AI analyzes the voice data using, for example, deep learning or machine learning algorithms and transcribes it. For example, the generation AI takes the voice data as input and outputs text data. The generation AI uses voice recognition technology to recognize words and phrases in the voice data and converts it into text. For example, the generation AI analyzes the voice data and converts the voice "Tell me what the doctor said" into text "Tell me what the doctor said". The generation AI can also remove noise from the voice data and generate clear voice data. For example, the generation AI removes background sounds and environmental sounds from the voice data and converts only the contents of the medical examination into text. As a result, the use of the generation AI improves the accuracy of transcription of the voice data. Some or all of the above-mentioned processing in the transcription unit may be performed using, for example, AI, or may be performed without using AI. For example, the transcription unit can perform transcription using an AI model that takes audio data as input and outputs text data.

[0036] The summary creation unit can create a summary of the diagnosis result based on the text transcribed by the AI. The summary creation unit uses the generation AI to create a summary of the diagnosis result based on the text generated by the transcription unit. The generation AI creates a summary of the diagnosis result using, for example, a text generation AI (for example, LLM). The generation AI takes the transcribed text as input and outputs a summary of the diagnosis result. For example, the generation AI receives a prompt such as "Please summarize the main points of the diagnosis result" and extracts the main points of the transcribed text to create a summary. The generation AI can also develop an algorithm for automatically generating a summary of the diagnosis result. For example, the generation AI extracts important points of the diagnosis result and creates a summary based on the extracted points. As a result, the generation AI is used to automate the creation of a summary of the diagnosis result. A part or all of the above-mentioned processing in the summary creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the summary creation unit can create a summary using an AI model that takes the transcribed text as input and outputs a summary of the diagnosis result.

[0037] The answering unit can analyze the question and provide an answer from the recorded text. The answering unit can analyze the user's question and provide an answer from the recorded text. The answering unit can analyze the user's question using, for example, natural language processing technology and provide an appropriate answer. For example, the answering unit can analyze the question "Tell me what the doctor said" and provide an answer by extracting a summary of the diagnosis from the recorded text. The answering unit can also analyze the question "Tell me what medicine was prescribed" and provide an answer by extracting information on the prescribed medicine from the recorded text. This makes it possible to provide an appropriate answer to the user's question from the recorded text. Part or all of the above-mentioned processing in the answering unit may be performed, for example, using AI or may be performed without using AI. For example, the answering unit can provide an answer using an AI model that inputs the user's question and outputs an answer from the recorded text.

[0038] The medical examination content recording system includes a noise removal unit that removes noise from the recorded voice. The noise removal unit removes noise from the recorded voice. The noise removal unit uses, for example, a noise canceling technique for removing background sounds and environmental sounds. For example, the noise removal unit analyzes the environmental sounds of the examination room in real time and applies an optimal noise removal filter. When the examination room is quiet, the noise removal filter is set low, and when the examination room is noisy, the noise removal filter is set high. In addition, the noise removal unit can also adjust the noise removal filter in real time when the environmental sounds of the examination room fluctuate. This improves the quality of the voice data by noise removal. A part or all of the above-mentioned processing in the noise removal unit may be performed using, for example, AI, or may be performed without using AI. For example, the noise removal unit can perform noise removal using an AI model that inputs the environmental sounds of the examination room and outputs an optimal noise removal filter.

[0039] The medical examination content recording system includes a question analysis unit that analyzes a user's question and provides an appropriate answer. The question analysis unit analyzes the user's question and provides an appropriate answer, for example, by using natural language processing technology. For example, the question analysis unit analyzes the question of "tell me what the doctor said" and provides an answer by extracting a summary of the diagnosis result from the recorded text. The question analysis unit can also analyze the question of "tell me what medicine was prescribed" and provide an answer by extracting information on the prescribed medicine from the recorded text. This makes it possible to provide an appropriate answer to the user's question. A part or all of the above-mentioned processing in the question analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the question analysis unit can provide an answer by using an AI model that receives the user's question as an input and outputs an answer from the recorded text.

[0040] The recording unit can analyze the environmental sound of the examination room and automatically adjust the optimal microphone sensitivity. For example, when the examination room is quiet, the recording unit sets the microphone sensitivity low. For example, the recording unit analyzes the environmental sound of the examination room in real time and automatically adjusts the optimal microphone sensitivity. When the examination room is noisy, the microphone sensitivity can also be set high. For example, when the environmental sound of the examination room fluctuates, the recording unit adjusts the microphone sensitivity in real time. This improves the quality of the voice recording by automatically adjusting the microphone sensitivity according to the environmental sound of the examination room. A part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can automatically adjust the microphone sensitivity using an AI model that inputs the environmental sound of the examination room and outputs the optimal microphone sensitivity.

[0041] The recording unit can add a function of automatically highlighting when a specific keyword is spoken. The recording unit automatically highlights when a keyword such as "medicine" or "prescription" is spoken. For example, the recording unit analyzes voice data and adds a function of automatically highlighting when a specific keyword is spoken. It is also possible to automatically highlight when a keyword such as "next appointment" or "points to note" is spoken. For example, the recording unit detects a specific keyword in the voice data and highlights it. It is also possible to automatically highlight when a keyword such as "test result" or "treatment method" is spoken. By this, important information is not overlooked by automatically highlighting when a specific keyword is spoken. Part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI or may be performed without using AI. For example, the recording unit can highlight using an AI model that inputs voice data and detects and highlights a specific keyword.

[0042] The recording unit can select the optimal voice recording setting in consideration of the user's location information. For example, when the user is in a consultation room, the recording unit selects the optimal voice recording setting for the consultation room. For example, the recording unit selects the optimal voice recording setting in consideration of the user's location information. When the user is in a waiting room, the recording unit can also select the optimal voice recording setting for the waiting room. For example, the recording unit inputs the user's location information and outputs the optimal voice recording setting. When the user is at home, the recording unit can also select the optimal voice recording setting for the home. This improves the quality of the voice recording by selecting the optimal voice recording setting according to the user's location information. Part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI or may be performed without using AI. For example, the recording unit can select the voice recording setting using an AI model that inputs the user's location information and outputs the optimal voice recording setting.

[0043] The recording unit can preferentially record specific diagnostic contents by referring to the user's past medical history. The recording unit preferentially records related diagnostic contents, for example, based on diagnostic contents received by the user in the past. For example, the recording unit preferentially records specific diagnostic contents by referring to the user's past medical history. Based on advice received by the user in the past, related advice can also be preferentially recorded. For example, the recording unit takes the user's past medical history as an input and preferentially records specific diagnostic contents. Based on medicines prescribed by the user in the past, information on related medicines can also be preferentially recorded. This allows related diagnostic contents to be preferentially recorded by referring to the user's past medical history. Part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can record diagnostic contents using an AI model that takes the user's past medical history as an input and preferentially records specific diagnostic contents.

[0044] The transcription unit can automatically recognize technical terms in the contents of the medical examination and convert them into appropriate notation. The transcription unit can automatically recognize technical terms in the contents of the medical examination and convert them into appropriate notation using a generation AI. The generation AI can recognize technical terms using, for example, deep learning or machine learning algorithms and convert them into appropriate notation. For example, the generation AI can automatically recognize technical terms such as "high blood pressure" and "diabetes" and convert them into appropriate notation. The generation AI can also automatically recognize names of medical devices such as "MRI" and "CT scan" and convert them into appropriate notation. Furthermore, the generation AI can automatically recognize names of medicines such as "antibiotics" and "steroids" and convert them into appropriate notation. This improves the accuracy of transcription by automatically recognizing technical terms and converting them into appropriate notation. Some or all of the above-mentioned processing in the transcription unit may be performed, for example, using AI, or may be performed without using AI. For example, the transcription unit can recognize technical terms and convert them into appropriate notation using an AI model that inputs technical terms and outputs appropriate notation.

[0045] The transcription unit can identify the speaker of the audio and transcribe in a different format for each speaker. The transcription unit can use a generation AI to identify the speaker of the audio and transcribe in a different format for each speaker. The generation AI can use, for example, deep learning or machine learning algorithms to identify the speaker of the audio and transcribe in a different format for each speaker. For example, the generation AI can display the doctor's remarks in bold and the patient's remarks in normal font. The generation AI can also display the doctor's remarks in blue and the patient's remarks in black. Furthermore, the generation AI can also display the doctor's remarks in a large font size and the patient's remarks in a small font size. This makes it easier to organize information by transcribing in a different format for each speaker. Some or all of the above-mentioned processing in the transcription unit may be performed, for example, using AI or may be performed without using AI. For example, the transcription unit can perform transcription using an AI model that inputs audio data and transcribes in a different format for each speaker.

[0046] The transcription unit can determine the priority of transcription based on the importance of the contents of the medical examination. The transcription unit uses the generation AI to determine the priority of transcription based on the importance of the contents of the medical examination. The generation AI evaluates the importance of the contents of the medical examination using, for example, deep learning or machine learning algorithms, and determines the priority of transcription based on the evaluation. For example, the generation AI prioritizes transcription of important diagnostic contents. The generation AI can also prioritize transcription of important advice. Furthermore, the generation AI can also prioritize transcription of important drug information. As a result, important information can be prioritized by determining the priority of transcription based on the importance of the contents of the medical examination. Part or all of the above-mentioned processing in the transcription unit may be performed, for example, using AI, or may be performed without using AI. For example, the transcription unit can determine the priority of transcription using an AI model that inputs the importance of the contents of the medical examination and outputs the priority of transcription.

[0047] The transcription unit can apply different transcription algorithms depending on the category of the medical examination content. The transcription unit uses a generation AI to apply different transcription algorithms depending on the category of the medical examination content. The generation AI classifies the category of the medical examination content using, for example, deep learning or machine learning algorithms, and applies a transcription algorithm according to the category. For example, the generation AI applies an algorithm that is strong in technical terms to the diagnosis content. The generation AI can also apply an algorithm that summarizes concisely to advice. Furthermore, the generation AI can also apply an algorithm that emphasizes accuracy to drug information. As a result, by applying different transcription algorithms depending on the category of the medical examination content, the accuracy of the transcription is improved. Part or all of the above-mentioned processing in the transcription unit may be performed, for example, using AI, or may be performed without using AI. For example, the transcription unit can perform transcription using an AI model that inputs the category of the medical examination content and outputs a transcription algorithm.

[0048] The summary creation unit can adjust the level of detail of the summary based on the importance of the diagnostic content. The summary creation unit uses the generation AI to adjust the level of detail of the summary based on the importance of the diagnostic content. The generation AI evaluates the importance of the diagnostic content using, for example, deep learning or machine learning algorithms, and adjusts the level of detail of the summary based on the evaluation. For example, the generation AI creates a detailed summary for important diagnostic content. The generation AI can also create a detailed summary for important advice. Furthermore, the generation AI can also create a detailed summary for important drug information. As a result, important information can be grasped in detail by adjusting the level of detail of the summary based on the importance of the diagnostic content. Part or all of the above-mentioned processing in the summary creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the summary creation unit can adjust the level of detail of the summary using an AI model that inputs the importance of the diagnostic content and outputs the level of detail of the summary.

[0049] The summary creation unit can apply different summary creation algorithms depending on the category of the diagnostic content. The summary creation unit uses the generation AI to apply different summary creation algorithms depending on the category of the diagnostic content. The generation AI classifies the category of the diagnostic content using, for example, deep learning or machine learning algorithms, and applies the corresponding summary creation algorithm. For example, the generation AI applies an algorithm that is strong in technical terms to the diagnostic content. The generation AI can also apply an algorithm that summarizes concisely to advice. Furthermore, the generation AI can also apply an algorithm that emphasizes accuracy to drug information. Thereby, by applying different summary creation algorithms depending on the category of the diagnostic content, the accuracy of the summary is improved. Part or all of the above-mentioned processing in the summary creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the summary creation unit can create a summary using an AI model that inputs the category of the diagnostic content and outputs a summary creation algorithm.

[0050] The summary creation unit can determine the priority of the summary based on the time of submission of the diagnostic content. The summary creation unit uses the generation AI to determine the priority of the summary based on the time of submission of the diagnostic content. The generation AI evaluates the time of submission of the diagnostic content using, for example, deep learning or machine learning algorithms, and determines the priority of the summary based on the evaluation. For example, the generation AI prioritizes the creation of a summary of the most recent diagnostic content. The generation AI can also prioritize the creation of a summary of important diagnostic content. Furthermore, the generation AI can also prioritize the creation of a summary of diagnostic content whose submission deadline is approaching. As a result, important information can be summarized preferentially by determining the priority of the summary based on the time of submission of the diagnostic content. Part or all of the above-mentioned processing in the summary creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the summary creation unit can determine the priority of the summary using an AI model that inputs the time of submission of the diagnostic content and outputs the priority of the summary.

[0051] The summary creation unit can adjust the order of summaries based on the relevance of the diagnostic contents. The summary creation unit adjusts the order of summaries based on the relevance of the diagnostic contents using the generation AI. The generation AI evaluates the relevance of the diagnostic contents using, for example, deep learning or machine learning algorithms, and adjusts the order of summaries based on the evaluation. For example, the generation AI creates summaries with a high relevance. The generation AI can also create summaries with a high relevance of advice. Furthermore, the generation AI can also create summaries with a high relevance of drug information with a high relevance. This makes it easier to organize information by adjusting the order of summaries based on the relevance of the diagnostic contents. Part or all of the above-mentioned processing in the summary creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the summary creation unit can adjust the order of summaries using an AI model that inputs the relevance of the diagnostic contents and outputs the order of summaries.

[0052] The answering unit can select the optimal answer by referring to the user's past question history. The answering unit selects the optimal answer by referring to the user's past question history. The answering unit, for example, uses natural language processing technology to analyze the user's past question history and selects the optimal answer based on the analysis. For example, the answering unit provides a related answer based on the content of the user's past questions. The answering unit can also prioritize answers to content that the user has frequently asked in the past. Furthermore, the answering unit can also select the most appropriate answer from the user's past question history. This makes it possible to provide a related answer by referring to the user's past question history. A part or all of the above-mentioned processing in the answering unit may be performed, for example, using AI, or may be performed without using AI. For example, the answering unit can select an answer using an AI model that inputs the user's past question history and outputs an optimal answer.

[0053] The answer unit can customize the content of the answer based on the user's current health condition. The answer unit customizes the content of the answer based on the user's current health condition. The answer unit, for example, uses natural language processing technology to analyze the user's current health condition and customizes the content of the answer based on the analysis. For example, when the user inputs the current health condition, the answer unit customizes the answer based on the information. The answer unit can also provide an answer related to the user's current health condition based on the user's past health condition. Furthermore, the answer unit can also provide an answer including appropriate advice according to the user's health condition. As a result, a more appropriate answer is provided by customizing the content of the answer based on the user's current health condition. A part or all of the above-mentioned processing in the answer unit may be performed, for example, using AI, or may be performed without using AI. For example, the answer unit can customize the content of the answer using an AI model that inputs the user's health condition data and outputs the content of the answer.

[0054] The answering unit can select the optimal answer in consideration of the geographical location information of the user. The answering unit selects the optimal answer in consideration of the geographical location information of the user. The answering unit, for example, uses natural language processing technology to analyze the geographical location information of the user and selects the optimal answer based on the analysis. For example, when the user is in a hospital, the answering unit provides an answer related to the hospital. In addition, when the user is at home, the answering unit can also provide an answer that can be executed at home. Furthermore, when the user is out, the answering unit can also provide an answer that can be executed while out. This makes it possible to provide more appropriate information by providing an optimal answer according to the geographical location information of the user. Part or all of the above-mentioned processing in the answering unit may be performed, for example, using AI, or may be performed without using AI. For example, the answering unit can select an answer using an AI model that inputs the geographical location information of the user and outputs an optimal answer.

[0055] The answering unit can provide a relevant answer by analyzing the user's social media activity. The answering unit can provide a relevant answer by analyzing the user's social media activity. The answering unit can, for example, use natural language processing technology to analyze the user's social media activity and provide a relevant answer based on the analysis. For example, the answering unit can provide a relevant answer based on health information shared by the user on social media. The answering unit can also provide a relevant answer based on information on medical professionals the user follows on social media. Furthermore, the answering unit can provide a relevant answer based on health topics in which the user has shown interest on social media. This allows more appropriate information to be provided by providing a relevant answer based on the user's social media activity. Part or all of the above-mentioned processing in the answering unit can be performed, for example, using AI or without AI. For example, the answering unit can provide an answer using an AI model that inputs the user's social media activity data and outputs a relevant answer.

[0056] The noise removal unit can analyze the environmental sound of the examination room in real time and apply an optimal noise removal filter. The noise removal unit can analyze the environmental sound of the examination room in real time and apply an optimal noise removal filter. The noise removal unit can analyze the environmental sound of the examination room using, for example, deep learning or machine learning algorithms, and apply an optimal noise removal filter based on the analysis. For example, the noise removal unit sets the noise removal filter low when the examination room is quiet. Also, the noise removal unit can set the noise removal filter high when the examination room is noisy. Furthermore, the noise removal unit can adjust the noise removal filter in real time when the environmental sound of the examination room fluctuates. As a result, the quality of the voice data is improved by applying the optimal noise removal filter according to the environmental sound of the examination room. A part or all of the above-mentioned processing in the noise removal unit may be performed using, for example, AI, or may be performed without using AI. For example, the noise removal unit can apply the noise removal filter using an AI model that inputs the environmental sound of the examination room and outputs the optimal noise removal filter.

[0057] The noise removal unit can adjust the intensity of noise removal based on the importance of the medical examination contents. The noise removal unit adjusts the intensity of noise removal based on the importance of the medical examination contents using the generation AI. The generation AI evaluates the importance of the medical examination contents using, for example, deep learning or machine learning algorithms, and adjusts the intensity of noise removal based on the evaluation. For example, the generation AI increases the intensity of noise removal for important diagnostic contents. The generation AI can also increase the intensity of noise removal for important advice. Furthermore, the generation AI can also increase the intensity of noise removal for important medicine information. As a result, important information can be preferentially removed by adjusting the intensity of noise removal based on the importance of the medical examination contents. Part or all of the above-mentioned processing in the noise removal unit may be performed, for example, using AI, or may be performed without using AI. For example, the noise removal unit can adjust the intensity of noise removal using an AI model that inputs the importance of the medical examination contents and outputs the intensity of noise removal.

[0058] The question analysis unit can select the optimal analysis method by referring to the user's past question history. The question analysis unit selects the optimal analysis method by referring to the user's past question history. The question analysis unit, for example, uses natural language processing technology to analyze the user's past question history and selects the optimal analysis method based on the analysis. For example, the question analysis unit selects the optimal analysis method based on the content of questions asked by the user in the past. The question analysis unit can also preferentially analyze content that the user has frequently asked in the past. Furthermore, the question analysis unit can also select the most appropriate analysis method from the user's past question history. Thereby, by referring to the user's past question history, a related analysis method can be selected. A part or all of the above-mentioned processing in the question analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the question analysis unit can select the analysis method using an AI model that inputs the user's past question history and outputs the optimal analysis method.

[0059] The question analysis unit can select the optimal analysis method in consideration of the geographical location information of the user. The question analysis unit selects the optimal analysis method in consideration of the geographical location information of the user. The question analysis unit, for example, uses natural language processing technology to analyze the geographical location information of the user and selects the optimal analysis method based on the information. For example, when the user is in a hospital, the question analysis unit preferentially analyzes questions related to the hospital. In addition, when the user is at home, the question analysis unit can also preferentially analyze questions that can be performed at home. Furthermore, when the user is out, the question analysis unit can also preferentially analyze questions that can be performed outside the home. This allows for more appropriate analysis by selecting the optimal analysis method according to the geographical location information of the user. Part or all of the above-mentioned processing in the question analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the question analysis unit can select the analysis method using an AI model that inputs the geographical location information of the user and outputs the optimal analysis method.

[0060] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0061] The medical examination content recording system can not only record the medical examination content of the user, but also provide supplementary information to deepen understanding of the medical examination content. For example, the recording unit can automatically search for medical background information and general health information related to the medical examination content and provide it to the user. This allows the user to understand the medical examination content more deeply. The recording unit can also provide videos and images related to the medical examination content. For example, providing videos about treatments and prevention methods for diagnosed diseases can help the user understand. Furthermore, the recording unit can provide other patients' experiences and reviews related to the medical examination content. This allows the user to refer to the experiences of other patients.

[0062] The recording unit can have a function of not only recording the contents of the medical examination, but also monitoring the health condition of the user. For example, the recording unit can collect biological data such as the user's heart rate, blood pressure, and body temperature using a sensor and record the data together with the contents of the medical examination. This allows the user's health condition to be understood comprehensively. The recording unit can also collect lifestyle data such as the user's amount of exercise and sleep patterns and associate the data with the contents of the medical examination. For example, if the user's lack of exercise is affecting the diagnosis results, providing this information can encourage the user to improve their lifestyle habits. Furthermore, the recording unit can also record the user's diet and evaluate the nutritional balance. This can support the user's health management.

[0063] The summary creation unit can not only create a summary of the diagnosis results, but also provide predictive information based on the diagnosis results. For example, the summary creation unit can predict future changes in the user's health condition based on the diagnosis results. This allows the user to obtain information that will be useful for future health management. The summary creation unit can also perform risk assessment based on the diagnosis results. For example, if the diagnosis results indicate a high risk of a specific disease, the summary creation unit can notify the user of the risk and suggest preventive measures. Furthermore, the summary creation unit can predict the effectiveness of treatment based on the diagnosis results. For example, it can predict how effective a specific treatment will be and suggest the most suitable treatment to the user.

[0064] The answering unit can not only provide answers to the user's questions, but also learn based on the content of the user's questions. For example, the answering unit can learn the content of questions frequently asked by the user and improve the accuracy of the answers based on the information. This makes it possible to provide answers that are more suitable for the user's needs. The answering unit can also acquire new knowledge based on the content of the user's questions. For example, when a user repeatedly asks questions about a specific illness, the latest information about the illness can be automatically collected and reflected in the answers. Furthermore, the answering unit can provide personalized advice based on the content of the user's questions. For example, the answering unit can support the user's health management by providing specific advice according to the user's health condition and lifestyle.

[0065] The medical examination content recording system can not only record the medical examination content of the user, but also provide a health management plan based on the medical examination content. For example, the recording unit can automatically create a health management plan suitable for the user based on the medical examination content. This allows the user to know a specific health management method based on the medical examination content. The recording unit can also provide a meal plan and an exercise plan based on the medical examination content. For example, the recording unit can support the user's health improvement by suggesting a meal menu and an exercise program suitable for the user based on the diagnosis result. Furthermore, the recording unit can also provide a stress management plan based on the medical examination content. For example, the recording unit can suggest a relaxation method or a stress relief method to reduce the stress felt by the user during the medical examination.

[0066] The recording unit can not only record the contents of the medical examination, but also provide an interactive function to help the user understand the contents of the medical examination. For example, when the user inputs a question about the contents of the medical examination, the recording unit can provide related information. This allows the user to have a deeper understanding of the contents of the medical examination. The recording unit can also provide quizzes and tests related to the contents of the medical examination. For example, the user can review the contents of the medical examination through quizzes about the diagnosis results and advice. Furthermore, the recording unit can provide a discussion forum about the contents of the medical examination. This allows the user to share information and exchange opinions with other patients and medical professionals.

[0067] The process flow of the first embodiment will be briefly described below.

[0068] Step 1: When the user is examined at a hospital, the recording unit uses the smartphone's microphone to record the doctor's diagnosis as audio. The recording unit can record audio using, for example, the smartphone's built-in microphone or an external microphone. Step 2: The transcription unit uses the generation AI to transcribe the audio recorded by the recording unit. The generation AI analyzes the audio data using, for example, deep learning or machine learning algorithms, and transcribes it. Step 3: The summary creation unit uses a generation AI to create a summary of the diagnosis result based on the text generated by the transcription unit. The generation AI creates a summary of the diagnosis result using, for example, a text generation AI (e.g., LLM). Step 4: The answering unit analyzes the user's question and provides an answer from the recorded text. The answering unit analyzes the user's question using, for example, natural language processing technology and provides an appropriate answer. As a result, the medical examination content recording system according to the embodiment allows the elderly to remember to record the medical examination content and accurately understand the diagnosis results and advice.

[0069] (Example 2) The medical examination content recording system according to the embodiment of the present invention is a system that allows elderly people to record the contents of their medical examination without forgetting them and to accurately understand the diagnosis results and advice. In this system, when a user visits a hospital, the user starts an app and uses the smartphone's microphone to record the doctor's diagnosis by voice. Next, the generation AI analyzes the recorded voice and transcribes it. The generation AI creates a summary of the diagnosis results based on the transcribed text. In addition, information such as the doctor's advice and prescribed medicine is also recorded. When a user forgets the diagnosis, advice, or prescribed medicine, the user can ask the app a question and the generation AI will provide an answer from the recorded text. For example, when the user asks "Tell me what the doctor said," the generation AI will display a summary of the diagnosis results. Also, when the user asks "Tell me what medicine was prescribed," the generation AI will display information about the prescribed medicine. This app prevents elderly people from forgetting the contents of their medical examination and allows them to accurately understand the diagnosis results and advice. In addition, by using the generation AI, the transcription of voice and the creation of summaries are automated, reducing the burden on the user. This allows the medical examination details recording system to ensure that elderly people do not forget to record the details of their medical examinations, and to accurately understand diagnosis results and advice.

[0070] The medical examination content recording system according to the embodiment includes a recording unit, a transcription unit, a summary creation unit, and an answering unit. The recording unit records the doctor's diagnosis by voice using a microphone of a smartphone when a user is examined at a hospital. The recording unit can record the voice using, for example, a smartphone's built-in microphone or an external microphone. The transcription unit uses a generation AI to transcribe the voice recorded by the recording unit. The generation AI analyzes the voice data using, for example, deep learning or machine learning algorithms to perform transcription. The summary creation unit uses a generation AI to create a summary of the diagnosis result based on the text generated by the transcription unit. The generation AI creates a summary of the diagnosis result using, for example, a text generation AI (for example, LLM). The answering unit analyzes the user's question and provides an answer from the recorded text. The answering unit analyzes the user's question using, for example, natural language processing technology and provides an appropriate answer. As a result, the medical examination content recording system according to the embodiment allows the elderly to record the medical examination content without forgetting it and to accurately understand the diagnosis result and advice. Some or all of the above-mentioned processing in the answering unit may be performed using, for example, AI, or may be performed without using AI. For example, the answering unit may provide an answer using an AI model that takes a user's question as input and outputs an answer from recorded text.

[0071] The recording unit records the doctor's diagnosis by voice using the smartphone's microphone when the user is examined at a hospital. The recording unit can record voice using, for example, the smartphone's built-in microphone or an external microphone. Specifically, the smartphone application is started and recording begins before the examination begins. During recording, the smartphone's microphone picks up surrounding sounds and records the conversation in the examination room in high quality. When using an external microphone, a microphone with a noise canceling function can be selected to reduce noise in the examination room and record clearer voice. The recorded voice data is stored in the smartphone's storage and used for subsequent processing. In addition, the recording unit can be equipped with a function to automate the start and stop of recording. For example, it can be set so that recording starts automatically when entering the examination room and stops when the examination ends. This allows the user to concentrate on the examination without being conscious of the recording operation. The recording unit can also be equipped with a function to automatically upload the recorded data to the cloud. This can save the storage capacity of the smartphone and ensure a backup of the data. The data stored in the cloud is encrypted to protect the user's privacy with security measures in place. The recorder thereby provides a reliable means for the user to reliably record the contents of a medical examination for later review.

[0072] The transcription unit transcribes the voice recorded by the recording unit using the generation AI. The generation AI, for example, analyzes voice data using deep learning or machine learning algorithms and performs transcription. Specifically, it receives voice data as input and converts the voice signal into text using a voice recognition model. The voice recognition model is pre-trained using a large amount of voice data and corresponding text data, and can convert voice into text with high accuracy. For example, the voice recognition model learns the pronunciation patterns of phonemes and words and detects these patterns from the voice data. Furthermore, the generation AI can analyze the voice data taking into account the context and insert appropriate punctuation and line breaks. This makes the generated text in a format that is easy to read and understand. The transcription unit can also convert voice into text in real time, and can generate text sequentially as the consultation progresses. This allows the user to check the transcribed text immediately after the consultation is completed. In addition, the transcription unit can support multiple languages ​​and can accurately transcribe consultation contents in different languages. This allows the consultation content recording system to be used for users and doctors who speak foreign languages. In addition, the transcription unit can automatically save the generated text and store it in a database for subsequent processing, allowing the transcription unit to convert audio data into text quickly and accurately, and efficiently record medical consultations.

[0073] The summary creation unit uses a generation AI to create a summary of the diagnosis result based on the text generated by the transcription unit. The generation AI creates a summary of the diagnosis result using, for example, a text generation AI (e.g., LLM). Specifically, it receives the transcribed text as input, extracts important information, and generates a concise and easy-to-understand summary. The text generation AI has learned a large amount of medical data and diagnostic reports and has knowledge of medical terminology and diagnostic processes. This allows it to accurately understand the diagnosis content and create an appropriate summary. For example, the text generation AI extracts important information such as the diagnosis result, treatment plan, and doctor's advice, and summarizes them concisely. In addition, the summary creation unit can customize the format and content of the summary according to the user's needs. For example, if a detailed summary is required, a summary containing more information can be generated, and if a concise summary is required, a summary containing only the most important points can be generated. In addition, the summary creation unit can automatically save the generated summary so that the user can check it later. This allows the user to easily understand the diagnosis result and the doctor's advice and quickly check the necessary information. Furthermore, the summary creation unit can link the generated summaries with other medical systems and electronic medical records. This allows diagnostic results and treatment plans to be shared across medical institutions, enabling consistent care for patients. This allows the summary creation unit to efficiently create summaries of diagnostic results, playing an important role in enabling users to accurately understand the contents of medical examinations.

[0074] The answering unit analyzes the user's question and provides an answer from the recorded text. The answering unit, for example, uses natural language processing technology to analyze the user's question and provide an appropriate answer. Specifically, the answering unit uses natural language processing technology to analyze the question entered by the user and understand the intent and content of the question. For example, the answering unit analyzes the keywords and context of the question and extracts related information from the recorded text. The answering unit can generate an optimal answer to the user's question using a generation AI. The generation AI has learned a large amount of medical data and diagnostic reports and has knowledge about medicine. This makes it possible to provide accurate and appropriate answers to the user's questions. For example, if a user asks, "What are the side effects of this medicine?", the answering unit extracts information about the corresponding medicine from the recorded text and provides information about the side effects. In addition, the answering unit can provide answers to the user's questions in real time. This allows the user to quickly obtain answers even if they have questions after the examination. In addition, the answering unit can collect user feedback and continuously improve the accuracy and quality of the answers. For example, the user can evaluate the answers provided and the generative AI model can be updated based on the evaluation. This allows the answering unit to continue to provide high-quality answers that meet the user's needs. The answering unit can also respond to multiple questions, and provide appropriate answers even if the user asks multiple questions in succession. This allows the answering unit to play an important role in helping the user accurately understand the contents of the examination and resolve any doubts.

[0075] The recording unit can record the sound using a microphone. The recording unit records the sound using, for example, a smartphone built-in microphone. For example, the smartphone built-in microphone can record conversations in the examination room with high quality. The recording unit can also record the sound using an external microphone. For example, the external microphone has a noise canceling function and can remove environmental sounds in the examination room to record the sound clearly. Furthermore, the recording unit can also record the sound using a Bluetooth microphone. For example, the Bluetooth microphone is connected wirelessly and can record the sound while the user is moving around freely. This makes it possible to easily record the sound by using the smartphone microphone. Some or all of the above-mentioned processing in the recording unit may be performed using, for example, AI, or may be performed without using AI. For example, the recording unit can automatically adjust the timing to start recording the sound using AI.

[0076] The transcription unit can analyze the voice data using AI and transcribe it. The transcription unit uses the generation AI to analyze the voice data recorded by the recording unit and transcribe it. The generation AI analyzes the voice data using, for example, deep learning or machine learning algorithms and transcribes it. For example, the generation AI takes the voice data as input and outputs text data. The generation AI uses voice recognition technology to recognize words and phrases in the voice data and converts it into text. For example, the generation AI analyzes the voice data and converts the voice "Tell me what the doctor said" into text "Tell me what the doctor said". The generation AI can also remove noise from the voice data and generate clear voice data. For example, the generation AI removes background sounds and environmental sounds from the voice data and converts only the contents of the medical examination into text. As a result, the use of the generation AI improves the accuracy of transcription of the voice data. Some or all of the above-mentioned processing in the transcription unit may be performed using, for example, AI, or may be performed without using AI. For example, the transcription unit can perform transcription using an AI model that takes audio data as input and outputs text data.

[0077] The summary creation unit can create a summary of the diagnosis result based on the text transcribed by the AI. The summary creation unit uses the generation AI to create a summary of the diagnosis result based on the text generated by the transcription unit. The generation AI creates a summary of the diagnosis result using, for example, a text generation AI (for example, LLM). The generation AI takes the transcribed text as input and outputs a summary of the diagnosis result. For example, the generation AI receives a prompt such as "Please summarize the main points of the diagnosis result" and extracts the main points of the transcribed text to create a summary. The generation AI can also develop an algorithm for automatically generating a summary of the diagnosis result. For example, the generation AI extracts important points of the diagnosis result and creates a summary based on the extracted points. As a result, the generation AI is used to automate the creation of a summary of the diagnosis result. A part or all of the above-mentioned processing in the summary creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the summary creation unit can create a summary using an AI model that takes the transcribed text as input and outputs a summary of the diagnosis result.

[0078] The answering unit can analyze the question and provide an answer from the recorded text. The answering unit can analyze the user's question and provide an answer from the recorded text. The answering unit can analyze the user's question using, for example, natural language processing technology and provide an appropriate answer. For example, the answering unit can analyze the question "Tell me what the doctor said" and provide an answer by extracting a summary of the diagnosis from the recorded text. The answering unit can also analyze the question "Tell me what medicine was prescribed" and provide an answer by extracting information on the prescribed medicine from the recorded text. This makes it possible to provide an appropriate answer to the user's question from the recorded text. Part or all of the above-mentioned processing in the answering unit may be performed, for example, using AI or may be performed without using AI. For example, the answering unit can provide an answer using an AI model that inputs the user's question and outputs an answer from the recorded text.

[0079] The medical examination content recording system includes a noise removal unit that removes noise from the recorded voice. The noise removal unit removes noise from the recorded voice. The noise removal unit uses, for example, a noise canceling technique for removing background sounds and environmental sounds. For example, the noise removal unit analyzes the environmental sounds of the examination room in real time and applies an optimal noise removal filter. When the examination room is quiet, the noise removal filter is set low, and when the examination room is noisy, the noise removal filter is set high. In addition, the noise removal unit can also adjust the noise removal filter in real time when the environmental sounds of the examination room fluctuate. This improves the quality of the voice data by noise removal. A part or all of the above-mentioned processing in the noise removal unit may be performed using, for example, AI, or may be performed without using AI. For example, the noise removal unit can perform noise removal using an AI model that inputs the environmental sounds of the examination room and outputs an optimal noise removal filter.

[0080] The medical examination content recording system includes a question analysis unit that analyzes a user's question and provides an appropriate answer. The question analysis unit analyzes the user's question and provides an appropriate answer, for example, by using natural language processing technology. For example, the question analysis unit analyzes the question of "tell me what the doctor said" and provides an answer by extracting a summary of the diagnosis result from the recorded text. The question analysis unit can also analyze the question of "tell me what medicine was prescribed" and provide an answer by extracting information on the prescribed medicine from the recorded text. This makes it possible to provide an appropriate answer to the user's question. A part or all of the above-mentioned processing in the question analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the question analysis unit can provide an answer by using an AI model that receives the user's question as an input and outputs an answer from the recorded text.

[0081] The recording unit can estimate the user's emotion and adjust the start timing of voice recording based on the estimated user's emotion. For example, when the user is nervous, the recording unit delays the start of voice recording until the user relaxes. For example, the recording unit captures the user's facial expression with a camera and estimates the user's emotion using an emotion estimation algorithm. When the user is relaxed, the voice recording can also be started immediately. For example, the recording unit records the user's voice and estimates the user's emotion using a voice analysis technique. When the user is impatient, the voice recording can also be started quickly. For example, the recording unit collects the user's biometric data (heart rate and electrodermal activity) with a sensor and estimates the user's emotion using an emotion estimation algorithm. This allows for more appropriate recording by adjusting the start timing of voice recording according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the recording unit may be performed using, for example, AI, or may be performed without using AI. For example, the recording unit can adjust the start timing of voice recording using an AI model that inputs user emotion data and outputs the start timing of voice recording.

[0082] The recording unit can analyze the environmental sound of the examination room and automatically adjust the optimal microphone sensitivity. For example, when the examination room is quiet, the recording unit sets the microphone sensitivity low. For example, the recording unit analyzes the environmental sound of the examination room in real time and automatically adjusts the optimal microphone sensitivity. When the examination room is noisy, the microphone sensitivity can also be set high. For example, when the environmental sound of the examination room fluctuates, the recording unit adjusts the microphone sensitivity in real time. This improves the quality of the voice recording by automatically adjusting the microphone sensitivity according to the environmental sound of the examination room. A part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can automatically adjust the microphone sensitivity using an AI model that inputs the environmental sound of the examination room and outputs the optimal microphone sensitivity.

[0083] The recording unit can add a function of automatically highlighting when a specific keyword is spoken. The recording unit automatically highlights when a keyword such as "medicine" or "prescription" is spoken. For example, the recording unit analyzes voice data and adds a function of automatically highlighting when a specific keyword is spoken. It is also possible to automatically highlight when a keyword such as "next appointment" or "points to note" is spoken. For example, the recording unit detects a specific keyword in the voice data and highlights it. It is also possible to automatically highlight when a keyword such as "test result" or "treatment method" is spoken. By this, important information is not overlooked by automatically highlighting when a specific keyword is spoken. Part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI or may be performed without using AI. For example, the recording unit can highlight using an AI model that inputs voice data and detects and highlights a specific keyword.

[0084] The recording unit can estimate the user's emotion and determine the priority of the voice to be recorded based on the estimated user's emotion. For example, when the user is nervous, the recording unit preferentially records important diagnostic contents. For example, the recording unit captures the user's facial expression with a camera and estimates the user's emotion using an emotion estimation algorithm. When the user is relaxed, all diagnostic contents can be recorded evenly. For example, the recording unit records the user's voice and estimates the user's emotion using a voice analysis technology. When the user is impatient, important advice can be preferentially recorded. For example, the recording unit collects the user's biometric data (heart rate and electrodermal activity) with a sensor and estimates the user's emotion using an emotion estimation algorithm. As a result, important information can be preferentially recorded by determining the priority of the voice according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the recording unit may be performed using, for example, AI, or may be performed without using AI. For example, the recording unit can determine the priority of voices using an AI model that inputs user emotion data and outputs the priority of voices.

[0085] The recording unit can select the optimal voice recording setting in consideration of the user's location information. For example, when the user is in a consultation room, the recording unit selects the optimal voice recording setting for the consultation room. For example, the recording unit selects the optimal voice recording setting in consideration of the user's location information. When the user is in a waiting room, the recording unit can also select the optimal voice recording setting for the waiting room. For example, the recording unit inputs the user's location information and outputs the optimal voice recording setting. When the user is at home, the recording unit can also select the optimal voice recording setting for the home. This improves the quality of the voice recording by selecting the optimal voice recording setting according to the user's location information. Part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI or may be performed without using AI. For example, the recording unit can select the voice recording setting using an AI model that inputs the user's location information and outputs the optimal voice recording setting.

[0086] The recording unit can preferentially record specific diagnostic contents by referring to the user's past medical history. The recording unit preferentially records related diagnostic contents, for example, based on diagnostic contents received by the user in the past. For example, the recording unit preferentially records specific diagnostic contents by referring to the user's past medical history. Based on advice received by the user in the past, related advice can also be preferentially recorded. For example, the recording unit takes the user's past medical history as an input and preferentially records specific diagnostic contents. Based on medicines prescribed by the user in the past, information on related medicines can also be preferentially recorded. This allows related diagnostic contents to be preferentially recorded by referring to the user's past medical history. Part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can record diagnostic contents using an AI model that takes the user's past medical history as an input and preferentially records specific diagnostic contents.

[0087] The transcription unit can estimate the user's emotion and adjust the accuracy of the transcription based on the estimated emotion of the user. For example, when the user is nervous, the transcription unit increases the accuracy of the transcription. For example, the transcription unit captures the user's facial expression with a camera and estimates the user's emotion using an emotion estimation algorithm. When the user is relaxed, the accuracy of the transcription can also be set to normal. For example, the transcription unit records the user's voice and estimates the user's emotion using a voice analysis technique. When the user is impatient, the accuracy of the transcription can also be increased. For example, the transcription unit collects the user's biometric data (heart rate and electrodermal activity) with a sensor and estimates the user's emotion using an emotion estimation algorithm. This allows for more accurate transcription by adjusting the accuracy of the transcription according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processes in the transcription unit may be performed using, for example, AI, or may be performed without using AI. For example, the transcription unit can adjust the accuracy of the transcription using an AI model that inputs user emotion data and outputs the accuracy of the transcription.

[0088] The transcription unit can automatically recognize technical terms in the contents of the medical examination and convert them into appropriate notation. The transcription unit can automatically recognize technical terms in the contents of the medical examination and convert them into appropriate notation using a generation AI. The generation AI can recognize technical terms using, for example, deep learning or machine learning algorithms and convert them into appropriate notation. For example, the generation AI can automatically recognize technical terms such as "high blood pressure" and "diabetes" and convert them into appropriate notation. The generation AI can also automatically recognize names of medical devices such as "MRI" and "CT scan" and convert them into appropriate notation. Furthermore, the generation AI can automatically recognize names of medicines such as "antibiotics" and "steroids" and convert them into appropriate notation. This improves the accuracy of transcription by automatically recognizing technical terms and converting them into appropriate notation. Some or all of the above-mentioned processing in the transcription unit may be performed, for example, using AI, or may be performed without using AI. For example, the transcription unit can recognize technical terms and convert them into appropriate notation using an AI model that inputs technical terms and outputs appropriate notation.

[0089] The transcription unit can identify the speaker of the audio and transcribe in a different format for each speaker. The transcription unit can use a generation AI to identify the speaker of the audio and transcribe in a different format for each speaker. The generation AI can use, for example, deep learning or machine learning algorithms to identify the speaker of the audio and transcribe in a different format for each speaker. For example, the generation AI can display the doctor's remarks in bold and the patient's remarks in normal font. The generation AI can also display the doctor's remarks in blue and the patient's remarks in black. Furthermore, the generation AI can also display the doctor's remarks in a large font size and the patient's remarks in a small font size. This makes it easier to organize information by transcribing in a different format for each speaker. Some or all of the above-mentioned processing in the transcription unit may be performed, for example, using AI or may be performed without using AI. For example, the transcription unit can perform transcription using an AI model that inputs audio data and transcribes in a different format for each speaker.

[0090] The transcription unit can estimate the user's emotion and adjust the transcription speed based on the estimated user's emotion. For example, when the user is nervous, the transcription unit increases the transcription speed. For example, the transcription unit captures the user's facial expression with a camera and estimates the user's emotion using an emotion estimation algorithm. When the user is relaxed, the transcription speed can also be set to normal. For example, the transcription unit records the user's voice and estimates the user's emotion using a voice analysis technology. When the user is impatient, the transcription speed can also be increased. For example, the transcription unit collects the user's biometric data (heart rate and electrodermal activity) with a sensor and estimates the user's emotion using an emotion estimation algorithm. This allows for more appropriate transcription by adjusting the transcription speed according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI can be a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processes in the transcription unit may be performed using, for example, AI, or may be performed without using AI. For example, the transcription unit can adjust the transcription speed using an AI model that inputs the user's emotion data and outputs the transcription speed.

[0091] The transcription unit can determine the priority of transcription based on the importance of the contents of the medical examination. The transcription unit uses the generation AI to determine the priority of transcription based on the importance of the contents of the medical examination. The generation AI evaluates the importance of the contents of the medical examination using, for example, deep learning or machine learning algorithms, and determines the priority of transcription based on the evaluation. For example, the generation AI prioritizes transcription of important diagnostic contents. The generation AI can also prioritize transcription of important advice. Furthermore, the generation AI can also prioritize transcription of important drug information. As a result, important information can be prioritized by determining the priority of transcription based on the importance of the contents of the medical examination. Part or all of the above-mentioned processing in the transcription unit may be performed, for example, using AI, or may be performed without using AI. For example, the transcription unit can determine the priority of transcription using an AI model that inputs the importance of the contents of the medical examination and outputs the priority of transcription.

[0092] The transcription unit can apply different transcription algorithms depending on the category of the medical examination content. The transcription unit uses a generation AI to apply different transcription algorithms depending on the category of the medical examination content. The generation AI classifies the category of the medical examination content using, for example, deep learning or machine learning algorithms, and applies a transcription algorithm according to the category. For example, the generation AI applies an algorithm that is strong in technical terms to the diagnosis content. The generation AI can also apply an algorithm that summarizes concisely to advice. Furthermore, the generation AI can also apply an algorithm that emphasizes accuracy to drug information. As a result, by applying different transcription algorithms depending on the category of the medical examination content, the accuracy of the transcription is improved. Part or all of the above-mentioned processing in the transcription unit may be performed, for example, using AI, or may be performed without using AI. For example, the transcription unit can perform transcription using an AI model that inputs the category of the medical examination content and outputs a transcription algorithm.

[0093] The summary creation unit can estimate the user's emotion and adjust the summary expression method based on the estimated user's emotion. The summary creation unit uses the generation AI to estimate the user's emotion and adjust the summary expression method based on the estimated user's emotion. The generation AI estimates the user's emotion using, for example, deep learning or machine learning algorithms and adjusts the summary expression method based on the estimation. For example, the generation AI creates a concise and easy-to-understand summary when the user is nervous. The generation AI can also create a detailed summary when the user is relaxed. Furthermore, the generation AI can also create a summary that covers the main points when the user is impatient. This allows a more appropriate summary to be provided by adjusting the summary expression method according to the user's emotion. The emotion estimation is realized using, for example, an emotion engine or a generation AI using an emotion estimation function. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the summary creation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the summary creation unit can adjust the way the summary is expressed using an AI model that takes a user's emotional data as input and outputs the way the summary is expressed.

[0094] The summary creation unit can adjust the level of detail of the summary based on the importance of the diagnostic content. The summary creation unit uses the generation AI to adjust the level of detail of the summary based on the importance of the diagnostic content. The generation AI evaluates the importance of the diagnostic content using, for example, deep learning or machine learning algorithms, and adjusts the level of detail of the summary based on the evaluation. For example, the generation AI creates a detailed summary for important diagnostic content. The generation AI can also create a detailed summary for important advice. Furthermore, the generation AI can also create a detailed summary for important drug information. As a result, important information can be grasped in detail by adjusting the level of detail of the summary based on the importance of the diagnostic content. Part or all of the above-mentioned processing in the summary creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the summary creation unit can adjust the level of detail of the summary using an AI model that inputs the importance of the diagnostic content and outputs the level of detail of the summary.

[0095] The summary creation unit can apply different summary creation algorithms depending on the category of the diagnostic content. The summary creation unit uses the generation AI to apply different summary creation algorithms depending on the category of the diagnostic content. The generation AI classifies the category of the diagnostic content using, for example, deep learning or machine learning algorithms, and applies the corresponding summary creation algorithm. For example, the generation AI applies an algorithm that is strong in technical terms to the diagnostic content. The generation AI can also apply an algorithm that summarizes concisely to advice. Furthermore, the generation AI can also apply an algorithm that emphasizes accuracy to drug information. Thereby, by applying different summary creation algorithms depending on the category of the diagnostic content, the accuracy of the summary is improved. Part or all of the above-mentioned processing in the summary creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the summary creation unit can create a summary using an AI model that inputs the category of the diagnostic content and outputs a summary creation algorithm.

[0096] The summary creation unit can estimate the user's emotion and adjust the length of the summary based on the estimated user's emotion. The summary creation unit uses the generation AI to estimate the user's emotion and adjust the length of the summary based on the estimated user's emotion. The generation AI estimates the user's emotion using, for example, deep learning or machine learning algorithms and adjusts the length of the summary based on the emotion. For example, the generation AI creates a short summary that is to the point when the user is nervous. The generation AI can also create a detailed summary when the user is relaxed. Furthermore, the generation AI can also create a short summary that is to the point when the user is impatient. This allows a more appropriate summary to be provided by adjusting the length of the summary according to the user's emotion. The emotion estimation is realized using, for example, an emotion engine or a generation AI, using an emotion estimation function. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the summary creation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the summary creation unit can adjust the length of the summary using an AI model that takes a user's emotional data as input and outputs the length of the summary.

[0097] The summary creation unit can determine the priority of the summary based on the time of submission of the diagnostic content. The summary creation unit uses the generation AI to determine the priority of the summary based on the time of submission of the diagnostic content. The generation AI evaluates the time of submission of the diagnostic content using, for example, deep learning or machine learning algorithms, and determines the priority of the summary based on the evaluation. For example, the generation AI prioritizes the creation of a summary of the most recent diagnostic content. The generation AI can also prioritize the creation of a summary of important diagnostic content. Furthermore, the generation AI can also prioritize the creation of a summary of diagnostic content whose submission deadline is approaching. As a result, important information can be summarized preferentially by determining the priority of the summary based on the time of submission of the diagnostic content. Part or all of the above-mentioned processing in the summary creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the summary creation unit can determine the priority of the summary using an AI model that inputs the time of submission of the diagnostic content and outputs the priority of the summary.

[0098] The summary creation unit can adjust the order of summaries based on the relevance of the diagnostic contents. The summary creation unit adjusts the order of summaries based on the relevance of the diagnostic contents using the generation AI. The generation AI evaluates the relevance of the diagnostic contents using, for example, deep learning or machine learning algorithms, and adjusts the order of summaries based on the evaluation. For example, the generation AI creates summaries with a high relevance. The generation AI can also create summaries with a high relevance of advice. Furthermore, the generation AI can also create summaries with a high relevance of drug information with a high relevance. This makes it easier to organize information by adjusting the order of summaries based on the relevance of the diagnostic contents. Part or all of the above-mentioned processing in the summary creation unit may be performed, for example, using AI, or may be performed without using AI. For example, the summary creation unit can adjust the order of summaries using an AI model that inputs the relevance of the diagnostic contents and outputs the order of summaries.

[0099] The answer unit can estimate the user's emotion and adjust the way of expressing the answer based on the estimated user's emotion. The answer unit estimates the user's emotion and adjusts the way of expressing the answer based on the estimated user's emotion. The answer unit estimates the user's emotion using, for example, natural language processing technology and adjusts the way of expressing the answer based on the emotion. For example, the answer unit provides a concise and easy-to-understand answer when the user is nervous. The answer unit can also provide a detailed answer when the user is relaxed. Furthermore, the answer unit can also provide an answer that covers the main points when the user is impatient. In this way, a more appropriate answer is provided by adjusting the way of expressing the answer according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the answer unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the answer unit can adjust the way the answer is expressed using an AI model that takes the user's emotional data as input and outputs the way the answer is expressed.

[0100] The answering unit can select the optimal answer by referring to the user's past question history. The answering unit selects the optimal answer by referring to the user's past question history. The answering unit, for example, uses natural language processing technology to analyze the user's past question history and selects the optimal answer based on the analysis. For example, the answering unit provides a related answer based on the content of the user's past questions. The answering unit can also prioritize answers to content that the user has frequently asked in the past. Furthermore, the answering unit can also select the most appropriate answer from the user's past question history. This makes it possible to provide a related answer by referring to the user's past question history. A part or all of the above-mentioned processing in the answering unit may be performed, for example, using AI, or may be performed without using AI. For example, the answering unit can select an answer using an AI model that inputs the user's past question history and outputs an optimal answer.

[0101] The answer unit can customize the content of the answer based on the user's current health condition. The answer unit customizes the content of the answer based on the user's current health condition. The answer unit, for example, uses natural language processing technology to analyze the user's current health condition and customizes the content of the answer based on the analysis. For example, when the user inputs the current health condition, the answer unit customizes the answer based on the information. The answer unit can also provide an answer related to the user's current health condition based on the user's past health condition. Furthermore, the answer unit can also provide an answer including appropriate advice according to the user's health condition. As a result, a more appropriate answer is provided by customizing the content of the answer based on the user's current health condition. A part or all of the above-mentioned processing in the answer unit may be performed, for example, using AI, or may be performed without using AI. For example, the answer unit can customize the content of the answer using an AI model that inputs the user's health condition data and outputs the content of the answer.

[0102] The answering unit can estimate the user's emotion and determine the priority of the answers based on the estimated user's emotion. The answering unit estimates the user's emotion and determines the priority of the answers based on the estimated user's emotion. The answering unit estimates the user's emotion using, for example, natural language processing technology and determines the priority of the answers based on the estimation. For example, the answering unit provides important answers preferentially when the user is nervous. Also, the answering unit can provide all answers equally when the user is relaxed. Furthermore, the answering unit can provide important answers preferentially when the user is impatient. In this way, important information can be provided preferentially by determining the priority of the answers according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the answering unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the answer unit can determine the priority of answers using an AI model that takes a user's emotional data as input and outputs the priority of answers.

[0103] The answering unit can select the optimal answer in consideration of the geographical location information of the user. The answering unit selects the optimal answer in consideration of the geographical location information of the user. The answering unit, for example, uses natural language processing technology to analyze the geographical location information of the user and selects the optimal answer based on the analysis. For example, when the user is in a hospital, the answering unit provides an answer related to the hospital. In addition, when the user is at home, the answering unit can also provide an answer that can be executed at home. Furthermore, when the user is out, the answering unit can also provide an answer that can be executed while out. This makes it possible to provide more appropriate information by providing an optimal answer according to the geographical location information of the user. Part or all of the above-mentioned processing in the answering unit may be performed, for example, using AI, or may be performed without using AI. For example, the answering unit can select an answer using an AI model that inputs the geographical location information of the user and outputs an optimal answer.

[0104] The answering unit can provide a relevant answer by analyzing the user's social media activity. The answering unit can provide a relevant answer by analyzing the user's social media activity. The answering unit can, for example, use natural language processing technology to analyze the user's social media activity and provide a relevant answer based on the analysis. For example, the answering unit can provide a relevant answer based on health information shared by the user on social media. The answering unit can also provide a relevant answer based on information on medical professionals the user follows on social media. Furthermore, the answering unit can provide a relevant answer based on health topics in which the user has shown interest on social media. This allows more appropriate information to be provided by providing a relevant answer based on the user's social media activity. Part or all of the above-mentioned processing in the answering unit can be performed, for example, using AI or without AI. For example, the answering unit can provide an answer using an AI model that inputs the user's social media activity data and outputs a relevant answer.

[0105] The noise removal unit can estimate the user's emotion and adjust the intensity of noise removal based on the estimated user's emotion. The noise removal unit estimates the user's emotion and adjusts the intensity of noise removal based on the estimated user's emotion. The noise removal unit estimates the user's emotion using, for example, a natural language processing technique and adjusts the intensity of noise removal based on the emotion. For example, the noise removal unit increases the intensity of noise removal when the user is nervous. In addition, the noise removal unit can also set the intensity of noise removal to normal when the user is relaxed. Furthermore, the noise removal unit can also increase the intensity of noise removal when the user is impatient. Thereby, the quality of the voice data is improved by adjusting the intensity of noise removal according to the user's emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the noise removal unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the noise removal unit can adjust the strength of noise removal using an AI model that inputs user emotion data and outputs the strength of noise removal.

[0106] The noise removal unit can analyze the environmental sound of the examination room in real time and apply an optimal noise removal filter. The noise removal unit can analyze the environmental sound of the examination room in real time and apply an optimal noise removal filter. The noise removal unit can analyze the environmental sound of the examination room using, for example, deep learning or machine learning algorithms, and apply an optimal noise removal filter based on the analysis. For example, the noise removal unit sets the noise removal filter low when the examination room is quiet. Also, the noise removal unit can set the noise removal filter high when the examination room is noisy. Furthermore, the noise removal unit can adjust the noise removal filter in real time when the environmental sound of the examination room fluctuates. As a result, the quality of the voice data is improved by applying the optimal noise removal filter according to the environmental sound of the examination room. A part or all of the above-mentioned processing in the noise removal unit may be performed using, for example, AI, or may be performed without using AI. For example, the noise removal unit can apply the noise removal filter using an AI model that inputs the environmental sound of the examination room and outputs the optimal noise removal filter.

[0107] The noise removal unit can estimate the user's emotion and determine the priority of noise removal based on the estimated user's emotion. The noise removal unit estimates the user's emotion and determines the priority of noise removal based on the estimated user's emotion. The noise removal unit estimates the user's emotion using, for example, a natural language processing technique and determines the priority of noise removal based on the estimation. For example, when the user is nervous, the noise removal unit prioritizes important voice removal. In addition, when the user is relaxed, the noise removal unit can also equally remove all voices. Furthermore, when the user is impatient, the noise removal unit can also prioritize important voice removal. In this way, by determining the priority of noise removal according to the user's emotion, important voice can be prioritized and removed. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the noise removal unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the noise removal unit can determine the priority of noise removal using an AI model that receives user emotion data as input and outputs the priority of noise removal.

[0108] The noise removal unit can adjust the intensity of noise removal based on the importance of the medical examination contents. The noise removal unit adjusts the intensity of noise removal based on the importance of the medical examination contents using the generation AI. The generation AI evaluates the importance of the medical examination contents using, for example, deep learning or machine learning algorithms, and adjusts the intensity of noise removal based on the evaluation. For example, the generation AI increases the intensity of noise removal for important diagnostic contents. The generation AI can also increase the intensity of noise removal for important advice. Furthermore, the generation AI can also increase the intensity of noise removal for important medicine information. As a result, important information can be preferentially removed by adjusting the intensity of noise removal based on the importance of the medical examination contents. Part or all of the above-mentioned processing in the noise removal unit may be performed, for example, using AI, or may be performed without using AI. For example, the noise removal unit can adjust the intensity of noise removal using an AI model that inputs the importance of the medical examination contents and outputs the intensity of noise removal.

[0109] The question analysis unit can estimate the user's emotion and adjust the accuracy of the question analysis based on the estimated user's emotion. The question analysis unit estimates the user's emotion and adjusts the accuracy of the question analysis based on the estimated user's emotion. The question analysis unit estimates the user's emotion using, for example, a natural language processing technique and adjusts the accuracy of the question analysis based on the emotion. For example, the question analysis unit increases the accuracy of the question analysis when the user is nervous. In addition, the question analysis unit can also set the accuracy of the question analysis to normal when the user is relaxed. Furthermore, the question analysis unit can also increase the accuracy of the question analysis when the user is impatient. This allows for more appropriate analysis by adjusting the accuracy of the question analysis according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the question analysis unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the question analysis unit can adjust the accuracy of question analysis using an AI model that inputs user emotion data and outputs the accuracy of question analysis.

[0110] The question analysis unit can select the optimal analysis method by referring to the user's past question history. The question analysis unit selects the optimal analysis method by referring to the user's past question history. The question analysis unit, for example, uses natural language processing technology to analyze the user's past question history and selects the optimal analysis method based on the analysis. For example, the question analysis unit selects the optimal analysis method based on the content of questions asked by the user in the past. The question analysis unit can also preferentially analyze content that the user has frequently asked in the past. Furthermore, the question analysis unit can also select the most appropriate analysis method from the user's past question history. Thereby, by referring to the user's past question history, a related analysis method can be selected. A part or all of the above-mentioned processing in the question analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the question analysis unit can select the analysis method using an AI model that inputs the user's past question history and outputs the optimal analysis method.

[0111] The question analysis unit can estimate the user's emotion and determine the priority of question analysis based on the estimated user's emotion. The question analysis unit estimates the user's emotion and determines the priority of question analysis based on the estimated user's emotion. The question analysis unit estimates the user's emotion using, for example, natural language processing technology and determines the priority of question analysis based on the estimation. For example, when the user is nervous, the question analysis unit preferentially analyzes important questions. In addition, when the user is relaxed, the question analysis unit can also analyze all questions equally. Furthermore, when the user is impatient, the question analysis unit can also analyze important questions preferentially. In this way, by determining the priority of question analysis according to the user's emotion, important questions can be preferentially analyzed. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the question analysis unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the question analysis unit can determine the priority of question analysis using an AI model that receives user emotion data as input and outputs the priority of question analysis.

[0112] The question analysis unit can select the optimal analysis method in consideration of the geographical location information of the user. The question analysis unit selects the optimal analysis method in consideration of the geographical location information of the user. The question analysis unit, for example, uses natural language processing technology to analyze the geographical location information of the user and selects the optimal analysis method based on the information. For example, when the user is in a hospital, the question analysis unit preferentially analyzes questions related to the hospital. In addition, when the user is at home, the question analysis unit can also preferentially analyze questions that can be performed at home. Furthermore, when the user is out, the question analysis unit can also preferentially analyze questions that can be performed outside the home. This allows for more appropriate analysis by selecting the optimal analysis method according to the geographical location information of the user. Part or all of the above-mentioned processing in the question analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the question analysis unit can select the analysis method using an AI model that inputs the geographical location information of the user and outputs the optimal analysis method.

[0113] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0114] The medical examination content recording system can not only record the medical examination content of the user, but also provide supplementary information to deepen understanding of the medical examination content. For example, the recording unit can automatically search for medical background information and general health information related to the medical examination content and provide it to the user. This allows the user to understand the medical examination content more deeply. The recording unit can also provide videos and images related to the medical examination content. For example, providing videos about treatments and prevention methods for diagnosed diseases can help the user understand. Furthermore, the recording unit can provide other patients' experiences and reviews related to the medical examination content. This allows the user to refer to the experiences of other patients.

[0115] The recording unit can have a function of not only recording the contents of the medical examination, but also monitoring the health condition of the user. For example, the recording unit can collect biological data such as the user's heart rate, blood pressure, and body temperature using a sensor and record the data together with the contents of the medical examination. This allows the user's health condition to be understood comprehensively. The recording unit can also collect lifestyle data such as the user's amount of exercise and sleep patterns and associate the data with the contents of the medical examination. For example, if the user's lack of exercise is affecting the diagnosis results, providing this information can encourage the user to improve their lifestyle habits. Furthermore, the recording unit can also record the user's diet and evaluate the nutritional balance. This can support the user's health management.

[0116] The transcription unit can not only transcribe the voice data, but also perform emotion analysis of the voice data. For example, the transcription unit can analyze the user's voice data and estimate the user's emotional state. This makes it possible to grasp the anxiety and stress felt by the user during the examination. The transcription unit can also analyze the doctor's voice data and estimate the doctor's emotional state. For example, the transcription unit can analyze the tone and language used when the doctor conveys the diagnosis and estimate the doctor's emotion, thereby evaluating the doctor's consideration for the user. Furthermore, the transcription unit can provide advice and support to the user based on the emotion analysis result of the voice data. For example, it can suggest a relaxation method to reduce the anxiety felt by the user during the examination.

[0117] The summary creation unit can not only create a summary of the diagnosis results, but also provide predictive information based on the diagnosis results. For example, the summary creation unit can predict future changes in the user's health condition based on the diagnosis results. This allows the user to obtain information that will be useful for future health management. The summary creation unit can also perform risk assessment based on the diagnosis results. For example, if the diagnosis results indicate a high risk of a specific disease, the summary creation unit can notify the user of the risk and suggest preventive measures. Furthermore, the summary creation unit can predict the effectiveness of treatment based on the diagnosis results. For example, it can predict how effective a specific treatment will be and suggest the most suitable treatment to the user.

[0118] The answering unit can not only provide answers to the user's questions, but also learn based on the content of the user's questions. For example, the answering unit can learn the content of questions frequently asked by the user and improve the accuracy of the answers based on the information. This makes it possible to provide answers that are more suitable for the user's needs. The answering unit can also acquire new knowledge based on the content of the user's questions. For example, when a user repeatedly asks questions about a specific illness, the latest information about the illness can be automatically collected and reflected in the answers. Furthermore, the answering unit can provide personalized advice based on the content of the user's questions. For example, the answering unit can support the user's health management by providing specific advice according to the user's health condition and lifestyle.

[0119] The medical examination content recording system can estimate the user's emotions and adjust the method of recording the medical examination content based on the estimated emotions. For example, if the user is nervous during the medical examination, the recording unit can adjust so as not to start voice recording until the user is relaxed. This allows the medical examination content to be recorded in a relaxed state. Also, if the user feels anxious during the medical examination, the recording unit can record the medical examination content in detail so that the user can check it later. Furthermore, if the user is impatient during the medical examination, the recording unit can record important diagnostic contents with priority, making it easier for the user to check them later. This allows more appropriate recording by adjusting the method of recording the medical examination content according to the user's emotions.

[0120] The medical examination content recording system can not only record the medical examination content of the user, but also provide a health management plan based on the medical examination content. For example, the recording unit can automatically create a health management plan suitable for the user based on the medical examination content. This allows the user to know a specific health management method based on the medical examination content. The recording unit can also provide a meal plan and an exercise plan based on the medical examination content. For example, the recording unit can support the user's health improvement by suggesting a meal menu and an exercise program suitable for the user based on the diagnosis result. Furthermore, the recording unit can also provide a stress management plan based on the medical examination content. For example, the recording unit can suggest a relaxation method or a stress relief method to reduce the stress felt by the user during the medical examination.

[0121] The recording unit can estimate the user's emotion and adjust the recording format of the medical examination contents based on the estimated emotion. For example, if the user is nervous during the examination, the recording unit can record the medical examination contents in a concise format. This makes it easier for the user to check later. Also, if the user is relaxed during the examination, the recording unit can record the medical examination contents in detail. This allows the user to understand the medical examination contents more deeply. Furthermore, if the user is impatient during the examination, the recording unit can record important points with emphasis. This makes it possible to prevent the user from missing important information. The estimation of emotions is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. Some or all of the above-mentioned processing in the recording unit may be performed using, for example, an AI, or may be performed without using an AI.

[0122] The recording unit can not only record the contents of the medical examination, but also provide an interactive function to help the user understand the contents of the medical examination. For example, when the user inputs a question about the contents of the medical examination, the recording unit can provide related information. This allows the user to have a deeper understanding of the contents of the medical examination. The recording unit can also provide quizzes and tests related to the contents of the medical examination. For example, the user can review the contents of the medical examination through quizzes about the diagnosis results and advice. Furthermore, the recording unit can provide a discussion forum about the contents of the medical examination. This allows the user to share information and exchange opinions with other patients and medical professionals.

[0123] The recording unit can estimate the user's emotion and adjust the recording frequency of the medical examination contents based on the estimated emotion. For example, if the user is nervous during the examination, the recording unit can record the medical examination contents frequently to make it easier for the user to check later. Also, if the user is relaxed during the examination, the recording unit can record the medical examination contents at a normal frequency. Furthermore, if the user is impatient during the examination, the recording unit can record important points with emphasis so that the user does not miss important information. This allows more appropriate recording by adjusting the recording frequency of the medical examination contents according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. A part or all of the above-mentioned processing in the recording unit may be performed using, for example, an AI, or may be performed without using an AI.

[0124] The process flow of the second embodiment will be briefly described below.

[0125] Step 1: When the user is examined at a hospital, the recording unit uses the smartphone's microphone to record the doctor's diagnosis as audio. The recording unit can record audio using, for example, the smartphone's built-in microphone or an external microphone. Step 2: The transcription unit uses the generation AI to transcribe the audio recorded by the recording unit. The generation AI analyzes the audio data using, for example, deep learning or machine learning algorithms, and transcribes it. Step 3: The summary creation unit uses a generation AI to create a summary of the diagnosis result based on the text generated by the transcription unit. The generation AI creates a summary of the diagnosis result using, for example, a text generation AI (e.g., LLM). Step 4: The answering unit analyzes the user's question and provides an answer from the recorded text. The answering unit analyzes the user's question using, for example, natural language processing technology and provides an appropriate answer. As a result, the medical examination content recording system according to the embodiment allows the elderly to remember to record the medical examination content and accurately understand the diagnosis results and advice.

[0126] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0127] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0128] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0129] Each of the multiple elements including the above-mentioned recording unit, transcription unit, summary creation unit, answer unit, noise removal unit, and question analysis unit is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the recording unit records the diagnosis content by voice using the microphone of the smart device 14. The transcription unit is realized by the specific processing unit 290 of the data processing device 12 and transcribes the recorded voice. The summary creation unit is realized by the specific processing unit 290 of the data processing device 12 and creates a summary of the diagnosis result based on the transcribed text. The answer unit is realized by the specific processing unit 290 of the data processing device 12 and provides an answer to the user's question from the recorded text. The noise removal unit is realized by the control unit 46A of the smart device 14 and removes noise from the recorded voice. The question analysis unit is realized by the specific processing unit 290 of the data processing device 12 and analyzes the user's question and provides an appropriate answer. The correspondence between each part and the device or control part is not limited to the above-mentioned example, and various modifications are possible.

[0130] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0131] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0132] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0133] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0134] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0135] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0136] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0137] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0138] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0139] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0140] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0141] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0142] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0143] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0144] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0145] Each of the multiple elements including the above-mentioned recording unit, transcription unit, summary creation unit, answer unit, noise removal unit, and question analysis unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the recording unit records the diagnosis content by voice using the microphone of the smart glasses 214. The transcription unit is realized by the specific processing unit 290 of the data processing device 12 and transcribes the recorded voice. The summary creation unit is realized by the specific processing unit 290 of the data processing device 12 and creates a summary of the diagnosis result based on the transcribed text. The answer unit is realized by the specific processing unit 290 of the data processing device 12 and provides an answer to a user's question from the recorded text. The noise removal unit is realized by the control unit 46A of the smart glasses 214 and removes noise from the recorded voice. The question analysis unit is realized by the specific processing unit 290 of the data processing device 12 and analyzes the user's question and provides an appropriate answer. The correspondence between each part and the device or control part is not limited to the above-mentioned example, and various modifications are possible.

[0146] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0147] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0148] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0149] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0150] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0151] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0152] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0153] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0154] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0155] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0156] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0157] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0158] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0159] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0160] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0161] Each of the multiple elements including the above-mentioned recording unit, transcription unit, summary creation unit, answer unit, noise removal unit, and question analysis unit is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the recording unit records the diagnosis contents by voice using the microphone of the headset type terminal 314. The transcription unit is realized by the specific processing unit 290 of the data processing device 12 and transcribes the recorded voice. The summary creation unit is realized by the specific processing unit 290 of the data processing device 12 and creates a summary of the diagnosis result based on the transcribed text. The answer unit is realized by the specific processing unit 290 of the data processing device 12 and provides an answer to a user's question from the recorded text. The noise removal unit is realized by the control unit 46A of the headset type terminal 314 and removes noise from the recorded voice. The question analysis unit is realized by the specific processing unit 290 of the data processing device 12 and analyzes the user's question and provides an appropriate answer. The correspondence between each part and the device or control part is not limited to the above-mentioned example, and various modifications are possible.

[0162] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0163] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0164] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0165] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0166] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0167] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0168] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0169] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0170] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0171] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0172] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0173] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0174] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0175] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0176] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0177] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0178] Each of the multiple elements including the above-mentioned recording unit, transcription unit, summary creation unit, answer unit, noise removal unit, and question analysis unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the recording unit records the diagnosis content by voice using a microphone of the robot 414. The transcription unit is realized by the specific processing unit 290 of the data processing device 12 and transcribes the recorded voice. The summary creation unit is realized by the specific processing unit 290 of the data processing device 12 and creates a summary of the diagnosis result based on the transcribed text. The answer unit is realized by the specific processing unit 290 of the data processing device 12 and provides an answer to a user's question from the recorded text. The noise removal unit is realized by the control unit 46A of the robot 414 and removes noise from the recorded voice. The question analysis unit is realized by the specific processing unit 290 of the data processing device 12 and analyzes the user's question and provides an appropriate answer. The correspondence between each part and the device or control part is not limited to the above-mentioned example, and various modifications are possible.

[0179] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0180] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0181] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0182] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out on emotion map 400 you go, the more visible the emotions become (the more they are expressed in behavior).

[0183] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0184] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0185] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0186] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0187] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0188] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0189] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0190] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0191] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0192] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0193] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The above specific processes are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0194] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0195] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0196] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0197] (Appendix 1) A recording unit for recording audio; a transcription unit that transcribes the voice recorded by the recording unit; a summary creation unit that creates a summary of the diagnosis result based on the text generated by the transcription unit; An answering unit that provides an answer to a user's question based on the summary created by the summary creating unit. A system characterized by: (Appendix 2) The recording section is Record audio using a microphone 2. The system of claim 1, (Appendix 3) The transcription department is Analyze voice data using AI and transcribe it 2. The system of claim 1, (Appendix 4) The summary creation department: Create a summary of diagnostic results based on AI-transcribed text 2. The system of claim 1, (Appendix 5) The answer section is Analyze questions and provide answers from recorded text 2. The system of claim 1, (Appendix 6) Equipped with a noise reduction unit that removes noise from recorded audio 2. The system of claim 1, (Appendix 7) Equipped with a question analysis unit that analyzes user questions and provides appropriate answers 2. The system of claim 1, (Appendix 8) The recording unit is To estimate a user's emotion and adjust a start timing of voice recording based on the estimated user's emotion. 2. The system of claim 1, (Appendix 9) The recording unit is Analyzes the ambient sound in the examination room and automatically adjusts the microphone sensitivity to the optimum level 2. The system of claim 1, (Appendix 10) The recording unit is Add a feature to automatically highlight certain keywords when they are spoken 2. The system of claim 1, (Appendix 11) The recording unit is Estimate the user's emotions and prioritize the audio to be recorded based on the estimated user's emotions. 2. The system of claim 1, (Appendix 12) The recording unit is Select optimal voice recording settings taking into account the user's location 2. The system of claim 1, (Appendix 13) The recording unit is Refer to the user's past medical history and record specific diagnoses with priority 2. The system of claim 1, (Appendix 14) The transcription unit includes: Estimate user sentiment and adjust transcription accuracy based on estimated user sentiment 2. The system of claim 1, (Appendix 15) The transcription unit includes: Automatically recognizes medical terminology and converts it into appropriate notation 2. The system of claim 1, (Appendix 16) The transcription unit includes: Identify the speaker of audio and transcribe it in a different format for each speaker 2. The system of claim 1, (Appendix 17) The transcription unit includes: Infer user sentiment and adjust transcription speed based on the estimated user sentiment 2. The system of claim 1, (Appendix 18) The transcription unit includes: Prioritize transcription based on importance of encounters 2. The system of claim 1, (Appendix 19) The transcription unit includes: Apply different transcription algorithms depending on the category of encounter 2. The system of claim 1, (Appendix 20) The summary creation unit Infer user emotions and adjust the way the summary is presented based on the estimated user emotions 2. The system of claim 1, (Appendix 21) The summary creation unit Adjust the summary detail based on the importance of the diagnosis 2. The system of claim 1, (Appendix 22) The summary creation unit Apply different summary algorithms depending on diagnostic category 2. The system of claim 1, (Appendix 23) The summary creation unit Infer user sentiment and adjust the length of the summary based on the estimated user sentiment 2. The system of claim 1, (Appendix 24) The summary creation unit Prioritize summaries based on when diagnostics were submitted 2. The system of claim 1, (Appendix 25) The summary creation unit Adjust the order of summaries based on diagnostic relevance 2. The system of claim 1, (Appendix 26) The answering section: Infer the user's emotions and adjust the way the answer is presented based on the inferred user emotions 2. The system of claim 1, (Appendix 27) The answering section: Select the best answer by looking at the user's past question history 2. The system of claim 1, (Appendix 28) The answering section: Customize answer content based on the user's current health status 2. The system of claim 1, (Appendix 29) The answering section: Infer user emotions and prioritize answers based on the inferred user emotions 2. The system of claim 1, (Appendix 30) The answering section: Select the best answer based on the user's geographic location 2. The system of claim 1, (Appendix 31) The answering section: Analyze your social media activity to provide relevant answers 2. The system of claim 1, (Appendix 32) The noise removal unit includes: Estimate user emotion and adjust noise reduction strength based on the estimated user emotion. 2. The system of claim 1, (Appendix 33) The noise removal unit includes: Analyzes the ambient sound in the examination room in real time and applies the optimal noise reduction filter 2. The system of claim 1, (Appendix 34) The noise removal unit includes: Estimate user emotion and determine priority of noise removal based on the estimated user emotion. 2. The system of claim 1, (Appendix 35) The noise removal unit includes: Adjust noise reduction strength based on importance of the diagnosis 2. The system of claim 1, (Appendix 36) The question analysis unit: Estimate user's emotions and adjust the accuracy of question analysis based on the estimated user's emotions. 2. The system of claim 1, (Appendix 37) The question analysis unit: Select the optimal analysis method by referring to the user's past question history 2. The system of claim 1, (Appendix 38) The question analysis unit: Estimate user's emotions and determine the priority of question analysis based on the estimated user's emotions. 2. The system of claim 1, (Appendix 39) The question analysis unit: Select the best analysis method based on the user's geographic location 2. The system of claim 1, [Explanation of symbols]

[0198] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A recording unit for recording audio; A transcription unit that transcribes the voice recorded by the recording unit; a summary creation unit that creates a summary of the diagnosis result based on the text generated by the transcription unit; An answering unit that provides an answer to a user's question based on the summary created by the summary creating unit. A system characterized by:

2. The recording unit is Record audio using a microphone 2. The system of claim 1 .

3. The transcription department is Analyze voice data using AI and transcribe it 2. The system of claim 1 .

4. The summary creation department: Create a summary of diagnostic results based on text transcribed by AI 2. The system of claim 1 .

5. Equipped with a noise reduction unit that removes noise from recorded audio 2. The system of claim 1 .

6. Equipped with a question analysis unit that analyzes user questions and provides appropriate answers 2. The system of claim 1 .

7. The recording unit is To estimate a user's emotion and adjust a start timing of voice recording based on the estimated user's emotion.

2. The system of claim 1 .

Images