Data processing system

Abstract

To provide a data processing system allowing a medical worker to efficiently select an appropriate diagnosis and treatment method.SOLUTION: The data processing system includes a necklace-type terminal including a sensor configured to detect biological data of a wearer, a microphone configured to collect an interaction between the wearer and a medical worker and convert the interaction into voice data, a collection unit configured to collect an output of each of the sensor and the microphone, and a communication unit configured to transmit the output of each of the sensor and the microphone collected by the collection unit to an external device, and a data processing device. The data processing device includes an input unit that receives biological data collected by the sensor and voice data collected by the microphone, a processing unit that acquires and infers medically important information from the voice data based on a change in an emotion of the wearer estimated from at least one of the biological data and the voice data, and acquires a diagnosis support information as an inference result, and an output unit that outputs the diagnosis support information acquired by the processing unit to a preset output destination.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The techniques of this disclosure relate to data processing systems. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the medical field, it is desirable to select more appropriate diagnoses and treatment methods, and to do so, medical professionals need to process and evaluate a large amount of information necessary to select a patient's diagnosis and treatment method. However, it is elaborate, difficult, and very time-consuming for medical professionals to process and evaluate a large amount of information at once. While the above-mentioned conventional technology can generate chatbot utterances corresponding to user utterances, it is not possible to process a large amount of information from the dialogue between a medical professional and a patient during a medical interview and evaluate a diagnosis or treatment method. [Means for solving the problem]

[0005] A first aspect of the technology of the present disclosure is a data processing system including: a necklace-type terminal including a sensor that detects biometric data of a wearer; a microphone that picks up a conversation between the wearer and a medical professional and converts it into audio data; a collection unit that collects the outputs of the sensor and the microphone; and a communication unit that transmits the outputs of the sensor and the microphone collected by the collection unit to an external device; and a data processing device, wherein the data processing device includes: an input unit that accepts the biometric data collected by the sensor and the audio data picked up by the microphone; a processing unit that acquires and infers medically important information from the audio data based on a pattern of changes in the wearer's emotions estimated from at least one of the biometric data and the audio data, and acquires the inference result as diagnostic support information; and an output unit that outputs the diagnostic support information acquired by the processing unit to a predetermined output destination.

[0006] A second aspect of the technology disclosed herein is a data processing system according to the first aspect, wherein the processing unit inputs a prompt including the wearer's emotional change pattern and the voice data into a data generation model, and obtains the diagnostic assistance information based on the output of the data generation model.

[0007] A third aspect of the technology of the present disclosure is the data processing system of the second aspect, wherein the processing unit, each time the collected dialogue is obtained, uses an emotion engine that estimates the wearer's emotion to estimate the emotion of the wearer based on the biometric data, and estimates the time-series change in the estimated emotion of the wearer as a change pattern of the wearer's emotion.

[0008] A fourth aspect of the technique of the present disclosure is a data processing system according to any one of the first to third aspects, wherein one or more types of diseases and an evaluation result for each of the diseases are acquired as the diagnostic assistance information.

[0009] A fifth aspect of the technique of the present disclosure is a data processing system according to the fourth aspect, wherein one or more treatment methods are acquired for each of the diseases as the diagnostic support information. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a necklace-type terminal according to a first embodiment. [Figure 3] 1 is a side view showing the configuration of a necklace-type terminal according to a first embodiment. [Figure 4] 1 is a top view showing the configuration of a necklace-type terminal according to a first embodiment. FIG. [Figure 5] 2 is a schematic diagram showing the functional configuration of a control unit of the necklace-type terminal according to the first embodiment. [Figure 6] 2 shows a schematic functional configuration of a specific processing unit of the data processing device according to the first embodiment. [Figure 7] 3 shows an example of an operational flow of a specific process performed by the data processing device according to the first embodiment. [Figure 8] 10 is a diagram illustrating an example of an operational flow of a specific process performed by a data processing device according to a second embodiment. [Figure 9] 10 shows an example of an operation flow when updating a data generation model according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an example of an embodiment of a data processing device, a data processing method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0012] First, the terms used in the following description will be explained.

[0013] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), or an APU (Accelerated Processing Unit).

[0014] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0015] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0016] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0017] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0018] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to a first embodiment of the present disclosure.

[0019] 1, the data processing system 10 includes a data processing device 12 and a necklace-type terminal 14. An example of the data processing device 12 is a server. In this embodiment, the data processing device 12 is an example of a "data processing device" according to the technology of the present disclosure, and the necklace-type terminal 14 is an example of a "necklace-type terminal" according to the technology of the present disclosure.

[0020] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0021] The necklace type terminal 14 includes a computer 36, a microphone 38, a sensor 39, a speaker 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 38, the speaker 40, and the camera 42 are also connected to the bus 52.

[0022] The user 20 who wears the necklace-type terminal 14 may be, for example, a patient whose health condition is to be diagnosed, or may be a normal user.

[0023] The microphone 38 picks up the voice uttered by the user 20 who is wearing the necklace-type terminal 14, as well as sounds around the user 20. The microphone 38 also receives instructions and the like from the user 20 by receiving the voice uttered by the user 20. The microphone 38 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 40 outputs audio in accordance with instructions from the processor 46. The speaker 40 is, for example, a directional speaker, and outputs audio toward the ears of the user 20.

[0024] The sensor 39 is a sensor that detects biological data of the user 20 who is wearing the necklace-type terminal. For example, the sensor 39 is a heart rate sensor or a blood oxygen sensor.

[0025] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0026] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0027] FIG. 2 shows an example of the main functions of the data processing device 12 and the necklace-type terminal 14.

[0028] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores a data generation model 58. The data generation model 58 is used by the specific processing unit 290. The storage 32 also includes a data accumulation unit 57.

[0030] In the necklace-type terminal 14, the data collection process is performed by the processor 46. A data collection program 60 is stored in the storage 50. The processor 46 reads the data collection program 60 from the storage 50 and executes the read data collection program 60 on the RAM 48. The data collection process is realized by the processor 46 operating as the control unit 46A in accordance with the data collection program 60 executed on the RAM 48.

[0031] As shown in FIGS. 3 and 4, the necklace type terminal 14 includes multiple microphones 38, multiple sensors 39, multiple speakers 40, and multiple cameras 42. FIGS. 3 and 4 show an example in which two microphones 38 are arranged so as to be located in front of the user 20 when the user 20 wears the necklace type terminal 14. FIGS. 3 and 4 also show an example in which two sensors 39 are arranged so as to be located on the right and left sides of the user 20 when the user 20 wears the necklace type terminal 14. FIGS. 3 and 4 also show an example in which two speakers 40 are arranged so as to be located on the right and left rear sides of the user 20 when the user 20 wears the necklace type terminal 14. FIGS. 3 and 4 also show an example in which two cameras 42 are arranged so as to be located on the right and left front sides of the user 20 when the user 20 wears the necklace type terminal 14. FIGS. 3 and 4 also show an example in which two sensors 39 are arranged inside the necklace type terminal 14 so as to come into contact with the neck of the user 20 when the user 20 wears the necklace type terminal 14.

[0032] Next, the processing of the control unit 46A when the necklace-type terminal 14 performs a data collection process for collecting data will be described.

[0033] In the data collection process of this embodiment, biometric data of the user is collected in real time. Furthermore, not only biometric data but also all situational data surrounding the user is collected. This makes it possible to detect early signs of, for example, Alzheimer's disease and dementia. It also makes it possible to monitor the user's health condition (for example, heart disease).

[0034] As shown in FIG. 5, the control unit 46A includes a data collection unit 100 and a communication unit 102.

[0035] The data collection unit 100 collects the output of each of the microphone 38, the sensor 39, and the camera 42.

[0036] The communication unit 102 transmits the outputs of the microphone 38 , the sensor 39 , and the camera 42 collected by the data collection unit 100 to the data processing device 12 .

[0037] Next, a description will be given of the processing of the specific processing unit 290 when the data processing device 12 performs the specific processing to acquire a response corresponding to a user utterance.

[0038] In the identification process of this embodiment, a response corresponding to a user utterance picked up by the microphone 38 of the necklace-type terminal 14 is acquired using the data generation model 58.

[0039] As shown in FIG. 6, the specific processing unit 290 includes an input unit 292, a processing unit 294, and an output unit 296.

[0040] The input unit 292 stores the outputs of the microphone 38 , the sensor 39 , and the camera 42 received from the necklace-type terminal 14 in the data storage unit 57 .

[0041] The input unit 292 acquires the user's utterance received by the necklace type terminal 14. Specifically, the input unit 292 acquires the user's utterance picked up by the microphone 38 of the necklace type terminal 14.

[0042] The processing unit 294 performs a specific process using the data generation model 58. Specifically, a prompt including a user utterance is input to the data generation model 58 to obtain a generation result. At this time, the prompt may further include outputs from the sensor 39 and the camera 42 collected by the data collection unit 100.

[0043] The output unit 296 transmits the result of the identification processing to the necklace type terminal 14. In the necklace type terminal 14, the control unit 46A causes the speaker 40 to output the result of the identification processing. In this way, a response corresponding to the user utterance picked up by the microphone 38 is output to the user 20 by the speaker 40. The microphone 38 further acquires the user utterance in response to the result of the identification processing. The control unit 46A transmits voice data indicating the user utterance acquired by the microphone 38 to the data processing device 12. In the data processing device 12, the identification processing unit 290 acquires the user utterance.

[0044] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generative AI models. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0045] The outputs of the microphone 38, the sensor 39, and the camera 42 stored in the data storage unit 57 are used, for example, to diagnose the health condition of the user 20. In this case, the outputs of the microphone 38, the sensor 39, and the camera 42 stored in the data storage unit 57 may be transmitted to a terminal on the medical institution side. Alternatively, the data processing device 12 may analyze the outputs of the microphone 38, the sensor 39, and the camera 42 stored in the data storage unit 57 to diagnose the health condition of the user 20.

[0046] Next, the operation of the data processing system 10 will be described.

[0047] First, an example of the flow of the data collection process will be described.

[0048] When the user 20 is wearing the necklace-type terminal 14, the data collection unit 100 sequentially collects the outputs of the microphone 38, the sensor 39, and the camera 42. The communication unit 102 sequentially transmits the outputs of the microphone 38, the sensor 39, and the camera 42 collected by the data collection unit 100 to the data processing device 12.

[0049] Next, an example of the flow of the identification process will be described with reference to Fig. 7. Here, it is assumed that the input unit 292 of the data processing device 12 sequentially acquires the outputs of the microphone 38, the sensor 39, and the camera 42 received from the necklace-type terminal 14 and stores them in the data accumulation unit 57.

[0050] In step S300, the processing unit 294 determines whether a predetermined trigger condition is satisfied. Specifically, the trigger condition may be that a specific word (for example, the name of an agent installed in the necklace type terminal 14) or phrase (for example, "Hi! XXX" (XXX is the name of the agent)) is included in the user utterance picked up by the microphone 38.

[0051] If the trigger condition is met in step S300 (step S300; Yes), the data processing system 10 proceeds to step S301. On the other hand, if the trigger condition is not met in step S300 (step S300; No), the data processing system 10 ends the identification process.

[0052] In step S301, the processing unit 294 generates a prompt by adding an instruction sentence for obtaining a result of a specific process to text representing a user utterance picked up by the microphone 38.

[0053] For example, a prompt such as "The user is saying the following: XXX. Please respond as an agent." (XXX is the user's utterance) may be generated. Alternatively, the outputs of the sensor 39 and the camera 42 may be added to the prompt to generate a prompt such as "This is biometric data representing the user's heart rate and video data representing the user's surroundings. The user is also saying the following: XXX. Please respond as an agent." (XXX is the user's utterance).

[0054] In step S303, the processing unit 294 inputs the generated prompt to the data generation model 58, and obtains the result of the specific process based on the output of the data generation model 58.

[0055] In step S304, the output unit 296 outputs the result of the identification process to the necklace-type terminal 14, and the identification process ends.

[0056] [Second embodiment] Next, a second embodiment will be described. In the second embodiment, as the specific processing described in the first embodiment, for example, a conversation between a medical professional and a patient during a medical interview is collected in real time in a medical setting. Then, in order to efficiently determine an appropriate diagnosis and treatment method from the biometric data of the patient during the medical interview and audio data including the collected content of the conversation, medically important information is acquired and inferred, and the inference result is acquired as diagnostic support information. Here, a medical professional refers to a person engaged in medical care, such as a doctor or a nurse. Note that in this embodiment, "inference" also refers to, for example, analysis, classification, prediction, and / or summarization.

[0057] "Diagnostic support information" refers to information that can assist medical professionals in making diagnoses and determining treatment methods. It refers to medically important information obtained during interviews between medical professionals and patients, organized and evaluated for easy understanding by doctors. Specifically, for example, it refers to information such as one or more types of diseases that the patient may be suffering from, evaluation results for each disease, one or more treatment methods for each disease, and evaluation results for each treatment method. Here, "evaluation results" refer to, for example, values ​​indicating the likelihood of the disease and values ​​indicating the effectiveness of the treatment methods. Furthermore, "medically important information" refers to information indicating symptoms, such as physical conditions and changes, that the patient consciously or unconsciously feels, and is information linked to a specific disease. Note that this embodiment will also be described using the configuration and symbols of the data processing system 10 in the first embodiment described above.

[0058] In this embodiment, the user 20, who is a patient, wears the necklace-type terminal 14. As described above, the necklace-type terminal 14 includes a sensor 39 that detects biometric data of the user 20, who is a patient and wears the necklace-type terminal 14, in real time, a microphone 38 that picks up the conversation between the user 20 and a medical professional and converts it into audio data, a data collection unit 100 that collects the outputs of the sensor 39 and the microphone 38, and a communication unit 102 that transmits the outputs of the sensor 39 and the microphone 38 collected by the data collection unit 100 to the data processing device 12, which is an external device.

[0059] Here, the sensor 39 detects data such as the heart rate, pulse rate, respiratory rate, blood pressure, and perspiration of the user 20 wearing the necklace-type terminal 14 as biometric data.

[0060] The data processing device 12 in this embodiment includes an input unit 292 that receives biometric data collected by the sensor 39 and voice data picked up by the microphone 38, a processing unit 294 that acquires and infers medically important information from the voice data during the conversation based on a pattern of emotional changes estimated from at least one of the biometric data and voice data of the user 20, and acquires the inference results as diagnostic support information, and an output unit 296 that outputs the acquired diagnostic support information to a pre-set output destination.

[0061] Here, the preset output destination may be, for example, a medical professional's terminal or a terminal of a medical institution. By accessing their own terminal or a terminal of a medical institution, medical professionals can check the diagnostic support information, which is the inference result of inferring medically important information. This allows medical professionals to efficiently select appropriate diagnosis and treatment methods by checking the diagnostic support information.

[0062] In order for healthcare professionals to select appropriate diagnoses and treatment methods, not only words but also the information conveyed by the patient's emotions during dialogue between the patient and healthcare professional are important. Therefore, the pattern of changes in a patient's emotions and the content of the dialogue are medically important information, and analyzing this information can enable more appropriate diagnoses and treatments for individual patients. In other words, the information healthcare professionals need may be obtained through fluctuations in a patient's emotions, and analyzing these fluctuations may provide important insights for healthcare professionals to determine appropriate diagnoses and treatment methods.

[0063] Therefore, the processing unit 294 estimates the emotional change pattern of the user 20 based on at least one of the biometric data of the user 20 wearing the necklace-type terminal 14 output from the sensor 39 and the audio data captured from the conversation between the user 20 and a medical professional. Here, the emotional change pattern may be a value indicating a change in the absolute value of an emotional value that represents the intensity of an emotion, a value indicating a change in an index value that represents any of joy, anger, sadness, and pleasure, or a value indicating a trajectory of the amount of change in an emotional value that changes within a predetermined period of time. The emotional change pattern serves as an index for estimating changes in the emotion of the user 20, such as tension, relaxation, agitation, fatigue, excitement, relief, anxiety, anger, sadness, etc.

[0064] As an example, the processing unit 294 estimates the emotion of the user 20 based on at least one of the collected dialogue and biometric data using an emotion engine that estimates the emotion of the user 20, and estimates the time-series changes in the estimated emotion of the user 20 as an emotion change pattern. The processing unit 294 can capture subtle changes in emotion from "human speech and behavior," "changes in pulse rate," "sweating patterns," etc.

[0065] The processing unit 294 inputs the estimated emotion change pattern and a prompt including voice data during the dialogue between the user 20 and the healthcare professional into the data generation model 58, and acquires diagnostic assistance information based on the output of the data generation model 58. The data generation model 58 infers the input emotion change pattern of the user 20 and the voice data during the dialogue between the user 20 and the healthcare professional in accordance with the instructions given by the prompt, and outputs the diagnostic assistance information that is the inference result in a data format such as voice data and / or text data.

[0066] For example, the processing unit 294 may add the estimated emotion change pattern of the user 20 and the collected voice data to a prompt, generate a prompt saying, "This is voice data including the user's emotion change pattern and a conversation with a healthcare professional. Please extract medically important information, organize it so that it is easy to understand, and evaluate it," input the generated prompt to the data generation model 58, and acquire, as diagnostic support information, an evaluation result in which the medically important information has been organized and evaluated based on the output of the data generation model 58. Alternatively, the processing unit 294 may add the outputs of the sensor 39 and the microphone 38 to the prompt, and generate a prompt saying, "This is voice data including the user's biometric data and a conversation with a healthcare professional. Please extract medically important information, organize it so that it is easy to understand, and evaluate it."

[0067] The processing unit 294 may acquire, in the voice data during a conversation with a medical professional, an utterance such as when there is a change in the emotion of the user 20 who is a patient and wears the necklace-type terminal 14, or when the pattern of change in the emotion of the user 20 is greater than a preset threshold, as medically important information. Specifically, for example, when the user 20 who is a patient and wears the necklace-type terminal 14 utters "I can't sleep" during a medical interview with a medical professional, if there is a change in the emotion of the user 20 or if the pattern of change in emotion becomes greater than a preset threshold, the processing unit 294 acquires the utterance of the user 20 at that time, "I can't sleep," as medically important information.

[0068] Furthermore, the processing unit 294 may estimate changes in the emotions of the user 20 based on the speaking speed, intonation, tone of voice, etc. in the voice data collected by the microphone 38. Specifically, when the user 20 utters "My lower abdomen hurts" during a medical interview with a medical professional, if the tone of the user's voice increases, the processing unit 294 may acquire the utterance "My lower abdomen hurts" as medically important information.

[0069] In this embodiment, when the processing unit 294 acquires, as medically important information, a plurality of symptoms including, for example, "My lower abdomen hurts," it acquires, as diagnostic support information, one or more types of diseases that may cause the plurality of symptoms including, for example, "My lower abdomen hurts." Specifically, as diagnostic support information, it acquires, as an example, an inference result such as "Because there is pain in the right lower abdomen, there is a possibility of appendicitis, pancreatitis, etc." At this time, the processing unit 294 may acquire, for each disease, an evaluation result that evaluates the possibility of the disease. For example, the evaluation result can be displayed using labels such as 00% or large, medium, small, etc.

[0070] Furthermore, when a medical professional determines a diagnosis for a patient who is the user 20 from the output diagnostic support information, the processing unit 294 may generate a prompt such as "Please suggest some optimal treatment methods for △△ (△△ is the name of the disease)," input the generated prompt to the data generation model 58, and acquire multiple treatment methods as diagnostic support information based on the output of the data generation model 58. In this case, the processing unit 294 acquires multiple treatment methods via the data generation model 58 by referring to the patient's past medical records and the latest research results, etc., which are stored in advance in the data accumulation unit 57, for example.

[0071] Next, an example of the operation of the specific processing unit 290 in this embodiment will be described with reference to FIG.

[0072] Here, when the patient user 20 wears the necklace-type terminal 14, the data collection unit 100 sequentially collects the outputs of the microphone 38 and the sensor 39. The communication unit 102 sequentially transmits the outputs of the microphone 38 and the sensor 39 collected by the data collection unit 100 to the data processing device 12.

[0073] In step S401, the processing unit 294 receives, via the input unit 292, the biometric data collected by the sensor 39 and the voice data including the content of the conversation picked up by the microphone .

[0074] In step S402, the processing unit 294 estimates a pattern of change in the emotion of the user 20 based on at least one of the collected dialogue and the biometric data using the emotion engine.

[0075] In step S403, the processing unit 294 extracts medically important information from the emotional change pattern estimated from the biometric data of the user 20 and the voice data during the conversation between the user 20 and the medical staff.

[0076] In step S404, the processing unit 294 analyzes the extracted medically important information, organizes it into an easily understandable format for medical professionals, and evaluates it. Specifically, the processing unit 294 infers the specific type of disease, the evaluation result of the possibility of the disease, and the like.

[0077] In step S405, the processing unit 294 determines the inference result as the specific type of disease, the evaluation result assessing the possibility of the disease, etc., and outputs the inference result as one or more diagnostic support information to a pre-set output destination.

[0078] In step S406, the processing unit 294 accepts input from the medical professional via the input unit 292. As an example, the input unit 292 accepts input content via a keyboard (not shown) or a mouse (not shown). Note that if the medical professional is wearing the necklace-type terminal 14, speech picked up by the microphone 38 may be accepted. Also, if the medical professional is not wearing the necklace-type terminal 14, speech picked up by an external microphone (not shown) or the like may be accepted.

[0079] In step S407, the processing unit 294 determines whether the medical professional has decided on a diagnosis. Specifically, the processing unit 294 determines that the medical professional has decided on a diagnosis when, for example, the medical professional inputs a disease name via the input unit 292.

[0080] If a diagnosis is determined in step S407, in step S408, the processing unit 294 acquires a plurality of treatment methods for the disease diagnosed by the medical professional and outputs them to a preset output destination, thereby completing the identification process.

[0081] In this way, by checking the output diagnostic support information, medical professionals can save the trouble of processing the large amount of information obtained during patient interviews, allowing them to more efficiently select appropriate diagnostic and treatment methods.

[0082] When the diagnostic assistance information is output to a preset output destination by the process described above, a medical professional will check the output content and select a diagnosis result and treatment method for the user 20. However, if the determination accuracy of the data generation model 58 is not high, even if the type of disease, treatment method, etc. are output as diagnostic assistance information, the type of disease and treatment method that are actually output may not be optimal. Therefore, it is expected that the medical professional will actually update the data generation model 58 by feeding back the diagnosis result of the user 20 and the selected treatment method to the data generation model 58, thereby improving the determination accuracy.

[0083] The operation when updating the data generation model 58 will be described with reference to the flowchart of FIG.

[0084] When the processing unit 294 outputs diagnostic assistance information to a preset output destination, it receives a feedback result regarding the effectiveness of the diagnostic assistance information (step S501). Then, the processing unit 294 evaluates the determination accuracy of the data generation model 58 based on the received feedback result (step S502). If the determination accuracy is equal to or greater than a predetermined threshold (step S503; YES), the processing unit 294 ends the processing. On the other hand, if the determination accuracy is less than the predetermined threshold (step S503; NO), the processing unit 294 performs deep learning based on the evaluation result to update the data generation model 58 (step S504).

[0085] By carrying out the processing as described above, the data generation model 58 undergoes machine learning again based on the feedback results every time diagnostic assistance information is output, thereby improving the accuracy of judgment.

[0086] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[0087] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0088] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0089] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0090] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0091] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0092] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[0093] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0094] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0095] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0096] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0097] 10 Data Processing System 12 Data Processing Device 14 Necklace-type device 38 Microphone 39 Sensors 40 speakers 42 Camera 46A Control Unit 100 Data Collection Department 102 Communications Department 290 Special Processing Department 292 Input section 294 Processing Section 296 Output Section< / url:>

Claims

1. a sensor for detecting biometric data of the wearer; a microphone that picks up a conversation between the wearer and a medical professional and converts it into voice data; a collection unit that collects outputs from the sensors and the microphones; a communication unit that transmits the outputs of the sensor and the microphone collected by the collection unit to an external device; and a data processing device; and a data processing system comprising: The data processing device includes: an input unit that receives the biological data collected by the sensor and the audio data collected by the microphone; a processing unit that acquires and infers medically important information from the voice data based on a change in the wearer's emotion estimated from at least one of the biometric data and the voice data, and acquires the inference result as diagnostic support information; an output unit that outputs the diagnostic assistance information acquired by the processing unit to a preset output destination, Data processing system.

2. the processing unit inputs a prompt including the emotion change pattern of the wearer and the voice data into a data generation model, and acquires the diagnostic assistance information based on an output of the data generation model.

10. The data processing system of claim 1.

3. 3. The data processing system according to claim 1, wherein the processing unit estimates the wearer's emotion based on the biometric data using an emotion engine that estimates the wearer's emotion each time a captured conversation is obtained, and estimates a time-series change in the estimated wearer's emotion as a change pattern of the wearer's emotion.

4. The data processing system according to claim 1 , wherein the processing unit acquires, as the diagnostic assistance information, one or more types of diseases and an evaluation result for each of the diseases.

5. The data processing system according to claim 4 , wherein the processing unit acquires one or more treatment methods for each of the diseases as the diagnostic assistance information.

Images