HEALTH MANAGEMENT SYSTEM, HEALTH MANAGEMENT DEVICE, HEALTH MANAGEMENT PROGRAM, AND HEALTH MANAGEMENT METHOD

Abstract

Provided are a health management system, a health management device, a health management program, and a health management method, with which anyone, anywhere, can extremely easily measure and understand his or her own state of health, in a short period of time, non-invasively, and without informing another person.

Description

[Technical field]

[0001] The present invention relates to a health management system, a health management device, a health management program, and a health management method. [Background technology]

[0002] Since the World Health Organization (WHO) advocated healthy lifespan in 2000, there has been growing interest in not simply extending lifespan, but in extending the period during which people can live independently and healthily both physically and mentally.

[0003] In Japan, the difference between average life expectancy and healthy life expectancy (the period of time during which daily life is not restricted by health problems) was 8.84 years for men and 12.34 years for women in 2016 (see non-patent document 1).

[0004] Maintaining the health of citizens is the basis of individual happiness, and at the same time, it prevents a decline in productivity due to the worsening of illness, prevents an increase in the period during which medical expenses and nursing care costs are incurred, and curbs the increase in social security burdens.

[0005] On the other hand, except for special cases such as athletes, individuals often only realize the importance of health when they become ill. In other words, since people are not normally very health conscious, early detection can be difficult.

[0006] For example, the attendance rate for semi-mandatory company health checkups (including health checkups, medical examinations, and medical checkups) is 77%, while the attendance rate for self-employed individuals, family employees, homemakers, contract employees, etc. is around 50%. There are various reasons for not attending, such as not having the time, the cost, or not knowing that health checkups existed, but ultimately it can be said that this is due to low health literacy and an underestimation of the importance of preventing or detecting illness early.

[0007] On the other hand, there is a problem that mental and neurological disorders are rarely included in the test items even if people undergo health checkups. This is because there are no highly accurate biomarkers and expensive equipment is required to perform the tests. Furthermore, although there are questionnaires for depression such as the BDI and PHQ-9 that are commonly used worldwide, there are limitations to their objectivity due to the possibility of the subject's conscious concerns about the evaluation, unconscious defenses, and the subject's subjectivity.

[0008] In addition, people with mental or neurological disorders may not want others to know that they may have the illness, or may feel reluctant to visit a psychiatrist, so it is thought that they need to be able to manage their own health.

[0009] In recent years, technologies have been disclosed that estimate emotions and mental states by analyzing spoken voice (see Patent Documents 1 and 2), making it possible to measure and quantify a person's state by analyzing voice.

[0010] Furthermore, there are also situations where voice is actively used for purposes other than interpersonal communication, such as the disclosure of a technology that performs personal authentication using voiceprints and grants the right to access devices (see Patent Document 3) and a voice recognition technology that enables smart home appliances and other devices to be operated by voice (see Patent Document 4).

[0011] In addition, with the spread of smartphones, everyone carries a calling device, making it possible to speak whenever necessary.

[0012] Furthermore, if voice is recorded and stored as electronic data, it does not deteriorate like blood or urine, and therefore has the advantage that it can be analyzed retroactively at any time if necessary. [Prior art documents] [Patent documents]

[0013] [Patent Document 1] JP 2007-296169 A [Patent Document 2] International Publication No. 2006 / 132159 [Patent Document 3] US Publication No. 2016 / 0119338 [Patent Document 4] JP 2014-206642 A [Non-patent literature]

[0014] [Non-Patent Document 1] Ministry of Health, Labour and Welfare Science Research Grant FY2018 Research Report, pp. 26-39 “Study on the calculation and evaluation of nationwide trends in healthy life expectancy” Summary of the Invention [Problem to be solved by the invention]

[0015] In order to increase the rate of medical examinations that lead to the prevention and early detection of diseases, there is a need for tests that are easy to perform by oneself, inexpensive, and do not require the user to make time for them in their daily lives. Therefore, the object of the present invention is to provide a health management system, a health management device, a health management program, and a health management method that allow anyone, anywhere, to measure and understand their own health condition in a short time, non-invasively, and without the knowledge of others.

[0016] The inventors discovered that by using the voice as a biomarker indicating the state of the mind and body, it is possible to continuously manage one's health from a healthy state to contracting an illness, recovering, and then returning to a healthy state, and thus arrived at the present invention. [Means for solving the problem]

[0017] The health management system according to the present application is a health management system comprising a first server that continuously stores voice data acquired from a user, a second server that stores user identification data, a third server that stores output data for the user, and an estimation device that infers a predetermined disease using an acoustic feature extracted from the voice data as an input, the estimation device comprising an extraction unit that extracts an acoustic feature based on the voice data, an estimation unit that infers the predetermined disease using the acoustic feature as an input, and a selection unit that selects output data of the third server based on an estimation result of the estimation unit and the user identification data, and the selection unit: The system is characterized in that, when the estimation unit estimates that the user is not suffering from any disease, it is configured to select first output data from the third server for the user before the disease, and when the estimation unit estimates that the user is suspected of suffering from any disease, it is configured to select second output data from the third server for the user who is suspected of suffering from the disease, and after the user is diagnosed with any disease by a doctor's diagnosis, the identification data of the second server is updated, third output data from the third server for the user after the disease has been developed, and the third output data is continuously selected even after remission. Effect of the Invention

[0018] The present invention provides a health management system, a health management device, a health management program, and a health management method that allow anyone, anywhere, to measure and understand their own health condition in a short amount of time, non-invasively, and without others knowing. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a hardware configuration diagram showing an example of a computer that realizes the functions of a health management system 200 of the present invention. [Diagram 2] FIG. 2 is a configuration diagram of a health management system 200 according to an embodiment of the present application. [Figure 3A] FIG. 3A is a diagram showing a feature equation F(a) for calculating one or more acoustic features. [Figure 3B] FIG. 3B is an image showing that the intensity of a certain acoustic feature varies for each disease. [Figure 3C] FIG. 3C shows a distribution image of disease prediction values ​​obtained by combinations of three acoustic parameters. [Figure 4A] FIG. 4A is a graph showing an ROC curve indicating the performance of separating healthy subjects or subjects with a specific disease from others. [Figure 4B] FIG. 4B is a graph showing an ROC curve indicating the performance of separating healthy subjects or subjects with a specific disease from others. [Diagram 5] FIG. 5 shows example sentences read by each patient and healthy subject for use as learning data. [Figure 6] FIG. 6 is a diagram showing the results of estimating whether a user is healthy, belongs to the dementia group, or belongs to the mood disorder group. [Figure 7] FIG. 7 shows the estimation results of the estimation program on a phrase-by-phrase basis in the classification of major depression and healthy subjects. [Figure 8A] FIG. 8A shows the results of collating for each person the judgment that each phrase was answered as either major depression or healthy based on the estimation program. [Figure 8B] FIG. 8B shows the results of collating for each person the judgment that each phrase was answered as either major depression or healthy based on the estimation program. [Figure 9] FIG. 9 shows the results when the healthy judgment rate was calculated for each individual, and individuals with a healthy judgment rate of 60% or more were presumed to be healthy individuals, and individuals with a healthy judgment rate of less than 60% were presumed to be major depression patients. [Figure 10] FIG. 10 shows the estimation results of the estimation program on a phrase-by-phrase basis in classifying dementia with Lewy bodies from healthy subjects. [Figure 11A] FIG. 11A is a diagram showing the estimation results of Lewy body dementia. [Figure 11B] FIG. 11B is a diagram showing the estimation results of Lewy body dementia. [Figure 12]Figure 12 shows the results when the healthy determination rate was calculated for each individual, and those with a healthy determination rate of 60% or more were presumed to be healthy, and those with a healthy determination rate of less than 60% were determined to have Lewy body dementia. [Figure 13] FIG. 13 is a sequence diagram showing the operation of the health management system 200. [Figure 14] FIG. 14 is a sequence diagram showing the operation of the health management system 200. [Figure 15] FIG. 15 is a schematic diagram showing an overall view of the health management system 200. As shown in FIG. [Figure 16] FIG. 16 is a diagram showing an example of registering a user using a voice platform device or another system (transmission / reception, output of unique determination results) or an embedded system. [Figure 17] FIG. 17 is a diagram illustrating the flow of user registration. [Figure 18] FIG. 18 is a diagram for explaining the flow of storing audio data. [Figure 19] FIG. 19 is a diagram for explaining an example of a voice analysis of a user whose identification information has been registered. [Figure 20] FIG. 20 is a diagram for explaining an example regarding the determination. [Figure 21] FIG. 21 is a diagram for explaining an example related to action support. [Figure 22A] FIG. 22A is a diagram for explaining the flow of the system. [Figure 22B] FIG. 22B is a diagram for explaining the flow of the system. [Figure 23] FIG. 23 is a diagram explaining data transmission and reception, a voice platform, a voice analysis device, and an embedded system. [Figure 24] FIG. 24 is a diagram illustrating the audio platform device. [Diagram 25] FIG. 25 is a diagram illustrating a voice analysis device. [Figure 26] FIG. 26 is a diagram for explaining another system. [Figure 27A] FIG. 27A is a diagram for explaining another system. [Figure 27B] FIG. 27B is a diagram illustrating another system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, a detailed description will be given of a form for carrying out the disease inference system and disease inference method according to the present application (hereinafter, referred to as "embodiment") with reference to the drawings. Note that the disease inference system and disease inference method according to the present application are not limited to this embodiment. In addition, the same parts in each of the following embodiments are given the same reference numerals, and duplicated descriptions are omitted. [1. Program]

[0021] A health management system 200 according to an embodiment is realized by a computer 100 having a configuration as shown in Fig. 1, for example. An example will be described below. Fig. 1 is a hardware configuration diagram showing an example of a computer that realizes the functions of the health management system 200. The computer 100 has a CPU 101, a RAM 102, a ROM 103, a HDD 104, a communication interface (I / F) 105, an input / output interface (I / F) 106, and a media interface (I / F) 107.

[0022] The CPU 101 operates and controls each unit based on a program stored in the ROM 103 or the HDD 104. The ROM 103 stores a boot program executed by the CPU 101 when the computer 100 is started up, programs that depend on the hardware of the computer 100, and the like.

[0023] The HDD 104 stores programs executed by the CPU 101, data used by the programs, etc. The communication interface 105 receives data from other devices via the network N and sends it to the CPU 101, and transmits data generated by the CPU 101 to other devices.

[0024] The CPU 101 controls output devices such as a display and a printer, and audio input devices such as a microphone, and input devices such as a keyboard and a mouse, via the input / output interface 106. The CPU 101 acquires data from the input devices via the input / output interface 106. The CPU 101 also outputs generated data to the output devices via the input / output interface 106.

[0025] Media interface 107 reads a program or data stored in recording medium 108 and provides it to CPU 101 via RAM 102. CPU 101 loads the program from recording medium 108 onto RAM 102 via media interface 107, and executes the loaded program. Recording medium 108 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0026] For example, when the computer 100 functions as the health management system 200 according to the embodiment, the CPU 101 of the computer 100 executes a program loaded on the RAM 102 to realize the function of the control unit. In addition, the data in the storage unit is stored in the HDD 104. The CPU 101 of the computer 100 reads and executes these programs from the recording medium 108, but as another example, these programs may be obtained from another device. [2. Structure of the Health Management System]

[0027] Next, the configuration of a health management system 200 according to an embodiment will be described with reference to Fig. 2. As shown in Fig. 2, the health management system 200 is connected to a user terminal 201 and a web server 202 via a network N so as to be able to communicate with each other via wired or wireless communication. The health management system 200 may also be connected to a plurality of other user terminals 201 and a plurality of web servers 202. The web server 202 includes some of the functions of the health management system 200, and servers of health checkup facilities and medical facilities.

[0028] As shown in FIG. 2, the health management system 200 includes a communication unit 203, a first server 204, a second server 205, a third server 206, and a fourth server 210. Each of the first to fourth servers functions as a memory unit. Also, a part of the fourth server functions as a control unit. The communication unit 203 is realized by, for example, a NIC (Network Interface Card) or the like. The communication unit 203 is connected to the network N by wire or wirelessly, and transmits and receives information to and from the user terminal 201.

[0029] The storage unit is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.

[0030] The control unit is realized, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing various programs stored in the storage device of the health management system 200 using a RAM as a working area. The control unit is realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0031] As described above, the functions of the storage unit and the control unit are realized by the cooperation of the first to fourth servers in Fig. 2. Here, the first to fourth servers will be described.

[0032] The first server 204 has a function of continuously storing voice data acquired from the user. The second server 205 has a function of storing user identification data, and the user log is continuously updated through communication with the user terminal 201 and the web server 202.

[0033] The third server 206 includes the first to third distribution data (207 to 209). The third server 206 has a function of storing distribution data for users. The function of the third server may be substituted by an external web server 202 via a communication unit 203.

[0034] The fourth server 210 includes a calculation unit 211, an estimation unit 212, a selection unit 213, and a storage unit 214. The fourth server performs estimation processing to infer a predetermined disease using an acoustic feature calculated based on the voice data of the first server as an input, and also performs selection and distribution processing of distribution data stored in the third server, thereby functioning as an estimation device.

[0035] The first to fourth servers are assigned for the sake of convenience in order to explain the functions of the health management system 200, and these functions may be performed by part of the same server or by multiple servers.

[0036] The estimation program and distribution program stored in the fourth server 210 are executed using the RAM 102 in Fig. 1 as a working area, and thereby operate as a calculation unit 211, an estimation unit 212, and a distribution unit 213. The calculation unit 211, the estimation unit 212, and the distribution unit 213 function as a control unit in the fourth server. Meanwhile, the storage unit 214 stores the estimation program, the distribution program, and data of a second acoustic parameter described later.

[0037] Here, the calculation unit 211 calculates an acoustic feature based on the voice data. The estimation unit 212 estimates a predetermined disease using the acoustic feature calculated by the calculation unit 211 as an input. The selection unit 213 selects distribution data to be stored in the third server based on the estimation result of the estimation unit 212 and with reference to the identification data of the second server.

[0038] Here, the health management system 200 executes an estimation process for estimating the disease information of the user based on an estimation program and a distribution program stored in the fourth server 210, and a distribution process for selecting and distributing voice data, video data, or text data (hereinafter, voice data, video data, or text data will be simply referred to as "distribution data") stored in the third server 206 based on the estimated information. The distribution data can be any content, such as characters, figures, symbols, voice, video, games, or any other content, as long as it is capable of arousing the interest of the user and widely disseminating the information contained in the distribution data.

[0039] The distribution data stored in the third server 206 includes data including distribution content that prompts the user to input voice, and distribution data related to the user's health management, including a message that prompts the user to visit a medical institution or undergo a health check. The distribution data related to the user's health management is distribution data related to information for enhancing cognitive and social skills that determine an individual's motivation and ability to access, understand, and use information on how to improve and maintain better health, and includes, for example, information for enhancing health literacy. The distribution data may also be a numerical result of the estimation process of the fourth server.

[0040] In the third server 206 in Fig. 2, the first distribution data 207, the second distribution data 208, and the third distribution data 209 are conveniently classified and stored. The first distribution data 207 includes health information for users before they become ill. The second distribution data 208 includes health information including a message encouraging a user who is estimated by the estimation unit 212 of the fourth server 210 to be suspected of having any disease to visit a medical institution or undergo a health check. The third distribution data 209 includes health information for users after they become ill.

[0041] The first to third distribution data (207 to 209) will be described in more detail. The user to whom the first distribution data 207 is distributed is assumed to be a person who is relatively good at health care, disease prevention, and health promotion in daily life. Therefore, the first distribution data 207 includes information on health promotion such as information encouraging regular medical checkups and recreational information suited to the situation and place where the user terminal 201 is used. The frequency at which the first distribution data 207 is distributed is less than the frequency at which the second and third distribution data are distributed. For example, the first distribution data 207 may be distributed at the timing when the user inputs voice via the user terminal 201.

[0042] The user to whom the second distribution data 208 is distributed is assumed to be a person suspected of suffering from some mental or neurological disorder. Therefore, the second distribution data 208 includes information encouraging the user to visit a specific medical institution. However, there is a risk that the user will not admit that he or she is suspected of suffering from a mental or neurological disorder. In addition, even if the patient is aware of his or her condition, there is a risk that the patient will not receive medical care due to prejudice against mental illness and reluctance to visit a psychiatrist or psychosomatic clinic. Therefore, the second distribution data 208 may be configured to distribute distribution data for persistently persuading the patient to seek social services such as those of a medical institution more frequently than the first distribution data.

[0043] In addition, since mental and neurological diseases have common symptoms, it may be difficult for an individual to identify the disease. For example, it is difficult to distinguish between Alzheimer's dementia and frontotemporal dementia, Alzheimer's dementia and Lewy body dementia, Lewy body dementia and Parkinson's disease, and bipolar disorder and major depression. In addition, in the case of depression in the elderly, attention, concentration, judgment, and memory are reduced due to depressive symptoms such as depressed mood and inhibition of thinking, and there is pseudodementia that appears to be dementia at first glance, and it is necessary to distinguish whether it is a type of depression or a type of dementia in order to receive appropriate treatment. The present invention is advantageous in that it can provide distribution data by distinguishing medical institutions recommended according to the identified disease based on the disease estimated via the estimation unit 212 as in the present invention.

[0044] The user to whom the third distribution data 209 is distributed is assumed to be a person who has a history of suffering from some mental or nervous disease, or a person who is undergoing treatment for some mental or nervous disease. In general, it is difficult to completely cure a mental or nervous disease. Therefore, although it cannot be said that the disease is completely cured, it is common to continue taking medicine orally and undergo follow-up observation by a doctor to prevent recurrence even after the symptoms have disappeared after the condition has improved and entered a state of mild remission. Therefore, the third distribution data 209 includes a message, etc., that encourages confirmation of the symptoms of the disease by continuous voice input.

[0045] Even after the third distribution data 209 has been distributed to the user, if the estimation unit 212 subsequently estimates that there is a suspicion of recurrence of the disease or that the user is suspected of suffering from another mental or neurological disease, the second distribution data 208 is distributed.

[0046] In this manner, the selection unit 213 of the fourth server 210 selects and distributes the first to third distribution data in accordance with the estimation result of the estimation unit 212 and the health condition of the user.

[0047] In addition, after a doctor diagnoses that the user has any disease, the identification data of the second server 205 is updated from a server of the medical institution to which the doctor belongs via the web server 202, or by the user himself / herself via the user terminal 201. After the updating of the identification data is completed, the selection unit 213 is configured to select the second or third distribution data (208, 209) for the user.

[0048] Returning to the explanation of the health management system 200, the health management system 200 is an information processing device such as a web server that collects voice data of a user using a user terminal 201 used by the user as a voice input means, while performing data communication via the user terminal 201. For example, when the health management system 200 receives a distribution request for requesting data distribution from the user terminal 201, it selects target distribution data based on the disease estimation result based on the voice data of the user using the user terminal 201, and transmits the selected distribution data to the user terminal 201.

[0049] The user's identification data is stored in the second server 205, the log is continuously updated via the user terminal 201, and the medical examination institution or medical institution continuously updates the medical examination results and diagnosis results via the web server 202. The user's identification data includes health information having personal information including whether the user has ever suffered from a specific disease, and various information indicating the user's profile and background, such as the user's gender, age, address, hobbies, interests, behavioral history, and web page browsing history. The health information having personal information includes personal information including the user's medical examination results or the results of a doctor's diagnosis of a disease.

[0050] The user terminal 201 is a communication terminal such as a smartphone or tablet that has a means for inputting voice and a means for outputting distribution data, and can communicate with any server device via a wireless communication network such as 3G (Generation) or LTE (Long Term Evolution). The user terminal 201 may be an information processing device such as a desktop PC (Personal Computer), a notebook PC, a netbook, or a server device, as long as it has a means for inputting voice and a means for outputting distribution data.

[0051] The user terminal 201 may have some of its functions built into an electronic device having a means for inputting voice or a means for outputting distribution data. For example, a mobile device such as an automobile, a hardware device for operating a computer game such as a home game machine, or an amusement facility such as a karaoke booth or a running machine may have a function for inputting voice or a function for outputting distribution data.

[0052] The health management system 200 can display web content that prompts the user to input voice via the user terminal 201. The web content that prompts the user to input voice is distribution content stored in the third server 206, and includes text, image, video, animation, or audio playback. In such a case, the user terminal 201 transmits a web content distribution request to a web server that distributes the web content automatically via an application or by operation of the user. The distribution request may be automatically transmitted to the web server when the user starts an application on the user terminal 201.

[0053] The health management system 200 delivers web content that prompts the user to input voice to the user terminal 201. The user terminal 201 then displays the web content received from the health management system 200.

[0054] For example, when a user is playing a game via the user terminal 201, a character in the game may be configured to deliver a message via the application encouraging the user to input voice.

[0055] When a user inputs voice via the user terminal 201, the health management system 200 stores the voice data in the first server 204 of the distribution server 10. Then, when a distribution request for requesting distribution of data stored in the third server 206 is received, a disease estimation process is performed based on the voice data of the user using the user terminal 201, and further a process of selecting distribution data is performed, and the selected distribution data is transmitted to the terminal device 100. 3. Processing Executed by Health Management System 200

[0056] When distributing data stored in the third server 206, the health management system 200 selects the distribution data to be distributed from among the first to third distribution data (207 to 209) stored in the third server based on the estimation result of the estimation unit 212 of the fourth server 210 and the user identification data stored in the second server 205.

[0057] In other conventional technologies, the distribution data related to health management to be distributed is based only on the user's identification data stored in the second server 205. To give a more specific example, in the conventional technologies, if the user to be distributed is male, health information similar to the health information provided to many males is distributed.

[0058] However, there may be cases where there is no log of attribute information for a newly registered user. In addition, there may be cases where the attribute information of a user becomes unknown, such as when cookies or the like are deleted and an ID (identifier) ​​for identifying a user becomes unknown. In such cases, the conventional technology is unable to select appropriate distribution data to be distributed.

[0059] In addition, for users who do not receive regular health checkups or who do not visit a medical institution even when they develop a mental or neurological disorder, the delivery data that the health management system 200 should select is not desirable, and there is a risk that the information provided to improve health literacy may actually have a negative impact on the users.

[0060] As described above, in the conventional technology, when there is no user log stored in the second server 205 or when the user's attribute information is unknown, it is not possible to select appropriate distribution data. Even if a user develops some kind of mental or neurological disease and the abnormality can be detected from the user's behavior history obtained via the user terminal 201, if the predicted disease cannot be identified to some extent, there is a risk that incorrect health information will be selected and provided, causing adverse effects on the user.

[0061] Therefore, the health management system 200 calculates the acoustic feature of the user using a calculation unit 211 of the fourth server 210 that calculates the acoustic feature from the voice data stored in the first server 204. Then, the distribution server 10 executes an estimation process in which an estimation unit 212 of the fourth server estimates the disease of the user based on the calculated feature.

[0062] Furthermore, when a user whose attribute information is unknown inputs voice data, the health management system 200 calculates acoustic features of the voice data using the calculation unit 211 of the fourth server 210. The health management system 200 also executes an estimation process in which an estimation unit 212 of the fourth server estimates a disease of the user based on the calculated features. The health management system 200 then executes a distribution process in which a selection unit 213 selects distribution data to be distributed using the estimated disease information, and distributes the data. [4. Calculation and Estimation Sections]

[0063] Here, the calculation unit 211 and the estimation unit 212 used by the health management system 200 when estimating a disease will be described. [4-1. Construction of estimation model]

[0064] We will explain how the estimation model of the fourth server of the health management system is constructed. First, multiple acoustic parameters are extracted from the subject's voice data. Acoustic parameters are parameterized characteristics of sound transmission. For example, acoustic parameters include the zero-crossing rate and Hurst exponent. The zero-crossing rate is calculated as the number of times per unit time that the sound pressure waveform of the voice crosses the reference pressure, as a measure of the intensity of changes in the voice waveform. The Hurst exponent indicates the correlation of changes in the voice waveform.

[0065] The acoustic parameters are classified into first acoustic parameters and second acoustic parameters. Here, the first acoustic parameters are acoustic parameters extracted from the voice of a user whose specific disease is to be estimated. The first acoustic parameters are extracted based on the voice data stored in the first server 204.

[0066] The second acoustic parameters are acoustic parameters recorded in advance in a database. Data on the second acoustic parameters is stored in the storage unit 214 of the fourth server 210 or another server that substitutes for the function of the database. The second acoustic parameters are extracted from voice data of a patient with Alzheimer's disease, Lewy body dementia, Parkinson's disease, major depression, atypical depression, or bipolar disorder, and each acoustic parameter is associated in advance with each disease.

[0067] Specific examples of the acoustic parameters used in the present invention include the following items. 1) Volume envelope (attack time, decay time, sustain level, release time) 2) Waveform fluctuation information (Shimmer, Jitter) 3) Zero-crossing rate 4) Hurst exponent 5) VOT (Voice Onset Time) 6) Statistics of the intra-utterance distribution of certain coefficients of the Mel-frequency cepstrum (1st quartile, median, 3rd quartile, 95th percentile, arithmetic mean, geometric mean, difference between the 3rd quartile and the median, etc.) 7) Statistics of the intrautterance distribution of the rate of change of the frequency spectrum (1st quartile, median, 3rd quartile, 95th percentile, arithmetic mean, geometric mean, difference between the 3rd quartile and the median, etc.) 8) Within-utterance distribution statistics for the time variation of a certain coefficient of the Mel-frequency cepstrum (1st quartile, median, 3rd quartile, 95th percentile, arithmetic mean, geometric mean, difference between the 3rd quartile and the median, etc.) 9) Statistics on the intra-utterance distribution of the time variation of a certain coefficient of the Mel-frequency cepstrum (1st quartile, median, 3rd quartile, 95th percentile, arithmetic mean, geometric mean, difference between the 3rd quartile and the median, etc.) 10) Squared error for quadratic regression approximation of intra-utterance time variation of frequency spectrum with 90% roll-off 11) Arithmetic errors for quadratic regression approximation of intra-utterance time variation of frequency spectrum centroid Other parameters include pitch rate, probability of being voiced, power in a given range of frequencies, scale, speech rate (number of morae in a given period of time), pauses and intervals, and volume.

[0068] Such acoustic parameters are extracted for each speech phrase using acoustic parameter extraction software (e.g., openSMILE, etc.). In the present invention, 7440 acoustic parameters were extracted, acoustic parameters dependent on the recording environment were first removed, and one or more acoustic parameters suitable for expressing the health condition were selected from the remaining parameters.

[0069] The estimation program stored in the fourth server 210 has a learning function based on artificial intelligence and performs estimation processing using the learning function. Neural network-type deep learning may be used, or reinforcement learning that partially strengthens the learning field may be used, or other techniques such as genetic algorithms, cluster analysis, self-organizing maps, and ensemble learning may be used. Of course, other artificial intelligence techniques may be used. In ensemble learning, a classification algorithm may be created using a method that combines boosting and decision trees.

[0070] In the stage of creating the estimation program, the algorithm creator uses a stepwise method to select one or more of the optional acoustic parameters to be used as the variable f(n) from among the above-mentioned second acoustic parameter items, in order to find the best combination. Next, a coefficient is assigned to the selected optional acoustic parameter to create one or more acoustic features. Furthermore, these acoustic features are combined to create the acoustic feature F(a).

[0071] There are three types of stepwise methods: forward, backward, and incremental. Any of them may be used. The regression analysis used in the stepwise method includes linear classification such as linear discriminant and logistic regression analysis. The variables f(n) and their coefficients, i.e., the coefficients xn of the equation F(a) shown in Figure 3A, are called regression coefficients and are weights given to the function f(n).

[0072] After the creator of the learning algorithm selects the regression coefficients, the quality may be improved by machine learning to increase the estimation accuracy based on disease information accumulated in a database.

[0073] FIG. 3A shows a feature equation F(a) for calculating one or more acoustic features. A predicted value of the subject's disease can be calculated based on the equation F(a). Here, f(n) is any one or more second acoustic parameters arbitrarily selected from the above acoustic parameter items (1) to (11). xn is a regression coefficient specific to the disease. f(n) and xn may be stored in advance in the storage unit 214 of the fourth server 210 as part of the estimation program. The regression coefficient of the feature F(a) may be improved during the machine learning process of the estimation program.

[0074] 2 calculates a feature quantity for distinguishing between healthy subjects and subjects with a disease, or between diseases, based on the combination of the second acoustic parameters. From the feature quantity, a predicted value of the disease of the subject is calculated by performing scoring for calculating a reference range and how far the subject's value is from the reference range.

[0075] FIG. 3B is an image showing that the intensity of a certain acoustic feature varies for each disease. It shows that the subject had the highest score for disease A. Therefore, the predicted value for the subject's disease A is calculated to be higher than that of the other disease groups. In addition, by setting the intensity of 50 as the threshold, for example, the subject can be classified into a group of diseases A, D, and E, and a group of diseases B and C.

[0076] In Fig. 3B, the predicted value of the disease is calculated based on the intensity of one acoustic feature, but in reality, it is difficult to classify diseases using only one acoustic feature. Therefore, the disease may be classified by calculating the feature F(a) obtained by combining several acoustic features.

[0077] Based on the feature F(a), which is a combination of these acoustic features, we calculate the predicted disease value for the voice of the labeled subject and obtain the distribution of the predicted values ​​for each disease. This makes it possible to classify each disease.

[0078] FIG. 3C is a distribution image of disease predictive values ​​(denoted as "mental values" in FIG. 3C) obtained by combining three acoustic parameters.

[0079] From Figure 3C, it can be seen that the distribution of predicted values ​​for the Lewy body dementia patient group can be separated from the distribution of predicted values ​​for the patient groups with other diseases and the healthy control group. The present invention sets a combination of acoustic parameters for each disease so that it can be distinguished from other diseases, calculates a feature F(a), and uses the feature F(a), which is a combination of acoustic features, as an input to determine which disease the voice of each subject corresponds to.

[0080] Another method is to extract features F(a), which are a combination of acoustic features for each disease, from the voice of each patient, determine which disease features the patient has more of, and compare the predicted values ​​of the diseases to infer which disease the patient is suffering from.

[0081] In this case, the predicted value of a disease can be regarded as the degree of the disease. By comparing and recalculating the predicted values ​​of each disease, it is possible to express the probability of having each disease.

[0082] In this way, feature F(a), which is a combination of acoustic features associated with each disease, is extracted from the voices of patients with six diseases - Alzheimer's disease, dementia with Lewy bodies, Parkinson's disease, major depression, atypical depression, and bipolar disorder - as well as the voices of healthy individuals, and a predicted value for each disease is calculated.

[0083] Furthermore, a prediction program may be created from the voices of patients with 10 diseases, including four more diseases, namely vascular dementia, frontotemporal dementia, cyclothymia, and dysthymia, for the target diseases.

[0084] Finally, by analyzing the voice spoken by the subject, it is estimated whether the subject has one of the above-mentioned 6 to 10 diseases or is healthy.

[0085] Regarding the estimation flow of the estimation program, as described above, the feature (a) for each individual disease may be extracted and a disease prediction value may be calculated for each disease. Alternatively, a combination of acoustic features for a group of diseases may be created first, and the feature F(a) for the group of diseases may be used as an input to the estimation unit. Input and estimation may be performed in multiple stages to finally estimate each disease or healthy state. [4-2. Estimation accuracy]

[0086] 4A and 4B are graphs of ROC curves showing the separation performance between healthy subjects or specific diseases and others. The horizontal axis shows 1-specificity, and the vertical axis shows sensitivity. In other words, the horizontal axis shows the false positive rate, and the vertical axis shows the true positive rate. The ROC curves in both Figs. 4A and 4B show high true positive rates at low false positive rates.

[0087] In addition, the AUC (area under an ROC curve) was higher than 0.5 in all cases, and a significant difference from random classification was confirmed. The diseases for which the separation performance was verified were dementia with Lewy bodies, dementia of the Alzheimer's type, Parkinson's disease, major depression, bipolar disorder, and non-specific depression. The AUC in each ROC curve was 0.794 for dementia with Lewy bodies, 0.799 for dementia of the Alzheimer's type, 0.771 for Parkinson's disease, 0.869 for major depression, 0.86 for bipolar disorder, and 0.86 for non-specific depression. The diseases that can be predicted using the present invention are not limited to those mentioned above. The AUC results confirm that the accuracy of disease prediction is high.

[0088] The prediction program of the present invention can classify dementia with Lewy bodies as healthy or as having another disease.

[0089] The prediction program of the present invention can classify Alzheimer's disease as healthy or as having another disease.

[0090] The prediction program of the present invention can classify Parkinson's disease from healthy subjects or from other diseases.

[0091] The prediction program of the present invention can classify major depression as healthy or as having another disease.

[0092] The prediction program of the present invention can classify bipolar disorder from health or from other diseases.

[0093] The prediction program of the present invention can classify non-specific depression as healthy or as having another disease.

[0094] The prediction program of the present invention can classify a healthy state into the above-mentioned diseases.

[0095] The estimation program created as described above can be used without any restrictions by individuals suspected of having a psychiatric or neurological disorder, or individuals presumed to be healthy. It can also be used as a tool for doctors to examine the subjects, or as a test item for health checkups and medical examinations, simply by acquiring the subjects' voices.

[0096] The voice data analysis device of the present invention can be equipped with several different voice analysis programs.

[0097] For example, if a healthy person wants to know their daily health condition, MIMOSYS (registered trademark of PST Corporation) can tell them the level of vitality from their "vitality pressure" and "activity value." MIMOSYS is a product equipped with a voice analysis program that uses voice analysis to calculate the type and intensity of emotions expressed by the user and the level of vitality.

[0098] In addition, if the user is aware of a decline in their vitality, or if the MIMOSYS measurements indicate a trend toward a decline in vitality, it is preferable to use an analysis program that determines the possibility of a mental or neurological disorder.

[0099] Examples of such voice analysis programs include programs that estimate whether a person is healthy, has a possibility of having a dementia group disease, or has a possibility of having a mood disorder group disease.

[0100] Furthermore, if the user is considered to be highly likely to have some kind of psychiatric or neurological disorder, it is preferable to use a program that predicts the presence of any of the psychiatric or neurological disorders.

[0101] Here, specific examples of psychiatric and neurological disorders include dementia with Lewy bodies, Alzheimer's disease, and Parkinson's disease. In addition, vascular dementia can be included. In addition, mood disorders include major depression, bipolar disorder, and atypical depression. In addition, cyclothymia and dysthymia can be included.

[0102] In addition, the disease to be predicted is preferably a mental or neurological disease, but any disease that affects speech may be possible to predict, such as heart disease or respiratory disease. [5. Example of Estimation Program Creation]

[0103] In the above item 4, the performance as a classifier of the estimation program executed in the estimation unit 212 of the fourth server 210 has been sufficiently proven. In this item 5, an example of creating an estimation program will be introduced. [5-1. Example of Estimation Program Creation 1]

[0104] In this section 5-1, we will explain a prediction program for predicting whether a user is healthy, in the dementia group, or in the mood disorder group when diseases are classified into the dementia group, which includes dementia with Lewy bodies, Alzheimer's disease, and Parkinson's disease, and the mood disorder group, which includes major depression, bipolar disorder, and atypical depression. First, we will explain the procedure for creating the prediction program.

[0105] The speech samples used for training were 15 patients with Alzheimer's disease, 12 patients with bipolar disorder, 14 patients with atypical depression, 15 patients with Lewy body dementia, 15 patients with major depression, 15 patients with Parkinson's disease, and 15 healthy individuals.

[0106] The test data consisted of speech samples from five patients with Alzheimer's disease, four patients with bipolar disorder, five patients with atypical depression, five patients with dementia with Lewy bodies, five patients with major depression, five patients with Parkinson's disease, and five healthy subjects. The data was randomly divided into training data and test data.

[0107] The speech used for the training data was obtained by each patient and healthy subject speaking the 17 sentences shown in Figure 5 (numbers 1 to 13 spoken twice each, and numbers 14 to 17 spoken once each), resulting in 30 utterances from each person.

[0108] Characteristic acoustic parameters were extracted from each of these utterances, and the characteristics of the acoustic parameters, such as frequency and intensity, were correlated with the subject's disease, resulting in the six diseases and seven types of healthy subjects, as well as various associated acoustic parameter groups.

[0109] By selecting characteristic acoustic parameters for each disease and further combining the selected acoustic parameters, we created an estimation program 10 based on features to infer for each person's spoken sentence whether they have one of the six diseases or are healthy, and then to infer whether they have a dementia group, a mood disorder group, or are healthy.

[0110] As test data, eight phrases (16 in total, two for each phrase) were used from the 17 sentences in Figure 5: 2 "aiueokakikukeko", 3 "It's a fine day today", 6 "I'm very healthy", 8 "I have an appetite", 9 "I'm calm", 10 "I'm angry", 12 "Let's look up and walk", and 13 "Let's do our best". For each utterance, we estimated whether it was a disease or healthy. Among the estimated diseases, those that were estimated to be Alzheimer's disease, Lewy body dementia, and Parkinson's disease were determined to be diseases in the dementia group. In addition, those that were estimated to be major depression, bipolar disorder, and atypical depression were determined to be diseases in the mood disorder group.

[0111] Here, the 16 utterances from each person were tallied to see whether they were judged to be dementia disease, mood disorder disease, or healthy, and the one that received the most judgments was regarded as the final judgment result. (Result 1) Patients with Alzheimer's disease (patient code AD03):

[0112] Thirteen were judged to be dementia group, one was judged to be mood disorder group, and two were judged to be healthy. Therefore, they were ultimately estimated to be dementia group. This was consistent with the dementia group to which the actual disease Alzheimer's disease belongs.

[0113] The results of the similar assessment of 34 people are shown in Figure 6. The accuracy rate of the prediction program was 85.3%. This fully proves that the program is capable of predicting whether a user is healthy, in the dementia group, or in the mood disorder group, when classified into a dementia group, including dementia with Lewy bodies, Alzheimer's disease, and Parkinson's disease, and a mood disorder group, including major depression, bipolar disorder, and atypical depression. [5-2. Example of Estimation Program Creation 2]

[0114] In this section 5-2, as examples of creating a prediction program, a prediction program that classifies major depression as healthy or another disease, and a prediction program that classifies Lewy body dementia as healthy or another disease will be described. Note that a similar procedure can be used to create a program that classifies Alzheimer's disease, Parkinson's disease, bipolar disorder, or atypical depression as healthy or a disease other than the selected disease, but this will be omitted because it is obvious from the results of section 4 above. [5-2-1. Example of creating a prediction program for major depression]

[0115] First, we will explain the procedure for creating the estimation program (5-2-1A) for classifying major depression and healthy subjects.

[0116] We extracted 7,440 acoustic features using openSMILE, a software that extracts acoustic features, for each phrase spoken by 20 healthy subjects and 20 patients with major depression (approximately 30 utterances from each subject). We excluded features that showed significant differences depending on the facility where the speech was acquired, and selected 1,547 features. We then used a method that combined boosting and decision trees to generate disease prediction values ​​from the features to distinguish between healthy subjects and patients with major depression, and created a classification algorithm. The estimation results for each phrase are shown in Figure 7A.

[0117] As shown in Figure 7A, the prediction program for classifying major depression from healthy controls had a sensitivity of 81.4%, a specificity of 85.5%, and an accuracy rate of 83.5% in determining whether major depression was present.

[0118] The results of determining whether each phrase was major depression or healthy for each person based on the prediction program are shown in Figures 8A and 8B.

[0119] Furthermore, the healthy judgment rate for each person was calculated, and those with a healthy judgment rate of 60% or more were estimated to be healthy, and those with a healthy judgment rate of less than 60% were estimated to be major depression patients. The results are shown in Figure 9. All 20 healthy people were estimated to be healthy, and all 20 major depression patients were estimated to be major depression patients. In other words, the specificity, sensitivity, and accuracy rate were all 100%.

[0120] Next, we will explain the procedure for creating the estimation program 5-2-1B, which classifies major depression and other psychiatric and neurological disorders.

[0121] The prediction program 5-2-1B extracts and predicts major depression among psychiatric and neurological disorders that at least include Alzheimer's disease, Lewy body dementia, Parkinson's disease, major depression, atypical depression, and bipolar disorder, which have similar symptoms.

[0122] The types of utterances used in the analysis were the 17 types of utterances (phrases) shown in Figure 5.

[0123] First, 7,440 acoustic features were extracted for each spoken phrase using openSMILE, a software that extracts acoustic features. Data with significant differences depending on the facility where the voice was acquired were excluded, and a classification algorithm was created using the features of 1,547 pieces of data by a method that combined boosting and decision trees, and a disease prediction value was generated to distinguish between major depression and other diseases.

[0124] In addition, 2,236 speech phrases from the five patients other than major depression were used, while 940 speech phrases from patients with major depression were used. In order to avoid an imbalance in the amount of data, pseudo-data of the speech of patients with major depression was created using the Synthetic Minority Over-sampling Technique (SMOTE), resulting in 3,139 speech phrases from patients with major depression.

[0125] The data was divided into 10 parts, and 10-fold cross-validation of the classification algorithm of the estimation program 5-2-1B was performed.

[0126] The results of the estimation based on the 5-2-1B estimation program are shown in Figure 7B. When using the 5-2-1B estimation program to distinguish between major depression and other psychiatric and neurological disorders, the recall rate for major depression was 91.1%, the recall rate for other psychiatric and neurological disorders was 93.8%, and the accuracy rate was 92.7%. The AUC was 0.977. [5-2-2. Example of creating a prediction program for Lewy body dementia]

[0127] First, we will explain the procedure for creating a prediction program for classifying Lewy body dementia and healthy subjects.

[0128] The procedure for creating the estimation program 5-2-2A was the same as that for the estimation program in item 5-2-1 above, except that the speech phrases of 20 healthy subjects and 20 patients with dementia with Lewy bodies were used as the voices. The estimation results for each phrase are shown in Figure 10A.

[0129] The prediction program 5-2-2A had a sensitivity of 81.5%, a specificity of 83.1%, and an accuracy rate of 82.2% in determining whether or not someone had Lewy body dementia.

[0130] Based on the estimation program 5-2-2A, the results of estimating whether each phrase was dementia with Lewy bodies or healthy for each person are shown in Figures 11A and 11B.

[0131] Furthermore, the healthy judgment rate for each person was calculated, and those with a healthy judgment rate of 60% or more were estimated to be healthy, and those with a healthy judgment rate of less than 60% were estimated to be patients with dementia with Lewy bodies. The results are shown in Figure 12. All 20 healthy people were estimated to be healthy, and of the 20 patients with dementia with Lewy bodies, 19 were estimated to have dementia with Lewy bodies, and 1 was estimated to be a healthy person. In other words, the specificity was 100%, the sensitivity was 95%, and the accuracy rate was 97.5%.

[0132] Next, we explain the procedure for creating a prediction program (5-2-2B) for classifying Lewy body dementia from other psychiatric and neurological disorders.

[0133] The inference program 5-2-2B extracts and infers dementia with Lewy bodies from among psychiatric and neurological disorders that at least include Alzheimer's disease, dementia with Lewy bodies, Parkinson's disease, major depression, atypical depression, and bipolar disorder, which have similar symptoms. The types of utterances used for analysis were 17 types of utterances (phrases), the same as in the inference program 1.

[0134] First, 7,440 acoustic features were extracted for each spoken phrase using openSMILE, a software that extracts acoustic features. Features that showed significant differences depending on the facility where the speech was acquired were excluded, and a classification algorithm was created using 1,547 features by combining boosting and decision trees, generating disease prediction values ​​for distinguishing between Lewy body dementia and other diseases.

[0135] In addition, 2,987 speech phrases from the five patients other than Lewy body dementia were used, while 583 speech phrases from patients with Lewy body dementia were used.In order to avoid an imbalance in the amount of data, pseudo-data of the speech of patients with Lewy body dementia was created using the Synthetic Minority Over-sampling Technique (SMOTE), resulting in 2,696 speech phrases from patients with Lewy body dementia.

[0136] The data was divided into 10 parts, and 10-fold cross-validation was performed on the classification algorithm of the estimation program 5-2-2B.

[0137] The results of the estimation based on the 5-2-2B estimation program are shown in Figure 10B. When using the 5-2-2B estimation program to distinguish between dementia with Lewy bodies and other psychiatric and neurological disorders, the recall rate for dementia with Lewy bodies was 88.6%, the recall rate for other psychiatric and neurological disorders was 89.8%, and the accuracy rate was 89.2%. The AUC was 0.959.

[0138] By using a procedure similar to that described above, it is possible to combine a prediction program that classifies a specific disease from the other five diseases with a prediction program that classifies the specific disease from healthy individuals, thereby making it possible to predict with a high probability whether a person has Alzheimer's disease, dementia with Lewy bodies, Parkinson's disease, major depression, atypical depression, or bipolar disorder, or whether they are a healthy individual. 6. Operation of Health Management System 200

[0139] 13 and 14 are sequence diagrams showing the operation of the health management system 200. Each will be explained in turn below.

[0140] FIG. 13 is a sequence diagram showing the operation of the health management system 200 when the user has not yet visited a medical institution or when the user has gone into remission after visiting a medical institution once.

[0141] First, in step S1001, the user's identification information or a delivery request input from the user terminal 201 is transmitted to the second server 205 via the network N and the communication unit 203. A user who has achieved remission from a psychiatric or neurological disorder may include this fact in the identification information.

[0142] Next, in step S1002, the second server transmits to the third server 206 a message notifying that a distribution request has been made from the user terminal 201.

[0143] Next, in step S1003, when third server 206 receives the distribution request, it transmits distribution data to user terminal 201. This distribution data includes a message prompting voice input. The distribution data is transmitted to user terminal 201 via communication unit 203 and network N, and output by a means for outputting distribution data.

[0144] Next, in step S 1004 , the voice data input by the user via the voice input means of the user terminal 201 is transmitted to the first server 204 .

[0145] Next, in step S1005, the first server stores a copy of the received voice data and transmits the voice data to the fourth server.

[0146] Next, in step S1006, the fourth server pre-processes the received voice data to calculate a first acoustic feature, and calculates a feature F(a) in the calculation unit 211 based on the second acoustic feature previously stored in the storage unit 214 and an estimation program. The estimation unit 212 performs a process of estimating a psychiatric / neurological disorder based on the feature F(a) calculated in the calculation unit 211.

[0147] Here, when the estimation unit 212 estimates that the user does not suffer from any psychiatric or neurological disorder, the first distribution data 207 of the third server 206 is distributed via the selection unit 213. On the other hand, when the estimation unit 212 estimates that the user is suspected of suffering from any psychiatric or neurological disorder, the second distribution data 208 of the third server 206 is distributed via the selection unit 213.

[0148] In addition, if the identification information of a user registered in the second server 205 includes a history of suffering from a psychiatric or neurological disorder but a record indicating that the user is already in remission, and if the estimation unit 212 estimates that the user is not suffering from any psychiatric or neurological disorder, the third distribution data 209 of the third server 206 is distributed via the selection unit 213.

[0149] Next, in step S1007, third server 206 transmits predetermined distribution data to user terminal 201. User terminal 201 outputs the received distribution data from an output means, and ends the process.

[0150] 13, the distribution data distributed from the third server 206 is changed depending on how the identification information of the user registered in the second server 205 and the voice data of the user are estimated by the estimation unit 212 of the fourth server 210. This is useful in that health information suited to the individual health condition of the user can be distributed as distribution data. In addition, by repeatedly executing a part or the whole of the steps shown in FIG. 13, it is possible to check the change in condition in chronological order.

[0151] Next, we move on to the explanation of Fig. 14. Fig. 14 is a sequence diagram after a user is diagnosed by a doctor, and includes mutual communication between the health management system 200 and a server of a medical institution in addition to the functions of the health management system 200 in Fig. 13. Note that in Fig. 14, the steps will be explained on the assumption that the user has been diagnosed with some kind of mental or neurological disorder.

[0152] First, in step S2001, the medical institution server 202M, which is a type of the web server 202, accesses the second server 205 via the network N and the communication unit 203, and updates the user's identification information.

[0153] Next, in step S2002, the second server 205 transmits a message to the third server 206 indicating that the user's identification information has been updated.

[0154] Next, in step S2003, when the medical institution server 202M requests the user to input voice, or when the user terminal accesses the health management system 200, the third server 206 sends a distribution message to the user terminal 201 prompting the user to input voice.

[0155] Next, in step S 2004 , the user's voice data input via the voice input means is transmitted to the first server 204 .

[0156] Next, in step S2005, the first server 204 copies and stores the voice data, and also transmits the voice data to the fourth server 210.

[0157] Next, in step S2006, the fourth server 210 pre-processes the received voice data to calculate a first acoustic feature, and calculates a feature F(a) in the calculation unit 211 based on the second acoustic feature previously stored in the storage unit 214 and an estimation program. The estimation unit 212 performs a process of estimating a psychiatric / neurological disorder based on the feature F(a) calculated in the calculation unit 211.

[0158] Here, the processes of steps S2001 to S2006 may be repeated. For example, in FIG. 14, the user has already been diagnosed with some kind of mental or neurological disease, but even after the diagnosis, the user can continue to input the prognosis as voice data periodically and the estimation process can be continued in the fourth server. The voice data in the first server and the results of the estimation process in the fourth server are stored each time. The stored data can be used as a basis for determining the prognosis in the diagnosis at the next visit to the hospital.

[0159] In addition, after the diagnosis, it is possible to check the voice data of the first server before the diagnosis and the result of the estimation process of the fourth server acquired in the step of Fig. 13. This can increase the accuracy of the diagnosis.

[0160] Next, in step S2007, the third server 206 selects distribution data via the selection unit based on the result of the estimation process of the fourth server 210, and transmits the distribution data to the prognostic user. The second or third distribution data is selected as the distribution data.

[0161] In addition, the second or third distribution data may contain question data for the user, including a medical questionnaire provided by a doctor. In that case, proceed to step S2007. The operations from step S2007 onwards may be repeatedly executed independently of the operations from step S2001 to step S2006.

[0162] Next, in step S2007, the medical questionnaire is transmitted to user terminal 201. Next, when the user completes inputting the medical questionnaire from user terminal 201, in step S2008, the input data is transmitted to second server 205 and stored.

[0163] Next, in step S2009, the input data is transmitted to the medical institution server 202M. The operation of step S2009 may be combined with the operation of step S2008 performed by the second server 205. The medical institution server 202M outputs the input data from an output means and ends the process.

[0164] 13 or 14 as described above, the user can use the health management system 200 as a personal health management tool before the onset of the illness. After the onset of a psychiatric or neurological illness, the user can use the health management system 200 as a diagnostic support tool for doctors. After a user has fully recovered or is in remission, the user can use the health management system 200 as a personal health management tool, as a monitoring tool by a doctor as necessary.

[0165] So far, an example of the operation of the health management system 200 has been introduced, but the present invention is not limited to this. In addition, in Fig. 14, the medical institution server 202M is taken up as an example of using the combination of the web server 202 and the health management system 200, but instead of a medical institution using this system, a market institution may use the health management system 200 including the medical institution server 202M for the purpose of conducting a consumer trend survey. 7. Features of the Present Invention

[0166] The health management system 200 can display the output every time an utterance is acquired, and can grasp the health condition each time, but it can also acquire voice continuously, compile and analyze it, and view changes over time, as shown in Figures 13 and 14. For example, it is possible to view the history of a person's health periodically, such as once a month or once every three months, or whenever a person experiences some kind of health problem.

[0167] As a result, the health management system has the following advantages over conventional systems:

[0168] (1) By looking at changes over time, you can see trends in your health condition, and if there is a trend toward illness, you can take preventive measures.

[0169] (2) By continuously recording audio, for example, even once a week, it is possible to grasp the weekly health status. This is much more frequent than a health check once or twice a year, so that the timing for prevention and early detection is not missed.

[0170] (3) In the case of mental illnesses such as depression, various factors such as genetic factors, personal qualities (stress tolerance, resilience, etc.), the physical environment of the residential area, and close human relationships such as family and colleagues at work overlap to cause the illness to develop. If the cause is not removed, there is a high risk of recurrence even if the illness is cured. Therefore, by analyzing the illness over time, it is possible to determine the timing at which the health condition deteriorates, and by considering the life events at that time, it is possible to identify the main cause. This makes it easier to remove the cause, making it less likely for the illness to recur after recovery or remission.

[0171] (4) As dementia diseases such as Alzheimer's progress, the patient's awareness of the disease decreases and they tend to stop acknowledging that they have the disease. This makes it more difficult to receive treatment and the disease progresses more quickly, and increases the mental burden on family and others around the patient. Since anyone is more likely to develop dementia as they age, if the patient is aware of the risk of dementia progressing while the disease is still mild, they can actively receive training to prevent the progression of dementia and prepare for the future. It also allows the family to prepare for the future and make it easier to deal with the disease. [8. Audio capture scene]

[0172] 2, it has been explained that the user terminal 201 includes a communication terminal such as a smartphone or tablet having a means for inputting voice and a means for outputting distribution data. Here, other examples of the means for inputting voice will be introduced.

[0173] (1) When the user is semi-forced to speak and audio is obtained - Confirm that you have clocked in by voice instead of using a time card. When teleworking, certain utterances are made from home etc. as a signal to start work. The same applies when work ends. - Used as a pre-work examination (to understand health status) for drivers, pilots, etc. in the transport industry. During follow-up of treatment, patients can record audio when instructed by their doctor. For example, audio can be recorded when taking medicine after each meal.

[0174] (2) When a user is in a situation where he or she normally speaks and that voice is to be collected Capture audio of call center operator conversations. Acquire audio recordings of customer service conversations at convenience stores, fast food restaurants, and other retail establishments. Acquiring audio of announcers, voice actors, presenters, station staff, etc. speaking in the course of business. When smart home appliances are operated by voice, the voice is acquired. -Captures speech when chatting within a team in online games, etc. When playing smartphone app games, set it so that spells are cast using actual audio. -Capture audio from radio conversations used by police officers, security companies, taxis, forestry workers, etc. -Capture the audio of medical and caregivers speaking to patients, etc. - Acquire audio when introducing products at the counters of insurance or financial stores.

[0175] (3) When obtaining speech that a user voluntarily speaks in order to obtain benefits by providing incentives to the user In smartphone games, set it up so that you can obtain items that can be used in the game by making sounds. When accessing a coupon distribution site, set it up so that you can receive more preferential coupons than usual by speaking.

[0176] (4) When a situation arises where a user is given the opportunity to speak by accessing the user from another person Daycare staff, security companies, etc. will periodically call elderly people living alone to check on their health and record the audio of the responses they make. The conversational robot periodically speaks to a user who owns it as a pet, and the voice response is recorded. If the user is hospitalized, their body temperature, pulse, and blood pressure will be measured and their speech will be recorded.

[0177] (5) Suggest speaking to the user in places the user regularly visits and capture audio. For health- and beauty-conscious customers at gyms, yoga salons, swimming schools, beauty salons, dance studios, etc., audio recording will be performed as part of health management, similar to blood pressure and other measurements.

[0178] (6) When the user voluntarily and regularly records speech for the purpose of managing his or her own health. By setting up a variety of voice capture situations, it is possible to create an environment in which voice capture can be continued. For each person, even if the voice capture situation changes, voice can be continuously captured as long as any of the above conditions are met. [9. An example of distribution data]

[0179] An example of the data distributed from the third server 206 will be introduced. [9-1. Output when healthy]

[0180] In the health management system 200, when the patient is healthy, the voice analysis engine is used in a non-medical field, so it is considered that the engine for determining the risk of illness and mental and physical conditions is mainly used, rather than an engine for determining illness. Specifically, for example, a voice determination engine that classifies the patient into three categories: "healthy," "presenting depressive symptoms," and "presenting cognitive decline."

[0181] It is also possible to combine an engine that determines whether a person is "healthy" or "possibly ill" with an engine that determines whether the possible illness is "presenting depressive symptoms" or "presenting cognitive decline." 9-2. Examples of quantification by the calculation unit 211 or the estimation unit 212 and examples of conversion into language

[0182] The captured voice is analyzed and classified into three categories: "healthy," "exhibiting depressive symptoms," and "exhibiting cognitive decline." The confidence level for which category the voice belongs is calculated and quantified so that the sum of these values ​​equals 1.

[0183] In this case, if the value representing "health" is the "H value," the confidence level of a depressed state is the "M value," and the confidence level of a dementia state is the "C value," when analyzing the voice of a certain user, the analysis result by the analysis device (engine) will be, for example, H value 0, M value 0.75, C value 0.25, H value 0.5, M value 0.25, C value 0.25, etc.

[0184] The numerical values ​​obtained by the analysis device may be output as they are, but in order to make them easier for the user to understand, they may be converted into some kind of words and then displayed.

[0185] For example, if both the M and C values ​​are 0.3 or less, the system judges the user to be healthy and outputs the message, "Your body and mind are in a healthy state." If the M or C value is between 0.3 and 0.5, the system suggests trying a program to maintain, strengthen, or improve your physical and mental health.

[0186] For example, examples of phrases that can be converted into language include "Exercise is a great way to relieve stress. How about going to the gym or dancing?", "If you have something worrying, try using mindfulness to relieve it", "Massages, beauty treatments, exfoliation, and other activities that involve contact with the skin are soothing", "Let's go on a trip to change your mood!", and "If you let out your childlike inner self, you will be able to make the most of your natural talents. How about a party or a disco?"

[0187] If the M or C value exceeds 0.5, feedback is given to the user for the classification exceeding 0.5. For example, if the M value exceeds 0.5, feedback is given that can trigger awareness, such as "Are you feeling weak mentally?", "Are you under a lot of stress?", "It seems like you're trying too hard", "Are you having trouble with interpersonal relationships?", and "It's important not to push yourself too hard." [9-3. If you suspect you may be ill]

[0188] Furthermore, if the M or C value exceeds 0.5, it can be subjected to further analysis to determine the presence or absence of disease, and further feedback can be provided based on the numerical value.

[0189] For example, if the certainty level indicated by the analysis device that determines the illness is 0.7 or higher, and the illness is one in which depressive symptoms are the main symptom, such as major depression, bipolar disorder, or atypical depression, the device can add feedback such as, "We recommend that you visit a mental clinic or a psychosomatic medicine specialist."Furthermore, if the certainty level is 0.7 or higher and the illness is one in which cognitive or motor decline due to brain dysfunction is the main symptom, such as Alzheimer's disease, Lewy body dementia, or Parkinson's disease, the device can add feedback such as, "We recommend that you visit a neurologist." [9-4. When used by doctors]

[0190] When used by doctors, the engine is used as a regular diagnostic device to determine illness. By also using an engine that determines the severity of the patient's physical and mental condition or the severity of the illness, treatment progress can be monitored. In addition, by enabling data to be acquired over time through voice input from the patient, the effort required for doctors and nurses to visit patients' homes in person can be reduced, which can lead to reduced medical costs. [10. Overall System Description]

[0191] 15 is a schematic diagram showing an overall image of the health management system 200. The main system flow will be explained using steps S3001 to S3012.

[0192] First, in step S3001, voice input from the user is performed in various usage scenarios. In addition to voice input from the user terminal 201, the various usage scenarios include the scenarios listed in the section "8. Situations in which voice is acquired." In the health management system 200 having first to fourth servers, the input voice data is stored mainly in the first server 204. At the same time, the identification information input by the user via the user terminal 201 is stored in the second server 205 of the health management system 200.

[0193] Next, based on the input voice data, the fourth server 210 performs voice analysis, measurement, and estimation processing, and in step S3002, the estimation result is transmitted to the user terminal 201. The estimation result may also be transmitted to the non-medical service institution 202A via the web server 202 (S3003). In this case, in step S3004, the estimation result is transmitted from the non-medical service institution 202A to the third server 206. Note that in FIG. 15, some of the functions of the third server 206 are described separately from the health management system 200. Then, in step S3005, a message recommending a visit to a medical institution is transmitted from the third server 206 to the user terminal 201.

[0194] Furthermore, in steps S3006 and S3007, management information relating to the user's identification information is transmitted and received between the user terminal 201 and the health management system 200.

[0195] The user then undergoes a medical interview by visiting the facility, etc.

[0196] Next, in step S3008, the health management system 200 can transmit the voice data stored in the first server 204 and the estimated result data stored in the fourth server 210 to a facility. The facility can use the data during interview, examination, and diagnosis.

[0197] After the examination and diagnosis, the facility provides the diagnosis results to the health management system 200 (step S3009). During treatment and during the prognosis period, the health management system 200 transmits and receives the voice data stored in the first server 204 and the estimated result data stored in the fourth server 210 (steps S3010 and S3011), which can be used for treatment. Even after the patient goes into remission, the progress can be monitored by periodically acquiring the user's voice data.

[0198] In step S3012, the voice data and the time-series data of the estimation result can be fed back and used in various usage scenarios. [11. Other Commentary]

[0199] Other aspects of the present invention will be explained below. Note that the present invention is not limited to the following explanation.

[0200] FIG. 16 shows an example of registering a user using a voice platform device or another system (transmission / reception, output of unique determination results) or an embedded system.

[0201] The process for using this device is as follows: (1) input the voice and other data of the registered user and store it in the user data storage device; (2) select the stored voice and other associated data; (3) Select the service you wish to use, (4) The voice data is analyzed using a voice data analysis device used for the service, and the results are stored in a user data storage device. (5) Based on the analysis results, search for the discrimination results from the discrimination master prepared in advance based on other data, (6) Transmit the results of the discrimination and any analysis results or other data required by the service. (7) The service provider or system receives and provides to the user output or services that are useful for behavioral support.

[0202] For example, currently, sports gyms and diet programs provide services aimed at improving physical health. At these health promotion facilities, goals are set and action plans are drawn up through services from activity trainers and other professionals. On the other hand, some organizations, particularly athletes, aim to achieve effective goals by improving not only the physical aspects but also the mental aspects. By using an audio platform device or a system with the functions of an audio platform device built in, it is possible to encourage the use of information that is effective in improving the mental aspects.

[0203] FIG. 17 explains the flow of user registration. The user checks the explanation of data usage from the consent screen for starting use, and if the user agrees to the contents, presses the start use button. After pressing the start use button, the initial registration screen is displayed. The user enters the necessary information (age, sex, medical history, etc.) displayed on the initial registration screen and presses the input completion button. When the input completion button is pressed, if a code that identifies the device used for input can be obtained, it is entered as the terminal identification number mainly used. After pressing the input completion button, the user selects "free speech" or "free speech" displayed on the screen, and inputs voice following one or both of the input guides. When it is confirmed that the voice required for voiceprint extraction has been input, a screen indicating completion of voice input is displayed, and the input data is sent to the voice platform device as confirmed information by pressing the confirmation button.

[0204] For example, when a user wishes to use a voice platform device or a system incorporating the functions of a voice platform device to formulate a training plan that also includes mental aspects, the user registers. At this time, voice is acquired as user information.

[0205] FIG. 18 explains the flow of voice data storage. A user registers for the first time or inputs voice data at any time directly to the voice platform device, or inputs voice acquired by another system via the voice platform device. After completing the confirmation at the time of input, the voice data is sent from the state temporarily stored in the voice platform device to the data storage device. The transmitted voice and associated data are received by the data storage device constituting the voice platform device and stored in the database of the data storage device together with the voice acquisition date and time, the terminal used, and data associated with the use. The stored voice and other data cannot be searched, viewed, output, or deleted by anyone other than the user himself or the service or service provider approved by the user. Data sets that are difficult to uniquely identify individuals as stated in the consent at the time of registration can be permitted to be viewed and messages sent and received by businesses that have obtained permission to use the voice platform device from businesses that provide services using the voice platform device. When a user requests the voice platform operator to suspend or terminate use in a predetermined procedure and the process is completed, the data that can identify an individual can be erased (logically or physically) based on consent, or if the voice platform operator determines that the user has violated the terms of use. Data that cannot uniquely identify individuals may continue to be used by voice platform operators as statistical information for those operators.

[0206] The data stored on this platform includes sensitive personal information, and the data storage environment will include measures to protect personal information, such as encryption and data distribution.

[0207] For example, voice data is input as part of an activity record when applying for use at a sports gym, when making a plan, during the execution of the plan, and when the plan is completed. If the voice platform device is used with a mobile terminal device such as a smartphone, it is possible to save and utilize voices captured by the user's own device in addition to those captured by the microphone at the sports gym.

[0208] Examples of audio acquisition methods include the following:

[0209] (1) When users receive health promotion services, voice input is used to create activity plans aimed at improving not only their physical condition but also their mental health.

[0210] (2) When a user wishes to store or utilize voice or other data, the user inputs the data using the functions of the voice platform device, or using other apps, systems, etc. that utilize the functions of the voice platform device, or via wearable devices, etc. to which they are connected.

[0211] (3) With the user's consent, employers etc. obtain the user's voice in advance during business activities (such as call center response records, communication radio communications in the transportation industry, or singers recording or performing live, or actors or voice actors performing, or medical, nursing care, or health promotion services providing interviews, nursing care, greetings, or counseling, or selling insurance products, or when serving customers at a retail store, etc.), convert it into a form that can be handled by the voice platform device, and input it using the functions of the voice platform device.

[0212] (4) When businesses provide or implement incentives for the purpose of improving marketing or customer loyalty or obtaining information, or when using a system in an application, etc. to purchase cards or virtual items to be used in a game by lottery (so-called gacha), or when using the functions of a device that is mainly operated by voice, such as a smart speaker, or when operating a device with a voice input device such as a smartphone, or when operating an application on a device such as a computer or smartphone,

[0213] (5) Using voice for amusements (such as karaoke) or provision of services that involve voice input, or inputting voice obtained in connection with the explanation or provision of services (such as voice obtained as part of a health check or for the purpose of understanding health conditions when purchasing insurance products, or for the purpose of formulating plans for nursing care or health promotion services and evaluating the performance of such services).

[0214] (6) Audio is collected when you try on or wear clothing or accessories, when you ride in a vehicle such as a motor vehicle, when you view or live in a home in connection with a real estate transaction, or when you try out or use other services or goods.

[0215] (7) Services provided via the Internet (online shopping, remote medical care, counseling, education, brokerage of financial products, support services for finding employment or changing jobs, matching apps for arranging meetings with members of the opposite sex, etc.).

[0216] (8) When communicating the voice content or status of the information sender, the voice is input using a device such as a computer or smartphone.

[0217] (9) Daily voice input will be performed in order to monitor changes in the patient's mental health or dementia at a more appropriate time when the condition relapses after remission.

[0218] FIG. 19 illustrates an example of voice analysis of a user who has registered identification information. When a user requests the use of the voice platform device's functions or other service provider's services using voice data and associated data that have been previously saved or registered or imported when using the service, and the service provider accepts the request, the voice and other data required for the service are transmitted to the analysis device that uses the data required for the user's use of the service as an analysis data set. After confirming permission to use the service, the analysis device performs preprocessing and analysis using the received data set, and outputs the analysis result. The output analysis result data set is transmitted to the voice platform device. The voice platform device receives the analysis result of the analysis data set transmitted by the voice platform device, checks the result, and if the analysis is completed at the expected service level, a completion notice is transmitted to the analysis device, and if it is not completed, a reanalysis request is transmitted to the analysis device, or if it is due to a defect in the analysis data set, a notice is transmitted to the voice platform device, or if it is an unexpected error of any device, a predetermined message based on an error log is transmitted to the operator providing the platform device's service, the user, and the administrator of the analysis device.

[0219] For example, the voice input in each scene is stored as the person's own voice in the data storage device of the voice platform device. The stored voice is stored together with a voiceprint value that indicates the person's identity by a voiceprint verification device that can be executed when the voice is acquired or when the voice is saved. The stored data is extracted from the data storage device and a data set required for analysis is created by a service selected by the user, and the analysis data set is transmitted from the voice platform device to an analysis device.

[0220] The analysis device preprocesses and analyzes the received analysis data set, and transmits the analysis results. By viewing the analysis results, the mental state (mental value) and the state of cognitive function (cognitive value) can be confirmed in chronological order for the voice that was saved in chronological order.

[0221] FIG. 20 explains an example of discrimination. When a user requests the use of the functions of a voice platform device or other service provider's services using the analysis results of voice data and associated data that have been previously saved or registered or imported when using a service, and the service provider accepts the request, the voice analysis result and other data required for the service are extracted as a discrimination data set and transmitted to a discrimination device that uses data required for discrimination based on discrimination conditions set based on the user's request for service use. The discrimination device searches for a corresponding discrimination result from a discrimination result data storage device that corresponds to data such as a voice analysis result that has been registered in advance, and outputs the data discrimination result to an output destination and voice platform data storage device that are predetermined for the service.

[0222] For example, based on the analysis results data, indicators and information are identified to better understand the state of mental health and cognitive function. By checking the chronological status of the user from the results of the identification, the user can consider solutions or services that should be taken into account in the future based on objective information.

[0223] Fig. 21 explains an example related to behavioral support. When a user requests the use of the functions of the voice platform device or the services of other service providers using the discrimination results of the voice data and associated data stored in the voice platform device or registered or imported when using the service, and the service provider accepts the request, the voice analysis results and the data and discrimination results required for other services are extracted as an information utilization data set and transmitted to an information utilization method selection device used in the service based on the information utilization conditions set or selected by the user or set in advance by the service provider as data constituting a part of the service.

[0224] The information utilization method selection device searches for a corresponding information utilization method from an information utilization method data storage device that corresponds to the received data set from pre-registered information utilization methods, and outputs the information utilization method selection result to an output destination predetermined by the service and to the voice platform data storage device.

[0225] The result of information utilization based on the output result is received. The received information utilization result is transmitted to the voice data platform data storage device. The voice data platform data storage device stores the received information utilization result as a result of the selected information utilization method.

[0226] For example, by displaying search results for information such as recommended training menus, counseling that is also effective for disease prevention, or feedback on diet (and other information held by platform operators and service providers), based on the results of the discrimination, it is possible to obtain information on the progress of plans, opportunities for new support, or opportunities to take action to prevent disease.

[0227] Figures 22A and 22B explain the flow of the system. When introducing the embedded system or when changes to the registered information occur, the medical facility or other service provider registers facility information, staff information, etc. In addition, it is possible to share data and perform mutual input and output by combining the conditions of the examination and other services used by the embedded system from a combination of predefined settings, settings by an administrator, or connection from a system already owned by the service provider.

[0228] The user is someone who has previously registered as a user of the service or facility that uses this invention (in the case of medical care, this means that the user's information is registered as a patient at the medical facility and a patient card or the like has been issued, or that the user has joined social insurance and been issued a social insurance number, etc.).

[0229] By storing medical card-related data and social insurance-related data in the voice data platform device, it will be possible to perform identity verification and data exchange by using devices such as smartphones, IC card technology, and contactless communication technology.

[0230] When the user first starts using the embedded system, the user can check the terms of use and decide whether or not to accept them. After accepting the terms of use, the user's information is registered by the user himself / herself or his / her relatives, or by the medical facility staff requested by the user or his / her relatives, or by the system. If the user information is stored in a hospital system or the like in advance, it is also possible to register the user information after receiving prior consent. If the user inputs user information or voice, etc. using a device such as a smartphone, it is also possible to obtain information on the input device. In addition, identity verification is possible by incorporating a function to verify the identity of the user using voiceprints. The consent to use and the results of the registration of the user information are stored in the data storage device of the embedded system, and the results are output to the user and the service provider as electronic data, paper, or other format.

[0231] After the user is registered, the service provider, who has received instructions from the user or a relative, etc., either in writing or using an embedded system, etc., selects a service (analysis, discrimination, conditions for information utilization, etc.) and inputs voice data, etc., by capturing voice data, etc., stored in advance in a voice platform device or using a voice input device such as a microphone of the device.

[0232] After inputting the voice data to be analyzed, the data is checked to see if it meets the conditions of the service. If it does, it is sent to the data storage device of the embedded system; if it does not, a corresponding message or the like is output and the input screen is displayed again.

[0233] After confirming that the voice data has been input correctly, the voice data is output to a predetermined analysis device in accordance with the terms of the service. After confirming that the voice data has been received correctly, the analysis device performs pre-processing necessary for analysis, analyzes the voice data, and outputs the analysis results to a data storage device.

[0234] The discrimination device extracts data necessary for discrimination, including the stored analysis results, from the data storage device, and selects a corresponding data discrimination result from a previously set discrimination result data storage device. The data discrimination result is output to an information utilization method selection device and data storage device set in the service. The information utilization method selection device selects an information utilization method based on the contents previously set in the service based on the input data discrimination result, and outputs the selection result to the output destination and data storage device defined in the information utilization method. The information utilization result is received by the information utilization device and output to the data storage destination.

[0235] The data stored in the data storage device of the embedded system can be retrieved at the request of the user using a pre-registered smartphone terminal, an IC card with a magnetic storage device, contactless communication, etc. In addition, the results of using a specific service from the embedded system with the user's consent can also be retrieved via the embedded system's functions.

[0236] The health care system of the present invention can be used in the following situations, for example. A patient who had been told by people around him that he had been feeling low recently, had low motivation in everything he did, and was making more and more simple mistakes, was taken by his family to a nearby general internal medicine clinic. General internal medicine clinics are not specialists in mental illnesses or dementia, so it can be difficult to decide on a treatment plan, or it can take a long time to arrive at a treatment plan that will be effective. However, when the patient visited this general internal medicine clinic, which has a built-in system, he was instructed to take a voice test as well as a simple measurement of cognitive function during the preliminary interview. Since the patient's motivation was declining, he and his family members confirmed the terms of use and agreed to the voice test. Since the family members handled everything except the patient's voice input, the burden on the patient was low, and the doctor checked the results of the voice test before entering the examination room.

[0237] At the time of the examination, the patient's cognitive function was also impaired, but based on the results of the voice test, the patient was diagnosed with a high possibility of major depression, a depressive disorder, and a treatment plan was made, which the patient and her family agreed to. Except when visiting the facility, the patient's voice was recorded using a treatment support app linked to the voice platform device on her smartphone before and after taking medication. During hospital visits, the voice data and changes in condition outside the hospital were confirmed, and the time-series voice data was imported from the smartphone to the built-in system using a non-contact communication function, and then re-examined using the built-in system. At that time, the voice data and medical data obtained during past hospital visits were saved as the patient's data in the data storage device on the voice platform device. Since the condition outside the hospital suggested that the treatment effect may be lower than expected, the treatment plan was revised and the prescription was changed. When the symptoms improved and gradual improvement was seen at work, the patient was transferred to a distant location. In order to continue treatment, the voice platform device was used to search for hospitals that could be visited from the new address and that used the voice platform device, a referral letter was created at the current hospital, and the history of treatment to date was saved as data on the patient's own voice platform device. The data allowed the patient to continue treatment without interruption at the new hospital. The treatment was more effective and the patient's symptoms improved, so the timing for tapering was detected, which would be difficult to detect with a regular examination alone. When symptoms were stable, the patient was allowed to adjust the dosage themselves, and was able to tapering between visits while checking the audio data. The detailed tapering, which took into account daily fluctuations, was effective, and the patient went into remission. Although the relapse rate of mental illnesses is high after remission, the patient was able to use the audio platform device to grasp his condition chronologically, and was able to find activities that had a positive effect on his mental health through data, which helped him to remain stable. However, when his workload temporarily increased, he noticed a drop in his mental health values, so he went back to the hospital. Because the consultation was received before his symptoms worsened, he went into remission with shorter visits, mainly cognitive behavioral therapy.

[0238] FIG. 23 explains data transmission and reception, the voice platform, the voice analysis device, and the embedded system.

[0239] "300. Other Systems (Data Transmission Only)" acquires voice data that identifies the user and transmits it to the voice platform device. Voice data can be acquired from smartphones, smart speakers, karaoke, business radios, etc. The results can be confirmed on the voice platform device.

[0240] In "400. Other systems (transmission and reception, output of unique judgment results)," this device acquires voice and outputs the judgment results. The judgment results can be converted to the device's own judgment results. This applies to systems that have individual requirements for training and care budget planning support systems and attendance and other business management systems.

[0241] With the "100. Voice Platform Device," the device itself can record, select services, check results, check history, view information, and support secondary data use. If there are specific constraints or requirements for voice acquisition or feedback of classification results, data determined by this system based on the results of receiving and analyzing voice data acquired outside of this device can be output to a specific external system. Note that even when processing outside of this device, the consent of the user must be obtained by this device or the procedures and functions provided by this device.

[0242] "200. Voice analysis device" has the function of pre-processing and analyzing the voice data received from the voice platform device, and transmitting the analysis results to the voice platform device in a predetermined format. The analysis determines whether or not to carry out the analysis process based on the service information received along with the voice data. After the analysis is carried out, the results such as the possibility of a specific disease, mental values, cognitive values, etc. are transmitted, and the execution history of the process up until the voice platform device has completed reception is saved. After analysis, the voice data is automatically deleted.

[0243] In the case of "500. Embedded System", when a direct connection to the voice platform device is not desired due to restrictions such as portability of medical data, the intention of the service provider, or various regulations, the functions of this device can be embedded in an individual system to obtain voice and output the results of discrimination. When information managed in a system of a medical facility or the like is to be taken out using an emergency connection using NFC or Bluetooth (registered trademark) technology, it is possible to import the information into the voice platform device by outputting it in a form that meets the format specifications that can be input into the voice platform device.

[0244] The data output at each step to the data storage destination of the voice platform device and the embedded system is stored as a record linking the extraction and selection results for each user. The stored results can be reused within the scope of the individual's consent.

[0245] The stored data, excluding information that can identify individuals to the extent that consent for use has been obtained, can be searched, viewed, or extracted as statistical information only if a third party has permission from the operator of the voice platform device. For users who meet certain conditions set by a third party without being identified as individuals, if the user so wishes, it is possible to disclose or provide some or all of the data along with the purpose of use of the personal information, or to respond to requests for disclosure from operators, etc., or, for example, to recruit volunteers for clinical trials, via the voice data platform.

[0246] FIG. 24 describes the voice platform device. This device is composed of a voice acquisition unit, an acquired voice confirmation unit, an acquired voice transmission unit, a data storage unit, an analysis data set extraction unit, an analysis data set transmission unit, an analysis result data reception unit, an analysis result discrimination device (including discrimination target data extraction, discrimination result search, and discrimination result output, etc.), an information utilization method selection device (including selection target data extraction, information utilization method search, and search result output, etc.), and a user and administrator use screen of each device, etc. In addition, the data stored in the data storage unit includes, for example, user data consisting of user data, service selection data, voice data, service utilization history, analysis data, discrimination data, information utilization method selection result, and information utilization implementation data, etc. Various master data consisting of service providers, information utilization methods, and service price lists, etc. Analysis engine utilization data consisting of analysis engine types, connection methods, connection settings, and connection history, etc. Secondary utilization data consisting of public minimum datasets, browsing history, data utilization application history, data utilization approval history, data lending period, and lending data deletion history, etc. Various data such as transaction data, master data, various settings, and utilization execution history can be cited. This voice platform device is a new device that enables highly secure and unprecedented utilization of data, mainly voice, by the individual himself / herself, or by a person or business authorized by the individual.

[0247] The voice analysis device is explained in Fig. 25. It is composed of a voice analysis engine execution and management environment, a voice input device for analysis, a voice preprocessing device for analysis, a voice analysis device, an analysis result output device, an output destination reception result confirmation device, an analysis data set and analysis result deletion device, an analysis engine usage record acquisition device, a usage history acquisition and complement device, a device for calculating and outputting costs based on usage records by user and engine, and a connection method setting device by connection destination.

[0248] To provide an environment in which multiple analysis engines can be selected and executed upon request, personal information can be analyzed according to the request of the individual, and after the analysis, data analysis can be performed in a multifaceted manner without identifying individuals by having only a portion of the user data held by the requesting system or service.

[0249] 26, 27A and 27B, other systems will be described.

[0250] A system that has some of the functions of a voice platform device (acquiring voice data, checking voice data, discriminating, searching for ways to utilize information, etc.). A form of use intended for cases where unique security and usability requirements must be met, or for the purpose of providing services that utilize systems already owned by individual service providers. After acquisition and confirmation, the voice is sent to the voice platform, and the results of analysis, discrimination, and searching for ways to utilize the information are received in a data format predetermined by the voice platform device.

[0251] When voice is acquired according to a unique specification, the acquired voice can be output according to a predetermined specification, enabling voice data input to the voice platform device. After the voice data is input, the functions of the voice platform device can be used.

[0252] This refers to a usage pattern when a network connection to a voice platform device is not possible, such as in a medical facility, and is an independent system device that handles everything from obtaining consent to use to outputting results. Data input from the voice platform device and data output to the voice platform device are not performed via the Internet, but rather using wireless technology such as a smartphone (Bluetooth (registered trademark), This can be achieved via a mobile phone (NFC, AirDrop, etc.) or an IC card with memory.

[0253] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be embodied in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the forms described in the Disclosure of the Invention section.

[0254] This application claims priority to Japanese Patent Application No. 2019-232504, filed on December 24, 2019, and incorporates by reference all of the contents of that Japanese application. [Explanation of symbols]

[0255] 200 Health Management System 204 First Server 205 Second Server 206 Third Server 210 4th Server 211 Calculation section 212 Estimation Department 213 Selection Section

Claims

1. a first server for continuously storing voice data acquired from a user; a second server for storing user identification data; a third server for storing output data for users; and estimation device, A health management system comprising: The estimation device includes: A calculation unit that calculates an acoustic feature based on the speech data; An estimation unit that estimates a predetermined disease using the acoustic feature as an input; and a selection unit that selects output data of the third server based on an estimation result of the estimation unit and the identification data of the user; Equipped with The selection unit is when the estimation unit estimates that there is no suspicion of the subject suffering from the predetermined disease, the selection unit selects, from the third server, first output data including distribution data for healthy individuals; When the estimation unit estimates that the patient is suspected of having the predetermined disease, the selection unit selects second output data including a message from the third server encouraging the patient to visit a medical institution, After the user is diagnosed by a doctor as suffering from a predetermined disease, the identification data of the second server is updated, and the selection unit is configured to select third output data including distribution data for the user after the user is diagnosed with the disease from the third server, The estimation unit estimates at least one disease from a group of diseases including Alzheimer's disease, Lewy body dementia, Parkinson's disease, atypical depression, and bipolar disorder. Health management system.

2. The health management system according to claim 1 , wherein the estimation unit is further capable of estimating major depression.

3. The estimation unit is capable of classifying the subject into a dementia group including Lewy body dementia, Alzheimer's disease, and Parkinson's disease, and a mood disorder group including major depression, bipolar disorder, and atypical depression. The health management system according to claim 2 .

4. The health management system according to claim 1 or 2, wherein the estimation unit is capable of estimating Lewy body dementia.

5. The health management system according to claim 1 , wherein the estimation unit is capable of estimating Alzheimer's disease.

6. The health management system according to claim 1 , wherein the estimation unit is capable of estimating Parkinson's disease.

7. The health management system according to claim 1 , wherein the estimation unit is capable of estimating bipolar disorder.

Images