Method for collecting voice data for constructing brain health diagnosis index

Description

A method for collecting voice data to establish a brain health diagnostic index.

[0001] The present invention relates to a method for controlling and standardizing a voice data collection environment and method, which is a biomarker used in various brain health diagnoses such as cognitive impairment, dementia, and depression, and more specifically, to a method for collecting voice data for establishing a brain health diagnosis index, which can ensure consistency and validity of data by setting and controlling standards for the surrounding environment for collecting voice data, i.e., microphone status, surrounding noise status, soundproofing status, user posture, etc., and can increase the accuracy of various brain health diagnoses by setting and checking standards for the current physical condition of a user who needs to collect voice data and providing data that reflects the user's voice characteristics and disease characteristics of the voice. In particular, the present invention relates to a voice data collection method for establishing a brain health diagnosis index, which can increase the accuracy of brain health diagnoses by collecting user's voice data multiple times in relation to a plurality of images while sequentially displaying them, and also can increase the accuracy of user-related voice data by collecting the same set of voice data from multiple users for common sentences and / or collecting voice data in a state where user emotions are excluded as much as possible.

[0002] When it comes to brain health, various diseases exist, including cognitive impairment, Alzheimer's disease (dementia), and depression. Cognitive impairment refers to a decline in memory, attention, language skills, visuospatial abilities, and judgment. The degree of cognitive impairment ranges from very mild to severe. Dementia occurs when cognitive impairment is so severe that it interferes with daily life or social activities. Alzheimer's Disease (AD) is a neurodegenerative disease characterized by a long-term and gradual decline in cognitive function. It is a common form of dementia that first progresses to mild cognitive impairment (MCI) and then to AD. Early detection of Alzheimer's disease is essential for timely treatment, which can slow cognitive decline and reduce the risk of developing dementia. Depression is a condition characterized by a variety of cognitive and psychosomatic symptoms, primarily loss of motivation and feelings of sadness, which lead to a decline in daily functioning. Depressive disorder is a serious illness that causes changes in emotions, thoughts, physical state, and behavior.

[0003] Voice has various characteristics, including pitch, volume, rate, rhythm, fluency, articulation, pronunciation, and tone. These features are also the components that allow us to perceive what others are saying. These sounds are generated through various circuits in the brain, passing through various locations, and are then heard by our ears. This is why voices differ in various diseases and conditions related to brain health. These differences can be used to diagnose various disorders. However, to develop such voice biomarkers, data consistency and validity are crucial. To ensure this, secure technology and well-controlled environment for data collection are crucial.

[0004] Identifying specific characteristics of each negative biomarker for diagnosing different brain health diseases can vary greatly depending on the nature of the disease and patient criteria. However, when it comes to establishing the raw reference data, or diagnostic criteria, the collection method must be quantifiable and consistent with established standards.

[0005] (Patent Document) Domestic Patent No. 10-2274072 (registered on July 1, 2021) "Method and Device for Determining the Degree of Dementia in a User"

[0006] The prior patent disclosed in the above <patent document> also only discloses the concept of visualizing the characteristics of the voice input by the user and using this to determine the degree of dementia.

[0007] Therefore, the need to collect voice data that can ensure data consistency and validity by controlling and standardizing the voice data collection environment and method from the voice data collection stage is increasing.

[0008] The present invention has been devised to solve the above problems.

[0009] The purpose of the present invention is to provide a method for collecting voice data for establishing a brain health diagnosis index that can ensure consistency and validity of data by controlling and standardizing the voice data collection environment and method to enable diagnosis of various brain health conditions using voice data as a biomarker.

[0010] Another object of the present invention is to provide a method for collecting voice data for establishing a brain health diagnosis index that can ensure consistency and validity of data by setting and controlling standards for the surrounding environment for collecting voice data, i.e., microphone status, ambient noise status, soundproofing status, user posture, etc.

[0011] Another object of the present invention is to provide a method for collecting voice data to establish a brain health diagnosis index that can increase the accuracy of various brain health diagnoses by setting and checking criteria for the current physical condition of a user who needs to collect voice data and providing data reflecting the user's voice characteristics and disease characteristics of the voice.

[0012] Another object of the present invention is to provide a voice data collection method for constructing a brain health diagnosis index that can increase the accuracy of user-related voice data by collecting user voice data multiple times while sequentially showing multiple images, and also increase the accuracy of brain health diagnosis by collecting the same set of voice data from multiple users for common sentences and / or collecting voice data in a state where user emotions are excluded as much as possible.

[0013] Another object of the present invention is to provide a method for collecting voice data for constructing a brain health diagnosis index that can increase the accuracy of brain health diagnosis by collecting and providing image data including the user's eyes moving in conjunction with a moving object shown on a display along with acquisition of voice data.

[0014] Another object of the present invention is to provide a method for collecting voice data for establishing a brain health diagnosis index that can increase the accuracy of brain health diagnosis by preprocessing the collected voice data to remove noise other than the user's voice and setting the user's voice characteristics and disease characteristics of the user's voice and reflecting them in the voice data and providing the data.

[0015] In order to achieve the above-mentioned purpose, the present invention is implemented by an embodiment having the following configuration.

[0016] According to one embodiment of the present invention, a method for collecting voice data for constructing a brain health diagnosis index according to the present invention is characterized by including a step (S1) of checking, using a program of a device, whether the user's current physical condition is suitable for collecting voice data; a step (S3) of checking whether the user's surrounding environment is suitable for collecting voice data; a step (S5) of collecting user-related metadata; and a step (S7) of providing content to the user to collect voice data.

[0017] According to another embodiment of the present invention, the step S1 in the present invention is characterized by including a step (S11) of checking whether a disease suffered by the user is suitable for voice data collection based on information input from the user, and a step (S12) of checking whether the user's voice quality is suitable for voice data collection based on information collected through the user's natural speech.

[0018] According to another embodiment of the present invention, the step S3 in the present invention is characterized by including a step (S31) of checking whether a frequency band and a maximum sound pressure range that can be received by a microphone used for voice data collection are appropriate for voice data collection based on information input from a user, a step (S32) of checking whether the ambient environmental noise of a place where voice data is to be collected is 20 dB or less, and a step (S33) of checking whether a place where voice data is to be collected based on information input from a user is a soundproof place with two or fewer walls, more than 80% of which are made of glass, and an area of ​​more than 1.2 m2.

[0019] According to another embodiment of the present invention, the step S5 in the present invention is characterized by collecting metadata related to the user's age, the user's annual education level, past medical conditions, and medications being taken based on information input from the user.

[0020] According to another embodiment of the present invention, the step S7 in the present invention is characterized by including a step (S71) of controlling and setting a user posture and environment suitable for collecting voice data, a step (S72) of identifying user voice characteristics based on information collected through the user's natural speech, and a step (S73) of providing content to the user to collect a plurality of voice data.

[0021] According to another embodiment of the present invention, the step S73 of the present invention includes a step (S731) of collecting voice data of users multiple times while a plurality of images are sequentially displayed on a device display using a program of the device, a step (S732) of collecting the same set of voice data for a plurality of users while a common sentence is displayed on the device display using the program of the device, and a step (S733) of collecting voice data in a state excluding emotions for images or sentences displayed on the device display using the program of the device, and in steps S731 and S732, re-recorded voice data is not collected, and in step S733, re-recorded voice data can also be collected and used.

[0022] According to another embodiment of the present invention, the step S7 in the present invention is characterized in that it additionally includes a step (S74) of collecting image data including the user's eyes in conjunction with a moving object being shown on the display during the process of collecting voice data through the step S73.

[0023] According to another embodiment of the present invention, another method of the present invention further comprises a step (S9) of providing voice or image data collected for brain health diagnosis; wherein step S9 comprises a preprocessing step (S91) of removing noise other than the user's voice from the voice data, a step (S92) of reflecting the user's voice characteristics identified through step S72 into the voice data, and a step (S93) of setting the disease characteristics of the user's voice based on data related to the user's past medical illness or medication being taken identified through step S5 and reflecting them into the voice data.

[0024] According to another embodiment of the present invention, the voice data provided in the step S9 of the present invention is voice data for diagnosing Alzheimer's disease, and is provided by extracting a disfluency feature based on a segmented voice pattern after performing voice segmentation on the voice data, and the disfluency feature includes a static feature, a participant feature, and an interviewer feature, and the static feature includes a sum of the lengths of static segments in a voice recording (Len_S), a variance of the lengths of static segments in a voice recording (Var_S), an average of the lengths of static segments in a voice recording (Avg_S), a maximum length of the lengths of static segments in a voice recording (Max_S), a ratio of static segments to the entire voice length (Rat_S), a number of static segments in a voice recording (Num_S), a number of static segments between consecutive participant segments (Num_PS), an average of the lengths of static segments between consecutive participant segments (Avg_PS), and a variance of the lengths of static segments between consecutive participant segments (Var_PS), and the participant feature includes a ratio of participant segments to the entire voice length (Rat_P), a number of participant segments in a voice recording, and a number of participant segments in a voice recording. The interviewer features include a number (Num_P), and the interviewer features include a number of interviewer segments (Num_I) in the voice recording, and a number of speaker changes (Num_chn) between participant segments and interviewer segments in the voice recording.

[0025] The present invention can obtain the following effects through the combination and use of the configuration described above and the following examples.

[0026] The present invention has the effect of ensuring consistency and validity of data by controlling and standardizing the voice data collection environment and method to enable diagnosis of various brain health conditions using voice data as a biomarker.

[0027] The present invention has the effect of ensuring consistency and validity of data by setting and controlling standards for the surrounding environment for collecting voice data, i.e., microphone status, surrounding noise status, soundproofing status, user posture, etc.

[0028] The present invention has the effect of increasing the accuracy of various brain health diagnoses by setting and checking standards for the current physical condition of a user who must collect voice data and providing data reflecting the user's voice characteristics and voice disease characteristics.

[0029] The present invention has the effect of increasing the accuracy of user-related voice data by collecting user voice data multiple times while sequentially displaying multiple images, and also increasing the accuracy of brain health diagnosis by collecting the same set of voice data from multiple users for common sentences and / or collecting voice data in a state where user emotions are excluded as much as possible.

[0030] The present invention has the effect of increasing the accuracy of brain health diagnosis by collecting and providing image data including the user's eyes moving in conjunction with the display showing a moving object along with acquisition of voice data.

[0031] The present invention has the effect of increasing the accuracy of brain health diagnosis by preprocessing the collected voice data to remove noise other than the user's voice, setting the user's voice characteristics and disease characteristics of the user's voice, and reflecting these in the voice data and providing the data.

[0032] Figure 1 is a step diagram of a voice data collection method according to one embodiment of the present invention.

[0033] Figure 2 is a reference diagram of the non-fluency characteristics of voice data provided by the present invention.

[0034] Hereinafter, preferred embodiments of a voice data collection method for establishing a brain health diagnosis index according to the present invention will be described in detail with reference to the attached drawings. It should be noted that identical components are represented by the same reference numerals throughout the drawings, as much as possible. Unless otherwise defined, all terms in this specification have the same general meaning as those terms understood by a person skilled in the art to which the present invention pertains, and if there is a conflict with the meaning of a term used in this specification, the definitions used in this specification shall apply. Throughout the specification, when a part is said to "include" a certain component, this does not mean that other components are excluded, but rather that other components may be further included, unless specifically stated otherwise. In addition, terms such as "... part" and "... module" described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware, software, or a combination of hardware and software.

[0035]

[0036] Referring to FIG. 1, a method for collecting voice data for establishing a brain health diagnosis index according to one embodiment of the present invention may include a step (S1) of checking, using a program of a device, whether the user's current physical condition is suitable for collecting voice data; a step (S3) of checking whether the user's surrounding environment is suitable for collecting voice data; a step (S5) of collecting user-related metadata; and a step (S7) of providing content to the user to collect voice data.

[0037] The above step S1 is a process of first checking whether the user's current physical condition is suitable for collecting voice data. If the user has a disease that he or she is currently suffering from or his or her current throat condition, which may affect the voice data to be collected, the voice data as a voice biomarker may be adversely affected. In such cases, the step S1 may include a step (S11) of checking whether the disease the user is suffering from is suitable for collecting voice data based on information input by the user, and a step (S12) of checking whether the user's voice quality is suitable for collecting voice data based on information collected through the user's natural speech.

[0038] The above step S11 is a process of checking whether the disease the user is suffering from is suitable for voice data collection based on information input by the user using a device program. By utilizing a program installed on a device such as a smartphone that the user can easily carry around, the user is requested to provide information on the relevant disease, and based on the information input by the user, it is checked whether the disease the user is currently suffering from or has suffered from in the past may affect voice data collection and thus whether voice data collection is possible. For example, if the disease the user is currently suffering from based on the information input by the user is a sore throat, pneumonia, vocal cord nodules, etc., these are diseases that may significantly affect the reliability of voice data, and in such cases, it can be determined that voice data collection is not possible.

[0039] The above step S12 is a process of checking whether the user's voice quality is appropriate for collecting voice data based on information collected through the user's natural speech. Even if it is confirmed that the user does not have a disease that may affect voice data collection based on the information input through the above step S11, the user's current throat condition may have a negative effect on voice data collection without the user's awareness. Therefore, in step S12, voice data is collected from the user through natural speech using a program installed on the device the user is carrying, and it is checked whether the user's voice quality determined based on this is appropriate for collecting voice data. For example, if irregular noises (due to the current throat condition) are confirmed in the voice collected through natural speech through step S12, it may be determined that voice data cannot be collected.

[0040] The above step S3 is a process of checking whether the user's surrounding environment is suitable for collecting voice data. If the location or facility where the user is currently located to collect voice data may have a negative impact on the voice data to be collected, the quality of the voice data as a voice biomarker will be affected. Therefore, the step S3 is a process of checking whether voice data collection is possible in such cases. To this end, the step S3 may more specifically include a step (S31) of checking whether the frequency band and maximum sound pressure range that the microphone used for voice data collection can receive are suitable for voice data collection based on information input from the user, a step (S32) of checking whether the surrounding environmental noise of the location where voice data is to be collected is 20 dB or less, and a step (S33) of checking whether the location where voice data is to be collected is a soundproof location with two or fewer walls, more than 80% of which are made of glass, and an area of ​​more than 1.2 m2, based on information input from the user.

[0041] The above step S31 is a process of checking whether the frequency range and maximum sound pressure range that the microphone can receive are appropriate for voice data collection based on information input from the user. In other words, it is a process of verifying the condition and performance of the microphone used for voice data collection. The basic structure of the audio equipment (microphone) used for voice recording is composed of an input section, a processing section, an amplification section, and an output section. When a human voice is input, the vibration called sound is converted into an electrical signal and stored. At this time, the frequency range that can be received and the maximum sound pressure range that can be received are different for each microphone. A microphone that can stably collect sound without distortion by covering the frequency range and maximum sound pressure range of the entire sound range that can be produced by the human voice and can collect unique sound waves without any processing or amplification is necessary to enable the collection of good voice data as a voice biomarker. Accordingly, in the above step S31, based on information input from the user, it is possible to check whether the microphone used for voice data collection is suitable for voice data collection by checking whether the microphone can collect frequencies in the range of 350 Hz to 17 KHz, that is, all sound ranges, and whether all sounds can be recorded without distortion or deformation up to a maximum sound pressure range of 120 dB.

[0042] The above step S32 is a process of checking whether the ambient noise of the place where voice data is to be collected is 20 dB or less. Using a program installed on a device carried by the user (such as a smartphone with a microphone function), the user checks whether the ambient noise level in the place where the user's voice is to be recorded is 20 dB or less, which can minimize the effect on the quality of the voice data obtained.

[0043] The above step S33 is a process for checking whether the place where voice data is to be collected is a soundproof place with two or fewer walls made of glass, more than 80% of which are capable of blocking out external noise, and a floor area of ​​at least 1.2㎡, based on information input from the user. In other words, the place (space) where voice data is to be collected must be a soundproof place that can minimize external influence during the collection process in order to secure the quality of the collected voice data, and is therefore a process for verifying the recording location. Therefore, in the above step S33, it is possible to check the soundproofing status of the place where voice data is to be collected, based on information input from the user, that is, whether the place is a soundproof place with two or fewer walls made of glass, more than 80% of which are capable of blocking out external noise, and a floor area of ​​at least 1.2㎡.

[0044] The above S5 step is a process of collecting user-related metadata. In order to diagnose brain health for each user more accurately, not only the standardized (standardized) voice data collected through the present invention must be collected, but also the medical or educational information related to the brain health (disease) to be diagnosed for each user must be supported for utilization and interpretation. In the S5 step, metadata related to the user's age, annual education level, past medical conditions, and medications being taken can be collected based on information input by the user using a program installed on the device the user carries. That is, for example, when diagnosing cognitive impairment using voice data collected from a user with hearing loss, correction of the obtained voice data according to the hearing loss disease is required, and voice data related to reading collected from an illiterate user must be excluded from future diagnosis. In this way, the correction of the voice data can be reflected when diagnosing brain health conditions for each user by utilizing the user-specific metadata collected through the S5 step.

[0045] The above step S7 is a process of collecting voice data by providing content to the user (using a program installed on a device carried by the user), and for this purpose, the step S7 may more specifically include a step (S71) of controlling and setting a user posture and environment appropriate for collecting voice data, a step (S72) of identifying the user's voice characteristics based on information collected through the user's natural speech, a step (S73) of providing content to the user to collect a plurality of voice data, and a step (S74) of collecting image data including the user's eyes in conjunction with a moving object shown on the display during the process of collecting voice data through the step S73.

[0046] The above step S71 is a process of controlling and setting an appropriate user posture and environment for collecting voice data. First, in order to obtain standardized (standardized) voice data to be used for brain health diagnosis, it is important to collect voice data in a posture in which the user straightens the upper body, raises the face, and does not strain the neck. Therefore, the user's posture is guided and controlled using a program on the device carried by the user to ensure that the user assumes an appropriate posture for collecting voice data. In addition, it is important to record the user's speech at a distance of about 10 to 15 cm from the microphone that records the voice data. Similarly, an appropriate environment for collecting voice data can be controlled and set by guiding and controlling an appropriate distance from the microphone using a program on the device carried by the user.

[0047] The above step S72 is a process of identifying the user's voice characteristics based on information collected through the user's natural speech. Since there are individual differences in the unique characteristics of each user's voice or the characteristics of a voice disease that has affected the voice due to past medical history, etc., by identifying these unique voice characteristics or voice disease characteristics of each user through the above step S72, the collected voice data can be reflected (neutralized) so that it is not contaminated by these user-specific voice characteristics.

[0048] The above step S73 is a process of collecting multiple voice data by providing content to the user, and especially in the process of collecting standardized (standardized) voice data for establishing a brain health diagnosis index, the involvement of the collector needs to be minimized. However, in the existing voice data collection process, multiple collectors frequently get involved in the voice data acquisition process, which inevitably increases the contamination of the collected voice data. In particular, the present invention utilizes conversational AI technology for voice collection of a device program to enable consistent voice data to be collected. To this end, the above step S73 may more specifically include a step (S731) of sequentially displaying multiple images on the device display using the device program and collecting user voice data related thereto multiple times, a step (S732) of collecting the same set of voice data for multiple users related to common sentences displayed on the device display using the device program, and a step (S733) of collecting voice data for images or sentences shown on the device display while excluding emotions using the device program.

[0049] The above step S731 is a process of collecting user voice data related to multiple images sequentially displayed on the device display using the device's program multiple times. If voice data for multiple image contents sequentially displayed through step S731 is collected multiple times for a specific user, the voice characteristics of the specific user can be more accurately identified. In other words, the repeated collection of voice data for the same specific user allows for the acquisition of voice data that contains the unique voice characteristics of the specific user as is, excluding unnecessarily mixed noise, etc. (such as voices recorded by artificially processing (deforming) by the user).

[0050] The above step S732 is a process of collecting the same set of voice data for multiple users related to a common sentence displayed on the device display using the device's program. If the same set of voice data is collected under the same conditions for the common sentence content that is given identically to multiple users through step S732, the voice characteristics that differentiate between users can be more accurately identified. In other words, the voice data set acquired for multiple users under the same conditions / environment when the same content (sentence) is given can identify the voice data characteristics that differentiate between users, and in particular, can obtain a voice data set that can be helpful in distinguishing the voice characteristics that differentiate between users with brain health diseases such as cognitive impairment, Alzheimer's disease, and depression and users with healthy brains.

[0051] The above step S733 is a process of collecting voice data in a state where emotions are excluded for images or sentences shown on the device display using the device's program, which enables the collection of neutral voice data in a state where emotions are excluded as much as possible, and preferably, in the process of providing content such as prose in which reflection of emotions can be minimized, voice data can be obtained by guiding (controlling) so that speech can be made in a state where emotions are excluded as much as possible. The voice data obtained through the S733 step can be utilized, in particular, for the diagnosis of brain health diseases such as depression.

[0052] The process of collecting multiple voice data through content provision like this is carried out by dividing the content into content that can be re-recorded and content that cannot be re-recorded depending on the nature of the content provided when the user uses the program once. In steps S731 and S732, re-recorded voice data is not collected, and in step S733, re-recorded voice data can also be collected and used. That is, in order to more accurately identify the voice characteristics of a specific user, voices that have been recorded after being artificially processed (deformed) by the user must be excluded. In steps S731 and S732, if the same recording process is repeated in a single recording, the possibility of artificial processing / deformation increases, so this is excluded. However, in step S733, voice data spoken in a state where emotions are excluded as much as possible is required, and therefore, voice data with minimal emotional reflection can be obtained and used through re-recording. The method proposed in the present invention acquires and utilizes voice data by distinguishing between non-recordable and re-recordable data based on the nature of the content provided. This allows for the acquisition and provision of data that significantly enhances the accuracy of diagnosis and identification. Furthermore, it is desirable that the above-described steps be performed in a manner that is as convenient and easy for the user as possible, based on functionality and order.

[0053] The above step S74 is a process of collecting image data including the user's eyes in conjunction with the moving object shown on the display during the process of collecting voice data through the above step S73. In this way, the image data of the user's face / actions, etc. shown during the process of collecting voice data, especially the facial image data including the user's eye movements reacting to the content shown on the display, can be provided together with the voice data in the future when diagnosing brain health diseases and can be utilized for more accurate diagnosis, etc. In particular, for users whose voice data has shown to have relatively low reliability, if the facial image data including the eye movements provided together in this way is utilized, a more accurate diagnosis can be made based on the facial expressions, eye movements, and behavioral characteristics shown by the patient with depression, cognitive impairment, dementia, etc.

[0054] The (voice / video) data collected in this way can be used to diagnose a single brain health disease, diagnose a single disease per content, diagnose multiple brain health-related diseases with a single content, or diagnose a single disease by collecting various voice data using multiple contents.

[0055] Below, the validity of the method for collecting voice data, etc. according to the present invention was evaluated through experiments.

[0056] - Experimental purpose: The purpose of this study was to verify the effectiveness of the voice data collection method proposed in the present invention by comparing it with other collection methods and comparing the accuracy of hearing loss and cognitive impairment diagnosis based on voice data collected by each method.

[0057] - Voice data collection method: Voice data was collected using a voice data collection method (Comparative Examples 1 to 6) under different conditions compared to the voice data collection method (Example 1) proposed in the present invention.

[0058] In Example 1, voice data was collected in a soundproof space (a space with at least two walls made of glass, with at least 80% of the walls being able to block out external noise, and with a floor area of ​​at least 1.2 m2) with ambient noise less than 20 dB and a distance of 10 cm between the user's mouth and the microphone, while limiting the volume of the recorded voice to between 50 and 120 dB. (At this time, the user participating in the recording was not currently suffering from a disease such as a sore throat.)

[0059] In contrast, in Comparative Examples 1 to 6, voice data was collected under one different condition compared to Example 1, and the details are as shown in [Table 1] below. (In all examples, 10 voice data were collected, and the 10 voice data collected at this time were voice data collected according to different content provision.)

[0060] Ambient noise Distance from microphone Location (soundproofing) Sound level Disease presence Data number Example 120 dB or less About 10 cm Soundproofed room 50~120 dB None 10 Comparative example Over 120 dB About 10 cm Soundproofed room 50~120 dB None 10 Comparative example 220 dB or less About 20 cm Soundproofed room 50~120 dB None 10 Comparative example 320 dB or less About 10 cm Bathroom (soundproofing ×) 50~120 dB None 10 Comparative example 420 dB or less About 10 cm Soundproofed room Less than 50 dB None 10 Comparative example 520 dB or less About 10 cm Soundproofed room Exceed 120 dB None 10 Comparative example 620 dB or less About 10 cm Soundproofed room 50~120 dB Sore throat 10

[0061] - Model used for hearing loss and cognitive impairment diagnosis (prediction): First, the model used for hearing loss diagnosis (prediction) is the VGGish model composed of CNN. This hearing loss model first converts the input voice data into mel-spectogram format and then inputs the converted data into the VGGish model. Then, the output features of VGGish are input into the LSTM model to diagnose (predict) hearing loss. Second, the model used for cognitive impairment diagnosis (prediction) is a deep learning model based on the 1D Swin Transformer. This model first converts the input voice data into wav2vec and then inputs it into the 1D Swin Transformer model. The output features of the model are used to predict cognitive impairment through MLP. - Experimental results: As described above, the results of diagnosing (predicting) hearing loss and cognitive impairment using the voice data collected through Example 1 and Comparative Examples 1 to 6 can be seen in [Table 2] below.

[0062] Hearing loss prediction performance Cognitive impairment prediction performance Example 190%80% Comparative example 170%60% Comparative example 280%70% Comparative example 350%60% Comparative example 460%50% Comparative example 580%70% Comparative example 680%80%

[0063] As can be seen from the analysis results, the prediction accuracy utilizing voice data collected using the method of the present invention was the highest. In particular, the prediction accuracy for hearing loss was around 90%, and the prediction accuracy for cognitive impairment was also around 80%. This confirms that voice data collected through the method of the present invention, through standardization and control of the voice data collection environment, user status, and posture, can be effectively and usefully applied to various brain health diagnoses. In particular, as in Comparative Example 1, voice recording in a place where the ambient noise in which voice recording is performed is over 20 dB, and as in Comparative Example 3, voice recording in a place where soundproofing is not provided, resulted in a significant decrease in the accuracy of diagnosis (prediction) for both hearing loss and cognitive impairment, confirming that ambient noise and soundproofing are important factors in obtaining voice data as an indicator of brain health diagnosis. In addition, it was confirmed that the user's voice size in Comparative Examples 4 and 5, especially when the sound is too low, significantly decreases the accuracy of prediction for hearing loss and cognitive impairment, and it was confirmed that the user's throat condition in Comparative Example 6 has a significant effect on prediction of hearing loss, etc.

[0064]

[0065] Meanwhile, a method for collecting voice data for constructing a brain health diagnosis index according to another embodiment of the present invention may additionally include a step (S9) of providing voice or image data collected for brain health diagnosis, as illustrated in FIG. 1.

[0066] The above step S9 is a process for providing voice or image data collected for brain health diagnosis, and more specifically, may include a preprocessing step (S91) of removing noise other than the user's voice from the voice data, a step (S92) of reflecting the user's voice characteristics identified through the step S72 into the voice data, and a step (S93) of setting the disease characteristics of the user's voice based on data related to the user's past medical conditions or medications being taken identified through the step S5 and reflecting them into the voice data.

[0067] The above S91 step is a process of removing noise other than the user's voice from the voice data. Biomarkers used for brain health diagnosis, etc. must use only pure user voice data. Therefore, noise other than the user's voice, such as ambient noise included in the recording process, is removed through this preprocessing process and then provided for purposes such as diagnosis.

[0068] The above step S92 is a process of reflecting the user voice characteristics identified through the above step S72, etc., into the voice data. By reflecting the user voice characteristics identified through the process of collecting the user's voice data for multiple image contents sequentially displayed multiple times based on the information collected through the user's natural speech through the above step S72, or through the process of collecting the user's voice characteristics for multiple image contents sequentially displayed through the above step S731, into the voice data, it functions as a biomarker used for more accurate brain health diagnosis. In other words, it prevents the voice data from being contaminated by the reflection of characteristics such as a specific user's unique speech habits into the voice data.

[0069] The above step S93 is a process of setting the disease characteristics of the user's voice based on the user's past medical conditions or medication-related data identified through the above step S5, and reflecting them in the voice data. That is, for example, when diagnosing cognitive impairment using voice data collected from a user with hearing loss, correction of the obtained voice data according to the hearing loss is required, and voice data related to reading collected from illiterate users should be excluded from future diagnosis. In this way, by utilizing the user-specific metadata collected through the above step S5, the voice data provided for diagnosing brain health diseases of each user in the above step S93 can be provided by reflecting the user-specific voice disease characteristics.

[0070] Meanwhile, a major example of voice data provided in the above S9 step may be voice data for diagnosing Alzheimer's Disease (AD). In this case, voice segmentation may be performed on the voice data, and then disfluency features may be extracted and provided based on the segmented voice pattern.

[0071] Voice segmentation is the process of dividing input voice data into voice segments (speech parts) and static segments (pause parts) using a neural network (e.g., Voice Activity Detection (VAD) algorithm or a pre-trained audio neural network (PANN)). The segmented voice segments can then be further divided into participant segments (participating in brain health diagnosis) and interviewer segments.

[0072] Disfluency features based on segmented speech patterns in this way can include static features, participant features, and interviewer features (additionally metadata features), as illustrated in Fig. 2.

[0073] First, the above static features can be seen as the most important features for AD detection, because abnormal speech interruptions, longer durations of silence, and greater number of pauses in speech are symptoms that appear in Alzheimer's disease (AD) patients. The above static features basically include statistical features. That is, they can include the sum of the lengths of static segments in a voice recording (Len_S), the variance of the lengths of static segments in a voice recording (Var_S), the average of the lengths of static segments in a voice recording (Avg_S), the maximum length of static segments in a voice recording (Max_S), the ratio of static segments to the total voice length (Rat_S), and the number of static segments in a voice recording (Num_S). Among the above static features, the sum of the lengths of static segments in a voice recording (Len_S), the average of the lengths of static segments in a voice recording (Avg_S), the ratio of static segments to the total voice length (Rat_S), and the number of static segments in a voice recording (Num_S) are static features that are considered to increase the accuracy of AD diagnosis through statistical features related to static segments in the input voice data. In particular, among the above static features, the maximum length of static segments in a voice recording (Max_S) and the variance of the lengths of static segments in a voice recording (Var_S) are features that are considered to enable more accurate AD detection using static features while minimizing the influence of abnormal data that may be included in the voice data.

[0074] Meanwhile, in the above static features, only features related to static features in the participant voice data, i.e., features induced by the participant (P) among the speakers, can be additionally included. Specifically, the number of static segments between consecutive participant segments (Num_PS), the average length of static segments between consecutive participant segments (Avg_PS), and the variance of the length of static segments between consecutive participant segments (Var_PS) can be additionally included. The number of static segments between consecutive participant segments (Num_PS) can be used to identify the characteristics of the number of static segments in the participant's continuous speech process, that is, how often pauses (silence pauses) appear in the speech process. The average length of the static segments between consecutive participant segments (Avg_PS) can be used to identify the characteristics of how long the static segments, i.e., pauses (silence pauses, voiceless pauses are easier to extract from speech signals (speech data) than filled pauses such as 'uh', 'um', etc., and can be used for analysis regardless of language) continue on average in the participant's continuous speech process. The variance of the length of the static segments between consecutive participant segments (Var_PS) can be used to identify the characteristics of how evenly the length of the static segments, i.e., pauses (silence pauses), appear in the participant's continuous speech process. Therefore, by utilizing these characteristics, it is possible to identify the characteristics of how slowly the original speech is made (even in the absence of AD disease). This can improve the accuracy of AD diagnosis by filtering out participants with a speech tendency to have regular pauses between pronunciations.

[0075] The above participant features include the proportion of participant segments in the total speech length (Rat_P) and the number of participant segments in the speech recording (Num_P), and the above interviewer features include the number of interviewer segments in the speech recording (Num_I) and the number of speaker changes between participant segments and interviewer segments in the speech recording (Num_chn). The interviewer features do not include the proportion of interviewer segments (Rat_I) to avoid unnecessary duplication (i.e., Rat_S+Rat_P+Rat_I is always 1).

[0076] In particular, the above interviewer characteristics serve as an important factor in increasing the accuracy of AD diagnosis, because the interviewer's (investigator's) intervention occurs more frequently during the interview (investigation) process for AD patients, and therefore, not only the number of interviewer segments in the voice recording (Num_I), but especially the number of speaker changes between the participant segment and the interviewer segment in the voice recording (Num_chn) can be viewed as disfluency characteristics that contribute to increasing the accuracy of AD diagnosis.

[0077] In this way, in the present invention, preprocessing and / or reflection of user-specific voice characteristics, etc. can be provided for diagnosis of brain health diseases of each user through voice data collected through the collection method of the present invention, and in particular, Alzheimer's (AD) can be accurately diagnosed using the disfluency features extracted from the user's voice data as described above.

[0078]

[0079] In the above, the applicant has described various embodiments of the present invention, but such embodiments are only examples of implementing the technical idea of ​​the present invention, and any change or modification that implements the technical idea of ​​the present invention should be interpreted as falling within the scope of the present invention.

Claims

1. A method for manufacturing functional yogurt, characterized by including a raw milk input step of filtering and inputting raw milk, a primary sterilization step of sterilizing raw milk, a functional addition step of improving the function of yogurt by adding colostrum or whey, a primary cooling step of cooling raw milk, a lactic acid bacteria input step of inputting lactic acid bacteria into raw milk, a fermentation step of fermenting raw milk by lactic acid bacteria, a final cooling step of finally cooling the fermented yogurt, and a filling step of filling the cooled yogurt into a container.

2. In the first paragraph, the first sterilization step Sterilize the raw milk at a temperature of 90℃ or higher and 100℃ or lower. The above function addition step is, A method for manufacturing functional yogurt, characterized by comprising a functional material introduction step of introducing functional materials such as colostrum or whey powder, a stirring step of mixing the introduced functional materials with raw milk, and an additional sterilization step of sterilizing the raw milk with the functional materials introduced at a temperature of 70°C or higher and 75°C or lower.

3. In the second paragraph, the functional material input step A method for manufacturing functional yogurt, characterized in that when whey powder is added, tangerine peel is additionally added.

4. In the second paragraph, the functional material input step A method for manufacturing functional yogurt, characterized in that when whey powder is added, one of red cabbage, honey, yuzu, and citrus fruits containing glucose is additionally added.

5. In the fourth paragraph, the functional material input step A method for manufacturing functional yogurt characterized by adding additional ginseng.

6. In the second paragraph, the functional material input step A method for manufacturing functional yogurt, characterized in that when whey powder is added, any one of citrus peel, avocado, papaya, and olive containing alpha tocopherol is additionally added.

7. A functional yogurt characterized by being manufactured by the manufacturing method of any one of claims 1 to 6.

8. It includes a container for manufacturing yogurt by sterilizing, fermenting, and cooling raw milk and functional materials, and a process container formed to surround the container and sterilizing and cooling the raw milk contained in the container through the flow of water. The above process receptor is, A functional yogurt manufacturing device characterized by comprising an inner channel that forms a space in contact with the receiving container to allow water to flow, an outer channel that is partitioned to the outside of the inner channel to form a space in which water flows, and a partition plate that partitions the inner channel and the outer channel.

9. In the 8th paragraph, the process receiver A functional yogurt manufacturing device characterized in that the sterilization or cooling temperature of raw milk contained in the container is controlled by controlling the position where process water is injected among the inner and outer channels.

10. In the 9th paragraph, the partition plate The upper part of the above process receiver is formed to have a lower height than the process receiver so that the inner and outer channels are connected to each other. A functional yogurt manufacturing device characterized in that the process water container includes an inlet formed to penetrate from the outside toward the outside flow path so that process water is supplied.

11. In the 10th paragraph, the process receiver It includes a discharge hole for discharging process water and a discharge control means for controlling discharge of process water through the discharge hole. The above discharge hole is, It includes an inner discharge hole formed to penetrate a lower point of the above-mentioned partition plate, and an outer discharge hole formed to face the inner discharge hole and to penetrate a lower point of the process receiver. The above emission control means, A functional yogurt manufacturing device characterized by comprising an inner opening / closing stopper for opening / closing the inner discharge port, an outer opening / closing stopper for opening / closing the outer discharge port, a connecting support member for connecting the inner opening / closing stopper and the outer opening / closing stopper to allow them to be raised / lowered as a unit, and an elevating cylinder connected to the lower portion of the outer opening / closing stopper for raising / lowering the inner opening / closing stopper and the outer opening / closing stopper.