Method and device for determining the degree of dementia of a user

Abstract

The present invention is used to determine the degree of dementia of a user. Content is output through a user terminal, the user's voice related to the acquired content is received through a microphone of the user terminal, the voice is visualized to generate a spectrogram, and the degree of dementia of the user is determined based on the spectrogram through a convolutional neural network (CNN) and a deep neural network (DNN).

Description

Technical Field

[0001] The present invention relates to a technology for determining the degree of dementia of a user, and in particular, to a device and method for determining the degree of dementia of a user based on user voice. Background Art

[0002] Dementia can prevent or delay further decline in cognitive function through early diagnosis and appropriate treatment, but there are currently problems with early diagnosis of this disease. Because it is necessary to visit specialized medical institutions such as hospitals, and because a relatively large number of patients who are admitted to the hospital due to worsening sensory amnesia have already developed mild cognitive impairment (MCI) or Alzheimer's disease (AD), neurocognitive function tests (SNSB-II, CERAD-K, etc.) used for diagnosis require medical personnel with sufficient experience and skills to obtain high reliability, and magnetic resonance imaging (MRI) or single photon emission scanning (SPECT), positron emission tomography (PET), cerebrospinal fluid examination, etc. are not only expensive to diagnose, but also cause great inconvenience to patients receiving diagnosis. Summary of the invention

[0003] Technical issues

[0004] An object of the present invention is to provide a method and apparatus for determining the degree of dementia of a user.

[0005] Another object of the present invention is to provide an apparatus and method for determining the degree of dementia of a user based on the user's voice.

[0006] Technical Solution

[0007] According to one embodiment, the method for determining the degree of dementia provided by the present invention is executed by an electronic device and is used to determine the degree of dementia of a user, and includes the following steps: outputting a first content pre-produced for determining the degree of dementia of the user through a user terminal; receiving a first voice of the user related to the acquired first content through a microphone of the user terminal; outputting a pre-produced second content through the user terminal; receiving a second voice of the user related to the acquired second content through the microphone; generating a first spectrogram by visualizing at least one characteristic of the first voice; generating a second spectrogram by visualizing at least one characteristic of the second voice; inputting the first spectrogram into a pre-updated first convolutional neural network (CNN) to generate a preset number of multiple first features related to the first voice; inputting the second spectrogram into a pre-updated second convolutional neural network to generate a preset number of multiple second features related to the second voice; determining a preset number of multiple target features from the multiple first features and the multiple second features; and inputting the multiple target features into a pre-updated deep neural network (DNN) to determine the degree of dementia of the user, and outputting the determined dementia degree through the user terminal.

[0008] The first content may include instructions for receiving the first voice.

[0009] The first content may be one of content that allows the user to follow a sentence, content that allows the user to guess the name of an output image, content that explains an output image, content for judging language fluency, content for numerical calculation, and content that guides story telling.

[0010] The step of generating a first spectrogram by visualizing at least one characteristic of the first speech may include the step of generating the first spectrogram related to the first speech by using an audio feature extraction tool (librosa tool).

[0011] The size of the first spectrogram may be the same as the size of the second spectrogram.

[0012] The first convolutional neural network mentioned above can be pre-updated based on the VGG16 model.

[0013] The first convolutional neural network includes an input layer and five convolutional layer blocks, but does not include a fully connected layer and a softmax, thereby generating a plurality of first features related to the first spectrogram.

[0014] The above-mentioned method for determining the degree of dementia may also include the step of updating the above-mentioned first convolutional neural network.

[0015] The step of updating the above-mentioned first convolutional neural network may include the following steps: receiving a first test speech of a test user related to the above-mentioned first content; generating a first test spectrogram by visualizing at least one feature of the above-mentioned first test speech, and marking the above-mentioned first test spectrogram with the benchmark truth (GT, ground truth) dementia level of the above-mentioned test user; inputting the above-mentioned first test spectrogram into the first fully convolutional neural network to determine the first test dementia level of the above-mentioned test user, the above-mentioned first fully convolutional neural network including an input layer, at least one convolutional layer block, a fully connected layer and a softmax; and updating the above-mentioned first fully convolutional neural network based on the above-mentioned first test dementia level and the above-mentioned benchmark truth dementia level, the above-mentioned first convolutional neural network only including the above-mentioned input layer and the above-mentioned at least one convolutional layer block among the multiple layers of the updated above-mentioned first fully convolutional neural network.

[0016] The above-mentioned method for determining the degree of dementia may also include the following step: after multiple convolutional neural networks including the above-mentioned first convolutional neural network and the above-mentioned second convolutional neural network are updated, updating the above-mentioned deep neural network.

[0017] The step of updating the above-mentioned deep neural network may include the following steps: determining a preset number of multiple test target features from a preset number of multiple first test features generated based on a first test spectrogram and a preset number of second test features generated based on a plurality of second test spectrograms, wherein the multiple test target features are marked with a baseline truth dementia level of the test user; inputting the multiple test target features into the above-mentioned deep neural network to determine a second test dementia level of the test user; and updating the above-mentioned deep neural network based on the second test dementia level and the baseline truth dementia level.

[0018] The step of updating the above-mentioned deep neural network also includes the step of verifying the above-mentioned test target features through the K-fold cross-validation method, where the above-mentioned K is a natural number greater than 2. If multiple of the above-mentioned test target features are not verified, the above-mentioned first convolutional neural network and the above-mentioned second convolutional neural network can be updated again.

[0019] The step of verifying the multiple test target features through the K-fold cross-validation method may include the following steps: dividing the multiple test target feature sets into the K groups; updating the K initial deep neural networks based on the K groups to generate K test deep neural networks; and verifying the multiple test target features based on the accuracy of the K test deep neural networks.

[0020] According to another embodiment, the electronic device provided by the present invention is used to determine the degree of dementia of a user, and includes: a memory having a program for determining the degree of dementia of the user recorded therein; and a processor for executing the program, wherein the program executes the following steps: outputting a first content pre-produced for determining the degree of dementia of the user through a user terminal; receiving a first voice of the user related to the acquired first content through a microphone of the user terminal; outputting a pre-produced second content through the user terminal; receiving a second voice of the user related to the acquired second content through the microphone; generating a first spectrogram by visualizing at least one characteristic of the first voice; generating a second spectrogram by visualizing at least one characteristic of the second voice; inputting the first spectrogram into a pre-updated first convolutional neural network (CNN) to generate a preset number of multiple first features related to the first voice; inputting the second spectrogram into a pre-updated second convolutional neural network to generate a preset number of multiple second features related to the second voice; determining a preset number of multiple target features from the multiple first features and the multiple second features; and inputting the second spectrogram into a pre-updated deep neural network (DNN) to generate a preset number of multiple first features related to the first voice. network) inputs a plurality of the target features to determine the user's dementia degree, and outputs the determined dementia degree through the user terminal.

[0021] According to another embodiment, the convolutional neural network updating method provided by the present invention is executed by an electronic device, and is used to update the convolutional neural network used to determine the user's dementia level, which includes the following steps: outputting a first content pre-produced for determining the user's dementia level through a user terminal; receiving a first test voice of a test user related to the above-mentioned first content; generating a first test spectrogram by visualizing at least one characteristic of the above-mentioned first test voice, and the above-mentioned first test spectrogram is marked with the benchmark truth (GT, ground truth) dementia level of the above-mentioned test user; inputting the above-mentioned first test spectrogram into a fully convolutional neural network to determine the test dementia level of the above-mentioned test user, the above-mentioned full convolutional neural network includes an input layer, at least one convolutional layer block, a fully connected layer and a softmax; and updating the above-mentioned full convolutional neural network based on the above-mentioned test dementia level and the above-mentioned benchmark truth dementia level, the above-mentioned convolutional neural network only includes the above-mentioned input layer and the above-mentioned at least one convolutional layer block among the multiple layers of the updated above-mentioned full convolutional neural network.

[0022] According to another embodiment, the electronic device provided by the present invention is used to update a convolutional neural network (CNN) used to determine the user's dementia level, and includes: a memory recording a program for updating the above-mentioned convolutional neural network; and a processor for executing the above-mentioned program, wherein the above-mentioned program performs the following steps: outputting a first content pre-produced for determining the user's fever dementia level through a user terminal; receiving a first test voice of a test user related to the above-mentioned first content; generating a first test spectrogram by visualizing at least one characteristic of the above-mentioned first test voice, wherein the above-mentioned first test spectrogram is marked with a reference truth (GT, groundtruth) dementia level of the above-mentioned test user; inputting the above-mentioned first test spectrogram into a fully convolutional neural network to determine the first test dementia level of the above-mentioned test user, wherein the above-mentioned fully convolutional neural network includes an input layer, at least one convolutional layer block, a fully connected layer and a softmax; and updating the above-mentioned fully convolutional neural network based on the above-mentioned test dementia level and the above-mentioned reference truth dementia level, wherein the above-mentioned convolutional neural network only includes the above-mentioned input layer and the above-mentioned at least one convolutional layer block among the multiple layers of the updated above-mentioned fully convolutional neural network.

[0023] Effects of the Invention

[0024] The present invention can provide a device and method for determining the degree of dementia of a user.

[0025] The present invention can provide a device and method for determining the degree of dementia of a user based on the user's voice. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a structural diagram of a system for determining the degree of dementia of a user according to an example of the present invention.

[0027] Figure 2 This figure shows a screen output to a user terminal in order to determine the degree of dementia of the user according to an example of the present invention.

[0028] Figure 3 4 is a structural diagram of an electronic device for determining the degree of dementia of a user according to an embodiment of the present invention.

[0029] Figure 4 The present invention is a flowchart of a method for determining the degree of dementia of a user according to an embodiment of the present invention.

[0030] Figure 5 The following shows the pre-made contents of one example of the present invention.

[0031] Figure 6 The following shows the pre-made contents according to another example of the present invention.

[0032] Figure 7A spectrogram generated based on speech according to an example of the present invention is shown.

[0033] Figure 8 An example of a fully convolutional neural network and a partial convolutional neural network that can determine the degree of dementia of a user according to the present invention is shown.

[0034] Fig. 9 Features generated for each of a plurality of speech sounds and target features determined based on the features are shown as an example of the present invention.

[0035] Fig.10 An example of a deep neural network for determining the degree of dementia of a user according to the present invention is shown.

[0036] Fig.11 Two classification steps are shown in one example of the present invention to improve the accuracy of determining the degree of dementia.

[0037] Fig.12 This is a flowchart of a method for updating a fully convolutional neural network according to an example of the present invention.

[0038] Fig.13 This is a flowchart of a method for updating a deep neural network according to an example of the present invention.

[0039] Fig.14 The flowchart of the method for determining the characteristics of a test target according to an example of the present invention.

[0040] Fig.15 The figure is a flow chart of a method for screening sub-features according to an example of the present invention.

[0041] Fig.16 The flowchart of a method for verifying test target characteristics according to an example of the present invention.

[0042] Fig.17 This is a K-fold cross-validation method for verifying target features according to an example of the present invention. DETAILED DESCRIPTION

[0043] Hereinafter, multiple embodiments are described in detail with reference to the accompanying drawings. However, the scope of protection of the invention is not limited or restricted to these embodiments. The same reference numerals recorded in each of the drawings represent the same components.

[0044] The following description may apply various changes to the embodiments. The embodiments described below do not limit the implementation methods, and should be understood to include all changes, equivalent technical solutions and alternative technical solutions resulting therefrom.

[0045] In the embodiments, the terms used are only used to illustrate specific embodiments and are not intended to limit the embodiments. Unless otherwise clearly indicated in the context, singular expressions include plural expressions. In this specification, terms such as "including" or "having" are only used to specify the existence of features, numbers, steps, operations, structural elements, components or combinations thereof recorded in this specification, and do not preclude the existence or additional possibility of one or more other features, numbers, steps, operations, structural elements, components or combinations thereof.

[0046] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs. Terms defined in commonly used dictionaries should be interpreted as having the same meanings as those in the context of the relevant technology and should not be interpreted in an idealized or overly formal sense unless clearly defined in this specification.

[0047] Furthermore, in the process of describing with reference to the drawings, the same structural elements are given the same reference numerals regardless of the reference numerals and repeated descriptions thereof are omitted. In the process of describing the embodiments, when it is determined that the detailed description of the related known technology may unnecessarily confuse the subject matter of the embodiments, its detailed description will be omitted.

[0048] Figure 1 This is a structural diagram of a system for determining the degree of dementia of a user according to an example of the present invention.

[0049] According to an embodiment of the present invention, the user dementia degree determination system may include: an electronic device 110 for determining the user's dementia degree; a user terminal 120 for outputting content; and a monitoring terminal 130 of a medical institution. For example, the electronic device 110 may be a server.

[0050] The electronic device 110 may provide the user terminal 120 with content that is pre-made for determining the degree of dementia of the user. For example, the content may be content for obtaining a voice from the user. Figure 5 and Figure 6 Detailed description.

[0051] The user terminal 120 can communicate with the electronic device 110 through an offline connection or an online connection. The electronic device 110 provides content to the user terminal 120, and the user terminal 120 outputs the content to the user through a display. The user terminal 120 can obtain the user's voice as a reaction to the content through a microphone and transmit the obtained voice to the electronic device 110.

[0052] The electronic device 110 may determine the user's dementia level based on the obtained user voice, and transmit the determined dementia level to the user terminal 120 .

[0053] The user terminal 120 may be a mobile terminal such as a tablet computer, a smart phone, etc. When the user terminal 120 is a mobile terminal, the user can measure the degree of dementia at a low cost without being restricted by time and place.

[0054] The electronic device 110 may transmit the user's voice and the degree of dementia to the monitoring terminal 130. For example, the user of the monitoring terminal 130 may be a doctor, and the doctor may adjust the content provided to the user based on the transmitted information.

[0055] Below, refer to Figures 2 to 17 Details the method used to determine the user's level of dementia.

[0056] Figure 2 This figure shows a screen output to a user terminal in order to determine the degree of dementia of the user according to an example of the present invention.

[0057] The following screen ( Figure 2 210 to 240 in the figure may be screens of an application for determining the degree of dementia. For example, a user of the electronic device 110 may create an application for distribution, and the user may execute the application through the user terminal 120.

[0058] The first screen 210 is a start screen of the application.

[0059] The second screen 220 shows functions supported by the application.

[0060] The third screen 230 is an example of content provided to the user. A plurality of contents may be provided to the user.

[0061] The third screen 240 is the determined user's dementia level. For example, normal, mild cognitive impairment (MCI), or Alzheimer's disease (AD) determined as the user's dementia level may be output. In addition to the attention level of each disease, a comprehensive judgment may also be output.

[0062] Figure 3 4 is a structural diagram of an electronic device for determining the degree of dementia of a user according to an embodiment of the present invention.

[0063] The electronic device 300 includes a communication unit 310, a processor 230, and a memory 330. For example, the electronic device 300 may be a reference Figure 1 An electronic device 110 is illustrated.

[0064] The communication unit 310 is connected to the processor 320 and the memory 330 for sending and receiving data. The communication unit 310 is connected to other external devices and can send and receive data. In the following content, the expression of sending and receiving "A" refers to sending and receiving "information (information) or data representing A".

[0065] The communication unit 310 may be implemented by a circuit within the electronic device 300. For example, the communication unit 310 may include an internal bus and an external bus. As another example, the communication unit 310 may be a structural element that connects the electronic device 300 to an external device. The communication unit 310 may be an interface. The communication unit 310 receives data from an external device and may transmit data to the processor 320 and the memory 330.

[0066] The processor 320 is used to process the data received by the communication unit 310 and the data stored in the memory 330. The "processor" can be a data processing device, which is implemented by hardware having a circuit, and the above circuit includes a physical structure for performing a target operation. For example, the target operation may include a code or instructions contained in a program. For example, a data processing device implemented by hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA).

[0067] The processor 320 may execute computer readable code (eg, software) stored in a memory (eg, the memory 330 ) and instructions called by the processor 320 .

[0068] The memory 330 is used to store data received by the communication unit 310 and data processed by the processor 320. For example, the memory 330 can store a program (or application, software). The stored program is encoded to be able to determine the degree of dementia of the user, and therefore, can be a set of processor executable syntax.

[0069] According to an embodiment of the present invention, the memory 330 may include a volatile memory, a non-volatile memory, a random access memory (RAM), a flash memory, a hard disk drive, and an optical disk drive.

[0070] The memory 330 stores an instruction set (eg, software) for operating the electronic device 300 . The instruction set for operating the electronic device 300 may be executed by the processor 320 .

[0071] The communication unit 310, the processor 320 and the memory 330 will be referred to below. Figures 4 to 17 Detailed description.

[0072] Figure 4 The present invention is a flowchart of a method for determining the degree of dementia of a user according to an embodiment of the present invention.

[0073] The following steps 410 to 450 are referred to as Figure 3 The electronic device 300 of the description is executed.

[0074] In step 410, the electronic device 300 outputs the content pre-made for determining the degree of dementia of the user through the user terminal (e.g., the user terminal 120). The content is output to the user terminal, and the user responds to the content. The user terminal can generate a voice as the above response through a microphone. The generated voice can be a data file.

[0075] When multiple contents are provided to the user, voices may be generated for the multiple contents respectively. The multiple contents of one example of the present invention can be described by the following Table 1.

[0076] Table 1

[0077]

[0078]

[0079] In step 420, the electronic device 300 receives the user's voice related to the content through the microphone of the user terminal. In the case where a plurality of contents are provided, a plurality of voices may be received.

[0080] When multiple contents are formed, step 410 and step 420 may be repeatedly performed. By repeatedly performing step 410 and step 420, voices of multiple contents may be received. For example, when the multiple contents include the first to eleventh contents, the first to eleventh voices corresponding thereto may be received.

[0081] In step 430, the electronic device 300 may generate a spectrogram of the speech by visualizing at least one characteristic of the received speech. For example, the electronic device 300 may generate a spectrogram of the speech by using an audio feature extraction tool (librosatool). The spectrogram may be a mel spectrogram.

[0082] For example, for the first to eleventh speech, the first to eleventh spectrograms can be generated respectively. Figure 7 Detailed description.

[0083] In step 440, the electronic device 300 generates a preset number of features of the speech by inputting the spectrogram into a pre-updated convolutional neural network (CNN) corresponding to the spectrogram. The convolutional neural network used to generate features may be different based on the content. For example, when there are 11 contents, there are convolutional neural networks corresponding to the 11 contents respectively, and the 11 convolutional neural networks may be named as a convolutional network set. In the following content, the term "update" may include the meaning of the term "training" and may be used interchangeably.

[0084] According to one embodiment of the present invention, the convolutional neural network may be pre-updated based on the VGG16 model. The convolutional neural network may be part of a fully convolutional neural network including an input layer, at least one convolutional layer block, a fully connected layer, and a softmax. For example, the convolutional neural network includes an input layer and at least one convolutional layer block, but may not include a fully connected layer and a softmax. Since the convolutional neural network does not include a fully connected layer and a softmax, the degree of dementia is not output as a result of the input spectrogram, but a preset number of features for calculating the degree of dementia may be output. For the fully convolutional neural network and the partial convolutional neural network, the following will refer to Figure 8 Detailed description.

[0085] For example, the electronic device 300 may input a first spectrogram into a pre-updated first convolutional neural network to generate a preset number of first features related to the first speech, and may input a second spectrogram into a pre-updated second convolutional neural network to generate a preset number of second features related to the second speech. As a specific example, when 11 speech sounds are received, if 4608 features are generated for one speech sound, a total of 50688 features may be generated.

[0086] In step 450, the electronic device 300 may determine a target feature from the features of the plurality of voices. The determined target feature may be a label for diagnosing dementia. As a specific example, 10 features may be determined from a total of 50,688 features. The features determined as target features may be predetermined as labels. The labels may be determined by referring to Figures 12 to 16 The convolutional neural network update steps and deep neural network (DNN) update steps described are predetermined.

[0087] In step 460, the electronic device 300 may input the target feature into the pre-updated deep neural network to determine the user's dementia level. For example, the determined dementia level may be normal or abnormal. Abnormal may include mild cognitive impairment (MCI) or Alzheimer's disease (AD). In order to further accurately determine the user's dementia level, two classification steps may be performed. Fig.11 Detailed description of the two classification steps performed to improve the accuracy of determining the degree of dementia.

[0088] In step 470 , the electronic device 300 may output the determined dementia degree through the user terminal.

[0089] Figure 5 The following shows the pre-made contents of one example of the present invention.

[0090] For example, the content 500 provided to the user may be content for guessing the names of the output images 520, 530, 540. In addition to the plurality of images 520, 530, 540, the content 500 may include instructions 510 (instructions) for the user to speak a voice related to the content 500. The instructions 510 may be expressed as a task or as a voice output. As the user speaks the names of the images 520, 530, 540, a voice may be generated.

[0091] Figure 6 The following shows the pre-made contents according to another example of the present invention.

[0092] Different from Figure 5 In the example shown, the content 600 provided to the user may be content that explains the output image 620. In addition to the image 620, the content 600 may include instructions 610 that cause the user to speak a voice related to the content 600. The instructions 610 may be expressed as a task or as a voice output. As the user explains or describes the situation of the image 620, a voice may be generated.

[0093] Figure 7 A spectrogram generated based on speech according to an example of the present invention is shown.

[0094] According to one embodiment of the present invention, the electronic device 300 can generate a spectrogram 700 of speech through an audio feature extraction tool. The horizontal axis of the spectrogram 700 is the time axis, and the vertical axis can be the frequency axis. The spectrogram 700 represents the amplitude difference based on the change of the time axis and the frequency by the difference of the printing density / display color. The display color of the corresponding position can be determined based on the size of the amplitude difference of the change. For example, a display color legend 710 based on the size of the amplitude difference can be output together with the spectrogram 700. In order to display a certain color, the R channel pixel value, the G channel pixel value, and the B channel pixel value of the corresponding coordinate can be determined.

[0095] For multiple voices, multiple spectrograms can be generated respectively. For example, a first spectrogram related to a first voice is generated, and a second spectrogram can be generated for a second voice. Although the ratio of the time axis to the frequency axis of the spectrogram can vary with the total time of each voice, the generated spectrograms can have the same size. For example, the size of the first spectrogram and the size of the second spectrogram can be the same, both 100×100.

[0096] Figure 8 An example of a fully convolutional neural network and a partial convolutional neural network that can determine the degree of dementia of a user according to the present invention is shown.

[0097] According to one embodiment of the present invention, the fully convolutional neural network 800 includes an input layer 810, a first convolutional layer block 820, a second convolutional layer block 830, a third convolutional layer block 840, a fourth convolutional layer block 850, a fifth convolutional layer block 860, a fully connected layer 870 and a softmax 880. The convolutional layer block may include at least one convolutional layer and a pooling layer.

[0098] The fully convolutional neural network 800 can be obtained by referring to the following Fig.12 The fully convolutional neural network update method described above is used to update the fully convolutional neural network. Fully convolutional neural networks with different contents can be updated in advance.

[0099] The partial convolutional neural network 805 only includes the input layer 810, the first convolutional layer block 820, the second convolutional layer block 830, the third convolutional layer block 840, the fourth convolutional layer block 850, and the fifth convolutional layer block 860, and may not include the fully connected layer 870 and the softmax 880. That is, the partial convolutional neural network 805 may be a convolutional neural network that removes the fully connected layer 870 and the softmax 880 from the full convolutional neural network 800 after the update of the full convolutional neural network 800 is completed. For example, in reference to Figure 3 In the above step 440 described above, the convolutional neural network may be a partial convolutional neural network 805.

[0100] Some convolutional neural networks 805 do not include the fully connected layer 870, and therefore can output multiple features of the spectrogram.

[0101] Fig. 9 Features generated for each of a plurality of speech sounds and target features determined based on the features are shown as an example of the present invention.

[0102] According to one embodiment of the present invention, a predetermined number of target features related to the speech are generated by a target convolutional neural network corresponding to the target speech. For example, the predetermined number of features may be 4608. When the number of the plurality of speech is n, the number of the generated overall features 900 may be 4608×n.

[0103] A preset number of target features 910 are determined from the overall features 900. The determined target features 910 may be preset labels for determining the degree of dementia. Fig.13 Step 1310 is described in detail.

[0104] Fig.10 An example of a deep neural network for determining the degree of dementia of a user according to the present invention is shown.

[0105] According to one embodiment of the present invention, the deep neural network for determining the degree of dementia of the user may include an input layer 1010, at least one hidden layer 1020, 1030, 1040 and an output layer 1050. For example, the deep neural network may be obtained by referring to Fig.13 A deep neural network updating method is described to update a deep neural network.

[0106] As an output for the target feature 910 input, the deep neural network may output the user's dementia level. The deep neural network may output one of a plurality of preset dementia levels. For example, the preset plurality of dementia levels may include normal, mild cognitive impairment (MCI), and Alzheimer's disease (AD).

[0107] Fig.11 Two classification steps are shown in one example of the present invention to improve the accuracy of determining the degree of dementia.

[0108] The method of stepwise determining the dementia degree through a plurality of models can improve the accuracy of determining the dementia degree, compared with the method of determining one of a plurality of dementia degrees through one model.

[0109] For example, compared to determining one of normal, mild cognitive impairment (MCI) and Alzheimer's disease (AD) through a model, the first classification step is used to determine normal or abnormal (mild cognitive impairment (MCI) and Alzheimer's disease (AD)), and the second classification step can be used to determine mild cognitive impairment (MCI) or Alzheimer's disease (AD).

[0110] In order to use the above method, it is necessary to prepare in advance a first convolutional neural network set and a first deep neural network for the first classification step and a second convolutional neural network set and a second deep neural network for the second classification step.

[0111] For example, steps 410 to 460 are performed for the first classification step, and when the user's dementia level is determined to be abnormal by the first classification step, steps 440 to 470 may be performed for the second classification step. When the user's dementia level is determined to be normal by the first classification step, the second classification step does not need to be performed. The first convolutional neural network set and the first deep neural network used in the first classification step and the second convolutional neural network set and the second deep neural network used in the second classification step may be different from each other.

[0112] Fig.12 This is a flowchart of a method for updating a fully convolutional neural network according to an example of the present invention.

[0113] According to one embodiment of the present invention, when executing Figure 4 Before the above step 410, the following step 1200 will be performed first. Step 1200 involves a full convolutional neural network update method, which may include the following steps 1210 to 1250.

[0114] In step 1210, the electronic device 300 outputs the content prepared in advance for determining the dementia degree of the user to the test user. For example, the electronic device 300 may output the content through the user terminal of the test user.

[0115] The test user may be a person whose degree of dementia is determined by a doctor's specialized diagnosis. For example, the test user may be a normal person, or may be a person suffering from mild cognitive impairment (MCI) or Alzheimer's disease (AD).

[0116] In step 1220, the electronic device 300 may receive a test voice of a user related to the test content through a microphone of the user terminal. In the case where a plurality of contents are provided, a plurality of test voices may be received.

[0117] In step 1230, the electronic device 300 generates a test spectrogram of the test speech by visualizing at least one characteristic of the test speech. The test spectrogram may be labeled with a ground truth (GT) dementia level of the test user.

[0118] In step 1240, the electronic device 300 may input the test spectrogram into a fully convolutional neural network to determine the test dementia degree of the test user. The fully convolutional neural network includes an input layer, at least one convolutional layer block, a fully connected layer, and a softmax. For example, the initial version of the fully convolutional neural network may be a VGG16 model.

[0119] Since the fully convolutional neural network includes a fully connected layer and a softmax, the fully convolutional neural network can determine the test dementia level. For example, the determined test dementia level may include normal, mild cognitive impairment (MCI) or Alzheimer's disease (AD).

[0120] According to one embodiment of the present invention, the first fully convolutional neural network corresponding to the first content can determine the test dementia level of the test user based only on the first test spectrogram, and the second fully convolutional neural network corresponding to the second content can determine the test dementia level of the test user based only on the second test spectrogram.

[0121] In step 1250, the electronic device 300 updates the fully convolutional neural network based on the test dementia level and the baseline truth dementia level. For example, in the case where there is a difference between the dementia level and the baseline truth dementia level, in order to update the fully convolutional neural network, back-propagation may be performed to set the difference as an error value. The fully convolutional neural network update method may be supervised learning.

[0122] exist Figure 8 In one embodiment, when the fully convolutional neural network 800 includes an input layer 810, a first convolutional layer block 820, a second convolutional layer block 830, a third convolutional layer block 840, a fourth convolutional layer block 850, a fifth convolutional layer block 860, a fully connected layer 870 and a softmax 880, only the third convolutional layer block 840, the fourth convolutional layer block 850, and the fifth convolutional layer block 860 may be updated without updating the remaining layers.

[0123] According to one embodiment of the present invention, the fully convolutional neural network can be repeatedly updated by multiple test users, and when the output accuracy of the updated fully convolutional neural network is above a preset critical value, the update of the fully convolutional neural network can be terminated.

[0124] According to one embodiment of the present invention, when multiple models are Fig.11 When the above method shown in the figure gradually determines the dementia model, the first full convolutional neural network set and the second full convolutional neural network set used for each classification step can be updated separately according to each classification step. For example, the first full convolutional neural network set is updated to determine normal or abnormal (mild cognitive impairment (MCI) and Alzheimer's disease (AD)), and the second full convolutional neural network set can be updated to determine mild cognitive impairment (MCI) or Alzheimer's disease (AD).

[0125] In step 440, the convolutional neural network used may be a neural network obtained by removing the fully connected layer and the softmax from the full convolutional neural network after the update of the full convolutional neural network is completed. That is, in step 440, the convolutional neural network used may be used as a feature extractor for the corresponding spectrogram.

[0126] Fig.13 This is a flowchart of a method for updating a deep neural network according to an example of the present invention.

[0127] According to one embodiment of the present invention, the following step 1300 involves a method for updating a deep neural network, Fig.12 After executing the above step 1200 in the manner shown in FIG. Figure 4 The above steps 410 shown above may be performed first. For example, after the update of the full convolutional neural network (or convolutional neural network) is completed, step 1300 may be performed.

[0128] Step 1300 may include the following steps 1310 to 1240 .

[0129] In step 1310, the electronic device 300 may determine a preset number of test target features from a preset number of first test features generated by a first convolutional neural network based on a first test spectrogram and a preset number of second test features generated by a second convolutional neural network based on a second test spectrogram. Although only the first test feature and the second test feature are recorded, for example, in the case where n test spectrograms are generated for n contents, the test target feature may be determined from the first test feature to the nth test feature. The test target feature may be a label for determining the degree of dementia. For the method of determining the test target feature, reference will be made to Fig.14 and Fig.15 Detailed description.

[0130] The test target feature is labeled with the test user's baseline truth dementia level.

[0131] In step 1320 , the electronic device 300 may verify the determined test target feature. For example, the test target feature may be verified by a K-fold cross validation method.

[0132] For the method of verifying the characteristics of the test target, the following will refer to Fig.15 and Fig.16 Detailed description.

[0133] In the case where the test target feature is verified, step 1330 may be performed. In the case where the test target feature is not verified, it is deemed that there is no need to update the convolutional neural network again, and therefore step 1200 may be performed again.

[0134] In step 1330, the electronic device 300 may input the test target feature into the deep neural network to determine the test dementia degree of the test user. In order to distinguish it from the test dementia degree determined by step 1240, the test dementia degree of step 1240 is named as the first test dementia degree, and the test dementia degree of step 1330 is named as the second test dementia degree. In the case of performing step 1330 for the first time, the deep neural network used may be an initial deep neural network or a basic deep neural network.

[0135] In step 1340, the electronic device 300 may update the deep neural network based on the second test dementia level and the baseline truth dementia level. For example, when there is a difference between the second test dementia level and the baseline truth dementia level, in order to update the deep neural network, back-propagation may be performed to set the difference as an error value. The deep neural network update method may be supervised learning.

[0136] According to one embodiment of the present invention, the deep neural network can be repeatedly updated by multiple test users, and when the output accuracy of the updated deep neural network is above a preset critical value, the update of the deep neural network can be terminated.

[0137] According to one embodiment of the present invention, when multiple models are Fig.11 When the above method shown in the figure gradually determines the dementia model, the first deep neural network and the second deep neural network used in each classification step can be updated separately according to each classification step. For example, the first deep neural network is updated to determine normal or abnormal (mild cognitive impairment (MCI) and Alzheimer's disease (AD)), and the second deep neural network can be updated to determine mild cognitive impairment (MCI) or Alzheimer's disease (AD).

[0138] Fig.14 The flowchart of the method for determining the characteristics of a test target according to an example of the present invention.

[0139] According to one embodiment of the present invention, referring to Fig.13 The illustrated step 1310 may include the following steps 1410 to 1450 .

[0140] In step 1410, the entire test feature including the first test feature and the second test feature is divided into a plurality of sub-feature sets. For example, when the entire test feature is 50688, the sub-feature sets may include 1000 test features respectively, and the 51st sub-feature set may include 688 test features. The entire test features may be assigned index numbers respectively, and the 1st sub-feature set includes the 1st test feature to the 1000th test feature.

[0141] In step 1420, a portion of the plurality of sub-feature sets (51) is screened. For example, 10 sets may be selected from the 1st to the 51st sub-feature sets. The selected 10 sub-feature sets include a total of 10,000 test features. The method for screening a portion of the sub-feature sets will be described below with reference to Fig.15 Detailed description.

[0142] In step 1430, the screened sub-features (eg, 10,000) may be divided into a plurality of sub-features. For example, when the screened features are 10,000, the sub-feature sets (50) may each include 200 test features.

[0143] In step 1440, a portion of the plurality of sub-feature sets (50) is screened. For example, 10 sets may be selected from the 1st sub-feature set to the 51st sub-feature set. The selected 10 sub-feature sets include a total of 2000 test features. The detailed description of step 1440 may also be applied to the following references. Fig.15 Step 1420 is described.

[0144] In step 1450, the test features included in the screened sub-feature set are determined as test target features. The determined test target features can be identified by respective index numbers.

[0145] The determined test target features can be used as labels for determining the degree of dementia of the user. For example, when the 56th feature, the 100th feature, and the 760th feature in the first feature and the 1st feature and the 2400th feature in the second feature are determined as test target features, Figure 4 The target features determined in the above step 450 include the 56th feature, the 100th feature, and the 760th feature in the first feature and the 1st feature and the 2400th feature in the second feature.

[0146] Fig.15 The figure is a flow chart of a method for screening sub-features according to an example of the present invention.

[0147] According to one embodiment of the present invention, referring to Fig.14 The illustrated step 1420 may include the following steps 1510 to 1540 .

[0148] In order to determine the test target characteristics, multiple user-related data is required. In the following content, the process of determining the test target characteristics will be explained using the data of 1,000 users as an example. The correct answer is also set in the data of the 1,000 users.

[0149] For example, 1000 users may be classified into 600 training data users, 200 authentication data users, and 200 test data users. For the 600 users, 50688 features of the 1st to 11th voices may be generated respectively, and 600 1st sub-feature sets with specific index numbers (e.g., 1 to 1000) may be generated. For example, 600 1st to 51st sub-feature sets for training data are generated. Similarly, 200 1st to 51st sub-feature sets for authentication data are generated, and 200 1st to 51st sub-feature sets for test data are generated.

[0150] In step 1510, an epoch of the initial deep neural network is performed based on 600 first sub-feature sets (first training data) as training data and 200 first sub-feature sets (first authentication data) as authentication data. The weighted values ​​of the edges or parameters of the nodes in the deep neural network are adjusted based on the 600 first sub-feature sets. The input first authentication data related results are output by the deep neural network with the adjusted weighted values. The output results may be 200. The operator may adjust the number of times the predetermined epoch is performed for learning with reference to the 200 output results.

[0151] In step 1520, a preset number of epochs are executed to the deep neural network. For example, 30 epochs may be executed. When the preset number of epochs are executed, one learning (or training) may be considered completed.

[0152] In step 1530, the first learning accuracy may be calculated based on the 200 first sub-feature sets of the test data (the first test data). For example, the first test data is input to the pre-learned deep neural network, and the accuracy of the 200 results may be calculated as the first learning accuracy.

[0153] As steps 1510 to 1530 are repeatedly performed for a preset number of times, additional learning accuracy rates may be calculated. Since the initial deep neural networks provided in step 1510 are different from each other, the results of deep neural network learning may be changed, thereby changing the learning accuracy rate based on multiple learnings. In the case where steps 1510 to 1530 are repeatedly performed 10 times, the first learning accuracy rate to the tenth learning accuracy rate may be calculated.

[0154] In step 1540 , a first average learning accuracy of the first training data is calculated. For example, the average of the first learning accuracy to the tenth learning accuracy may be calculated as the first average learning accuracy.

[0155] For example, when steps 1510 to 1540 are performed on a first sub-feature set including features from index number 1 to index number 1000, a first average learning accuracy of the first sub-feature set may be calculated.

[0156] As another example, when steps 1510 to 1540 are performed on a second sub-feature set including features from index number 1001 to index number 2000, a second average learning accuracy of the second sub-feature set may be calculated.

[0157] For the 51 sub-feature sets, the first average learning accuracy rate to the 51st average learning accuracy rate can be calculated. Among the 51 average learning accuracy rates, the top 10 sub-feature sets can be screened.

[0158] As another example, after 51 sub-feature sets are divided into a preset number of groups, the group average learning accuracy of the corresponding groups can be calculated. As a portion of the groups is selected from the plurality of groups based on the group average learning accuracy, the sub-feature sets within the selected groups can be screened.

[0159] In the case of selecting 10 sub-feature sets, 10000 index numbers are selected. Since the filtering is performed according to the sub-feature sets, the geographical characteristics between the features generated by the convolutional neural network based on the spectrogram can be automatically considered.

[0160] The descriptions of steps 1510 to 1540 are also applicable to the detailed description of step 1440 .

[0161] Fig.16 The flowchart of a method for verifying test target characteristics according to an example of the present invention.

[0162] According to one embodiment of the present invention, referring to Fig.13 The illustrated step 1320 may include the following steps 1610 to 1630 .

[0163] In step 1610, the electronic device 300 divides the test target feature set into K groups. The test target features determined for the test users can be defined as a set. For example, when there are 1,000 test users, there will be 1,000 test target feature sets, and the 1,000 sets can be divided into K groups. K is a natural number greater than 2. When K is 5, 5 groups each including 200 sets can be generated.

[0164] In step 1620, the electronic device 300 updates the initial deep neural network based on the K groups to generate K test deep neural networks. In the case of generating the first to fifth groups, the first test deep neural network is updated based on the second to fifth groups, the second test deep neural network is updated based on the first, third to fifth groups, the third test deep neural network is updated based on the first, second, fourth and fifth groups, the fourth test deep neural network is updated based on the first to third and fifth groups, and the fifth test deep neural network can be updated based on the first to fourth groups.

[0165] In step 1630, the electronic device 300 may verify the test target feature based on the accuracy of the K test deep neural networks. In the above embodiment, after the first group is input into the first test deep neural network to output the first group of results, the first accuracy of the output result may be calculated. Similarly, with respect to the second test deep neural network to the fourth test deep neural network, the second accuracy to the fourth accuracy may be calculated respectively.

[0166] When the calculated average value of the first to fifth accuracy rates is above the preset critical value, it can be determined that the test target feature has been verified. When the calculated average value of the first to fifth accuracy rates is less than the preset critical value, it can be determined that the test target feature has not been verified. In the case where the test target feature is not verified, the convolutional neural network used to extract the test feature can be updated again.

[0167] Fig.17 This is a K-fold cross-validation method for verifying target features according to an example of the present invention.

[0168] According to one example, the test target feature set 1710 can be divided into a first group 1701, a second group 1702, a third group 1703, a fourth group 1704, and a fifth group 1705. When the test target feature set 1710 includes 1000 sets, the first group 1701 to the fifth group 1705 each include 200 sets. Each set includes a test target feature of a specific test user.

[0169] The first test deep neural network 1720 may be updated based on the second to fifth groups 1702 to 1705. For example, the first test deep neural network 1720 may be updated 800 times based on 800 sets.

[0170] If the first group 1701 is input to the updated first test deep neural network 1720, the dementia degree of the test user of the first group 1701 may be determined. For example, the first test deep neural network 1720 may determine 200 second test dementia degrees of 200 sets.

[0171] The accuracy of the first test deep neural network 1720 can be calculated based on the 200 sets of each benchmark truth dementia level and the 200 second test dementia levels of the first group 1701. Similarly, the accuracy of the second test deep neural network to the fourth test deep neural network can be calculated. Finally, the test target feature can be verified based on the average of the accuracy calculation of the first test deep neural network to the fifth test deep neural network.

[0172] The device described above can be a hardware structural element, a software structural element and / or a combination of a hardware structural element and a software structural element. For example, the device and structural elements described in the embodiment can be implemented using one or more general-purpose computers or special-purpose computers of a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor (digital signal processor), a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor or any other device that can execute a response instruction. The processing device can execute an operating system (OS) and one or more software applications running on the above operating system. In addition, the processing device can also access, store, operate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device can be described as using one element, but it can be understood by a person of ordinary skill in the art that the processing device includes multiple processing elements and / or various types of processing elements. For example, the processing device can include multiple processors or include a processor and a controller. In addition, it can also be other processing configurations such as parallel processors.

[0173] Software may include a computer program, code, instructions, or a combination of more than one of them, and may configure a processing device to operate as required, or command the processing device independently or collectively. Software and / or data may be embodied as any type of machine, component, physical device, virtual device, computer storage medium or device, or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

[0174] The method according to the embodiment can be implemented and recorded in a computer-readable medium by a program instruction that can be executed by various computer devices. The above-mentioned computer-readable medium may include a single or multiple program instructions, data files, data structures, etc. The program instructions recorded on the above-mentioned medium may be specially designed and configured for the present embodiment, or may be known and available to the technicians in the field of computer software. Examples of computer-readable media include magnetic media (magnetic media) such as hard disks, floppy disks and tapes, optical recording media (optical media) such as CD-ROMs and DVDs, magneto-optical media (magneto-optical medium) such as floptical disks (floptical disks), and hardware devices such as ROMs, RAMs, flash memories, etc. that are specifically used to store and execute program instructions. Examples of program instructions include not only machine language codes such as those generated by compilers, but also high-level language codes that can be executed by computers using interpreters, etc. In order to perform the work of executing the embodiment, the above-mentioned hardware device may be configured to work by at least one software module, and vice versa.

[0175] Although the embodiments are described above with reference to the limited embodiments and drawings, a person skilled in the art of the present invention may make various modifications and variations based on the above. For example, the components of the system, structure, device, circuit, etc. that perform the techniques and / or combine or combine in a form different from the method described in an order different from the method described, or even if replaced or substituted by other structures or equivalent technical solutions, appropriate results can be achieved.

[0176] Therefore, other implementation modes, other examples and contents equivalent to the protection scope of the invention also fall within the protection scope of the present invention.

Claims

1. A method for determining the degree of dementia, executed by an electronic device, for determining the degree of dementia of a user, characterized in that: The steps include: outputting, through a user terminal, first content prepared in advance for determining a user's dementia degree; Receiving a first voice of the user related to the first content acquired through a microphone of the user terminal; outputting the pre-made second content through the user terminal; receiving a second voice of the user related to the second content acquired through the microphone; generating a first spectrogram by visualizing at least one characteristic of the first speech; generating a second spectrogram by visualizing at least one characteristic of the second speech; Inputting the first spectrogram into a pre-updated first convolutional neural network to generate a preset number of first features related to the first speech; Inputting the second spectrogram into a pre-updated second convolutional neural network to generate a preset number of second features related to the second speech; Determining a preset number of target features from the plurality of the first features and the plurality of the second features; as well as Inputting a plurality of the target features into a pre-updated deep neural network to determine the degree of dementia of the user, The above-mentioned dementia degree determined by the above-mentioned user terminal output, Among them, the first convolutional neural network is updated through the following steps: Receiving a first test voice of a test user related to the first content; generating a first test spectrogram by visualizing at least one characteristic of the first test speech, wherein the first test spectrogram is marked with a baseline truth dementia level of the test user; Inputting the first test spectrogram into a first fully convolutional neural network to determine a first test dementia degree of the test user, the first fully convolutional neural network comprising an input layer, at least one convolutional layer block, a fully connected layer and a softmax; and updating the first fully convolutional neural network based on the first test dementia level and the baseline truth dementia level, The above-mentioned first convolutional neural network only includes the above-mentioned input layer and the above-mentioned at least one convolutional layer block among the multiple layers of the updated above-mentioned first fully convolutional neural network.

2. The method for determining the degree of dementia according to claim 1, characterized in that: The first content includes an instruction for receiving the first voice. The first content is one of content that allows the user to repeat a sentence, content that allows the user to guess the name of an output image, content that explains an output image, content for judging language fluency, content for numerical calculation, and content that guides storytelling.

3. The method for determining the degree of dementia according to claim 1, characterized in that: The step of generating a first spectrogram by visualizing at least one characteristic of the first speech includes the following steps: generating the first spectrogram related to the first speech by an audio feature extraction tool.

4. The method for determining the degree of dementia according to claim 1, characterized in that: The first convolutional neural network mentioned above is pre-updated based on the VGG16 model.

5. The method for determining the degree of dementia according to claim 1, characterized in that: The above-mentioned first convolutional neural network includes an input layer and 5 convolutional layer blocks, but does not include a fully connected layer and a softmax, thereby generating multiple above-mentioned first features related to the above-mentioned first spectrogram.

6. The method for determining the degree of dementia according to claim 1, characterized in that: The method further includes the following step: updating the above-mentioned deep neural network after a plurality of convolutional neural networks including the above-mentioned first convolutional neural network and the above-mentioned second convolutional neural network are updated.

7. The method for determining the degree of dementia according to claim 6, characterized in that: The steps of updating the above deep neural network include the following steps: Determine a preset number of test target features from a preset number of first test features generated based on the first test spectrogram and a preset number of second test features generated based on the second test spectrogram, wherein the plurality of the test target features are marked with the baseline truth dementia level of the test user; Inputting a plurality of the test target features into the deep neural network to determine a second test dementia level of the test user; and The deep neural network is updated based on the second test dementia level and the baseline truth dementia level.

8. The method for determining the degree of dementia according to claim 7, characterized in that: The step of updating the above-mentioned deep neural network also includes the step of verifying the plurality of above-mentioned test target features by a K-fold cross-validation method, where the above-mentioned K is a natural number greater than 2, When multiple of the above-mentioned test target features are not verified, the above-mentioned first convolutional neural network and the above-mentioned second convolutional neural network are updated again.

9. The method for determining the degree of dementia according to claim 8, characterized in that: The step of verifying the multiple test target features by the K-fold cross validation method comprises the following steps: Divide multiple test target feature sets into K groups; Based on the K groups, K initial deep neural networks are updated respectively to generate K test deep neural networks; and Based on the accuracy of the K test deep neural networks, multiple test target features are verified.

10. A computer-readable recording medium, characterized in that: A program is stored, and the program is used to execute the method according to claim 1.

11. An electronic device for determining the degree of dementia of a user, characterized in that: include: a memory having recorded therein a program for determining the degree of dementia of a user; and A processor for executing the above program, The above program performs the following steps: outputting, through a user terminal, first content prepared in advance for determining a user's dementia degree; Receiving a first voice of the user related to the first content acquired through a microphone of the user terminal; outputting the pre-made second content through the user terminal; receiving a second voice of the user related to the second content acquired through the microphone; generating a first spectrogram by visualizing at least one characteristic of the first speech; generating a second spectrogram by visualizing at least one characteristic of the second speech; Inputting the first spectrogram into a pre-updated first convolutional neural network to generate a preset number of first features related to the first speech; Inputting the second spectrogram into a pre-updated second convolutional neural network to generate a preset number of second features related to the second speech; Determining a preset number of target features from the plurality of the first features and the plurality of the second features; and Inputting a plurality of the target features into a pre-updated deep neural network to determine the degree of dementia of the user, The above-mentioned dementia degree determined by the above-mentioned user terminal output, Among them, the first convolutional neural network is updated through the following steps: Receiving a first test voice of a test user related to the first content; generating a first test spectrogram by visualizing at least one characteristic of the first test speech, wherein the first test spectrogram is marked with a baseline truth dementia level of the test user; Inputting the first test spectrogram into a first fully convolutional neural network to determine a first test dementia degree of the test user, the first fully convolutional neural network comprising an input layer, at least one convolutional layer block, a fully connected layer and a softmax; and updating the first fully convolutional neural network based on the first test dementia level and the baseline truth dementia level, The above-mentioned first convolutional neural network only includes the above-mentioned input layer and the above-mentioned at least one convolutional layer block among the multiple layers of the updated above-mentioned first fully convolutional neural network.

12. A convolutional neural network updating method, executed by an electronic device, for updating a convolutional neural network used to determine the degree of dementia of a user, characterized in that: The steps include: outputting, through a user terminal, first content prepared in advance for determining a user's dementia degree; Receiving a first test voice of a test user related to the first content; generating a first test spectrogram by visualizing at least one characteristic of the first test speech, wherein the first test spectrogram is marked with a baseline truth dementia level of the test user; Inputting the first test spectrogram into a fully convolutional neural network to determine the test dementia degree of the test user, the fully convolutional neural network comprising an input layer, at least one convolutional layer block, a fully connected layer and a softmax; and Based on the above test dementia level and the above benchmark truth dementia level, the above fully convolutional neural network is updated. The above-mentioned convolutional neural network only includes the above-mentioned input layer and the above-mentioned at least one convolutional layer block among the multiple layers of the updated above-mentioned full convolutional neural network.

13. An electronic device for updating a convolutional neural network used to determine the degree of dementia of a user, characterized in that: include: A memory having recorded therein a program for updating the convolutional neural network; and A processor for executing the above program, The above program performs the following steps: outputting, through a user terminal, first content prepared in advance for determining a user's dementia degree; Receiving a first test voice of a test user related to the first content; generating a first test spectrogram by visualizing at least one characteristic of the first test speech, wherein the first test spectrogram is marked with a baseline truth dementia level of the test user; Inputting the first test spectrogram into a fully convolutional neural network to determine the test dementia degree of the test user, the fully convolutional neural network comprising an input layer, at least one convolutional layer block, a fully connected layer and a softmax; and Based on the above test dementia level and the above benchmark truth dementia level, the above fully convolutional neural network is updated. The above-mentioned convolutional neural network only includes the above-mentioned input layer and the above-mentioned at least one convolutional layer block among the multiple layers of the updated above-mentioned full convolutional neural network.

Images