A detection and early warning method and device, computer equipment and storage medium

Abstract

The embodiment of the application belongs to the technical field of medical detection, and relates to a detection and early warning method and device for Alzheimer disease, computer equipment and a storage medium. The voice collection equipment is carried in a hearing aid and the like, so that voice data can be collected in the normal language communication process of a wearer, data deviation caused by psychological factors in clinical diagnosis is avoided, and the collected data can truly reflect the cognitive state of the elderly. Meanwhile, the voice features of the user are extracted in real time, and the user is monitored and compared in two dimensions of long-time correlation of the voice features of the user and correlation of the voice features of the user and AD patients who have been diagnosed, so that the early symptoms of AD are detected and early warning is realized.

Description

Technical Field

[0001] This application relates to the field of medical testing technology, and in particular to a method, device, computer equipment, and storage medium for the detection and early warning of Alzheimer's disease. Background Technology

[0002] With the aging population trend across society, the health problems of the elderly are receiving increasing attention. Epidemiological surveys show that the prevalence of Alzheimer's disease (AD) among people aged 65 and above is approximately 4% to 8% in developed countries and 3% to 7% in my country. Based on this, it is estimated that there are currently about 6 to 8 million AD patients in my country. Currently, clinical diagnosis of AD mainly relies on a combination of various neuroimaging tools and neuropsychological scales, which is costly, complex, and has a certain impact on the health of the elderly, and early diagnosis of AD is not easy. Early diagnosis of AD symptoms is beneficial for timely drug or other interventions. Data cited in the 2011 AD diagnostic criteria indicates that if the onset of AD can be delayed by five years through early intervention, the number of AD patients could be reduced by 57% by 2050, and medical costs could be reduced by 45% (from $627 billion to $344 billion). Therefore, early diagnosis of AD can alleviate social and economic pressures to some extent.

[0003] In recent years, AD diagnosis technology based on speech or language information has also gained attention from the academic community. For example, Yilin et al. combined speech and linguistic features and used a dual-ended LSTM based on attention mechanism as a classifier to improve the accuracy of predicting AD from 62.15% to 70.75%.

[0004] However, the applicant found that due to the insidious onset of AD and concerns about personal privacy, the above-mentioned algorithms are either limited by the inability to collect unbiased speech signals or by the inability to achieve long-term early tracking and detection. As a result, it is difficult to guarantee the accuracy and robustness of the early warning of AD symptoms. It can be seen that traditional AD diagnostic technologies based on speech or language information have problems with low accuracy and robustness. Summary of the Invention

[0005] The purpose of this application is to propose a detection and early warning method, device, computer equipment, and storage medium for Alzheimer's disease, in order to solve the problems of low accuracy and robustness of traditional AD diagnosis technologies based on voice or language information.

[0006] To address the aforementioned technical problems, this application provides a detection and early warning method for Alzheimer's disease, employing the following technical solution:

[0007] Receive user voice data collected by the voice acquisition device from the target user;

[0008] Based on the trained acoustic-linguistic feature extraction model, a comprehensive feature extraction operation is performed on the user speech data to obtain acoustic-linguistic feature data;

[0009] The acoustic-linguistic feature data is stored in a feature database;

[0010] According to a preset cycle, the current cycle speech feature set and the previous cycle speech feature set corresponding to the target user are extracted from the feature database.

[0011] Based on the system similarity model, the current cycle speech feature set and the previous cycle speech feature set and performance feature change trend analysis operation are performed to obtain the feature change amount;

[0012] If the change in the feature is less than or equal to a preset change threshold, a signal indicating that no clinical diagnosis is required is output.

[0013] If the change in the feature is greater than the preset change threshold, a signal indicating that a clinical diagnosis is required will be output.

[0014] Furthermore, the acoustic-linguistic feature extraction model includes an acoustic feature extraction model and a linguistic feature extraction model. The step of performing comprehensive feature extraction on the user speech data based on the trained acoustic-linguistic feature extraction model to obtain acoustic-linguistic feature data specifically includes the following steps:

[0015] The acoustic feature extraction model is used to extract acoustic features from the user's speech data to obtain acoustic feature data.

[0016] The user speech data is subjected to linguistic feature extraction operation according to the linguistic feature extraction model to obtain linguistic feature data;

[0017] The acoustic feature data and the linguistic feature data are subjected to feature fusion operation to obtain the acoustic-linguistic feature data.

[0018] Furthermore, the acoustic feature data includes a target voiceprint feature vector and a target silence feature vector. The step of performing acoustic feature extraction on the user speech data according to the acoustic feature extraction model to obtain the acoustic feature data specifically includes the following steps:

[0019] The user's voice data is subjected to voiceprint feature extraction to obtain the target voiceprint feature vector.

[0020] The user's voice data is subjected to a silence feature extraction operation to obtain the target silence feature vector.

[0021] Furthermore, the step of performing voiceprint feature extraction on the user's voice data to obtain the target voiceprint feature vector specifically includes the following steps:

[0022] Mel-cepstral coefficient feature extraction is performed on the user's voice data to obtain a Mel-cepstral coefficient feature vector;

[0023] Based on the voiceprint characteristics of Alzheimer's disease, a voiceprint recognition operation is performed on the Mel-Cepstral Coefficient Feature Vector to obtain an initial voiceprint feature vector.

[0024] Principal component analysis is performed on the initial voiceprint feature vector to obtain a principal component dimension-reduced vector.

[0025] The principal component dimensionality reduction vector is reduced using a trained generative feature extractor to obtain the target voiceprint feature vector.

[0026] Furthermore, the step of performing linguistic feature extraction on the user speech data according to the linguistic feature extraction model to obtain linguistic feature data includes the following steps:

[0027] The user's voice data is extracted using speech recognition technology to obtain voice-text information.

[0028] The speech-text information is semantically extracted based on the linguistic feature extraction model to obtain speech-semantic information.

[0029] The speech semantic information is filtered based on the characteristics of Alzheimer's disease to obtain the linguistic feature data.

[0030] Furthermore, the step of performing feature fusion on the acoustic feature data and the linguistic feature data to obtain the acoustic-linguistic feature data specifically includes the following steps:

[0031] Perform a vector outer product operation on the target voiceprint feature vector, the target silence feature vector, and the linguistic feature data to obtain a vector outer product matrix;

[0032] The acoustic-linguistic feature data is obtained by performing a weighted pooling operation on the outer product matrix of the vectors.

[0033] Furthermore, after the step of performing comprehensive feature extraction on the user speech data based on the trained acoustic-linguistic feature extraction model to obtain acoustic-linguistic feature data, the following steps are also included:

[0034] The acoustic-linguistic feature data is subjected to detection and early warning operations based on the trained statistical model to obtain the detection and early warning probability.

[0035] If the detection and warning probability is less than or equal to the preset disease threshold, then output a signal that the patient does not have Alzheimer's disease.

[0036] If the detection and warning probability is greater than the preset disease threshold, then a signal indicating that the person has Alzheimer's disease is output.

[0037] To address the aforementioned technical problems, this application also provides a detection and early warning device for Alzheimer's disease, employing the following technical solution:

[0038] The data receiving module is used to receive user voice data collected by the voice acquisition device from the target user;

[0039] The comprehensive feature extraction module is used to perform comprehensive feature extraction operations on the user speech data based on the trained acoustic-linguistic feature extraction model to obtain acoustic-linguistic feature data;

[0040] The data storage module is used to store the acoustic-linguistic feature data into the feature database;

[0041] The data extraction module is used to extract the current period speech feature set and the previous period speech feature set corresponding to the target user from the feature database according to a preset period.

[0042] The trend analysis module is used to perform trend analysis on the current cycle speech feature set, the previous cycle speech feature set, and performance feature changes based on the system similarity model, and to obtain the feature change amount.

[0043] The first signal output module is used to output a signal that clinical diagnosis is not required if the change in the feature is less than or equal to a preset change threshold.

[0044] The second signal data module is used to output a signal requiring clinical diagnosis if the change in the feature is greater than the preset change threshold.

[0045] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0046] It includes a memory and an operator, wherein the memory stores computer-readable instructions, and the operator executes the computer-readable instructions to implement the steps of the detection and early warning method for Alzheimer's disease as described above.

[0047] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0048] The computer-readable storage medium stores computer-readable instructions, which, when executed by an operator, implement the steps of the detection and early warning method for Alzheimer's disease as described above.

[0049] This application provides a method for detecting and warning Alzheimer's disease, comprising: receiving user voice data of a target user collected by a voice acquisition device; performing comprehensive feature extraction on the user voice data according to a trained acoustic-linguistic feature extraction model to obtain acoustic-linguistic feature data; storing the acoustic-linguistic feature data in a feature database; extracting the current period voice feature set and the previous period voice feature set corresponding to the target user from the feature database according to a preset period; performing a system similarity model to analyze the current period voice feature set, the previous period voice feature set, and the performance feature change trend to obtain feature change amount; if the feature change amount is less than or equal to a preset change threshold, outputting a signal that no clinical diagnosis is required; if the feature change amount is greater than the preset change threshold, outputting a signal that clinical diagnosis is required. Compared with existing technologies, this application, by incorporating a voice acquisition device into hearing aids and other devices, enables the collection of voice data during normal language communication by the wearer. This avoids data bias caused by psychological factors in clinical diagnosis and ensures that the collected data truly reflects the cognitive state of the elderly. At the same time, it extracts the user's voice features in real time and performs long-term monitoring and comparison of the user from two dimensions: the long-term correlation of the user's own voice features and the correlation between the user and the voice features of diagnosed AD patients. This enables the detection and early warning of early AD symptoms. Attached Figure Description

[0050] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0052] Figure 2 This is a flowchart illustrating the implementation of the detection and early warning method for Alzheimer's disease provided in Embodiment 1 of this application;

[0053] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S202;

[0054] Figure 4yes Figure 3 A flowchart of a specific implementation of step S301;

[0055] Figure 5 yes Figure 4 A flowchart of a specific implementation of step S401;

[0056] Figure 6 yes Figure 3 A flowchart of a specific implementation of step S303;

[0057] Figure 7 yes Figure 2 A flowchart of a specific implementation method following step S202;

[0058] Figure 8 This is an overall flowchart of a specific implementation of the comprehensive feature extraction operation provided in Embodiment 1 of this application;

[0059] Figure 9 This is a schematic diagram of the structure of the detection and early warning device for Alzheimer's disease provided in Embodiment 2 of this application;

[0060] Figure 10 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0062] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0063] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0064] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0065] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0066] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0067] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0068] It should be noted that the detection and early warning method for Alzheimer's disease provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the detection and early warning device for Alzheimer's disease is generally set in the server / terminal device.

[0069] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0070] Example 1

[0071] Continue to refer to Figure 2 The diagram shows the implementation flowchart of the detection and early warning method for Alzheimer's disease provided in Embodiment 1 of this application. For ease of explanation, only the parts related to this application are shown.

[0072] The above-mentioned detection and early warning method for Alzheimer's disease includes steps S201, S202, S203, S204, S205, S206 and S207.

[0073] In step S201, the voice data of the target user collected by the voice acquisition device is received.

[0074] In this embodiment, the voice acquisition device includes a directional microphone and a Bluetooth module integrated on the hearing aid. The hearing aid acquires the user's voice in real time through the directional microphone and transmits the acquired voice data to the user's smartphone via the Bluetooth transmission protocol.

[0075] In step S202, a comprehensive feature extraction operation is performed on the user's speech data based on the trained acoustic-linguistic feature extraction model to obtain acoustic-linguistic feature data.

[0076] In this embodiment, an acoustic feature extraction model and a linguistic feature extraction model are established for speech signals, and the two are fused to train a comprehensive model for extracting AD-related feature vectors from the speech signals. The overall process is as follows: Figure 8 As shown, x-vector-plda and SL-rate (silence rate) are both acoustic features, while language-vector is a linguistic feature.

[0077] In step S203, the acoustic-linguistic feature data is stored in the feature database.

[0078] In this embodiment of the application, after obtaining the acoustic-linguistic feature data of the target user, the acoustic-linguistic feature data is stored in the feature database so that the data can be easily retrieved when performing trend analysis later.

[0079] In step S204, the current period speech feature set and the previous period speech feature set corresponding to the target user are extracted from the feature database according to a preset period.

[0080] In step S205, the current cycle speech feature set, the previous cycle speech feature set, and the performance feature change trend analysis are performed based on the system similarity model to obtain the feature change amount.

[0081] In this embodiment of the application, a system similarity model is used to analyze the changing trends of user AD characteristics, so as to determine whether there are significant changes in the user's AD-vector performance, thus providing assistance for medical diagnosis.

[0082] In this embodiment of the application, AD-vectors are extracted daily based on the collected voice signal data, using a preset period as the unit, and the data are arranged into a matrix AD, where N is the dimension of the AD-vector and M is the number of AD-vectors collected within the preset period.

[0083]

[0084] Where a1 represents an AD-vector extracted from the speech data of the preset period. Then, we perform singular value decomposition on this matrix, as shown in the following equation.

[0085]

[0086] The first K eigenvectors in U are retained in descending order of singular values. Let u1 = x1, and these K vectors form a matrix X. X is calculated every preset period.

[0087]

[0088] Simultaneously, K corresponding eigenvalues ​​are obtained: σ = diag{σ1, σ2, ..., σ...} K}

[0089] In the similarity model, each input consists of two matrices X1 and X2, and the output is the correlation S(X1, X2). First, the two X matrices are matched one-to-one with their feature vectors; the two matched feature vectors are called a pair. Simultaneously, μ... k Let μ represent the similarity of the k-th pair. k The calculation is performed using the Euclidean distance between the two vectors.

[0090]

[0091] Where x 1k Let x be the k-th eigenvector in matrix X1. 2k Let X be the k-th eigenvector in matrix X2.

[0092] Since there may be highly similar pairs between matrices X1 and X2, we need to consider this factor in our final matrix similarity calculation. We set a similarity threshold ∈ [x, y], where the similarity of the k-th pair μ... k When >∈, they are high-similarity pairs. Assume that among the K pairs, there are P high-similarity pairs. Let ω 1k and ω 2k Let ω represent the weight of each vector in matrices X1 and X2 when performing matrix similarity analysis, and assign values ​​to each weight as follows: 1k =σ 1k,ω 2k =σ 2k .

[0093] Therefore, the correlation between X1 and X2 can be given by the following formula:

[0094]

[0095] S represents the correlation between two systems X1 and X2.

[0096] The process of using the system correlation is as follows: First, record the AD-related feature representation matrix X0 extracted when the user first uses the system in the first week. Then, record the AD-related feature representation matrix X0 for each subsequent week. m We need to calculate S(X0,X) for each of the following weeks. m ) and S(X {m-1} ,X m ), thus obtaining two sets of AD-related feature matrix change characterization data sequences over time:

[0097] Ε(t)={S(X0,X1),S(X0,X2),…,S(X0,X m )}t∈(0,m)

[0098] Φ(t)={S(X0,X1),S(X1,X2),…,S(X m-1 ,X m )}t∈(0,m)

[0099] After obtaining the long-term similarity sequences E and Φ, we perform time-based curve fitting on these two time series to obtain continuous functions of E and Φ. We then differentiate these functions to obtain E'(t) and Φ'(t) for further analysis.

[0100] In step S206, if the feature change is less than or equal to a preset change threshold, a signal indicating that no clinical diagnosis is required is output.

[0101] In step S207, if the feature change is greater than a preset change threshold, a signal indicating that a clinical diagnosis is required is output.

[0102] In some optional implementations of this embodiment, after the user extracts the acoustic-linguistic feature data, the data can be stored in the device, making the recorded data a long-term feature time-series sequence. This facilitates the use of time-series analysis models to extract the changing characteristics of acoustic-linguistic features, enabling analysis of long-term time-series feature data sequences. It also facilitates the application of long-term time-series feature similarity models to this time-series feature sequence. This method is beneficial for analyzing the long-term changing trend of a user's Alzheimer's disease severity, for long-term patient monitoring, and for hospitals to track and analyze patient data over a long period.

[0103] This application provides a method for detecting and warning Alzheimer's disease, comprising: receiving user voice data of a target user collected by a voice acquisition device; performing comprehensive feature extraction on the user voice data according to a trained acoustic-linguistic feature extraction model to obtain acoustic-linguistic feature data; storing the acoustic-linguistic feature data in a feature database; extracting the current period voice feature set and the previous period voice feature set corresponding to the target user from the feature database according to a preset period; performing a system similarity model to analyze the current period voice feature set, the previous period voice feature set, and the performance feature change trend to obtain the feature change amount; if the feature change amount is less than or equal to a preset change threshold, outputting a signal that no clinical diagnosis is required; if the feature change amount is greater than the preset change threshold, outputting a signal that clinical diagnosis is required. Compared with existing technologies, this application, by incorporating a voice acquisition device into hearing aids and other devices, enables the collection of voice data during normal language communication by the wearer. This avoids data bias caused by psychological factors in clinical diagnosis and ensures that the collected data truly reflects the cognitive state of the elderly. At the same time, it extracts the user's voice features in real time and performs long-term monitoring and comparison of the user from two dimensions: the long-term correlation of the user's own voice features and the correlation between the user and the voice features of diagnosed AD patients. This enables the detection and early warning of early AD symptoms.

[0104] Continue reading Figure 3 , showed Figure 2 The flowchart of a specific implementation of step S202 is shown. For ease of explanation, only the parts relevant to this application are shown.

[0105] In some optional implementations of this embodiment, step S202 specifically includes: step S301, step S302 and step S303.

[0106] In step S301, acoustic feature extraction is performed on the user's speech data according to the acoustic feature extraction model to obtain acoustic feature data.

[0107] In step S302, linguistic feature extraction is performed on the user's speech data according to the linguistic feature extraction model to obtain linguistic feature data.

[0108] In step S303, a feature fusion operation is performed on the acoustic feature data and the linguistic feature data to obtain acoustic-linguistic feature data.

[0109] In this embodiment of the application, acoustic-linguistic feature data is divided into acoustic feature data and linguistic feature data.

[0110] In this embodiment of the application, the acoustic feature extraction operation may be to perform a voiceprint feature extraction operation on the user's voice data to obtain a target voiceprint feature vector, and then to perform a silence feature extraction operation on the user's voice data to obtain the target silence feature vector.

[0111] Continue reading Figure 4 , showed Figure 3 The flowchart of a specific embodiment of step S301 is shown. For ease of explanation, only the parts relevant to this application are shown.

[0112] In some optional implementations of this embodiment, step S301 specifically includes: step S401 and step S402.

[0113] In step S401, voiceprint feature extraction is performed on the user's voice data to obtain the target voiceprint feature vector.

[0114] In step S402, a silence feature extraction operation is performed on the user's voice data to obtain the target silence feature vector.

[0115] In this embodiment, voiceprint feature extraction first involves preliminary feature extraction from the input speech data (for user-input speech signal data, the MFCC feature vector is extracted first), followed by extraction of the user's AD-related x-vector (using any existing x-vector extraction technique). Then, principal component analysis (PCA) is performed on this x-vector to extract core features, resulting in an X-vector-PCA dimensionality-reduced vector. Finally, the X-vector-PCA dimensionality-reduced vector is further dimensionality-reduced using a trained PLDA feature extractor to extract core features related to AD representation, resulting in an N*1 dimensional feature vector x-vector-PLDA.

[0116] In this embodiment, the training process of the PLDA feature extractor is as follows: First, two sets of speech databases of users of the same age group are given, one set of AD patients and the other set of normal individuals, so that supervised classification can be performed in subsequent steps. During training, the first step is to perform principal component analysis on the entire speech database to find a dimensionality-reduced mapping space for extracting the core components of the X-VECTOR. The second step is to combine the PLDA with a classifier and train it, using supervised learning on the training dataset to make the PLDA tend to extract feature vectors related to AD representations in x-vector-PCA.

[0117] Continue reading Figure 5 , showed Figure 4 The flowchart of a specific embodiment of step S401 is shown. For ease of explanation, only the parts relevant to this application are shown.

[0118] In some optional implementations of this embodiment, step S401 specifically includes: step S501, step S502, step S503 and step S504.

[0119] In step S501, Mel-Cepstral Coefficient Feature Extraction is performed on the user's speech data to obtain the Mel-Cepstral Coefficient Feature Vector.

[0120] In step S502, the Mel-Cepstral Coefficient Feature Vector is subjected to voiceprint recognition operation based on the voiceprint characteristics of Alzheimer's disease to obtain the initial voiceprint feature vector.

[0121] In step S503, principal component analysis is performed on the initial voiceprint feature vector according to the principal component analysis technique to obtain the principal component dimensionality reduction vector.

[0122] In step S504, the principal component dimensionality reduction vector is reduced using the trained generative feature extractor to obtain the target voiceprint feature vector.

[0123] In this embodiment, firstly, based on Automatic Speech Recognition (ASR) technology, the text information of the speech signal of each object is extracted. After extracting the user's daily speech text information, we use a pre-trained linguistic feature extraction model based on BERT combined with a feedforward neural network to extract the AD-language-vector related to AD representation in the user's semantics.

[0124] In this embodiment, the training process of the linguistic feature extraction module is as follows: Utilizing the concept of transfer learning, we first introduce a pre-trained BERT model for natural language processing. This model uses a self-attention mechanism to encode the semantics of the text. After converting the original speech dataset into text information, the BERT model is used to obtain the text semantic encoding (encode-vector) for each data point. A feedforward neural network (DNN) is combined with a classifier, using the text semantic encoding (encode-vector) of all data as input and whether the user has Alzheimer's disease (AD) as output, to train the model, enabling the DNN to extract the feature vector AD-language-vector related to AD representation.

[0125] Continue reading Figure 6 , showed Figure 3 The flowchart of a specific embodiment of step S303 is shown. For ease of explanation, only the parts relevant to this application are shown.

[0126] In some optional implementations of this embodiment, step S303 specifically includes steps S601 and S602.

[0127] In step S601, a vector outer product operation is performed on the target voiceprint feature vector, the target silence feature vector, and the linguistic feature data to obtain a vector outer product matrix.

[0128] In step S602, a weighted pooling operation is performed on the outer product matrix of the vectors to obtain acoustic-linguistic feature data.

[0129] In this embodiment, a multimodal fusion algorithm from machine learning is used to fuse the three features mentioned above. Feature fusion involves inputting feature vectors from the three modalities and outputting a fused vector. We use a fusion method based on bilinear pooling. We first perform a vector outer product on the obtained x-vector-PLDA, SL-vector, and AD-language vector to obtain a matrix, and then perform weighted pooling on the matrix to obtain the final feature vector AD-vector.

[0130] Continue reading Figure 7 , showed Figure 2 The flowchart of a specific implementation after step S202 is shown. For ease of explanation, only the parts relevant to this application are shown.

[0131] In some optional implementations of this embodiment, after step S202, steps S701, S702, and S703 are also included.

[0132] In step S701, the acoustic-linguistic feature data is subjected to detection and early warning operations based on the trained statistical model to obtain the detection and early warning probability.

[0133] In step S702, if the detection warning probability is less than or equal to the preset disease threshold, a signal indicating that the patient does not have Alzheimer's disease is output.

[0134] In step S703, if the detection warning probability is greater than the preset disease threshold, a signal indicating that the patient has Alzheimer's disease is output.

[0135] In this embodiment, a statistical model is trained using AD-vectors extracted from an existing AD database, and this model is used to predict the probability of a user having AD. We construct a statistical distribution model based on AD-vectors extracted from the speech signal data of 50 AD patients and 50 normal individuals of the same age. Based on the AD-vectors extracted by the feature extraction module for each object, a Gaussian distribution is constructed for AD patients and normal individuals using maximum likelihood estimation, denoted as p(AD) and p(CC) respectively. After processing the AD-vectors extracted by the model, P(user|AD) and P(user|CC) can be calculated for each user. Therefore, we set the detection period to T, and calculate whether a user has AD within each detection period. Simultaneously, we set a threshold θ for determining the presence of AD; when P(user|AD) > θ, it is determined that the user has AD and needs to go to the hospital for intervention and treatment as soon as possible.

[0136] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium, and when executed, they can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0137] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0138] Example 2

[0139] Further reference Figure 9 As a response to the above Figure 2 The implementation of the method shown in this application provides an embodiment of a detection and early warning device for Alzheimer's disease, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0140] like Figure 9 As shown, the Alzheimer's disease detection and early warning device 200 of this embodiment includes: an acquisition module 401, an identification module 402, a calculation module 403, and a display module 404. Wherein:

[0141] The data receiving module 210 is used to receive user voice data of the target user collected by the voice acquisition device;

[0142] The comprehensive feature extraction module 220 is used to perform comprehensive feature extraction operations on user speech data based on the trained acoustic-linguistic feature extraction model to obtain acoustic-linguistic feature data;

[0143] Data storage module 230 is used to store acoustic-linguistic feature data into a feature database;

[0144] The data extraction module 240 is used to extract the current period speech feature set and the previous period speech feature set corresponding to the target user from the feature database according to a preset period.

[0145] The trend analysis module 250 is used to perform trend analysis on the current cycle speech feature set, the previous cycle speech feature set, and performance feature changes based on the system similarity model, and to obtain the feature change amount.

[0146] The first signal output module 260 is used to output a signal that does not require clinical diagnosis if the feature change is less than or equal to a preset change threshold.

[0147] The second signal data module 270 is used to output a clinical diagnostic signal if the feature change is greater than a preset change threshold.

[0148] In this embodiment, the voice acquisition device includes a directional microphone and a Bluetooth module integrated on the hearing aid. The hearing aid acquires the user's voice in real time through the directional microphone and transmits the acquired voice data to the user's smartphone via the Bluetooth transmission protocol.

[0149] In this embodiment, an acoustic feature extraction model and a linguistic feature extraction model are established for speech signals, and the two are fused to train a comprehensive model for extracting AD-related feature vectors from the speech signals. The overall process is as follows: Figure 8 As shown, x-vector-plda and SL-rate (silence rate) are both acoustic features, while language-vector is a linguistic feature.

[0150] In this embodiment of the application, after obtaining the acoustic-linguistic feature data of the target user, the acoustic-linguistic feature data is stored in the feature database so that the data can be easily retrieved when performing trend analysis later.

[0151] In this embodiment of the application, a system similarity model is used to analyze the changing trends of user AD characteristics, so as to determine whether there are significant changes in the user's AD-vector performance, thus providing assistance for medical diagnosis.

[0152] In this embodiment of the application, AD-vectors are extracted daily based on the collected voice signal data, using a preset period as the unit, and the data are arranged into a matrix AD, where N is the dimension of the AD-vector and M is the number of AD-vectors collected within the preset period.

[0153]

[0154] Where a1 represents an AD-vector extracted from the speech data of the preset period. Then, we perform singular value decomposition on this matrix, as shown in the following equation.

[0155]

[0156] The first K eigenvectors in U are retained in descending order of singular values. Let u1 = x1, and these K vectors form a matrix X. X is calculated every preset period.

[0157]

[0158] Simultaneously, K corresponding eigenvalues ​​are obtained: σ = diag{σ1, σ2, ..., σ...} K}

[0159] In the similarity model, each input consists of two matrices X1 and X2, and the output is the correlation S(X1, X2). First, the two X matrices are matched one-to-one with their feature vectors; the two matched feature vectors are called a pair. Simultaneously, μ... k Let μ represent the similarity of the k-th pair. k The calculation is performed using the Euclidean distance between the two vectors.

[0160]

[0161] Where x 1k Let x be the k-th eigenvector in matrix X1. 2k Let X be the k-th eigenvector in matrix X2.

[0162] Since there may be highly similar pairs between matrices X1 and X2, we need to consider this factor in our final matrix similarity calculation. We set a similarity threshold ∈ [x, y], where the similarity of the k-th pair μ... k When >∈, they are high-similarity pairs. Assume that among the K pairs, there are P high-similarity pairs. Let ω 1k and ω 2k Let ω represent the weight of each vector in matrices X1 and X2 when performing matrix similarity analysis, and assign values ​​to each weight as follows: 1k =σ 1k ,ω 2k =σ 2k .

[0163] Therefore, the correlation between X1 and X2 can be given by the following formula:

[0164]

[0165] S represents the correlation between two systems X1 and X2.

[0166] The process of using the system correlation is as follows: First, record the AD-related feature representation matrix X0 extracted when the user first uses the system in the first week. Then, record the AD-related feature representation matrix X0 for each subsequent week. m We need to calculate S(X0,X) for each of the following weeks. m ) and S(X {m-1} ,X m ), thus obtaining two sets of AD-related feature matrix change characterization data sequences over time:

[0167] Ε(t)={S(X0,X1),S(X0,X2),…,S(X0,X m )}t∈(0,m)

[0168] Φ(t)={S(X0,X1),S(X1,X2),…,S(X m-1 ,X m )}t∈(0,m)

[0169] After obtaining the long-term similarity sequences E and Φ, we perform time-based curve fitting on these two time series to obtain continuous functions of E and Φ. We then differentiate these functions to obtain E'(t) and Φ'(t) for further analysis.

[0170] In some optional implementations of this embodiment, after the user extracts the acoustic-linguistic feature data, the data can be stored in the device, making the recorded data a long-term feature time-series sequence. This facilitates the use of time-series analysis models to extract the changing characteristics of acoustic-linguistic features, enabling analysis of long-term time-series feature data sequences. It also facilitates the application of long-term time-series feature similarity models to this time-series feature sequence. This method is beneficial for analyzing the long-term changing trend of a user's Alzheimer's disease severity, for long-term patient monitoring, and for hospitals to track and analyze patient data over a long period.

[0171] In this embodiment, an Alzheimer's disease detection and early warning device 200 is provided, comprising: a data receiving module 210 for receiving user voice data of a target user collected by a voice acquisition device; a comprehensive feature extraction module 220 for performing comprehensive feature extraction on the user voice data according to a trained acoustic-linguistic feature extraction model to obtain acoustic-linguistic feature data; a data storage module 230 for storing the acoustic-linguistic feature data in a feature database; a data extraction module 240 for extracting the current period voice feature set and the previous period voice feature set corresponding to the target user from the feature database according to a preset period; a trend analysis module 250 for performing trend analysis on the current period voice feature set, the previous period voice feature set, and performance feature changes according to a system similarity model to obtain feature change amount; a first signal output module 260 for outputting a signal indicating that no clinical diagnosis is required if the feature change amount is less than or equal to a preset change threshold; and a second signal data module 270 for outputting a signal indicating that clinical diagnosis is required if the feature change amount is greater than the preset change threshold. Compared with existing technologies, this application, by incorporating a voice acquisition device into hearing aids and other devices, enables the collection of voice data during normal language communication by the wearer. This avoids data bias caused by psychological factors in clinical diagnosis and ensures that the collected data truly reflects the cognitive state of the elderly. At the same time, it extracts the user's voice features in real time and performs long-term monitoring and comparison of the user from two dimensions: the long-term correlation of the user's own voice features and the correlation between the user and the voice features of diagnosed AD patients. This enables the detection and early warning of early AD symptoms.

[0172] In some optional implementations of this embodiment, the acoustic-linguistic feature extraction model includes an acoustic feature extraction model and a linguistic feature extraction model. The aforementioned integrated feature extraction module 220 includes: an acoustic feature extraction submodule, a linguistic feature extraction submodule, and a feature fusion submodule, wherein:

[0173] The acoustic feature extraction submodule is used to perform acoustic feature extraction on user speech data according to the acoustic feature extraction model to obtain acoustic feature data;

[0174] The linguistic feature extraction submodule is used to perform linguistic feature extraction operations on user speech data according to the linguistic feature extraction model to obtain linguistic feature data.

[0175] The feature fusion submodule is used to perform feature fusion operations on acoustic feature data and linguistic feature data to obtain acoustic-linguistic feature data.

[0176] In some optional implementations of this embodiment, the acoustic feature data includes a target voiceprint feature vector and a target silence feature vector. The aforementioned acoustic feature extraction submodule includes: a voiceprint feature extraction unit and a silence feature extraction unit, wherein:

[0177] The voiceprint feature extraction unit is used to perform voiceprint feature extraction on user voice data to obtain the target voiceprint feature vector.

[0178] The silence feature extraction unit is used to extract silence features from user speech data to obtain the target silence feature vector.

[0179] In some optional implementations of this embodiment, the above-mentioned voiceprint feature extraction unit includes: a Mel-Cepstral Coefficient Feature Extraction Subunit, a Voiceprint Recognition Subunit, a Principal Component Analysis Subunit, and a Dimensionality Reduction Subunit, wherein:

[0180] The Mel-Cepstral Coefficient Feature Extraction Subunit is used to perform Mel-Cepstral Coefficient Feature Extraction on user speech data to obtain Mel-Cepstral Coefficient Feature Vectors.

[0181] The voiceprint recognition subunit is used to perform voiceprint recognition operations on the Mel-Cepstral Coefficient Feature Vector based on the voiceprint characteristics of Alzheimer's disease to obtain the initial voiceprint feature vector.

[0182] The principal component analysis subunit is used to perform principal component analysis on the initial voiceprint feature vector according to the principal component analysis technique to obtain the principal component dimension reduction vector.

[0183] The dimensionality reduction subunit is used to perform dimensionality reduction operations on the principal component dimensionality reduction vector based on the trained generative feature extractor to obtain the target voiceprint feature vector.

[0184] In some optional implementations of this embodiment, the linguistic feature extraction submodule includes: a text extraction unit, a semantic extraction unit, and a filtering unit, wherein:

[0185] The text extraction unit is used to perform text extraction operations on user voice data based on speech recognition technology to obtain voice text information;

[0186] The semantic extraction unit is used to perform semantic extraction operations on speech text information based on the linguistic feature extraction model to obtain speech semantic information.

[0187] The filtering unit is used to filter speech and semantic information based on the characteristics of Alzheimer's disease to obtain linguistic feature data.

[0188] In some optional implementations of this embodiment, the feature fusion submodule includes: a vector outer product unit and a weighted pooling pair unit, wherein:

[0189] The vector outer product unit is used to perform a vector outer product operation on the target voiceprint feature vector, the target silence feature vector, and linguistic feature data to obtain the vector outer product matrix.

[0190] Weighted pooling pairs are used to perform weighted pooling operations on vector outer product matrices to obtain acoustic-linguistic feature data.

[0191] In some optional implementations of this embodiment, the detection and early warning device 200 for Alzheimer's disease further includes: a detection and early warning module, a third signal output module, and a fourth signal output module, wherein:

[0192] The detection and early warning module is used to perform detection and early warning operations on acoustic-linguistic feature data based on a trained statistical model, and obtain the detection and early warning probability.

[0193] The third signal output module is used to output a signal that the patient does not have Alzheimer's disease if the detection and warning probability is less than or equal to the preset disease threshold.

[0194] The fourth signal output module is used to output a signal indicating that the patient has Alzheimer's disease if the detection and warning probability is greater than the preset disease threshold.

[0195] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 10 , Figure 10 This is a basic structural block diagram of the computer device in this embodiment.

[0196] The computer device 300 includes a memory 310, an operator 320, and a network interface 330 that are interconnected via a system bus. It should be noted that only the computer device 300 with components 310-330 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information operations according to pre-set or stored instructions, and its hardware includes, but is not limited to, micromanipulators, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0197] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0198] The memory 310 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 310 may be an internal storage unit of the computer device 300, such as the hard disk or memory of the computer device 300. In other embodiments, the memory 310 may also be an external storage device of the computer device 300, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, the memory 310 may also include both internal storage units and external storage devices of the computer device 300. In this embodiment, the memory 310 is typically used to store the operating system and various application software installed on the computer device 300, such as computer-readable instructions for Alzheimer's disease detection and early warning methods. Furthermore, the memory 310 can also be used to temporarily store various types of data that have been output or will be output.

[0199] In some embodiments, the operator 320 may be a central processing unit (CPU), controller, microcontroller, micromanipulator, or other data manipulation chip. The operator 320 is typically used to control the overall operation of the computer device 300. In this embodiment, the operator 320 is used to execute computer-readable instructions or operational data stored in the memory 310, such as executing the computer-readable instructions applied to the Alzheimer's disease detection and early warning method.

[0200] The network interface 330 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 300 and other electronic devices.

[0201] The computer device provided in this application, by incorporating a voice acquisition device into devices such as hearing aids, can collect voice data during normal language communication by the wearer, avoiding data bias caused by psychological factors in clinical diagnosis and ensuring that the collected data truly reflects the cognitive state of the elderly. At the same time, it extracts the user's voice features in real time and performs long-term monitoring and comparison of the user from two dimensions: the long-term correlation of the user's own voice features and the correlation between the user and the voice features of diagnosed AD patients, thereby realizing the detection and early warning of early symptoms of AD.

[0202] This application also provides another embodiment, namely, a computer-readable storage medium storing computer-readable instructions that can be executed by at least one operator to cause the at least one operator to perform the steps of the detection and early warning method for Alzheimer's disease as described above.

[0203] The computer-readable storage medium provided in this application, by incorporating a voice acquisition device into devices such as hearing aids, enables the acquisition of voice data during normal language communication by the wearer. This avoids data bias caused by psychological factors in clinical diagnosis and ensures that the acquired data truly reflects the cognitive state of the elderly. Simultaneously, it extracts the user's voice features in real time and performs long-term monitoring and comparison of the user from two dimensions: the long-term correlation of the user's own voice features and the correlation between the user and the voice features of diagnosed AD patients. This enables the detection and early warning of early AD symptoms.

[0204] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0205] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A method for detecting and warning Alzheimer's disease, the method being executed by a computer, characterized in that, Includes the following steps: Receive user voice data collected by the voice acquisition device from the target user; Based on the trained acoustic-linguistic feature extraction model, a comprehensive feature extraction operation is performed on the user speech data to obtain acoustic-linguistic feature data; The acoustic-linguistic feature data is stored in a feature database; According to a preset cycle, the current cycle speech feature set and the previous cycle speech feature set corresponding to the target user are extracted from the feature database. Based on the system similarity model, the current cycle speech feature set and the previous cycle speech feature set and performance feature change trend analysis operation are performed to obtain the feature change amount; If the change in the feature is less than or equal to a preset change threshold, a signal indicating that no clinical diagnosis is required is output. If the change in the feature is greater than the preset change threshold, a signal indicating that a clinical diagnosis is required will be output.

2. The detection and early warning method for Alzheimer's disease according to claim 1, characterized in that, The acoustic-linguistic feature extraction model includes an acoustic feature extraction model and a linguistic feature extraction model. The step of performing comprehensive feature extraction on the user speech data based on the trained acoustic-linguistic feature extraction model to obtain acoustic-linguistic feature data specifically includes the following steps: The acoustic feature extraction model is used to extract acoustic features from the user's speech data to obtain acoustic feature data. The user speech data is subjected to linguistic feature extraction operation according to the linguistic feature extraction model to obtain linguistic feature data; The acoustic feature data and the linguistic feature data are subjected to feature fusion operation to obtain the acoustic-linguistic feature data.

3. The detection and early warning method for Alzheimer's disease according to claim 2, characterized in that, The acoustic feature data includes a target voiceprint feature vector and a target silence feature vector. The step of performing acoustic feature extraction on the user speech data according to the acoustic feature extraction model to obtain the acoustic feature data specifically includes the following steps: The user's voice data is subjected to voiceprint feature extraction to obtain the target voiceprint feature vector. The user's voice data is subjected to a silence feature extraction operation to obtain the target silence feature vector.

4. The detection and early warning method for Alzheimer's disease according to claim 3, characterized in that, The step of performing voiceprint feature extraction on the user's voice data to obtain the target voiceprint feature vector specifically includes the following steps: Mel-cepstral coefficient feature extraction is performed on the user's voice data to obtain a Mel-cepstral coefficient feature vector; Based on the voiceprint characteristics of Alzheimer's disease, a voiceprint recognition operation is performed on the Mel-Cepstral Coefficient Feature Vector to obtain an initial voiceprint feature vector. Principal component analysis is performed on the initial voiceprint feature vector to obtain a principal component dimension-reduced vector. The principal component dimensionality reduction vector is reduced using a trained generative feature extractor to obtain the target voiceprint feature vector.

5. The detection and early warning method for Alzheimer's disease according to claim 2, characterized in that, The step of performing linguistic feature extraction on the user speech data according to the linguistic feature extraction model to obtain linguistic feature data includes the following steps: The user's voice data is extracted using speech recognition technology to obtain voice-text information. The speech-text information is semantically extracted based on the linguistic feature extraction model to obtain speech-semantic information. The speech semantic information is filtered based on the characteristics of Alzheimer's disease to obtain the linguistic feature data.

6. The detection and early warning method for Alzheimer's disease according to claim 3, characterized in that, The step of performing feature fusion on the acoustic feature data and the linguistic feature data to obtain the acoustic-linguistic feature data specifically includes the following steps: Perform a vector outer product operation on the target voiceprint feature vector, the target silence feature vector, and the linguistic feature data to obtain a vector outer product matrix; The acoustic-linguistic feature data is obtained by performing a weighted pooling operation on the outer product matrix of the vectors.

7. The detection and early warning method for Alzheimer's disease according to claim 1, characterized in that, After the step of performing comprehensive feature extraction on the user speech data based on the trained acoustic-linguistic feature extraction model to obtain acoustic-linguistic feature data, the following steps are also included: The acoustic-linguistic feature data is subjected to detection and early warning operations based on the trained statistical model to obtain the detection and early warning probability. If the detection and warning probability is less than or equal to the preset disease threshold, then output a signal that the patient does not have Alzheimer's disease. If the detection and warning probability is greater than the preset disease threshold, then a signal indicating that the person has Alzheimer's disease is output.

8. A detection and early warning device for Alzheimer's disease, the device being used to perform the steps of the detection and early warning method for Alzheimer's disease as described in any one of claims 1-7, characterized in that, include: The data receiving module is used to receive user voice data collected by the voice acquisition device from the target user; The comprehensive feature extraction module is used to perform comprehensive feature extraction operations on the user speech data based on the trained acoustic-linguistic feature extraction model to obtain acoustic-linguistic feature data; The data storage module is used to store the acoustic-linguistic feature data into the feature database; The data extraction module is used to extract the current period speech feature set and the previous period speech feature set corresponding to the target user from the feature database according to a preset period. The trend analysis module is used to perform trend analysis on the current cycle speech feature set, the previous cycle speech feature set, and performance feature changes based on the system similarity model, and to obtain the feature change amount. The first signal output module is used to output a signal that clinical diagnosis is not required if the change in the feature is less than or equal to a preset change threshold. The second signal data module is used to output a signal requiring clinical diagnosis if the change in the feature is greater than the preset change threshold.

9. A computer device, characterized in that, The device includes a memory and an operator, wherein the memory stores computer-readable instructions, and the operator executes the computer-readable instructions to implement the steps of the detection and early warning method for Alzheimer's disease as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by an operator, implement the steps of the detection and early warning method for Alzheimer's disease as described in any one of claims 1 to 7.

Images