Early prediction system for diabetic cognitive impairment based on speech recognition

Abstract

The present invention discloses an early prediction system for diabetic cognitive impairment based on speech recognition; it includes a speech acquisition module, a speech processing module, a feature extraction module, a large language model and a machine learning module; the large language model performs AI interaction with the patient, and the speech acquisition module acquires the patient's speech information, and transmits the acquired speech information to the speech processing module, the speech processing module processes the speech information, and transmits the processed speech information to the feature extraction module and the large language model, the feature extraction module transmits the extracted speech features to the machine learning model, and the machine learning model performs early evaluation and prediction of cognitive impairment according to the speech features. The present invention acquires the patient's speech, processes the patient's speech and extracts features, thereby facilitating the evaluation and prediction of the patient by analyzing and processing the features.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech analysis, and more specifically, particularly relates to an early prediction system for diabetic cognitive impairment based on speech recognition. Background Art

[0002] Since the 21st century, with the continuous intensification of the aging process of the population, diabetes has become one of the serious public health problems. Epidemiological survey data show that the prevalence of diabetes is 12.8%, of which the incidence rate of people over 60 years old has reached 25.0%, and the chronic complications caused by diabetes in the elderly have brought heavy economic and social burdens. Diabetes significantly increases the risk of cognitive dysfunction, and about 1 / 10 of dementia cases worldwide are related to diabetes. Cognitive dysfunction leads to a decrease in the self-management ability of diabetic patients and an increase in nursing dependence, which further aggravates the progression of the disease and forms a vicious circle. The clinical practice research database shows that the proportion of dementia-related deaths in the diabetic population is increasing year by year.

[0003] As the aging process of the population intensifies, the prevalence of cognitive impairment continues to increase. Diabetes significantly increases the risk of cognitive impairment. Data from community population studies show that the prevalence of dementia and mild cognitive impairment in the elderly population aged 60 and above is 6.0% and 15.5%, respectively, among which the prevalence of dementia and MCI in the diabetic population is 10.2% and 18.5%, respectively. Epidemiological data show that the risk of MCI and dementia in the diabetic population worldwide is 1.44 and 2.14 times that of the non-diabetic population, respectively, and diabetes also accelerates the transformation of MCI to dementia, shortening this process by 3.18 years. Compared with non-diabetic patients, diabetic patients have an accelerated decline in cognitive function after the onset of the disease, and are more likely to experience a decline in one or more cognitive domains such as memory, processing speed, and executive function, and the rate of decline is about 50% faster than normal aging.

[0004] Currently, the main method for detecting diabetes relies on blood sugar measurement, but this method requires obtaining blood samples, which is traumatic for patients and also requires specialized equipment, which is very expensive. In order to solve this detection pain point, an AI model detection method is proposed.

[0005] However, there are some problems with the existing technology: the existing AI model requires the patient's various parameters to be entered and then AI analysis and processing to be performed, which requires the patient to undergo a large number of tests and cannot be evaluated and predicted in a simple way. Therefore, we propose an early prediction system for diabetic cognitive impairment based on speech recognition. Summary of the invention

[0006] In view of the problems existing in the prior art, the purpose of the present invention is to provide an early prediction system for diabetic cognitive impairment based on speech recognition, which can collect the patient's voice, process the patient's voice and extract features, thereby facilitating the evaluation and prediction of the patient by analyzing and processing the features.

[0007] To achieve the above-mentioned object, the present invention provides the following technical solutions: an early prediction system for diabetic cognitive impairment based on speech recognition, comprising a speech acquisition module, a speech processing module, a feature extraction module, a large language model and a machine learning module;

[0008] The large language model implements AI interaction with the patient, and the voice collection module collects the patient's voice information and transmits the collected voice information to the voice processing module. After processing the voice information, the voice processing module transmits the processed voice information to the feature extraction module and the large language model. The feature extraction module transmits the extracted voice features to the machine learning model, and the large language model converts the voice information into text information, detects the patient's emotional information, and then transmits the text information and emotional information to the machine learning model. The machine learning model performs early assessment and prediction of cognitive impairment of the patient based on the voice features, text information and emotional information.

[0009] The voice collection module is used to collect the voice data of the patient's interaction with the system, and obtains the patient's voice data through simple AI dialogue, reading aloud, digital backtracking, naming objects, describing pictures and telling stories. The voice collection module collects the voice data of the patient's interaction with the system multiple times;

[0010] The speech processing module processes the speech data collected by the speech collection module, filters the noise in the speech data, performs enhancement processing on the speech data, and uses a deep learning algorithm to perform noise reduction processing on the patient's speech data;

[0011] The feature extraction module is used to extract features from the processed speech data, extract features of speech speed, intonation, and fluency from the processed speech data, and add more fine-grained acoustic features to extract pitch change features, frequency spectrum features, and formant features from the processed speech data;

[0012] The large language model is used to generate text and perform sentiment analysis on the processed voice data; the processed voice data is subjected to text analysis processing, and the dialogue system in the large language model is used to conduct dialogues with the patient to realize AI dialogues, or to generate text and pictures for the patient to read aloud and describe;

[0013] The machine learning model is used to process the extracted speech features for early assessment and prediction of cognitive impairment. The prediction results provide the following information:

[0014] Risk level of cognitive impairment: Based on the abnormalities of speech characteristics, the patient's cognitive function is assessed and a risk level or prediction result is given, including normal, mild cognitive impairment, and high risk;

[0015] Long-term monitoring and trend analysis: Conduct long-term tracking to record the changing trends of the patient's voice characteristics and promptly detect the deterioration of cognitive function.

[0016] Specifically, the voice collection module includes a buzzer, a microphone and a display. The voice collection module is connected to the large language model. The AI ​​dialogue content of the large language model is broadcast through the buzzer, and the reading content, digital backtracking sequence, named object pictures, description pictures and storytelling content of the large language model are displayed through the display, and the microphone is used to collect the patient's voice data.

[0017] Specifically, the deep learning algorithm uses a convolutional neural network, and the steps of the convolutional neural network for reducing the noise of the speech signal are as follows:

[0018] Convert speech information into a spectrogram, that is, convert speech information from a time domain signal into a time-frequency domain signal;

[0019] The convolution layer extracts key features of speech information by applying convolution kernels to the input spectrogram;

[0020] The key features are then processed through the pooling layer to reduce the size of the feature map and the computational complexity while retaining the main features of the speech information.

[0021] The main features are then mapped to the output space through the fully connected layer for classification or regression tasks. The fully connected layer maps the features extracted by convolution and pooling to the denoised and enhanced speech signal.

[0022] Specifically, the time domain signal is converted into the time-frequency domain signal by short-time Fourier transform:

[0023] Framing: Divide continuous speech information into multiple short segments according to the time window of 20-40ms;

[0024] Windowing: Apply a window function to each frame to reduce edge effects;

[0025] Fourier transform: Perform Fourier transform on each frame to obtain the spectrum information at that moment;

[0026] Obtaining the time-frequency diagram: The spectrum of each frame constitutes a time slice of the time-frequency diagram. The entire time-frequency diagram is obtained by Fourier transforming all frames.

[0027] The calculation formula of short-time Fourier transform is:

[0028]

[0029] Among them, x(τ) is the time domain signal, where τ is the integral variable, representing the change in time, w(t-τ) is the window function, representing the window of the current signal, X(t, f) is the output time-frequency signal, representing the signal strength at time t and frequency f, e -j2πfτ is the complex exponential term in the Fourier transform, which represents the complex spectral basis function at frequency f.

[0030] Specifically, the calculation formula of the convolutional layer is as follows:

[0031]

[0032] Among them, x i，j is the value of the input signal’s spectrum at position (i, j), w m，n is the weight of the convolution kernel at position (m, n), b is the bias term, and y i，j is the output after the convolution operation, M and N are the sizes of the convolution kernel, and θ is the noise of the environment;

[0033] The calculation formula of the pooling layer is as follows:

[0034]

[0035] Among them, the pooling window size is

[0036] The calculation formula of the fully connected layer is as follows:

[0037]

[0038] Among them, x is the output of the convolutional layer or pooling layer, W is the weight matrix of the fully connected layer, b is the bias term, and y is the output of the fully connected layer.

[0039] Specifically, the output of the fully connected layer is measured by the mean square error to measure the difference between the output and the original signal. The formula of the mean square error as the loss function is:

[0040]

[0041] Among them, y i is the target audio signal, the audio after noise reduction, is the output of the neural network, the filtered audio, and G is the number of data points.

[0042] Specifically, the feature extraction module extracts features through a long short-term memory network, and the calculation formula of the long short-term memory network is:

[0043] Forget Gate:

[0044] Input gate: i t =σ(W i x t +U i h t-1 +b i ),

[0045] Candidate memory cells:

[0046] Memory unit update:

[0047] Output gate: o t =σ(W o x t +U o h t-1 +b o ),

[0048] Hide status update:h t =o t tanh(C t ),

[0049] Where: W f , W i , W C , W o is the weight matrix of the input gate, forget gate, candidate memory unit and output gate; U f , U i , U C , U o is the weight matrix from the hidden layer to each gate; b f , b i , b C , b o is the bias term of each gate; C t is the state of the memory unit;

[0050] After extracting the speech speed, intonation, fluency, pitch change characteristics, frequency spectrum characteristics and formant characteristics, the feature information is used to generate a combined vector graph:

[0051]

[0052] Among them, h ts is a characteristic of speech speed, h tt It is a characteristic of intonation. tl is a characteristic of fluency, hty is the pitch change characteristic, h tp is the frequency spectrum characteristic, h tg is the formant characteristic;

[0053] By using the accumulation method, multiple feature points in each feature are accumulated:

[0054] in, is the new speech signal formed, v(h ij ) generates data in a combined vector map for feature information, including a combined vector map V 1 and combined vector graph V 2 ;

[0055] The combined vector graph is a process of fusing all features to form a new speech signal for subsequent tracking speech analysis.

[0056] Specifically, the machine learning model performs early assessment and prediction of cognitive impairment through support vector machines;

[0057] Find a hyperplane that separates samples of different categories;

[0058] The hyperplane equation is expressed as: Where S is the normal vector, the vector perpendicular to the hyperplane, and h t is the input sample, b n is the bias term;

[0059] According to the classification of the hyperplane, the features are divided into the upper and lower parts of the hyperplane, and the upper part of the hyperplane is the normal feature, and the lower part of the hyperplane is the abnormal feature;

[0060] According to the feature classification above and below the hyperplane, statistical features are performed, and normal, mild cognitive impairment, and high risk are determined based on the statistical feature data volume;

[0061] Set the total number of features to The feature above the hyperplane is R 1 , the feature below the hyperplane is R 2 ;

[0062] The calculation formula is as follows:

[0063] Among them, if the E value is below 50%, the patient is judged to be at high risk, if the E value is between 50% and 80%, the patient is judged to have mild cognitive impairment, and if the E value is above 80%, the patient is judged to be normal.

[0064] Specifically, the prediction calculation formula of the machine learning model is as follows:

[0065]

[0066] Where P(A|B) is the probability of event A occurring given that event B has occurred, P(B|A) is the probability of event B occurring given that event A has occurred, P(A) is the prior probability of event A, and P(B) is the marginal probability of event B.

[0067] The normal patients with diabetic cognitive impairment were classified as event B, and the mild cognitive impairment and high-risk patients with diabetic cognitive impairment were classified as event A.

[0068] Specifically, the machine learning model is electrically connected to an auxiliary module, and the auxiliary module includes a memory, a communicator and an alarm;

[0069] The memory is used to store the collected voice information, extracted feature information and diagnosis results;

[0070] The communicator is used to communicate with doctors and nurses to transmit the diagnosis results;

[0071] The alarm is used to give an audible and visual alarm when the diagnosis result is mild cognitive impairment and high risk.

[0072] Technical effects and advantages of the present invention:

[0073] When the present invention is used, a large language model is used to realize questioning of patients and generation of text images, which is convenient for patients to ask questions and answer and recite, and then the voice of the patient is collected through the voice collection module, and then the voice of the patient is analyzed and processed through the voice processing module. During the analysis and processing, the noise in the patient's voice information is filtered out, and then the patient is filtered and enhanced. The patient's voice information is subjected to noise reduction and enhancement processing by a deep learning algorithm, so as to improve the clarity of the patient's voice information and facilitate subsequent feature processing;

[0074] The machine learning model extracts features through long short-term memory networks, and uses a large language model to generate text and perform sentiment analysis on patients. The machine learning model generates a combined vector graph from feature information and converts it into a new voice signal. The new voice signal is combined with text to generate subsequent tracking voice analysis, which can prevent analysis errors of voice information and facilitate tracking and re-analysis, thereby improving the safety and accuracy of the analysis. The feature evaluation and processing is achieved through a support vector machine, which facilitates the classification of patients with diabetic cognitive impairment into normal, mild cognitive impairment, and high risk, as well as predictions based on the classification of diabetic cognitive impairment, thereby enabling early treatment of diabetic patients.

[0075] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 It is a schematic diagram of the system structure provided by the present invention. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0078] like Figure 1 As shown, the early prediction system of diabetic cognitive impairment based on speech recognition provided by the embodiment of the present invention includes a speech collection module, a speech processing module, a feature extraction module, a large language model and a machine learning module;

[0079] The large language model implements AI interaction with the patient, and the voice collection module collects the patient's voice information and transmits the collected voice information to the voice processing module. After processing the voice information, the voice processing module transmits the processed voice information to the feature extraction module and the large language model. The feature extraction module transmits the extracted voice features to the machine learning model, and the large language model converts the voice information into text information, detects the patient's emotional information, and then transmits the text information and emotional information to the machine learning model. The machine learning model conducts early assessment and prediction of cognitive impairment in patients based on the voice features, text information and emotional information.

[0080] The voice collection module is used to collect the voice data of the patient's interaction with the system. The voice data of the patient is obtained through simple AI dialogue, reading aloud, digital backtracking, naming objects, describing pictures and telling stories. The voice collection module collects the voice data of the patient's interaction with the system multiple times;

[0081] The speech processing module processes the speech data collected by the speech collection module, filters the noise in the speech data, enhances the speech data, and uses a deep learning algorithm to reduce the noise of the patient's speech data;

[0082] The feature extraction module is used to extract features from the processed speech data, extract the features of speech speed, intonation, and fluency from the processed speech data, and add more fine-grained acoustic features to extract pitch change features, frequency spectrum features, and formant features from the processed speech data;

[0083] The large language model is used to generate text and perform sentiment analysis on the processed speech data. The processed speech data is subjected to text analysis, and the dialogue system in the large language model is used to conduct dialogues with the patient, realize AI dialogues, or generate text and pictures for the patient to read and describe.

[0084] The machine learning model is used to process the extracted speech features for early assessment and prediction of cognitive impairment. The prediction results provide the following information:

[0085] Risk level of cognitive impairment: Based on the abnormalities of speech characteristics, the patient's cognitive function is assessed and a risk level or prediction result is given, including normal, mild cognitive impairment, and high risk;

[0086] Long-term monitoring and trend analysis: Conduct long-term tracking to record the changing trends of the patient's voice characteristics and promptly detect the deterioration of cognitive function.

[0087] In this embodiment, preferably, the voice collection module includes a buzzer, a microphone and a display. The voice collection module is connected to the large language model, and the AI ​​dialogue content of the large language model is broadcast through the buzzer, and the reading content, digital backtracking sequence, named object pictures, description pictures and storytelling content of the large language model are displayed through the display, and the microphone is used to collect the patient's voice data;

[0088] It should be noted that the question and answer information of the large language model is played through a buzzer, and the text and pictures of the large language model are displayed through a display, which is convenient for patients to recite or describe, and the patient's voice information is obtained through a microphone, which is convenient for subsequent analysis and processing of the voice information.

[0089] In this embodiment, preferably, the deep learning algorithm adopts a convolutional neural network, and the convolutional neural network performs the following steps to reduce the noise of the speech signal:

[0090] Convert speech information into a spectrogram, that is, convert speech information from a time domain signal into a time-frequency domain signal;

[0091] The convolution layer extracts key features of speech information by applying convolution kernels to the input spectrogram;

[0092] The key features are then processed through the pooling layer to reduce the size of the feature map and the computational complexity while retaining the main features of the speech information.

[0093] The main features are then mapped to the output space through the fully connected layer for classification or regression tasks. The fully connected layer maps the features extracted by convolution and pooling to the denoised and enhanced speech signal.

[0094] It should be noted that the patient's voice information is analyzed and processed through a deep learning algorithm, and the voice information is converted from a time domain signal to a time-frequency domain signal, which facilitates the convolution layer and the pooling layer to perform time-frequency domain signal analysis and processing, and facilitates the extracted features to be mapped to the noise-reduced and enhanced voice signal, thereby improving the accuracy of the voice signal.

[0095] In this embodiment, preferably, the time domain signal is converted into the time-frequency domain signal by short-time Fourier transform:

[0096] Framing: Divide continuous speech information into multiple short segments according to the time window of 20-40ms;

[0097] Windowing: Apply a window function to each frame to reduce edge effects;

[0098] Fourier transform: Perform Fourier transform on each frame to obtain the spectrum information at that moment;

[0099] Obtaining the time-frequency diagram: The spectrum of each frame constitutes a time slice of the time-frequency diagram. The entire time-frequency diagram is obtained by Fourier transforming all frames.

[0100] The calculation formula of short-time Fourier transform is:

[0101]

[0102] Among them, x(τ) is the time domain signal, where τ is the integral variable, representing the change in time, w(t-τ) is the window function, representing the window of the current signal, X(t, f) is the output time-frequency signal, representing the signal strength at time t and frequency f, e -j2πfτ It is the complex exponential term in Fourier transform, which represents the complex spectral basis function at frequency f;

[0103] It should be noted that the short-time Fourier transform is used to convert the time domain signal into a time-frequency signal, which facilitates the subsequent filtering and enhancement processing of the voice information.

[0104] In this embodiment, preferably, the calculation formula of the convolutional layer is as follows:

[0105]

[0106] Among them, x i，jis the value of the input signal’s spectrum at position (i, j), w m，n is the weight of the convolution kernel at position (m, n), b is the bias term, and y i，j is the output after the convolution operation, N and N are the sizes of the convolution kernel, and θ is the noise of the environment;

[0107] The calculation formula of the pooling layer is as follows:

[0108]

[0109] Among them, the pooling window size is

[0110] The calculation formula of the fully connected layer is as follows:

[0111]

[0112] Where x is the output of the convolutional layer or pooling layer, W is the weight matrix of the fully connected layer, b is the bias term, and y is the output of the fully connected layer;

[0113] It should be noted that the application of convolutional neural networks in audio processing, especially audio filtering, usually involves convolution operations on the time-frequency representation of audio signals. Through convolution operations, it can automatically learn to extract key audio information from noise, thereby achieving tasks such as noise suppression and signal enhancement.

[0114] In this embodiment, preferably, the difference between the output of the fully connected layer and the original signal is measured by the mean square error, and the formula of the mean square error as the loss function is:

[0115]

[0116] Among them, y i is the target audio signal, the audio after noise reduction, is the output of the neural network, the filtered audio, and G is the number of data points;

[0117] It should be noted that through the back-propagation algorithm and gradient descent method, the network weights (convolution kernels, fully connected layer weights, etc.) will be continuously adjusted during the training process to minimize the loss function, ultimately enabling the network to accurately perform audio filtering.

[0118] In this embodiment, preferably, the feature extraction module extracts features through a long short-term memory network, and the calculation formula of the long short-term memory network is:

[0119] Forget Gate:

[0120] Input gate: i t =σ(W i x t+U i h t-1 +b i ),

[0121] Candidate memory cells:

[0122] Memory unit update:

[0123] Output gate: o t =σ(W o x t +U o h t-1 +b o ),

[0124] Hide status update:h t =o t tanh(C t ),

[0125] Where: W f , W i , W C , W o is the weight matrix of the input gate, forget gate, candidate memory unit and output gate; U f , U i , U C , U o is the weight matrix from the hidden layer to each gate; b f , b i , b C , b o is the bias term of each gate; C t is the state of the memory unit;

[0126] After extracting the speech speed, intonation, fluency, pitch change characteristics, frequency spectrum characteristics and formant characteristics, the feature information is used to generate a combined vector graph:

[0127]

[0128] Among them, h ts is a characteristic of speech speed, h tt It is a characteristic of intonation. tl is a characteristic of fluency, h ty is the pitch change characteristic, h tp is the frequency spectrum characteristic, h tg is the formant characteristic;

[0129] By using the accumulation method, multiple feature points in each feature are accumulated:

[0130] in, is the new speech signal formed, v(h ij ) generates data in a combined vector map for feature information, including a combined vector map V 1 and combined vector graph V 2 ;

[0131] The combined vector graph is to fuse all the features to form a new speech signal for subsequent tracking speech analysis;

[0132] It should be noted that by introducing memory units and multiple gating mechanisms (input gate, forget gate, output gate) to control the flow of information, the network can retain or forget information over a longer time span, thereby effectively capturing the long-term dependencies in speech signals, and effectively capturing the timing characteristics of speech signals, improving the performance of tasks such as speech recognition and sentiment analysis, and generating a combined vector graph of feature information to facilitate the generation of new speech signals from features, which is convenient for subsequent tracking, analysis and processing.

[0133] In this embodiment, preferably, the machine learning model performs early assessment and prediction of cognitive impairment through a support vector machine;

[0134] Find a hyperplane that separates samples of different categories;

[0135] The hyperplane equation is expressed as: Where S is the normal vector, the vector perpendicular to the hyperplane, and h t is the input sample, b n is the bias term;

[0136] According to the classification of the hyperplane, the features are divided into the upper and lower parts of the hyperplane, and the upper part of the hyperplane is the normal feature, and the lower part of the hyperplane is the abnormal feature;

[0137] According to the feature classification above and below the hyperplane, statistical features are performed, and normal, mild cognitive impairment, and high risk are determined based on the statistical feature data volume;

[0138] Set the total number of features to The feature above the hyperplane is R 1 , the feature below the hyperplane is R 2 ;

[0139] The calculation formula is as follows:

[0140] If the value of E is below 50%, the patient is considered to be at high risk; if the value of E is between 50% and 80%, the patient is considered to have mild cognitive impairment; if the value of E is above 80%, the patient is considered to be normal.

[0141] It should be noted that the feature division is achieved through the support vector machine, which facilitates the division of feature information, and the level of cognitive impairment of patients is divided through the divided feature information.

[0142] In this embodiment, preferably, the prediction calculation formula of the machine learning model is as follows:

[0143]

[0144] Where P(A|B) is the probability of event A occurring given that event B has occurred, P(B|A) is the probability of event B occurring given that event A has occurred, P(A) is the prior probability of event A, and P(B) is the marginal probability of event B.

[0145] And the normal patients with diabetic cognitive impairment were classified as event B, and the mild cognitive impairment and high risk patients with diabetic cognitive impairment were classified as event A;

[0146] It should be noted that after the cognitive impairment is graded, it is convenient to perform predictive processing based on the patient's normal data and abnormal data, and to make predictions based on the patient's cognitive impairment to achieve early treatment of the patient.

[0147] In this embodiment, preferably, the machine learning model is electrically connected to an auxiliary module, and the auxiliary module includes a memory, a communicator and an alarm;

[0148] The memory is used to store the collected voice information, extracted feature information and diagnosis results;

[0149] The communicator is used to communicate with doctors and nurses to transmit the diagnosis results;

[0150] The alarm is used to give an audible and visual alarm when the diagnosis result is mild cognitive impairment and high risk;

[0151] It should be noted that the auxiliary module is used to assist the system in operation, improve the stability of the system, and store various information of the patient through the memory, so as to facilitate timely contact with doctors or nurses.

[0152] The specific operation process of the present invention is as follows:

[0153] When in use, the large language model implements AI interaction with the patient, and the voice collection module collects the patient's voice information and transmits the collected voice information to the voice processing module. The voice processing module uses a deep learning algorithm to perform noise reduction on the patient's voice data;

[0154] The deep learning algorithm uses a convolutional neural network. The steps of convolutional neural network for speech signal noise reduction are as follows:

[0155] Convert speech information into a spectrogram, that is, convert speech information from a time domain signal into a time-frequency domain signal;

[0156] The convolution layer extracts key features of speech information by applying convolution kernels to the input spectrogram;

[0157] The key features are then processed through the pooling layer to reduce the size of the feature map and the computational complexity while retaining the main features of the speech information.

[0158] The main features are then mapped to the output space through the fully connected layer for classification or regression tasks. The fully connected layer maps the features extracted by convolution and pooling to the denoised and enhanced speech signal.

[0159] The short-time Fourier transform is used to convert the time domain signal into the time-frequency domain signal:

[0160] Framing: Divide continuous speech information into multiple short segments according to the time window of 20-40ms;

[0161] Windowing: Apply a window function to each frame to reduce edge effects;

[0162] Fourier transform: Perform Fourier transform on each frame to obtain the spectrum information at that moment;

[0163] Obtaining the time-frequency diagram: The spectrum of each frame constitutes a time slice of the time-frequency diagram. The entire time-frequency diagram is obtained by Fourier transforming all frames.

[0164] The calculation formula of short-time Fourier transform is:

[0165]

[0166] Among them, x(τ) is the time domain signal, where τ is the integral variable, representing the change in time, w(t-τ) is the window function, representing the window of the current signal, X(t, f) is the output time-frequency signal, representing the signal strength at time t and frequency f, e -j2πfτ is the complex exponential term in the Fourier transform, which represents the complex spectral basis function at frequency f.

[0167] The calculation formula of the convolutional layer is as follows:

[0168]

[0169] Among them, x i，j is the value of the input signal’s spectrum at position (i, j), w m，n is the weight of the convolution kernel at position (m, n), b is the bias term, and y i，jis the output after the convolution operation, M and N are the sizes of the convolution kernel, and θ is the noise of the environment;

[0170] The calculation formula of the pooling layer is as follows:

[0171]

[0172] Among them, the pooling window size is

[0173] The calculation formula of the fully connected layer is as follows:

[0174]

[0175] Among them, x is the output of the convolutional layer or pooling layer, W is the weight matrix of the fully connected layer, b is the bias term, and y is the output of the fully connected layer.

[0176] The output of the fully connected layer is measured by the mean square error to measure the difference between the original signal and the mean square error as the loss function:

[0177]

[0178] Among them, y i is the target audio signal, the audio after noise reduction, is the output of the neural network, the filtered audio, and G is the number of data points;

[0179] After processing the speech information, the speech processing module transmits the processed speech information to the feature extraction module and the large language model;

[0180] The feature extraction module extracts features through the long short-term memory network. The calculation formula of the long short-term memory network is:

[0181] Forget gate: f t =σ(W f x t +U f h t-1 +b f ),

[0182] Input gate: i t =σ(W i x t +U i h t-1 +b i ),

[0183] Candidate memory cells:

[0184] Memory unit update:

[0185] Output gate: ot =σ(W o x t +U o h t-1 +b o ),

[0186] Hide status update:h t =o t tanh(C t ),

[0187] Where: W f , W i , W C , W o is the weight matrix of the input gate, forget gate, candidate memory unit and output gate; U f , U i , U C , U o is the weight matrix from the hidden layer to each gate; b f , b i , b C , b o is the bias term of each gate; C t is the state of the memory unit;

[0188] After extracting the speech speed, intonation, fluency, pitch change characteristics, frequency spectrum characteristics and formant characteristics, the feature information is used to generate a combined vector graph:

[0189]

[0190] Among them, h ts is a characteristic of speech speed, h tt It is a characteristic of intonation. tl is a characteristic of fluency, h ty is the pitch change characteristic, h tp is the frequency spectrum characteristic, h tg is the formant characteristic;

[0191] By using the accumulation method, multiple feature points in each feature are accumulated:

[0192] f t =σ(W f x t +U f h t-1 +b f ),in, is the new speech signal formed, v(h ij ) generates data in a combined vector map for feature information, including a combined vector map V 1 and combined vector graph V 2 ;

[0193] The combined vector graph is to fuse all the features to form a new speech signal for subsequent tracking speech analysis;

[0194] The feature extraction module transmits the extracted speech features to the machine learning model, and the large language model converts the speech information into text information, detects the patient's emotional information, and then transmits the text information and emotional information to the machine learning model. The machine learning model performs early assessment and prediction of cognitive impairment of the patient based on the speech features, text information and emotional information;

[0195] Machine learning models use support vector machines for early assessment and prediction of cognitive impairment;

[0196] Find a hyperplane that separates samples of different categories;

[0197] The hyperplane equation is expressed as: S·h t +b n = 0, where S is the normal vector, the vector perpendicular to the hyperplane, and h t is the input sample, h n is the bias term;

[0198] According to the classification of the hyperplane, the features are divided into the upper and lower parts of the hyperplane, and the upper part of the hyperplane is the normal feature, and the lower part of the hyperplane is the abnormal feature;

[0199] According to the feature classification above and below the hyperplane, statistical features are performed, and normal, mild cognitive impairment, and high risk are determined based on the statistical feature data volume;

[0200] Set the total number of features to S h t +b n = 0, the feature above the hyperplane is R 1 , the feature below the hyperplane is R 2 ;

[0201] The calculation formula is as follows: S·h t +b n =0;

[0202] Among them, if the E value is below 50%, the patient is judged to be at high risk, if the E value is between 50% and 80%, the patient is judged to have mild cognitive impairment, and if the E value is above 80%, the patient is judged to be normal.

[0203] The prediction calculation formula of the machine learning model is as follows:

[0204]

[0205] Where P(A|B) is the probability of event A occurring given that event B has occurred, P(B|A) is the probability of event B occurring given that event A has occurred, P(A) is the prior probability of event A, and P(B) is the marginal probability of event B.

[0206] And the normal patients with diabetic cognitive impairment were classified as event B, and the mild cognitive impairment and high risk patients with diabetic cognitive impairment were classified as event A;

[0207] The machine learning model extracts features through long short-term memory networks, and uses a large language model to generate text and perform sentiment analysis on patients. The machine learning model generates a combined vector graph from feature information and converts it into a new voice signal. The new voice signal is combined with text to generate subsequent tracking voice analysis, which can prevent analysis errors of voice information and facilitate tracking and re-analysis, thereby improving the safety and accuracy of the analysis. The feature evaluation and processing is achieved through a support vector machine, which facilitates the classification of patients with diabetic cognitive impairment into normal, mild cognitive impairment, and high risk, as well as predictions based on the classification of diabetic cognitive impairment, thereby enabling early treatment of diabetic patients.

[0208] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An early prediction system for diabetic cognitive impairment based on speech recognition, characterized in that: It includes speech acquisition module, speech processing module, feature extraction module, large language model and machine learning module; The large language model implements AI interaction with the patient, and the voice collection module collects the patient's voice information and transmits the collected voice information to the voice processing module. After processing the voice information, the voice processing module transmits the processed voice information to the feature extraction module and the large language model. The feature extraction module transmits the extracted voice features to the machine learning model, and the large language model converts the voice information into text information, detects the patient's emotional information, and then transmits the text information and emotional information to the machine learning model. The machine learning model performs early assessment and prediction of cognitive impairment of the patient based on the voice features, text information and emotional information. The voice collection module is used to collect the voice data of the patient's interaction with the system, and obtains the patient's voice data through simple AI dialogue, reading aloud, digital backtracking, naming objects, describing pictures and telling stories. The voice collection module collects the voice data of the patient's interaction with the system multiple times; The speech processing module processes the speech data collected by the speech collection module, filters the noise in the speech data, performs enhancement processing on the speech data, and uses a deep learning algorithm to perform noise reduction processing on the patient's speech data; The feature extraction module is used to extract features from the processed speech data, extract features of speech speed, intonation, and fluency from the processed speech data, and add more fine-grained acoustic features to extract pitch change features, frequency spectrum features, and formant features from the processed speech data; The large language model is used to generate text and perform sentiment analysis on the processed voice data; the processed voice data is subjected to text analysis processing, and the dialogue system in the large language model is used to conduct dialogues with the patient to realize AI dialogues, or to generate text and pictures for the patient to read aloud and describe; The machine learning model is used to process the extracted speech features for early assessment and prediction of cognitive impairment. The prediction results provide the following information: Risk level of cognitive impairment: Based on the abnormalities of speech characteristics, the patient's cognitive function is assessed and a risk level or prediction result is given, including normal, mild cognitive impairment, and high risk; Long-term monitoring and trend analysis: Conduct long-term tracking to record the changing trends of the patient's voice characteristics and promptly detect the deterioration of cognitive function.

2. The early prediction system for diabetic cognitive impairment based on speech recognition according to claim 1 is characterized by: The voice collection module includes a buzzer, a microphone and a display. The voice collection module is connected to the large language model. The AI ​​dialogue content of the large language model is broadcasted through the buzzer, and the reading content, digital backtracking sequence, named object pictures, description pictures and storytelling content of the large language model are displayed through the display. The microphone is used to collect the patient's voice data.

3. The early prediction system for diabetic cognitive impairment based on speech recognition according to claim 1 is characterized by: The deep learning algorithm uses a convolutional neural network. The steps of the convolutional neural network to reduce the noise of the speech signal are as follows: Convert speech information into a spectrogram, that is, convert speech information from a time domain signal into a time-frequency domain signal; The convolution layer extracts key features of speech information by applying convolution kernels to the input spectrogram; The key features are then processed through the pooling layer to reduce the size of the feature map and the computational complexity while retaining the main features of the speech information. The main features are then mapped to the output space through the fully connected layer for classification or regression tasks. The fully connected layer maps the features extracted by convolution and pooling to the denoised and enhanced speech signal.

4. The early prediction system for diabetic cognitive impairment based on speech recognition according to claim 3 is characterized by: The time domain signal is converted into a time-frequency domain signal by short-time Fourier transform: Framing: Divide continuous speech information into multiple short segments according to the time window of 20-40ms; Windowing: Apply a window function to each frame to reduce edge effects; Fourier transform: Perform Fourier transform on each frame to obtain the spectrum information at that moment; Obtaining the time-frequency diagram: The spectrum of each frame constitutes a time slice of the time-frequency diagram. The entire time-frequency diagram is obtained by Fourier transforming all frames. The calculation formula of short-time Fourier transform is: Among them, x(τ) is the time domain signal, where τ is the integral variable, representing the change in time, w(t-τ) is the window function, representing the window of the current signal, X(t, f) is the output time-frequency signal, representing the signal strength at time t and frequency f, e -j2πfτ is the complex exponential term in the Fourier transform, which represents the complex spectral basis function at frequency f.

5. The early prediction system for diabetic cognitive impairment based on speech recognition according to claim 3 is characterized by: The calculation formula of the convolutional layer is as follows: Among them, x i，j is the value of the input signal’s spectrum at position (i, j), w m，n is the weight of the convolution kernel at position (m, n), b is the bias term, and y i，j is the output after the convolution operation, M and N are the sizes of the convolution kernel, and θ is the noise of the environment; The calculation formula of the pooling layer is as follows: Among them, the pooling window size is p×q; The calculation formula of the fully connected layer is as follows: y=W·x+b, Among them, x is the output of the convolutional layer or pooling layer, W is the weight matrix of the fully connected layer, b is the bias term, and y is the output of the fully connected layer.

6. The early prediction system for diabetic cognitive impairment based on speech recognition according to claim 5 is characterized by: The output of the fully connected layer is measured by the mean square error to measure the difference between the original signal and the mean square error as the loss function: Among them, y i is the target audio signal, the audio after noise reduction, is the output of the neural network, the filtered audio, and G is the number of data points.

7. The early prediction system for diabetic cognitive impairment based on speech recognition according to claim 1 is characterized by: The feature extraction module extracts features through a long short-term memory network, and the calculation formula of the long short-term memory network is: Forget gate: f t =σ(W f x t +U f h t-1 +b f ), Input gate: i t =σ(W i x t +U i h t-1 +b i ), Candidate memory cells: Memory unit update: Output gate: o t =σ(W o x t +U o h t-1 +b o ), Hide status update:h t =o t tanh(C t ), Where: W f , W i , W C , W o is the weight matrix of the input gate, forget gate, candidate memory unit and output gate; U f , U i , U C , U o is the weight matrix from the hidden layer to each gate; b f , b i , b C , b o is the bias term of each gate; C t is the state of the memory unit; After extracting the speech speed, intonation, fluency, pitch change characteristics, frequency spectrum characteristics and formant characteristics, the feature information is used to generate a combined vector graph: Among them, h ts is a characteristic of speech speed, h tt It is a characteristic of intonation. tl is a characteristic of fluency, h ty is the pitch change characteristic, h tp is the frequency spectrum characteristic, h tg is the formant characteristic; By using the accumulation method, multiple feature points in each feature are accumulated: in, is the new speech signal formed, v(h ij ) generates data in a combined vector graph for feature information, including a combined vector graph V1 and a combined vector graph V2; The combined vector graph is a process of fusing all features to form a new speech signal for subsequent tracking speech analysis.

8. The early prediction system for diabetic cognitive impairment based on speech recognition according to claim 1 is characterized by: The machine learning model performs early assessment and prediction of cognitive impairment through support vector machine; Find a hyperplane that separates samples of different categories; The hyperplane equation is expressed as: S·h t +b n = 0, where S is the normal vector, the vector perpendicular to the hyperplane, and h t is the input sample, b n is the bias term; According to the classification of the hyperplane, the features are divided into the upper and lower parts of the hyperplane, and the upper part of the hyperplane is the normal feature, and the lower part of the hyperplane is the abnormal feature; According to the feature classification above and below the hyperplane, statistical features are performed, and normal, mild cognitive impairment, and high risk are determined based on the statistical feature data volume; Set the total number of features to K, the features above the hyperplane to R1, and the features below the hyperplane to R2; The calculation formula is as follows: Among them, if the E value is below 50%, the patient is judged to be at high risk, if the E value is between 50% and 80%, the patient is judged to have mild cognitive impairment, and if the E value is above 80%, the patient is judged to be normal.

9. The early prediction system for diabetic cognitive impairment based on speech recognition according to claim 8, characterized in that: The prediction calculation formula of the machine learning model is as follows: Where P(A|B) is the probability of event A occurring given that event B has occurred, P(B|A) is the probability of event B occurring given that event A has occurred, P(A) is the prior probability of event A, and P(B) is the marginal probability of event B. The normal patients with diabetic cognitive impairment were classified as event B, and the mild cognitive impairment and high-risk patients with diabetic cognitive impairment were classified as event A.

10. The early prediction system for diabetic cognitive impairment based on speech recognition according to claim 1, characterized in that: The machine learning model is electrically connected to an auxiliary module, wherein the auxiliary module includes a memory, a communicator and an alarm; The memory is used to store the collected voice information, extracted feature information and diagnosis results; The communicator is used to communicate with doctors and nurses to transmit the diagnosis results; The alarm is used to give an audible and visual alarm when the diagnosis result is mild cognitive impairment and high risk.

Images