Parkinson's speech recognition method and device based on PSO dynamic search classification network

Abstract

The present invention discloses a Parkinson's speech recognition method and device based on a PSO dynamic search classification network. The method comprises the following steps: acquiring speech data of a Parkinson's patient, editing the speech data, extracting speech features from the edited speech data, and preparing a speech feature data set; initializing parameters of a dynamic search classification network, and generating a classification network particle swarm by random initialization; training a classification network in each generation of the classification network particle swarm using the speech feature data set to obtain a fitness function value of each particle, updating the classification network particle swarm according to the fitness function value of each particle, cyclically training and updating the classification network until an optimal classification network is obtained, and performing Parkinson's speech recognition on the speech feature data to be recognized based on the optimal classification network obtained through training.

Description

Technical Field

[0001] The present invention is a Parkinson's speech recognition method based on a PSO (particle swarm optimization algorithm) dynamic search classification network structure, and relates to intelligent medical technology. Background Art

[0002] Parkinson's disease is a neurological disorder caused by the loss of dopamine-producing cells in the brain, which impairs brain function. As the disease progresses, coordination between the brain and various parts of the body becomes impaired. Nearly 90% of Parkinson's patients exhibit early symptoms of voice disorders. Furthermore, the speech signals of Parkinson's patients differ significantly from those of healthy individuals. Therefore, speech signals are used to assist in Parkinson's disease diagnosis. Currently, common methods for Parkinson's speech classification include support vector machines, random forests, multi-layer perceptrons, and neural networks.

[0003] The above methods have made great progress in Parkinson's classification for a certain type of speech signal. However, most of these methods extract common features of speech signals based on a certain type of speech signal and set classifiers in a targeted manner. The methods are not universal. Summary of the Invention

[0004] The problem to be solved by the present invention is the difficulty that speech signals of different languages and contents bring to Parkinson's speech recognition. The present invention provides a Parkinson's speech recognition method based on a PSO dynamic search classification network structure.

[0005] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:

[0006] In a first aspect, a Parkinson's speech recognition method is provided, comprising:

[0007] Step (1), obtaining speech data of Parkinson's patients;

[0008] Step (2), editing the acquired voice data to obtain edited voice data;

[0009] Step (3), extracting speech features from the clipped speech data to obtain a speech feature dataset;

[0010] Step (4), by dynamically searching the parameters of the classification network according to the preset conditions, randomly initializing and generating the classification network particle swarm;

[0011] Step (5), using the speech feature data set to iteratively train and update the classification network particle swarm to obtain a trained and updated dynamic search classification network model;

[0012] Step (6): inputting the speech feature data to be recognized into the trained and updated dynamic search classification network model, and determining the Parkinson's speech recognition result according to the output of the dynamic search classification network model.

[0013] In some embodiments, step (1), obtaining speech data of a Parkinson's patient, includes:

[0014] The speech data of Parkinson's patients include the King's College London Early and Late Parkinson's patients (MDVR-KCL) speech dataset, a speech dataset of healthy controls and / or an Italian Parkinson's speech dataset;

[0015] The MDVR-KCL speech dataset of early and late Parkinson's patients and healthy controls is in English and contains the reading of two texts.

[0016] The Italian Parkinson's speech dataset contains monosyllabic pronunciation data, including repeated pronunciations of the syllables 'a', 'i', 'e', 'o', 'u', 'ta', and 'pa'.

[0017] In some embodiments, in step (2), the acquired voice data is edited, including:

[0018] For the MDVR-KCL speech dataset of early and late Parkinson's patients and healthy controls, the speech was edited according to the sentence segmentation of the read text, and the blank speech segments with pauses were removed;

[0019] For the Italian Parkinson's speech dataset, each syllable pronunciation segment is clipped and the blank speech segments during the pause are removed.

[0020] In some embodiments, step (3), extracting speech features from the clipped speech data to obtain a speech feature dataset, includes:

[0021] The speech feature dataset includes 26 basic speech features, Mel-frequency cepstral coefficient features and 13 global features:

[0022] A set of 26 basic speech features were extracted from each speech signal using Praat acoustic analysis software;

[0023] Based on short-time Fourier transform, a Mel filter bank is used to extract the Mel cepstral coefficient features of the speech signal;

[0024] For speech signals, 13 global features are extracted, including mean, median, root mean square, square, maximum, minimum, first and third quartiles, interquartile range, standard deviation, skewness and kurtosis.

[0025] In some embodiments, step (4) is to randomly initialize and generate a classification network particle swarm by dynamically searching the parameters of the classification network according to preset conditions, including: initializing the population size to 20, the number of iterations to 10, the number of classification network layers to be in the range of [3, 10], requiring the first layer of the classification network to be a convolutional layer, and the last layer to be a fully connected layer, randomly initializing the structure of each classification network in the classification network particle swarm, and the number of main cycles to be 10 during the training of the classification network particle swarm.

[0026] In some embodiments, step (5) uses the speech feature data set to iteratively train and update the classification network particle swarm to obtain a trained and updated dynamic search classification network model;

[0027] The classification network in each generation of classification network particle swarm is trained using the speech feature dataset to obtain the fitness function value of each particle; the classification network particle swarm is updated according to the fitness function value of each particle, and this step is iteratively executed until the iteration stop condition is met, obtaining an optimal classification network as the dynamic search classification network model.

[0028] Furthermore, the classification network in each generation of classification network particle swarm is trained using the speech feature dataset to obtain the fitness function value of each particle, including:

[0029] The speech feature data set is used to train each classification network in the classification network particle swarm, calculate the loss function, and use the loss function to evaluate the classification network particle swarm to obtain the fitness function value of each particle. According to the result of comparing the fitness function value of each particle with the fitness function value of the global best classification network and the current particle best classification network, the global best classification network and the current particle best classification network are updated, and the classification network particle swarm is updated according to the global best classification network and the current particle best classification network.

[0030] In some embodiments, the method for updating the classification network includes: calculating the velocity v of the current particle by comparing the differences between network structures, and updating the classification network structure according to the following formula:

[0031] v i =v i +c1×rand()×(pbest i -x i )+c2×rand()×(gbest i -x i )

[0032] x i =x i +v i

[0033] Among them, gbesti is the global best classification network, pbest i is the current best particle classification network, i=1,2,…,N is the total number of particles in the group, v i is the velocity of the particle, x i is the current position of the particle, c1 and c2 are learning factors, and rand() is a random number between (0, 1).

[0034] In a second aspect, the present invention provides a Parkinson's speech recognition device, comprising a processor and a storage medium;

[0035] The storage medium is used to store instructions;

[0036] The processor is configured to operate according to the instructions to execute the steps of the method according to the first aspect.

[0037] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when executed by a processor.

[0038] The beneficial effects achieved by the present invention are:

[0039] This paper proposes a Parkinson's disease speech recognition method based on a PSO (Particle Swarm Optimization) algorithm that dynamically searches for a classification network structure. Using public datasets with diverse language content, a set of key feature combinations for Parkinson's disease speech recognition is extracted from the source speech files to construct a speech feature dataset. Using the PSO algorithm, a classification network is automatically constructed for each speech type, eliminating the need for manual design of classification networks for different languages. This method enables dynamic search of classification networks for Parkinson's disease speech recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0041] Figure 2 This is a detailed diagram of the PSO dynamic search classification network structure in an embodiment of the present invention;

[0042] Figure 3 This is a diagram of the optimal classification network structure obtained for the Italian Parkinson's speech dataset in the embodiment;

[0043] Figure 4 This is a diagram of the optimal classification network structure obtained in the embodiment for the speech dataset of MDVR-KCL early and late Parkinson's patients and healthy controls. DETAILED DESCRIPTION

[0044] The present invention will be further described below in conjunction with the accompanying drawings and examples. The following examples are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0045] In the description of the present invention, "several" means more than one, "plurality" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0046] In the description of the present invention, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the exemplary expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0047] Example 1

[0048] A Parkinson's speech recognition method based on a PSO dynamic search classification network, comprising:

[0049] Step (1), obtaining speech data of Parkinson's patients;

[0050] Step (2), editing the acquired voice data to obtain edited voice data;

[0051] Step (3), extracting speech features from the clipped speech data to obtain a speech feature dataset;

[0052] Step (4), by dynamically searching the parameters of the classification network according to the preset conditions, randomly initializing and generating the classification network particle swarm;

[0053] Step (5), using the speech feature data set to iteratively train and update the classification network particle swarm to obtain a trained and updated dynamic search classification network model;

[0054] Step (6): inputting the speech feature data to be recognized into the trained and updated dynamic search classification network model, and determining the Parkinson's speech recognition result according to the output of the dynamic search classification network model.

[0055] In some embodiments, step (1), obtaining speech data of a Parkinson's patient, includes:

[0056] The speech data of Parkinson's disease patients include the speech dataset of early and late Parkinson's disease patients from King's College London (MDVR-KCL, Mobile Device Voice Recordings at King's College London) and the speech dataset of healthy controls and / or the Italian Parkinson's disease speech dataset;

[0057] The MDVR-KCL speech dataset of early and late Parkinson's patients and the speech dataset of healthy controls are in English and contain the reading speech of two texts.

[0058] The Italian Parkinson's speech dataset contains monosyllabic pronunciation data, including repeated pronunciations of the syllables 'a', 'i', 'e', 'o', 'u', 'ta', and 'pa'.

[0059] In some embodiments, in step (2), the acquired voice data is edited, including:

[0060] For the MDVR-KCL speech dataset of early and late Parkinson's patients and healthy controls, the speech was edited according to the sentence segmentation of the read text, and the blank speech segments with pauses were removed;

[0061] For the Italian Parkinson's speech dataset, each syllable pronunciation segment is clipped and the blank speech segments during the pause are removed.

[0062] In some embodiments, step (3), extracting speech features from the clipped speech data to obtain a speech feature dataset, includes:

[0063] The speech feature dataset includes 26 basic speech features, Mel-frequency cepstral coefficient features and 13 global features:

[0064] A set of 26 basic speech features were extracted from each speech signal using Praat acoustic analysis software;

[0065] Based on short-time Fourier transform, a Mel filter bank is used to extract the Mel cepstral coefficient features of the speech signal;

[0066] For speech signals, 13 global features are extracted, including mean, median, root mean square, square, maximum, minimum, first and third quartiles, interquartile range, standard deviation, skewness and kurtosis.

[0067] In some embodiments, step (4) is to randomly initialize and generate a classification network particle swarm by dynamically searching the parameters of the classification network according to preset conditions, including: initializing the population size to 20, the number of iterations to 10, the number of classification network layers to be in the range of [3, 10], requiring the first layer of the classification network to be a convolutional layer, and the last layer to be a fully connected layer, randomly initializing the structure of each classification network in the classification network particle swarm, and the number of main cycles to be 10 during the training of the classification network particle swarm.

[0068] In some embodiments, step (5) uses the speech feature data set to iteratively train and update the classification network particle swarm to obtain a trained and updated dynamic search classification network model;

[0069] The classification network in each generation of classification network particle swarm is trained using the speech feature dataset to obtain the fitness function value of each particle; the classification network particle swarm is updated according to the fitness function value of each particle, and this step is iteratively executed until the iteration stop condition is met, obtaining an optimal classification network as the dynamic search classification network model.

[0070] Furthermore, the classification network in each generation of classification network particle swarm is trained using the speech feature dataset to obtain the fitness function value of each particle, including:

[0071] The speech feature data set is used to train each classification network in the classification network particle swarm, calculate the loss function, and use the loss function to evaluate the classification network particle swarm to obtain the fitness function value of each particle. According to the result of comparing the fitness function value of each particle with the fitness function value of the global best classification network and the current particle best classification network, the global best classification network and the current particle best classification network are updated, and the classification network particle swarm is updated according to the global best classification network and the current particle best classification network.

[0072] In some embodiments, the method for updating the classification network includes: calculating the velocity v of the current particle by comparing the differences between network structures, and updating the classification network structure according to the following formula:

[0073] v i =v i +c1×rand()×(pbest i -x i )+c2×rand()×(gbest i -x i )

[0074] x i =x i +v i

[0075] Among them, gbesti is the global best classification network, pbest i is the current best particle classification network, i=1,2,…,N is the total number of particles in the group, v i is the velocity of the particle, x i is the current position of the particle, c1 and c2 are learning factors, and rand() is a random number between (0, 1).

[0076] In this embodiment, the Pbest and Gbest of the classification network particle swarm are updated according to the fitness function value of each particle, and the classification network is updated according to Pbest and Gbest. Through continuous training and updating, the optimal classification network corresponding to the speech feature dataset is found. The optimal classification network for each speech dataset is found for Parkinson's disease recognition. The final classification network structure diagram is shown in the figure below. Figure 3 and Figure 4 shown.

[0077] In some embodiments, based on the above-mentioned optimal classification network, another part of the data in the above-mentioned speech feature dataset is used as a validation set for Parkinson's disease recognition.

[0078] Example 2

[0079] In a second aspect, this embodiment provides a Parkinson's speech recognition device, including a processor and a storage medium;

[0080] The storage medium is used to store instructions;

[0081] The processor is configured to operate according to the instructions to execute the steps of the method according to embodiment 1.

[0082] Example 3

[0083] In a third aspect, this embodiment provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in Example 1 are implemented.

[0084] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0085] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0086] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0088] The above is only an implementation example of the present invention, but is not limited to the above implementation mode. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A Parkinson's speech recognition method, characterized in that: include: Step (1) obtaining speech data of Parkinson's patients, including: The speech data of Parkinson's disease patients include the King's College London MDVR-KCL speech dataset of early and late Parkinson's disease patients, a speech dataset of healthy controls, and an Italian Parkinson's disease speech dataset; wherein the MDVR-KCL speech dataset of early and late Parkinson's disease patients and the speech dataset of healthy controls are in English and contain the speech readings of two paragraphs of text; wherein the Italian Parkinson's disease speech dataset contains monosyllabic pronunciation data, including repeated pronunciations of the syllables 'a', 'i', 'e', 'o', 'u', 'ta', and 'pa'; Step (2), editing the acquired voice data to obtain edited voice data; Step (3) extracting speech features from the clipped speech data to obtain a speech feature dataset, including: The speech feature dataset includes 26 basic speech features, Mel-frequency cepstral coefficient features and 13 global features: A set of 26 basic speech features were extracted from each speech signal using Praat acoustic analysis software; Based on short-time Fourier transform, a Mel filter bank is used to extract the Mel cepstral coefficient features of the speech signal; For speech signals, 13 global features are extracted, including mean, median, root mean square, square, maximum, minimum, first and third quartiles, interquartile range, standard deviation, skewness and kurtosis; Step (4) randomly initializes and generates a classification network particle swarm by dynamically searching the parameters of the classification network according to the preset conditions, including: initializing the population size to 20, the number of iterations to 10, the number of classification network layers to be in the range of [3, 10], requiring the first layer of the classification network to be a convolutional layer and the last layer to be a fully connected layer, and randomly initializing the structure of each classification network in the classification network particle swarm. When training the classification network particle swarm, the number of main cycles is 10; Step (5) iteratively trains and updates the classification network particle swarm using the speech feature data set to obtain a trained and updated dynamic search classification network model, including: using the speech feature data set to respectively train each classification network in the classification network particle swarm, calculating the loss function, and using the loss function to evaluate the classification network particle swarm to obtain the fitness function value of each particle, updating the global best classification network and the current particle best classification network according to the result of comparing the fitness function value of each particle with the fitness function value of the global best classification network and the current particle best classification network, and updating the classification network particle swarm according to the global best classification network and the current particle best classification network; Step (6): inputting the speech feature data to be recognized into the trained and updated dynamic search classification network model, and determining the Parkinson's speech recognition result according to the output of the dynamic search classification network model.

2. The Parkinson's speech recognition method according to claim 1, characterized in that Edit the acquired voice data, including: For the MDVR-KCL speech dataset of early and late Parkinson's patients and healthy controls, the speech was edited according to the sentence segmentation of the read text, and the blank speech segments with pauses were removed; For the Italian Parkinson's speech dataset, each syllable pronunciation segment is clipped and the blank speech segments during the pause are removed.

3. The Parkinson's speech recognition method according to claim 1, characterized in that Step (5), using the speech feature data set to iteratively train and update the classification network particle swarm to obtain a trained and updated dynamic search classification network model; The classification network in each generation of classification network particle swarm is trained using the speech feature dataset to obtain the fitness function value of each particle; the classification network particle swarm is updated according to the fitness function value of each particle, and this step is iteratively executed until the iteration stop condition is met, obtaining an optimal classification network as the dynamic search classification network model.

4. The Parkinson's speech recognition method according to claim 1, characterized in that The classification network update method includes: calculating the velocity v of the current particle by comparing the differences between network structures, and updating the classification network structure according to the following formula: v i =v i +c1×rand()×(pbest i -x i )+c2×rand()×(gbest i -x i ) x i =x i +v i Among them, gbest i is the global best classification network, pbest i is the current best particle classification network, i=1,2,…,N is the total number of particles in the group, v i is the velocity of the particle, x i is the current position of the particle, c1 and c2 are learning factors, and rand() is a random number between (0, 1).

5. A Parkinson's speech recognition device, characterized in that: including processors and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 4.

6. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images