Chinese speech decoding nursing system based on transfer learning

Abstract

The invention discloses a Chinese speech decoding nursing system based on transfer learning. The Chinese speech decoding nursing system comprises that an Inception-V3 neural network obtained by transfer learning is taken as a basis; the auxiliary classifier at the front layer position is deleted, anddeleting all network structures behind the auxiliary classifier at the middle layer position are deleted; an original full connection layer is deleted at the middle layer position of the neural network and then a new seven-layer residual neural network structure is added to construct a new convolutional neural network suitable for electroencephalogram time-frequency graph classification; only the residual neural network is trained by using the local first data set to obtain a trained first convolutional neural network, and then only the residual neural network is trained by using the local second data set to obtain a trained second convolutional neural network; the first convolutional neural network judges whether the paralyzed patient feels uncomfortable or not according to the electroencephalogram time-frequency diagram obtained by the signal preprocessing module, and if yes, the second convolutional neural network continues to be used for further judging the type of discomfort of the patient; the second convolutional neural network will judge whether the paralyzed patient is hungry or cold or other.

Description

technical field [0001] The invention relates to the field of Chinese speech decoding of EEG images, in particular to a Chinese speech decoding nursing system based on transfer learning. Background technique [0002] Stroke, cerebral infarction or massive blood loss are likely to cause paralysis and loss of speech function in patients. How to better take care of such patients has become a hot research issue in society. If patients can be allowed to "talk" to express their wishes and thoughts, such as whether they are cold or hungry, caregivers can take care of patients more specifically. [0003] In recent years, with the development of artificial intelligence technology and the advancement of brain-computer interface technology, it has become a reality to "decipher" brain signals and translate them into language. Researchers from Columbia University in the United States successfully converted brain waves into intelligible speech content by using artificial intelligence algo...

AI Technical Summary

Problems solved by technology

[0004] A prominent problem in the way of decoding Chinese speech from electroencephalogram (EEG) through deep neural network is that the sample data set is too small to meet the requirements of training deep neural network

It is easy to cause over-fitting of the deep neural network, that is, the accuracy of neural network training is very high, but the accuracy of this neural network is very low when actually tested.

[0005] Although the pre-trained Inception-V3 convolutional neural network on the ImageNet data set has a strong ability to classify pictures, but because the ImageNet data set is all pictures of real objects, and the constructed EEG such as non-real objects The pictures are very different. For this reason, it is necessary to modify the pre-trained Inception-V3 convolutional neural network, expand part of the network structure and retrain part of the weights before it can be used to recognize EEG.

Images