A passive hearing test method and system based on the Linus six-tone scale

Abstract

This invention proposes a passive hearing detection method and system based on the Lin's six tones, comprising: acquiring multi-channel EEG signals induced by passive listening to the Lin's six tones in a subject and preprocessing them; classifying hearing detection based on the preprocessed multi-channel EEG signals using a trained TCNet-Fusion EEG classification model; wherein, the TCNet-Fusion EEG classification model first performs spatiotemporal filtering on the multi-channel EEG signals to obtain a spatiotemporally filtered feature matrix; fully connects the spatiotemporally filtered feature matrix to obtain basic spatiotemporal features; then feeds these features into a temporal convolution residual module to obtain additional temporal features; fuses the additional temporal features obtained after the temporal convolution residual module with the fully connected features, and then performs classification through a neural network to obtain the hearing detection classification result. This method fully exploits the deep temporal, frequency, and spatial features of EEG signals induced by different spectral structures of the Lin's six tones, thereby achieving significant classification results for the six tones.

Description

Technical Field

[0001] This invention belongs to the field of hearing test technology, and in particular relates to a passive hearing test method and system based on the Linn six-tone scale. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In hearing assessment, the Linn six sounds (i.e., the audio elements / m / , / u / , / a / , / i / , / sh / , and / s / in any language pronunciation) are six widely used sound materials in the field of hearing testing. Their frequency range covers 250Hz to 8000Hz, with different elements having different frequency ranges. / m / and / u / belong to low-frequency sounds, / a / and / i / to mid-frequency sounds, and / sh / and / s / to high-frequency sounds. They can be used for hearing testing under various conditions, including assisted hearing, semi-assisted hearing, and auditory reconstruction. Current Linn six-sound hearing tests primarily use a question-and-answer method. For example, in clinical audiometry, a specific sound from the Linn six sounds is played, and the subject imitates the sound or chooses which sound they heard to assess their hearing ability. However, this method sometimes suffers from inaccurate or incomplete subjective perception by the subject, and some subjects may have difficulty understanding or cooperating with the test, leading to a lack of reference value in the test results.

[0004] Electroencephalogram (EEG) is an electrophysiological signal recording of brain neural activity on the scalp. Compared with other brain imaging methods, it has the advantages of being non-invasive, having high temporal resolution, and being convenient to detect. With the development of brain science, perceptual detection technology based on EEG brain-computer interfaces has also made significant progress. Deep learning, as a data-driven novel representation learning method, plays an important role in various brain-computer interfaces and intelligent systems. Passive hearing assessment does not require the subject to make subjective feedback to the sound stimulus; it only requires passive listening. The perception is then quantified and decoded based on the objective EEG signal patterns generated by the auditory system of the brain induced by the sound stimulus, thereby assessing the subject's hearing. This detection method, because it does not require subjective selection, is suitable for auditory assessment of various hearing conditions in any population.

[0005] Although deep neural networks have achieved certain results in EEG signal processing, for example, in the inventor's previous research, the invention patent number "2024101763263" was titled "Automatic Classification Method and System for Passive Pitch Perception EEG Across Subjects," which is an automatic classification method for pitch perception across subjects based on auditory EEG data and pitch feature information in natural audio. However, this method is completely different from the purpose and task of hearing detection for six types of audio with different spectral structure features of the "Ling Six Tones." This is because the pitch features of the sound and the spectral features of the Ling Six Tones stimulate the auditory brain network, resulting in completely different EEG signal expressions. Therefore, they cannot be substituted for each other when constructing EEG deep learning models to design corresponding detection and classification algorithms.

[0006] The invention patent "202210833107.9", entitled "Method and Device for Evaluating Hearing Based on Spatiotemporal Convolutional Neural Network", although it provides a deep learning algorithm based on auditory EEG spatiotemporal convolution, this patent is only the basic architecture of the auditory EEG deep learning algorithm and does not design a hearing detection method based on the EEG signals induced by "Lin's Six Sounds". Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, this invention provides a passive hearing detection method and system for the six Linn sounds, ultimately achieving high-performance classification of Linn six-sound evoked EEG signals.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0009] Firstly, a passive hearing test method based on the Lin six-tone scale is disclosed, including:

[0010] Multichannel EEG signals passively evoked by six audio frequencies of the Lin's Six Sounds were acquired, and the EEG signals were filtered and segmented preprocessed.

[0011] Based on the preprocessed multi-channel EEG signals, the trained TCNet-Fusion EEG classification model was used to perform passive hearing assessment classification of the Lin six sounds.

[0012] The TCNet-Fusion EEG classification model first performs spatiotemporal filtering on multi-channel EEG signals to obtain a spatiotemporally filtered feature matrix.

[0013] The spatiotemporally filtered feature matrix is ​​fully connected to obtain the fully connected features.

[0014] The fully connected features are fed into the temporal convolutional residual module to obtain additional temporal features, which allows for deeper mining of the spatiotemporally filtered features.

[0015] Finally, the additional temporal features obtained after the temporal convolution residual module are fused with the features after full connection, and then classified by a neural network to obtain the hearing assessment classification result.

[0016] As an optional technical solution, the specific steps for spatiotemporal filtering of multi-channel EEG signals are as follows:

[0017] Step 1: Perform time-domain filtering on the multi-channel EEG signals to obtain the time-domain filtered feature vector F. t ;

[0018] Step 2: Perform spatial domain filtering on the time-domain filtered feature vectors to obtain the spatiotemporally filtered feature matrix F. c ;

[0019] Step 3: Perform a fully connected operation on the spatiotemporal filtered feature matrix. The output of the fully connected operation is as follows:

[0020] Z = W * F

[0021] Where Z is the output of the fully connected layer, W is the weight matrix, and F is the feature matrix F after spatiotemporal filtering. c Vectors flattened by the Flatten layer.

[0022] As an optional technical solution, the fully connected feature Z is fed into the temporal convolutional residual module. The structure of the temporal convolutional residual module is mainly composed of n residual blocks stacked together. Each residual block includes two dilated causal convolutional layers and one 1×1 convolutional layer.

[0023] Furthermore, in dilated causal convolution, causal convolution means that when convolving the fully connected feature Z, the output of the t-th feature in the convolutional layer is only related to the t-th or earlier feature input in the previous layer, and is independent of the feature input after t. Dilated convolution skips some input features Z during convolution operations, thereby obtaining more time-series features and reducing computational cost. The 1×1 convolutional layer ensures that the additional time-series features output by the temporal convolution residual module have the same data format as the fully connected features.

[0024] Therefore, in the temporal convolutional residual module, the output of any i-th residual block Specifically:

[0025]

[0026] Where i = 1, ..., n, n represents the number of residual blocks in the temporal convolutional residual module. Therefore, the additional temporal features output by the temporal convolutional residual module are: The convolution kernel for the i-th residual block is related to the dilation factor d. During the dilated convolution process, (2d-2) zeros are added between two adjacent parameters in the one-dimensional convolution kernel, which expands the receptive field of the convolution operation by a factor of d. d is empirically set to 1, 2, 4 (when d=1, it is a normal convolution). In the construction of the TCNet-Fusion model, the weights in the convolution kernel are trained by the backpropagation algorithm. This is the output of the (i-1)th residual block; the initial input of the temporal convolutional residual module is the fully connected feature Z, i.e.

[0027] Therefore, the purpose of adding a temporal convolutional residual module is to extract deeper and additional temporal features from the input feature sequence at more time scales.

[0028] As an optional solution, the additional temporal features after passing through the temporal convolutional residual module... The feature Z is fused with the fully connected feature Z, and is represented as:

[0029]

[0030] Among them, F f It is a fusion feature after spatiotemporal filtering and temporal convolution residual processing. Adding a feature fusion layer can supplement the deeper additional temporal features of EEG signals induced by the six Linus sounds with different spectral characteristics, and further explore the useful information in the EEG signals.

[0031] As an optional approach, the fused features are classified using a neural network, specifically:

[0032] The fused features are fed into a fully connected layer whose output neurons are equal to the number of classification categories for weighting. The output neurons are then mapped using a softmax function to obtain the posterior probability corresponding to each category. The category with the highest posterior probability is selected as the final classification result.

[0033] Secondly, a passive hearing detection system based on the Lin six-tone scale is disclosed, comprising:

[0034] The data acquisition module is used to acquire passive multi-channel EEG signals induced by Lin's six sounds, and to perform filtering and segmented preprocessing of the EEG signals.

[0035] The hearing detection module is used to perform a six-class hearing assessment of EEG data based on preprocessed multi-channel EEG signals and using a trained TCNet-Fusion EEG classification model of the Lin six sounds.

[0036] The TCNet-Fusion EEG classification model specifically includes:

[0037] The time-domain filtering unit is used to perform time-domain filtering on multi-channel EEG signals to obtain time-domain filtered feature vectors.

[0038] The spatial filtering unit is used to perform spatial filtering on the time-domain filtered feature vector to obtain the spatiotemporally filtered feature matrix.

[0039] Fully connected units are used to flatten the spatiotemporally filtered feature matrix to obtain the fully connected features.

[0040] The temporal convolutional residual module unit is used to process the features after full connection to obtain additional temporal features that were not learned by the previous network structure;

[0041] The feature fusion unit is used to fuse the features after the spatiotemporal filtering fully connected with additional temporal features to obtain fused features;

[0042] The neural network classification unit is used to classify the fused features through a neural network to obtain the hearing assessment classification results.

[0043] Thirdly, a passive hearing detection device based on the Linn six-tone scale is disclosed, comprising:

[0044] The EEG data acquisition unit, EEG signal amplification unit, A / D conversion unit and main controller are connected in sequence, and the main controller is connected to the external controlled device.

[0045] The main controller is configured to include:

[0046] The time-domain filtering unit is used to perform time-domain filtering on multi-channel EEG signals to obtain time-domain filtered feature vectors.

[0047] The spatial filtering unit is used to perform spatial filtering on the time-domain filtered feature vector to obtain the spatiotemporally filtered feature matrix.

[0048] Fully connected units are used to flatten the spatiotemporally filtered feature matrix to obtain the fully connected features.

[0049] The temporal convolutional residual module unit is used to process the fully connected features to obtain additional temporal features that were not extracted earlier.

[0050] The feature fusion unit is used to fuse the features after the spatiotemporal filtering fully connected with additional temporal features to obtain fused features;

[0051] The neural network classification unit is used to classify the fused features through the neural network to obtain the hearing test classification results;

[0052] The main controller can convert the obtained hearing test classification results of the six Ringier tones into control commands for external controlled devices, thereby realizing the control commands for external controlled devices.

[0053] The above one or more technical solutions have the following beneficial effects:

[0054] This embodiment of the technical solution targets passive EEG signals induced by the six Lin's sounds. It fully considers the basic spatiotemporal characteristics and the more implicit temporal features of auditory event-related potentials (ERPs) induced by the differences of the six types of audio in the EEG signals. Based on spatiotemporal filtering and temporal convolution residual modules, the convolutional neural network architecture is optimized, resulting in a significant improvement in the passive hearing detection performance of evaluating the six Lin's sounds based on auditory EEG brain-computer interface.

[0055] This embodiment incorporates a temporal convolutional residual module to perform secondary extraction on the features after spatiotemporal filtering and full connection. This enables the extraction of additional temporal features from the input feature sequence at a deeper level across more time scales. The additional temporal features are then fused with the features after full connection to supplement the features and further uncover useful information in the EEG signals evoked by the six Linn sounds. Compared to basic spatiotemporal convolutional neural networks, the TCNet-Fusion model can improve the EEG classification accuracy of the six Linn sounds, thereby improving the accuracy of this passive hearing detection.

[0056] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0057] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0058] Figure 1 This is a flowchart of the passive EEG hearing detection method based on the TCNet-Fusion model in an embodiment of the present invention.

[0059] Figure 2 This is a schematic diagram of the temporal convolutional residual module structure in the TCNet-Fusion model in this embodiment of the invention;

[0060] Figure 3 This is a schematic diagram of the training process of the Lin's six-tone passive EEG classification TCNet-Fusion model in an embodiment of the invention.

[0061] Figure 4 This is a schematic diagram of the Lin six-tone passive hearing assessment device based on the TCNet-Fusion model in an embodiment of the present invention.

[0062] Figure 5This is a comparison chart of the hearing detection classification performance of the TCNet-Fusion model and the basic spatiotemporal filtering neural network model (EEGNet) in this embodiment of the invention. Detailed Implementation

[0063] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0064] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0065] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0066] Terminology Explanation:

[0067] Training: Input a set of data into the neural network, compare the output with the corresponding label of the data and calculate the error, obtain the gradient value of each parameter through the backpropagation algorithm and update it.

[0068] Deep Neural Networks: A deep neural network is an artificial neural network with multiple layers between the input and output layers, consisting of neurons, weights, biases, and other components. The weights and biases are obtained through training using the backpropagation algorithm.

[0069] The EEGNet model is a deep convolutional neural network model that has undergone temporal and spatial filtering. It consists of neurons, weights, biases, and other components. The weights and biases are obtained through backpropagation training.

[0070] Temporal Convolutional Residual Module: The temporal convolutional residual module is a convolutional neural network model that can extract deep temporal features from input feature sequences at multiple time scales. It consists of multiple residual blocks. The model includes neurons, weights, and other components. The weights are trained using the backpropagation algorithm.

[0071] The TCNet-Fusion model is a deep learning network based on the fusion of spatiotemporal convolutional deep learning and temporal convolutional residual modules. It possesses a stronger ability to extract spatiotemporal and frequency features of signals. The model internally includes neurons, weights, and biases. The weights and biases are trained using the backpropagation algorithm.

[0072] The Lin's Six Sounds are typical audio frequencies selected from high to low based on the concentrated energy regions of the audio spectrum. Each audio frequency has a different peak region of its spectral envelope and no significant difference in pitch characteristics. The EEG signals evoked by these frequencies are different from those evoked by pitch-induced EEG with the same spectral envelope but different pitch differences. These are different auditory tasks, and the EEG signals evoked by these frequencies will also have different characteristics. There is an existing basic model that mainly extracts the frequency and spatial characteristics of the EEG signals of each channel. However, the EEG signals of the Lin's Six Sounds need to rely on a more comprehensive feature expression of the EEG signals of each channel in terms of frequency, space, and time to significantly distinguish the corresponding six types of EEG signals.

[0073] Example 1

[0074] This embodiment discloses a passive hearing test method based on the Linn six-tone scale, referring to... Figure 1 Specifically, it includes the following process:

[0075] (1) Obtain multi-channel EEG signals induced by Lin's six sounds under specific paradigms (such as passive Oddball and improved Oddball, passive Go / Nogo, etc.), and perform EEG signal filtering and segmented preprocessing;

[0076] (2) Based on the preprocessed multi-channel EEG signals, the trained TCNet-Fusion EEG classification model is used to classify hearing detection.

[0077] The TCNet-Fusion EEG classification model first performs spatiotemporal filtering on multi-channel EEG signals to obtain a spatiotemporally filtered feature matrix. Then, it performs a fully connected operation on the spatiotemporally filtered feature matrix to obtain fully connected features. Finally, it feeds the fully connected features into a temporal convolutional residual module to obtain additional temporal features. Finally, it fuses the additional temporal features obtained after the temporal convolutional residual module with the fully connected features, and then performs classification through a neural network to obtain the hearing assessment classification result.

[0078] This embodiment of the sub-technical solution, based on the original basic spatiotemporal deep learning architecture, extracts more effective deep learning features from EEG signals induced by different spectral tones of the six Linn tones. It adds a temporal convolutional residual module and a feature fusion layer, thereby achieving high-performance classification results of passive EEG for the six Linn tones, which can be used for passive hearing detection of the six Linn tones.

[0079] In this embodiment, spatiotemporal filtering of multi-channel EEG signals is performed, and the specific steps are as follows:

[0080] The first step is to perform time-domain filtering on the multi-channel EEG signals to obtain the time-domain filtered feature vector F. t ;

[0081] The second step is to perform spatial domain filtering on the time-domain filtered feature vectors to obtain the spatiotemporally filtered feature matrix F.c .

[0082] The third step is to perform a fully connected operation on the spatiotemporally filtered feature matrix. The full connection is specifically as follows:

[0083] Z = W * F

[0084] Where Z is the output of the fully connected layer, W is the weight matrix, and F is the feature matrix F after spatiotemporal filtering. c Vectors flattened by the Flatten layer.

[0085] The fully connected feature Z is fed into the temporal convolutional residual module, as shown in the following example. Figure 2 Its structure is mainly composed of n residual blocks stacked together, each residual block including 2 layers of dilated causal convolution and 1×1 convolution layer.

[0086] Furthermore, in dilated causal convolution, causal convolution means that when convolving the fully connected feature Z, the output of the t-th feature in the convolutional layer is only related to the feature inputs of the previous layer up to or including the t-th feature, and is independent of the feature inputs after t. Dilated convolution skips some of the input features Z during the convolution operation, thereby obtaining more time-series features and reducing computational cost. The 1×1 convolutional layer ensures that the additional time-series features output by the temporal convolution residual module have the same data format as the features after full connection.

[0087] Therefore, in the temporal convolutional residual module, the output of any i-th residual block Specifically:

[0088]

[0089] Where i = 1, ..., n, n represents the number of residual blocks in the temporal convolutional residual module. Therefore, the additional temporal features output by the temporal convolutional residual module are: The convolution kernel for the i-th residual block is related to the dilation factor d. During the dilated convolution process, (2d-2) zeros are added between two adjacent parameters in the one-dimensional convolution kernel, which expands the receptive field of the convolution operation by a factor of d. d is empirically set to 1, 2, 4 (when d=1, it is a normal convolution). In the construction of the TCNet-Fusion model, the weights in the convolution kernel are trained by the backpropagation algorithm. This is the output of the (i-1)th residual block; the initial input of the temporal convolutional residual module is the fully connected feature Z, i.e.

[0090] Therefore, the purpose of adding a temporal convolutional residual module is to extract deeper and additional temporal features from the input feature sequence at more time scales.

[0091] Finally, the additional temporal features after passing through the temporal convolutional residual module are... The fusion with the fully connected feature Z can be expressed as:

[0092]

[0093] Among them, F f It is a fusion feature after spatiotemporal filtering and temporal convolution residual processing.

[0094] Adding a feature fusion layer can supplement the deeper temporal features of the Lin six-tone induced EEG signals with different spectral characteristics, and further extract useful information from the EEG signals.

[0095] The fused features are fed into a fully connected layer whose output neurons equal the number of classification categories for weighting. The output neurons are then mapped using a softmax function to obtain the network output P, ​​which is the posterior probability corresponding to each category. The category with the highest posterior probability is selected as the final classification result. As an example, the classification result could be: perception of different syllables in the Lin's Six Tones, etc.

[0096] In this embodiment, combined with Figure 3 The training process of the TCNet-Fusion model for classifying passive EEG signals evoked by the Lin's six sounds is as follows:

[0097] 1) Collect multi-channel auditory evoked EEG signals x(t) as the training set. Preprocess the training set data, including EEG signal filtering and segmentation. The number of channels is C, and the signal length is T. Set the maximum number of iterations Ne and the learning rate Lr.

[0098] As a specific implementation scheme, following the classic oddball paradigm, the subjects were played the six Linus six sounds / m / , / u / , / a / , / i / , / sh / , and / s / audio signals. The subjects remained awake and passively listened to the sounds, and the corresponding sound-evoked EEG signals were recorded and labeled. Then, a preprocessing operation was performed, namely, a 5th-order Butterworth filter was used to perform a 1-40Hz bandpass filter on the raw EEG signals, and the multi-channel auditory evoked EEG signals x(t) of the auditory stimulus part were segmented as the training set. The length of each EEG data segment was T=2000, the number of channels was C=59, there were 150 sample data, the number of residual blocks was n=2, the inflation factor was d=4, the maximum number of iterations was Ne=100, and the learning rate was Lr=0.0001.

[0099] 2) Train the TCNet-Fusion model for classifying Lin's six-sound evoked EEG patterns. Let i represent the iteration number, and start the following iterative loop from i=1:

[0100] a) Spatiotemporal filtering of multi-channel auditory evoked EEG signals yields the spatiotemporally filtered feature matrix F. c The obtained feature matrix is ​​fully connected to obtain the fully connected feature Z; the fully connected feature is then fed into the temporal convolutional residual module to obtain additional temporal features. Finally, the features after the residual block are fused with the features after the fully connected layer to obtain the fused feature F. f The fused features are fed into a fully connected layer whose output neuron is equal to the number of classification categories for weighting. The output neuron is then mapped using a softmax function to obtain the network output P, ​​which is the posterior probability corresponding to each category.

[0101] b) Compare the network output P with the corresponding label, and calculate the loss value of the current model using the network's cost loss function J.

[0102] c) Based on the obtained model loss value, calculate the gradient corresponding to each parameter in the network using the backpropagation algorithm.

[0103] d) Update all parameters of the model according to the learning rate Lr = 0.0001.

[0104] e) If i = Ne = 100, then the model training is complete; otherwise, let i + 1 and return to step a).

[0105] Combination Figure 5 To conduct experiments using a passive EEG sample set of the same subject listening to the six Lin sounds, this embodiment compares the automatic classification and detection results of passive EEG signals evoked by the six Lin sounds based on the TCNet-Fusion model with the detection results obtained by using only the basic spatiotemporal filtering neural network model (EEGNet) algorithm. It can be seen that the method in this embodiment can achieve a higher classification accuracy.

[0106] Thus, the experiment found that, for the "Lin's Six Sounds" EEG classification task, by adding a temporal convolutional residual module, it is possible to extract additional temporal features of Lin's Six Sounds EEG at more time scales and deeper levels on the basis of the previous basic deep network architecture; furthermore, by fusing spatiotemporal filtering features and additional temporal features, it is possible to mine more useful information of Lin's Six Sounds induced EEG for automatic classification.

[0107] Example 2

[0108] In one or more embodiments, a passive hearing detection system based on the Linn six-tone scale is disclosed, comprising:

[0109] The data acquisition module is used to acquire multi-channel EEG signals obtained by the Lin six-sound evoked in a specific paradigm, and to perform filtering and segmented preprocessing of the EEG signals.

[0110] The hearing assessment classification module is used to perform hearing assessment classification based on preprocessed multi-channel EEG signals and using a trained TCNet-Fusion EEG classification model.

[0111] Specifically, the Lin's six-sound evoked EEG classification TCNet-Fusion model includes:

[0112] The time-domain filtering unit is used to perform time-domain filtering on multi-channel EEG signals to obtain time-domain filtered feature vectors.

[0113] The spatial filtering unit is used to perform spatial filtering on the time-domain filtered feature vector to obtain the spatiotemporally filtered feature matrix.

[0114] Fully connected units are used to flatten the spatiotemporally filtered feature matrix to obtain the fully connected features.

[0115] The temporal convolutional residual module unit is used to process the fully connected features to obtain additional temporal features;

[0116] The feature fusion unit is used to fuse the fully connected features with additional temporal features to obtain fused features;

[0117] The neural network classification unit is used to classify the fused features through a neural network to obtain the hearing assessment classification results.

[0118] It should be noted that the specific implementation process of each of the above modules has been described in Example 1, and will not be detailed here.

[0119] Example 3

[0120] In one or more embodiments, a passive hearing detection device based on the Linn six-tone scale is disclosed, combined with... Figure 4 Specifically, it includes: an EEG data acquisition unit, an EEG signal amplification unit, an A / D conversion unit, and a main controller connected in sequence, with the main controller connected to an external controlled device.

[0121] The main controller contains the following:

[0122] The time-domain filtering unit is used to perform time-domain filtering on multi-channel EEG signals to obtain time-domain filtered feature vectors.

[0123] The spatial filtering unit is used to perform spatial filtering on the time-domain filtered feature vector to obtain the spatiotemporally filtered feature matrix.

[0124] Fully connected units are used to flatten the spatiotemporally filtered feature matrix to obtain the fully connected features.

[0125] The temporal convolutional residual module unit is used to process the fully connected features to obtain additional temporal features;

[0126] The feature fusion unit is used to fuse the fully connected features with additional temporal features to obtain fused features;

[0127] The neural network classification unit is used to classify the fused features through the neural network to obtain the hearing assessment classification results;

[0128] The main controller can convert the obtained hearing assessment classification results of the six Linn sounds into control commands for external controlled devices, thereby enabling control of these devices. For example, it can control the output of visual prediction results on the screen of the "Linn Six-Sound Evoked EEG Audiometry Analyzer," and then, based on a certain standard, inform the doctor of the subject's hearing level or sound discrimination ability.

[0129] Example 4

[0130] The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0131] Example 5

[0132] The purpose of this embodiment is to provide a computer-readable storage medium.

[0133] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above method.

[0134] The steps and methods involved in the apparatus or system of the above embodiments correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0135] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0136] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A passive hearing test method for the detection of the sixth sound of Lini, characterized in that, The method comprises the following steps: Obtain the six kinds of audio passive induced multi-channel electroencephalogram signals of Lin's six tones, and perform filtering and segmentation preprocessing on the electroencephalogram signals; Based on the preprocessed multi-channel electroencephalogram signals, the trained TCNet-Fusion electroencephalogram classification model is used to perform passive hearing evaluation classification of Lin's six tones; The TCNet-Fusion electroencephalogram classification model first performs space-time filtering on the multi-channel electroencephalogram signals to obtain a characteristic matrix after space-time filtering; The characteristic matrix after space-time filtering is fully connected to obtain a fully connected characteristic; The fully connected characteristic is sent to a time series convolution residual module to obtain additional time series characteristics; Finally, the additional time series characteristics obtained through the time series convolution residual module are fused with the fully connected characteristics, and then the neural network is used for classification to obtain the hearing evaluation classification result.

2. The passive hearing test method of claim 1, wherein the six tones of Lins are, The time-space filtering of the multi-channel electroencephalogram signal comprises the following specific steps: Step one: time domain filtering is performed on the multi-channel electroencephalogram signal to obtain a time domain filtered feature vector ; Step two: spatial filtering is performed on the time domain filtered eigenvector to obtain a time-space filtered eigenvector ; Step three: fully connect the characteristic matrix after space-time filtering, and the full connection output is as follows: wherein, is the output of the fully connected layer, is a weight matrix, is the spatio-temporally filtered feature matrix is passed through a layer flattening vector.

3. The passive hearing test method of claim 1, wherein the six tones of Linsly are, The full connection post-feature is sent into a timing convolution residual module, the structure of which is mainly composed of n residual block stacks, and each residual block includes 2 layers of dilated causal convolution and a 1×1 convolution layer. In the dilated causal convolution, the causal convolution refers to that when the convolution is performed on the full-connection feature Z, the output of the first feature in the convolution layer is only related to the input of the first feature in the previous layer and the input of the previous feature, and is not related to the input of the feature after t. Or ​​ Dilated convolution is a convolution operation that skips part of the input features Z to obtain more time series features and reduce the amount of calculation; The 1x1 convolution layer is used to ensure that the additional time series characteristics output by the time series convolution residual module and the fully connected characteristics have the same data form; In the temporal convolution residual module, the output of any i-th residual block Specifically: wherein, n represents the number of residual blocks in the temporal convolution residual module, therefore, the additional temporal features output by the temporal convolution residual module are ; is the convolution kernel of the i-th residual block, which is related to the dilation factor d. During the dilation convolution process, (2d-2) zeros are added between the two adjacent parameters in the one-dimensional convolution kernel, so that the receptive field of the convolution operation is expanded by d times. The value of d is empirically set to 1, 2, 4, and when is the ordinary convolution, and the weights in the convolution kernel are obtained by training through the back propagation algorithm during the construction of the TCNet-Fusion model; is the output of the i-1-th residual block; and the initial input of the temporal convolution residual module is the feature Z after full connection, i.e., ; The role of the time series convolution residual module is to extract deeper additional time series characteristics from the input feature sequence in more time scales.

4. The passive hearing test method of claim 1, wherein the six tones of Linsly are, The additional temporal features after the temporal convolution residual module Fusion with the full connection feature Z, which can be represented as: wherein, is the fusion feature after spatio-temporal filtering and temporal convolution residual processing, and the addition of the feature fusion layer can supplement the deeper additional temporal features of the forest's six sound-induced brain electrical signals of different spectral characteristics.

5. The passive hearing test method for the Lin six-tone scale as described in claim 1, characterized in that it integrates... The characteristics are classified by the neural network, and the specific process is as follows: The fusion features are sent to a fully connected layer with the number of output neurons equal to the number of classification categories for weighting, and the output neurons are mapped by a function to obtain a posterior probability corresponding to each category, and a category corresponding to a maximum value of the posterior probability is selected as a final classification result.

6. A passive hearing test system for the detection of the sixth Ling sound, characterized in that The method comprises the following steps: A data acquisition module is used to acquire passive multi-channel electroencephalogram signals induced by Lin's six tones, and perform filtering and segmentation preprocessing on the electroencephalogram signals; A hearing detection module is used to perform six-class hearing evaluation of electroencephalogram data based on the preprocessed multi-channel electroencephalogram signals, and use the trained TCNet-Fusion electroencephalogram classification model of Lin's six tones to perform the classification. The TCNet-Fusion electroencephalogram classification model comprises: A time domain filtering unit is used to perform time domain filtering on the multi-channel electroencephalogram signals to obtain a time domain filtering characteristic vector; A space domain filtering unit is used to perform space domain filtering on the time domain filtering characteristic vector to obtain a characteristic matrix after space-time filtering; A full connection unit is used to flatten the characteristic matrix after space-time filtering to obtain a fully connected characteristic; A time series convolution residual module unit is used to process the fully connected characteristic to obtain additional time series characteristics; A feature fusion unit is used to fuse the characteristic after space-time filtering and full connection with the additional time series characteristics to obtain a fusion characteristic; A neural network classification unit is used to classify the fusion characteristic through a neural network to obtain a hearing evaluation classification result.

7. A passive hearing test device for the detection of the sixth tone of Lincs, characterized in that, The method comprises the following steps: The brain electrical data acquisition unit, the electroencephalogram signal amplification unit, the A / D conversion unit and the main controller are connected in sequence, and the main controller is connected with an external controlled device; The main controller is configured to comprise: A time domain filtering unit is used to perform time domain filtering on the multi-channel electroencephalogram signals to obtain a time domain filtering characteristic vector; A space domain filtering unit is used to perform space domain filtering on the time domain filtering characteristic vector to obtain a characteristic matrix after space-time filtering; A full connection unit is used to flatten the characteristic matrix after space-time filtering to obtain a fully connected characteristic; a time sequence convolution residual module unit, configured to process the features after full connection to obtain additional time sequence features; a feature fusion unit, configured to fuse the features after the spatio-temporal filtering full connection and the additional time sequence features to obtain fused features; a neural network classification unit, configured to classify the fused features through a neural network to obtain a hearing detection classification result; the main controller can convert the obtained hearing detection classification result of the Lin's six-tone into a control instruction for an external controlled device to realize control of the external controlled device.

8. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the steps of the method of any one of claims 1-5.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to execute the steps of the method of any one of claims 1-5.

Images