Laryngeal electromyographic signal interpretation method and system based on self-supervised joint coding prediction, medium and equipment

Abstract

The embodiment of the invention discloses a laryngeal electromyographic signal interpretation method and system based on self-supervision joint coding prediction, a medium and equipment. The method comprises the steps that a laryngeal electromyographic signal data set and a corresponding instruction label are acquired; decomposing the signal into low-frequency and high-frequency components through wavelet transform; randomly generating a mask component for masking to obtain a visible group and a mask group; respectively extracting visible component features, target signal component features and mask vector features; performing joint coding on the mask vector feature and the visible component feature to obtain a target signal component prediction feature, and performing joint coding on the target signal component feature and the visible component feature to obtain a mask vector prediction feature; training a signal encoder based on the predicted and real features; combining the trained signal encoder with a classification module to construct an interpretation model and performing fine tuning training; the to-be-interpreted electromyographic signals are input into the fine-tuned signal interpretation model, a classification result is output, efficient feature learning is achieved through self-supervised joint coding prediction, and the generalization ability of the model is improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and human-computer interaction technology, and in particular to a method, system, medium and device for interpreting laryngeal electromyography signals based on self-supervised joint coding prediction. Background Technology

[0002] In recent years, traditional human-computer interaction methods such as voice input and gesture control have faced significant challenges in meeting the requirements of high energy efficiency, strong concealment, and universality due to inherent defects such as easy detection, high energy consumption, and poor environmental adaptability. They are particularly difficult to cope with the requirements of highly concealed and low-power fields such as military communications and medical rehabilitation.

[0003] With the continuous development of biosignal interaction technology, novel interaction technologies based on biosignals have provided new technical approaches to solving the aforementioned problems faced by traditional human-computer interaction. Among them, laryngeal electromyography signal decoding technology has become a research hotspot in the field of human-computer interaction due to its unique advantages such as being non-invasive, low-power, and highly discreet. This technology, by detecting the weak electrical signals of the laryngeal muscles during speech, can not only reconstruct speech function for special groups such as patients with aphasia and those with vocal cord injuries, but also realize discreet human-computer communication without the need for vocalization, which has important application value in the fields of medical assistance and special communication.

[0004] However, current techniques for decoding weak laryngeal electromyography (EMG) signals still face many bottlenecks, with the signal decoding stage being particularly problematic. Current mainstream decoding methods rely on supervised learning paradigms, which impose limitations: (1) Supervised learning requires large-scale, high-quality labeled datasets for model training, while the acquisition process of laryngeal EMG signals is complex, requiring simultaneous recording of EMG signals and corresponding speech content, resulting in extremely high labeling costs; (2) Due to significant differences in individual physiological structures, the distribution of EMG signal characteristics varies considerably among different subjects, leading to insufficient cross-user generalization ability of the model; (3) Existing methods are overly dependent on labeling quality, causing a sharp decline in model performance when data is scarce or labels contain noise, severely impacting practical application effectiveness. Summary of the Invention

[0005] Therefore, it is necessary to propose a method, system, medium, and device for interpreting laryngeal electromyography signals based on self-supervised joint coding prediction to address the above problems.

[0006] A method for interpreting laryngeal electromyographic signals based on self-supervised joint coding prediction, the method comprising:

[0007] A dataset of laryngeal electromyography (EMG) signals and a set of instruction labels are obtained. The dataset of laryngeal EMG signals includes several laryngeal EMG signals, and each laryngeal EMG signal corresponds to a fixed instruction label in the instruction label set.

[0008] The laryngeal electromyographic signal was decomposed into low-frequency and high-frequency components using wavelet transform.

[0009] Randomly generate a mask vector to mask the low-frequency and high-frequency components of the signal, and obtain the visible group signal components and the masked group signal components;

[0010] The visible component features of the visible group signal components, the target signal component features of the mask group signal components, and the mask vector features of the mask vector are extracted using a signal encoder, a mask signal encoder, and a vector encoder, respectively.

[0011] The mask vector features and the visible component features are jointly encoded to obtain the target signal component prediction features, and the target signal component features and the visible component features are jointly encoded to obtain the mask vector prediction features;

[0012] A signal encoder is trained based on predicted features and true features. The predicted features include the mask vector prediction features and the target signal component prediction features. The true features include the mask vector features and the target signal component features.

[0013] The trained signal encoder is combined with the classification module to construct a signal interpretation model. The laryngeal electromyography signal set is used as the training set, and the instruction label set is used as the label set. The signal interpretation model is then fine-tuned under label guidance.

[0014] The electromyographic signal to be interpreted is input into the finely tuned signal interpretation model, and the output instruction classification result is generated.

[0015] The training of the signal encoder based on predicted features and true features, wherein the predicted features include mask vector prediction features and target signal component prediction features, and the true features include mask vector features and target signal component features, specifically includes:

[0016] The pre-training loss function is determined based on the predicted features and the true features. The pre-training loss function is as follows:

[0017]

[0018] Where L is the pre-training loss function, F M For the characteristics of the target signal components, For the target signal component prediction features, F S For mask vector features, Predict features for mask vectors.

[0019] The signal encoder is trained based on the pre-trained loss function.

[0020] Specifically, the step of using the laryngeal electromyography signal set as the training set and the instruction label set as the label set to fine-tune the signal interpretation model under label guidance includes:

[0021] Unfreeze the parameters of the pre-trained signal encoder module;

[0022] The laryngeal electromyography (EMG) signal set is used as the training set, and the instruction label set is used as the label set. The laryngeal EMG signal is input into the signal interpretation model, and the interpretation result is output.

[0023] The loss function is calculated based on the interpretation results and instruction labels. The parameters of the signal encoder and classification module are updated, and the optimization is iteratively performed until fine-tuning training is completed.

[0024] Wherein, the loss function is based on Determined, among which, L cls Let A be the loss function. ij For the interpretation results, R ij Here, C represents the number of instruction categories corresponding to the laryngeal electromyography signal, and n represents the number of samples.

[0025] A laryngeal electromyography signal interpretation system based on self-supervised joint encoding prediction, the system comprising:

[0026] A laryngeal electromyography (EMG) signal dataset and instruction label set acquisition module is used to acquire a laryngeal EMG signal dataset and an instruction label set. The laryngeal EMG signal dataset includes several laryngeal EMG signals, and each laryngeal EMG signal corresponds to a fixed instruction label in the instruction label set.

[0027] A pre-training module is used to decompose the laryngeal electromyography signal into low-frequency and high-frequency components using wavelet transform; randomly generate a mask vector to mask the low-frequency and high-frequency components to obtain visible and masked signal components; use a signal encoder, a mask signal encoder, and a vector encoder to extract visible component features of the visible signal components, target signal component features of the masked signal components, and mask vector features of the mask vector, respectively; use the mask vector features as a cue to jointly encode with the visible component features to obtain predicted features of the target signal components, and use the target signal component features as a cue to jointly encode with the visible component features to obtain predicted features of the mask vector; train the signal encoder based on the predicted features and the true features, where the predicted features include mask vector prediction features and predicted features of the target signal components, and the true features include mask vector features and target signal component features.

[0028] The signal interpretation model construction module is used to combine the trained signal encoder with the classification module to build a signal interpretation model. The laryngeal electromyography signal set is used as the training set, and the instruction label set is used as the label set to perform label-guided fine-tuning training of the signal interpretation model.

[0029] The signal interpretation module is used to input the electromyographic signal to be interpreted into the finely tuned signal interpretation model and output the instruction classification result.

[0030] The pre-training module specifically includes:

[0031] The signal decomposition module is used to decompose the laryngeal electromyography signal into low-frequency and high-frequency components through wavelet transform.

[0032] The signal component masking module is used to randomly generate a masking vector to mask the low-frequency component and high-frequency component of the signal, thereby obtaining the visible group signal component and the masked group signal component.

[0033] The encoder module is used to extract the visible component features of the visible group signal components, the target signal component features of the mask group signal components, and the mask vector features of the mask vector.

[0034] The dual-branch mask prediction module uses mask vector features as a cue and co-encodes them with visible component features to obtain mask vector prediction features, and uses target signal component features as a cue and co-encodes them with visible component features to obtain target signal component prediction features.

[0035] The signal encoder training module is used to train a signal encoder based on predicted features and true features. The predicted features include mask vector prediction features and target signal component prediction features, and the true features include mask vector features and target signal component features.

[0036] Specifically, the encoder module includes:

[0037] A signal encoder is used to extract the visible component features of visible group signal components;

[0038] A mask signal encoder is used to extract the mask component features of the mask group signal components;

[0039] A vector encoder is used to extract the mask vector features of a mask vector.

[0040] Specifically, the dual-branch mask prediction module includes:

[0041] The target feature predictor is used to obtain the predicted features of the mask vector by jointly encoding the mask vector features with the visible component features, using the mask vector features as a cue.

[0042] A mask feature predictor is used to obtain the predicted features of the target signal components by jointly encoding them with the visible component features, using the target signal component features as cues.

[0043] A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method described above.

[0044] A computer device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method described above.

[0045] The embodiments of the present invention have the following beneficial effects:

[0046] This invention decomposes weak laryngeal electromyography (EMG) signals into multiple low-frequency and high-frequency components. By predicting the mask between different components, the model achieves self-supervised learning, overcoming the problem that existing laryngeal EMG signal interpretation techniques rely on large-scale training labels. This invention enables the training of network model parameters using only a large amount of unlabeled data, effectively avoiding the cost and labeling errors caused by relying on expensive manual labeling.

[0047] Furthermore, randomly generated mask vectors are used to mask the low-frequency and high-frequency components of the signal, obtaining the visible group signal components and the masked group signal components. By masking the signal components with randomly generated mask vectors, the model can learn the deep correlation features between signal components. Then, the visible component features of the visible group signal components, the target signal component features of the masked group signal components, and the mask vector features of the mask vector are extracted using a signal encoder, a mask signal encoder, and a vector encoder, respectively. The mask vector features are jointly encoded with the visible component features to obtain the mask vector prediction features, and the target signal component features are jointly encoded with the visible component features to obtain the target signal component prediction features. This achieves bidirectional prediction of the target component and the mask vector, overcoming the problem of insufficient feature learning in existing single-branch self-supervised methods. This allows the present invention to more comprehensively mine the deep correlation features between wavelet components of weak laryngeal electromyography signals, significantly improving the model's feature representation ability for weak laryngeal electromyography signals.

[0048] During the fine-tuning training phase, the encoder parameters are unfrozen to perform end-to-end supervised training of the signal encoder module and the classification module. This overcomes the problem of overfitting that traditional methods are prone to in scenarios with few samples, enabling the present invention to maintain superior interpretation performance even with small-scale labeled data, and has stronger practicality and generalization ability. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] in:

[0051] Figure 1 A flowchart illustrating an embodiment of a method for interpreting laryngeal electromyographic signals based on self-supervised joint coding prediction provided by the present invention;

[0052] Figure 2 A schematic diagram of an embodiment of a laryngeal electromyography signal interpretation system based on self-supervised joint coding prediction provided by the present invention;

[0053] Figure 3 This is a schematic diagram of the structure of an embodiment of the signal interpretation model provided by the present invention;

[0054] Figure 4 This is a schematic diagram of the structure of an embodiment of the pre-training module provided by the present invention;

[0055] Figure 5 A schematic diagram of the structure of an embodiment of the device provided by the present invention;

[0056] Figure 6 A schematic diagram of the structure of an embodiment of the medium provided by the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] like Figure 1 As shown, Figure 1 This is a flowchart illustrating an embodiment of a laryngeal electromyography (EMG) signal interpretation method based on self-supervised joint coding prediction provided by the present invention. The method includes:

[0059] S101: Obtain a laryngeal electromyography (EMG) signal dataset and a set of instruction labels. The laryngeal EMG signal dataset includes several laryngeal EMG signals, and each laryngeal EMG signal corresponds to a fixed instruction label in the instruction label set.

[0060] For example, a set of weak laryngeal electromyography (EMG) signals and a set of instruction tags are acquired. The set of weak laryngeal EMG signals includes multiple weak laryngeal EMG signals collected in the human larynx by an EMG sensor, and each weak laryngeal EMG signal corresponds to a fixed instruction tag in the instruction tag set.

[0061] S102: The laryngeal electromyography signal is decomposed into low-frequency and high-frequency components by wavelet transform.

[0062] For example, the weak laryngeal electromyography signal T is subjected to multiple wavelet transforms according to the following formula to obtain the low-frequency component of the decomposed signal. and high-frequency components of the signal

[0063]

[0064] S103: Randomly generate a mask vector to mask the low-frequency and high-frequency components of the signal, and obtain the visible group signal components and the masked group signal components.

[0065] For example, a mask vector is randomly generated, and the low-frequency components and high-frequency components of the signal are masked according to the mask vector. The unmasked components are regarded as visible group signal components, and the masked components are regarded as masked group signal components.

[0066] S104: Use a signal encoder, a mask signal encoder, and a vector encoder to extract the visible component features of the visible group signal components, the target signal component features of the mask group signal components, and the mask vector features of the mask vector, respectively.

[0067] S105: The mask vector prediction feature is obtained by jointly encoding the mask vector feature and the visible component feature, and the target signal component prediction feature is obtained by jointly encoding the target signal component feature and the visible component feature.

[0068] For example, a signal encoder, a mask signal encoder, and a vector encoder are used to extract the visible component features of the visible group signal components, the target signal component features of the mask group signal components, and the mask vector features of the mask vector, respectively.

[0069] Furthermore, the mask vector features are jointly encoded with the visible group signal component features to obtain vector cue coding features, and feature prediction is performed on the vector cue coding features to obtain the target signal component prediction features.

[0070] The target signal component features are used as cues and are jointly encoded with the visible component features to obtain signal cues coding features. Then, feature prediction is performed on the signal cues coding features to obtain mask vector prediction features.

[0071] S106: Train a signal encoder based on predicted features and true features. The predicted features include the mask vector prediction features and the target signal component prediction features. The true features include the mask vector features and the target signal component features.

[0072] For example, the L2 distance between the mask signal component features and the predicted mask signal component features, and between the mask vector features and the predicted mask vector features, is calculated. The L2 distance function expression is used as the pre-training loss function, and the obtained L2 distance is used as the training loss to guide the pre-training process. The L2 distance function expression is shown in the following formula:

[0073]

[0074] Where L is the pre-training loss function, F M For the characteristics of the target signal components, For the target signal component prediction features, F S For mask vector features, Predict features for mask vectors.

[0075] S107: Combine the trained signal encoder with the classification module to construct a signal interpretation model. Use the laryngeal electromyography signal set as the training set and the instruction label set as the label set to fine-tune the signal interpretation model under label guidance.

[0076] For example, a signal interpretation model based on a self-supervised joint encoding prediction framework is constructed, which includes a pre-trained signal encoder and a classification module.

[0077] Further, the parameters of the signal encoder module are unfrozen; the laryngeal electromyography (EMG) signal set is used as the training set, and the instruction label set is used as the label set. Weak laryngeal EMG signals from the laryngeal EMG signal set are input into the signal interpretation model. The signal interpretation model extracts features and classifies the weak laryngeal EMG signals to obtain the interpretation result. Based on the interpretation result and the instruction labels in the instruction label set, the loss function is calculated. The parameters of the signal encoder module and classification module in the signal interpretation model based on the self-supervised joint encoding prediction framework are updated, and the next iteration is performed until the fine-tuning training process ends. The loss function is shown in the following formula:

[0078]

[0079] Among them, L cls Let A be the loss function. ij For the interpretation results, R ij Here, C represents the number of instruction categories corresponding to the laryngeal electromyography signal, and n represents the number of samples.

[0080] S108: Input the electromyographic signal to be interpreted into the fine-tuned signal interpretation model and output the instruction classification result.

[0081] For example, a weak laryngeal electromyography (EMG) signal to be interpreted is acquired; the weak EMG signal to be interpreted is input into a finely tuned signal interpretation model to obtain the interpretation result.

[0082] As described above, this invention decomposes weak laryngeal electromyography (EMG) signals into multiple low-frequency and high-frequency components. By predicting the mask between different components, the model achieves self-supervised learning, overcoming the problem that existing laryngeal EMG signal interpretation techniques rely on large-scale training labels. This allows the invention to train network model parameters using only a large amount of unlabeled data, effectively avoiding the cost and labeling errors associated with expensive manual labeling.

[0083] Furthermore, randomly generated mask vectors are used to mask the low-frequency and high-frequency components of the signal, obtaining the visible group signal components and the masked group signal components. By masking the signal components with randomly generated mask vectors, the model can learn the deep correlation features between signal components. Then, the visible component features of the visible group signal components, the target signal component features of the masked group signal components, and the mask vector features of the mask vector are extracted using a signal encoder, a mask signal encoder, and a vector encoder, respectively. The mask vector features are jointly encoded with the visible component features to obtain the mask vector prediction features, and the target signal component features are jointly encoded with the visible component features to obtain the target signal component prediction features. This achieves bidirectional prediction of the target component and the mask vector, overcoming the problem of insufficient feature learning in existing single-branch self-supervised methods. This allows the present invention to more comprehensively mine the deep correlation features between wavelet components of weak laryngeal electromyography signals, significantly improving the model's feature representation ability for weak laryngeal electromyography signals.

[0084] During the fine-tuning training phase, the encoder parameters are unfrozen to perform end-to-end supervised training of the signal encoder module and the classification module. This overcomes the problem of overfitting that traditional methods are prone to in scenarios with few samples, enabling the present invention to maintain superior interpretation performance even with small-scale labeled data, and has stronger practicality and generalization ability.

[0085] like Figure 2 As shown, Figure 2 This is a schematic diagram of an embodiment of a laryngeal electromyography (EMG) signal interpretation system based on self-supervised joint coding prediction provided by the present invention. The system 10 includes:

[0086] The laryngeal electromyography signal dataset and instruction label set acquisition module 11 is used to acquire the laryngeal electromyography signal dataset and instruction label set. The laryngeal electromyography signal dataset includes several laryngeal electromyography signals, wherein each laryngeal electromyography signal corresponds to a fixed instruction label in the instruction label set.

[0087] The pre-training module 12 is used to decompose the laryngeal electromyography signal into low-frequency and high-frequency components using wavelet transform; randomly generate mask vectors to mask the low-frequency and high-frequency components to obtain visible and masked signal components; extract visible component features of the visible signal components, target signal component features of the masked signal components, and mask vector features of the mask vector using a signal encoder, a mask signal encoder, and a vector encoder, respectively; obtain mask vector prediction features by jointly encoding the mask vector features with the visible component features, and obtain target signal component prediction features by jointly encoding the target signal component features with the visible component features; train the signal encoder based on the prediction features and the true features, wherein the prediction features include mask vector prediction features and target signal component prediction features, and the true features include mask vector features and target signal component features.

[0088] The signal interpretation model construction module 13 is used to combine the trained signal encoder with the classification module to construct a signal interpretation model. The laryngeal electromyography signal set is used as the training set, and the instruction label set is used as the label set to perform label-guided fine-tuning training on the signal interpretation model.

[0089] The signal interpretation module 14 is used to input the electromyographic signal to be interpreted into the fine-tuned signal interpretation model and output the instruction classification result.

[0090] For example, in the laryngeal electromyography signal dataset and instruction label set acquisition module 11, the laryngeal electromyography signal dataset and instruction label set are acquired. The laryngeal electromyography signal dataset includes several laryngeal electromyography signals, wherein each laryngeal electromyography signal corresponds to a fixed instruction label in the instruction label set.

[0091] Further, in the pre-training module 12, the laryngeal electromyography signal is decomposed into low-frequency and high-frequency components using wavelet transform. A mask vector is randomly generated, and the low-frequency and high-frequency components are masked according to the mask vector. Unmasked components are considered visible signal components, and masked components are considered masked signal components. Visible component features of the visible signal components, target signal component features of the masked signal components, and mask vector features of the mask vector are extracted. Further, the mask vector features are jointly encoded with the visible signal component features to obtain vector cue encoding features, and feature prediction is performed on the vector cue encoding features to obtain mask vector prediction features. The target signal component features are jointly encoded with the visible signal component features to obtain signal cue encoding features, and feature prediction is performed on the signal cue encoding features to obtain mask signal component prediction features. A signal encoder is trained based on the predicted features and the true features. The predicted features include the mask vector prediction features and the target signal component prediction features, and the true features include the mask vector features and the target signal component features. Specifically, the L2 distance between the mask signal component features and the predicted features of the mask signal components, and between the mask vector features and the predicted features of the mask vector, is calculated. The L2 distance function expression is used as the pre-training loss function, and the obtained L2 distance is used as the training loss to guide the pre-training process.

[0092] Furthermore, in the signal interpretation model construction module 13, such as Figure 3 As shown, Figure 3 This is a schematic diagram of an embodiment of the signal interpretation model provided by the present invention. A signal interpretation model based on a self-supervised joint encoding prediction framework is constructed. The signal interpretation model includes a pre-trained signal encoder and a classification module. The parameters of the signal encoder module are unfrozen; a weak laryngeal electromyography (EMG) signal T from the laryngeal EMG signal set is input into the signal interpretation model. The signal interpretation model performs feature extraction and classification on the weak laryngeal EMG signal T to obtain the interpretation result A. The loss function L is calculated based on the interpretation result A and the instruction label R in the instruction label set. cls The parameters of the signal encoder module and classification module in the signal interpretation model based on the self-supervised joint coding prediction framework are updated and the next iteration is performed until the fine-tuning training process ends.

[0093] Furthermore, in the signal interpretation module 14, a weak laryngeal electromyography signal to be interpreted is acquired; the weak laryngeal electromyography signal to be interpreted is input into the fine-tuned signal interpretation model to obtain the interpretation result.

[0094] like Figure 4 As shown, Figure 4 This is a schematic diagram of a pre-training module according to an embodiment of the present invention. The pre-training module 12 includes:

[0095] The signal decomposition module is used to decompose laryngeal electromyography signals into low-frequency and high-frequency components using wavelet transform.

[0096] For example, a weak laryngeal electromyography (EMG) signal T is input into a signal decomposition module. Multiple wavelet transforms are performed on the weak laryngeal EMG signal T according to the following formula to obtain the low-frequency component of the decomposed signal. and high-frequency components of the signal

[0097]

[0098] The signal component masking module is used to randomly generate a mask vector to mask the low-frequency component and high-frequency component of the signal, thereby obtaining the visible group signal component and the masked group signal component.

[0099] For example, the low-frequency component and high-frequency component of the signal are input into the signal component masking module to randomly generate a mask vector, and the signal components are masked according to the mask vector. The unmasked components are regarded as visible group signal components, and the masked components are regarded as masked group signal components.

[0100] The encoder module is used to extract the visible component features of the visible group signal components, the target signal component features of the mask group signal components, and the mask vector features of the mask vector.

[0101] For example, the encoder module includes: a signal encoder, a mask signal encoder, and a vector encoder. The visible group signal components T... V , mask group signal component T M and mask vector T S The visible signal component features F are obtained by inputting the signal encoder, mask signal encoder, and mask vector encoder respectively. V Target signal component characteristics F M sum vector features F S .

[0102] The dual-branch mask prediction module uses mask vector features as a cue and co-encodes them with visible component features to obtain mask vector prediction features, and uses target signal component features as a cue and co-encodes them with visible component features to obtain target signal component prediction features.

[0103] For example, the dual-branch mask prediction module includes a target feature predictor and a mask feature predictor. The mask vector features are jointly encoded with the visible group signal component features as a cue to obtain vector cue encoded features. The obtained vector cue encoded features are then input into the target feature predictor to obtain the target signal component prediction features.

[0104] The obtained target signal component features are used as cues and jointly encoded with the visible component features to obtain signal cues coding features. The obtained signal cues coding features are then input into the mask feature predictor to obtain mask vector prediction features.

[0105] The signal encoder training module is used to train the signal encoder based on predicted features and true features. The predicted features include mask vector prediction features and target signal component prediction features, while the true features include mask vector features and target signal component features.

[0106] For example, the L2 distance between the mask signal component features and the mask signal component prediction features, and between the mask vector features and the mask vector prediction features, is calculated. The L2 distance function expression is used as the pre-training loss function, and the obtained L2 distance is used as the training loss to guide the pre-training process.

[0107] As described above, this invention decomposes weak laryngeal electromyography (EMG) signals into multiple low-frequency and high-frequency components. By predicting the mask between different components, the model achieves self-supervised learning, overcoming the problem that existing laryngeal EMG signal interpretation techniques rely on large-scale training labels. This allows the invention to train network model parameters using only a large amount of unlabeled data, effectively avoiding the cost and labeling errors associated with expensive manual labeling.

[0108] Furthermore, by constructing a dual-branch mask prediction module consisting of a target feature predictor branch and a mask feature predictor branch, bidirectional prediction of target component prediction and mask vector prediction is achieved, overcoming the problem of insufficient feature learning in existing single-branch self-supervised methods. This enables the present invention to more comprehensively mine the deep correlation features between wavelet components of weak laryngeal electromyography signals, significantly improving the model's feature representation ability for weak laryngeal electromyography signals.

[0109] Furthermore, a two-stage optimization strategy of "pre-training + fine-tuning" is adopted. In the pre-training stage, a general feature representation is learned through the joint prediction task of signal components. In the fine-tuning training stage, the signal encoder parameters are unfrozen for end-to-end supervised training. This overcomes the problem of overfitting in traditional methods in scenarios with few samples, enabling the present invention to maintain superior interpretation performance even with small-scale labeled data, and has stronger practicality and generalization ability.

[0110] like Figure 5 As shown, Figure 5 This is a schematic diagram of an embodiment of the device provided by the present invention. The device 20 includes a memory 21 and a processor 22. The memory 21 stores a computer program, and the processor 22 executes the computer program during operation to achieve, for example... Figure 1 The method shown.

[0111] The specific technical details of the laryngeal electromyography signal interpretation method based on self-supervised joint coding prediction implemented by the aforementioned device 20 when executing the computer program have been discussed in detail in the aforementioned method steps, and therefore will not be repeated here.

[0112] like Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of an embodiment of the medium provided by the present invention. The medium 30 stores at least one computer program 31, which is executed by the processor 22 to perform the following... Figure 1 The method shown is detailed above and will not be repeated here. In one embodiment, the medium 30 can be a storage chip, hard disk, portable hard disk, USB flash drive, optical disk, or other read / write storage device, or even a server, etc.

[0113] Furthermore, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific or sequential order shown to achieve the desired result. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0114] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer-readable storage media are basically similar to the method embodiments, and therefore described more simply; relevant parts can be referred to the descriptions of the method embodiments.

[0115] The apparatus, device, non-volatile computer-readable storage medium and method provided in the embodiments of this specification are corresponding. Therefore, the apparatus, device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, device and non-volatile computer storage medium will not be repeated here.

[0116] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0117] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components. Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0119] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0120] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0121] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0122] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0123] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0124] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0125] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0126] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0127] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for interpreting laryngeal electromyographic signals based on self-supervised joint coding prediction, characterized in that, The method includes: A dataset of laryngeal electromyography (EMG) signals and a set of instruction labels are obtained. The dataset of laryngeal EMG signals includes several laryngeal EMG signals, and each laryngeal EMG signal corresponds to a fixed instruction label in the instruction label set. The laryngeal electromyographic signal was decomposed into low-frequency and high-frequency components using wavelet transform. Randomly generate a mask vector to mask the low-frequency and high-frequency components of the signal, and obtain the visible group signal components and the masked group signal components; The visible component features of the visible group signal components, the target signal component features of the mask group signal components, and the mask vector features of the mask vector are extracted using a signal encoder, a mask signal encoder, and a vector encoder, respectively. The mask vector features and the visible component features are jointly encoded to obtain the target signal component prediction features, and the target signal component features and the visible component features are jointly encoded to obtain the mask vector prediction features; A signal encoder is trained based on predicted features and true features. The predicted features include the mask vector prediction features and the target signal component prediction features. The true features include the mask vector features and the target signal component features. The trained signal encoder is combined with the classification module to construct a signal interpretation model. The laryngeal electromyography signal set is used as the training set, and the instruction label set is used as the label set. The signal interpretation model is then fine-tuned under label guidance. The electromyographic signal to be interpreted is input into the finely tuned signal interpretation model, and the output instruction classification result is generated.

2. The method for interpreting laryngeal electromyographic signals based on self-supervised joint coding prediction according to claim 1, characterized in that, The signal encoder trained based on predicted features and true features, wherein the predicted features include mask vector prediction features and target signal component prediction features, and the true features include mask vector features and target signal component features, specifically including: The pre-training loss function is determined based on the predicted features and the true features. The pre-training loss function is as follows: Where L is the pre-training loss function, F M For the characteristics of the target signal components, For the target signal component prediction features, F S For mask vector features, Predict features for mask vectors; The signal encoder is trained based on the pre-trained loss function.

3. The method for interpreting laryngeal electromyographic signals based on self-supervised joint coding prediction according to claim 2, characterized in that, The step of using a set of laryngeal electromyography signals as a training set and a set of instruction labels as a label set to fine-tune the signal interpretation model under label guidance specifically includes: Unfreeze the parameters of the pre-trained signal encoder module; The laryngeal electromyography (EMG) signal set is used as the training set, and the instruction label set is used as the label set. The laryngeal EMG signal is input into the signal interpretation model, and the interpretation result is output. The loss function is calculated based on the interpretation results and instruction labels, the parameters of the signal encoder and classification module are updated, and the optimization is iteratively performed until fine-tuning training is completed.

4. The method for interpreting laryngeal electromyographic signals based on self-supervised joint coding prediction according to claim 3, characterized in that, The loss function is based on Determined, among which, L cls Let A be the loss function. ij For the interpretation results, R ij Here, C represents the number of instruction categories corresponding to the laryngeal electromyography signal, and n represents the number of samples.

5. A laryngeal electromyography signal interpretation system based on self-supervised joint encoding prediction, characterized in that, The system includes: A laryngeal electromyography (EMG) signal dataset and instruction label set acquisition module is used to acquire a laryngeal EMG signal dataset and an instruction label set. The laryngeal EMG signal dataset includes several laryngeal EMG signals, and each laryngeal EMG signal corresponds to a fixed instruction label in the instruction label set. A pre-training module is used to decompose the laryngeal electromyography signal into low-frequency and high-frequency components using wavelet transform; randomly generate a mask vector to mask the low-frequency and high-frequency components to obtain visible and masked signal components; extract visible component features of the visible signal components, target signal component features of the masked signal components, and mask vector features of the mask vector using a signal encoder, a mask signal encoder, and a vector encoder, respectively; use the mask vector features as a cue to jointly encode with the visible component features to obtain mask vector prediction features, and use the target signal component features as a cue to jointly encode with the visible component features to obtain target signal component prediction features; train the signal encoder based on the prediction features and the true features, where the prediction features include mask vector prediction features and target signal component prediction features, and the true features include mask vector features and target signal component features. The signal interpretation model construction module is used to combine the trained signal encoder with the classification module to build a signal interpretation model. The laryngeal electromyography signal set is used as the training set, and the instruction label set is used as the label set to perform label-guided fine-tuning training of the signal interpretation model. The signal interpretation module is used to input the electromyographic signal to be interpreted into the finely tuned signal interpretation model and output the instruction classification result.

6. The laryngeal electromyography signal interpretation system based on self-supervised joint coding prediction according to claim 5, characterized in that, The pre-training module specifically includes: The signal decomposition module is used to decompose the laryngeal electromyography signal into low-frequency and high-frequency components through wavelet transform. The signal component masking module is used to randomly generate a masking vector to mask the low-frequency component and high-frequency component of the signal, thereby obtaining the visible group signal component and the masked group signal component. The encoder module is used to extract the visible component features of the visible group signal components, the target signal component features of the mask group signal components, and the mask vector features of the mask vector. The dual-branch mask prediction module uses mask vector features as a cue and co-encodes them with visible component features to obtain the target signal component prediction features. The signal encoder training module is used to train a signal encoder based on predicted features and true features. The predicted features include mask vector prediction features and target signal component prediction features, and the true features include mask vector features and target signal component features.

7. The laryngeal electromyography signal interpretation system based on self-supervised joint coding prediction according to claim 6, characterized in that, The encoder module specifically includes: A signal encoder is used to extract the visible component features of visible group signal components; A mask signal encoder is used to extract the mask component features of the mask group signal components; A vector encoder is used to extract the mask vector features of a mask vector.

8. The laryngeal electromyography signal interpretation system based on self-supervised joint coding prediction according to claim 6, characterized in that, The dual-branch mask prediction module specifically includes: The target feature predictor is used to obtain the predicted features of the mask vector by jointly encoding the mask vector features with the visible component features, using the mask vector features as a cue. A mask feature predictor is used to obtain the predicted features of the target signal components by jointly encoding them with the visible component features, using the target signal component features as cues.

9. A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method as claimed in any one of claims 1 to 4.

10. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as claimed in any one of claims 1 to 4.

Images