Difficult airway detection method and electronic device and storage medium thereof

Abstract

An embodiment of the present invention provides a method for detecting a difficult airway. The method includes: extracting acoustic features and detecting voice activity on the user's voice audio; performing average pooling on the extracted acoustic features to obtain a first embedding vector; inputting the uttered frame after activity detection into a deep speaker model for a speaker characterizing a difficult airway to obtain a second embedding vector characterizing the difficult airway; and inputting the first embedding vector and the second embedding vector into a support vector machine to detect the difficult airway. An embodiment of the present invention also provides a difficult airway detection system. An embodiment of the present invention implements difficult airway detection based on voice technology. It solves the problems of cumbersomeness, high professional requirements and inaccuracy of traditional methods. Speaker embedding well characterizes the physical structure of the airway. Due to the convenient transmission of voice signals, users only need to use edge devices to perform detection, which greatly facilitates user detection.

Description

Technical Field

[0001] The present invention relates to the field of physiological structure detection, and in particular to a difficult airway detection method and electronic equipment and storage medium thereof. Background Art

[0002] A difficult airway refers to a situation in which the human airway has a special structure, making it difficult for conventionally trained anesthesiologists to perform mask ventilation or endotracheal intubation on the patient. Failure to detect a difficult airway during physical examination is the most important factor leading to major complications associated with long-term morbidity, accounting for 25% of anesthesia-related deaths. Accurate detection of potential difficult airways is key to reducing morbidity and mortality caused by failed airway management. However, existing clinical assessments of difficult airways are neither convenient nor accurate enough.

[0003] The extraction of several physical features used in clinical practice to detect difficult airways and the results of bedside screening tests rely on manual measurements and judgment. Recently, advanced imaging techniques such as computed tomography, magnetic resonance imaging, and ultrasound therapy have been used to assist in airway management. However, these procedures are often expensive, time-consuming, and invasive to the user due to radiation exposure.

[0004] To solve the above problems, usually use:

[0005] 1. Mallampati test observes the pharyngeal structure that the user can see and grades the user.

[0006] 2. The thyromental distance test is the distance from the thyroid cartilage notch to the chin process.

[0007] In the process of implementing the present invention, the inventors found that there are at least the following problems in the related art:

[0008] These technologies require professional medical staff to perform measurements or observations based on special equipment, which makes it difficult for users to perform efficient, rapid, and timely detection. At the same time, there are inevitably many errors and subjective judgments in the early warning of difficult airway intubation. Summary of the invention

[0009] In order to at least solve the problems in the prior art that difficult airway detection has high requirements on physicians, is difficult to popularize, and the test results are highly subjective.

[0010] In a first aspect, an embodiment of the present invention provides a method for detecting a difficult airway, comprising:

[0011] Perform acoustic feature extraction and voice activity detection on the user's voice audio;

[0012] The extracted acoustic features are average-pooled to obtain a first embedding vector;

[0013] Inputting the utterance frame after activity detection into a deep speaker model for representing a speaker with difficult airway to obtain a second embedding vector representing the difficult airway;

[0014] The first embedding vector is combined with the second embedding vector to be input into a support vector machine to detect difficult airway.

[0015] In a second aspect, an embodiment of the present invention provides a difficult airway detection system, comprising:

[0016] A speech processing program module is used to extract acoustic features and detect speech activity of the user's speech audio;

[0017] A first embedding determination program module, used for performing average pooling processing on the extracted acoustic features to obtain a first embedding vector;

[0018] A second embedding determination program module is used to input the utterance frame after activity detection into a deep speaker model for representing a speaker with difficult airway, so as to obtain a second embedding vector representing the difficult airway;

[0019] A detection program module is used to input the first embedding vector and the second embedding vector into a support vector machine to detect difficult airway.

[0020] According to a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the difficult airway detection method of any embodiment of the present invention.

[0021] In a fourth aspect, an embodiment of the present invention provides a storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the difficult airway detection method of any embodiment of the present invention are implemented.

[0022] The beneficial effects of the embodiments of the present invention are: difficult airway detection is achieved based on voice technology. The problems of cumbersomeness, high professional requirements and inaccuracy of traditional methods are solved. In addition, the proposed method based on speaker embedding shows the ability to identify difficult airways. Speaker embedding well characterizes the physical structure of the airway. More importantly, due to the convenient transmission of voice signals, users only need to use smart mobile devices to perform detection, which greatly facilitates the user's detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 is a flow chart of a difficult airway detection method provided by one embodiment of the present invention;

[0025] Figure 2 is a framework diagram of a difficult airway detection system of a difficult airway detection method provided by an embodiment of the present invention;

[0026] Figure 3 It is a basic block of a deep speaker model of a difficult airway detection method provided by an embodiment of the present invention;

[0027] Figure 4 This is a data diagram of different speech feature results of difficult airway detection of a difficult airway detection method provided by an embodiment of the present invention;

[0028] Figure 5 This is a data diagram showing the impact of different speaker embeddings on difficult airway detection in a difficult airway detection method provided by an embodiment of the present invention;

[0029] Figure 6 is a ROC curve data diagram of a difficult airway detection method provided by an embodiment of the present invention;

[0030] Figure 7 A data diagram showing comparison of the results of a difficult airway detection method provided by an embodiment of the present invention with those of a traditional clinical method in terms of difficult airway detection;

[0031] Figure 8 is a structural schematic diagram of a difficult airway detection system provided by an embodiment of the present invention;

[0032] Fig. 9 A schematic structural diagram of an embodiment of an electronic device for difficult airway detection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] like Figure 1 FIG. 1 is a flow chart of a difficult airway detection method provided by an embodiment of the present invention, comprising the following steps:

[0035] S11: extracting acoustic features and detecting voice activity of the user's voice audio;

[0036] S12: performing average pooling processing on the extracted acoustic features to obtain a first embedding vector;

[0037] S13: inputting the utterance frame after activity detection into a speaker model for representing a speaker with difficult airway, to obtain a second embedding vector representing the difficult airway;

[0038] S14: Inputting the first embedding vector and the second embedding vector into a support vector machine to detect difficult airway.

[0039] In this embodiment, since voice is easy to transmit and has a connection between the respiratory tract and pronunciation, voice detection can be used to allow users to record voice through mobile phones, pads and other terminals and upload them for detection.

[0040] For step S11, the method has two branches: one based on the original speech features of the user's speech audio and one based on speaker embedding. The original channel can directly use the speech features of the user's speech audio. At the same time, the speaker embedding-based channel extracts the speaker embedding for the final classifier. Figure 2 The framework of the difficult airway detection system is presented. Acoustic features are extracted from the original waveform of the user's speech audio, and the signal is converted from the original waveform to a frame feature with shape (T, D). Then a voice activity detection system is used to select a speech frame (voiced frame) (T', D) from the user's speech audio.

[0041] For step S12, the extracted acoustic features are averaged and pooled. In the case of embedding extracted by the mean pool, the acoustic features are averaged on the time axis to obtain a first embedding vector. This traditional signal processing-based method has high computational efficiency and is easy to verify.

[0042] In step S13, the utterance frame after activity detection is input into a deep speaker model for characterizing a speaker with a difficult airway, and the deep speaker model is trained by extracting basic acoustic features and speech frames, and then the deep speaker model is trained on a large speaker recognition dataset. The second embedding vector characterizing the difficult airway is obtained through the deep speaker model.

[0043] Specifically, the speaker model also includes a speaker model based on a traditional machine learning method, including:

[0044] The unsupervised modeling Gaussian mixture model-general background model is used to determine the supervector of the user's speech audio to distinguish the airway type of the user.

[0045] In this embodiment, the detection of difficult airway can also be interpreted as distinguishing people with difficult airway from ordinary people. Therefore, speaker recognition technology is used for this task. In fact, the traditional i-vector has been used to represent a series of acoustic features as a vector of fixed length. In order to extract the i-vector, a K-component GMM (Gaussian Mixed Model) is first estimated, expressed as a UBM (Universal Background Model) to model the common knowledge of the entire data space. Thereafter, a super vector is constructed by aggregating new GMM components adapted from GMM-UBM. Due to the existence of unsupervised modeling of GMM-UBM, existing large-scale data is applied to build the model. It is assumed that the super vector obeys a factor analysis model of the following form:

[0046] s=m+Tw

[0047] Where s is the supervector of the user's speech audio, m is the supervector of GMM-UBM, T is the transformation matrix estimated by the training data, and w is the i-vector vector representing the speaker information.

[0048] The speaker model includes a deep speaker model constructed by a deep neural network based on the acoustic features of the speaker, including:

[0049] The residual network constructed by the two-dimensional convolutional neural network is used to collect frame-level information, and

[0050] The time-delay neural network constructed by one-dimensional convolutional neural network module, residual enhanced one-dimensional convolutional neural network module, and squeeze and excitation network module performs speaker verification tasks based on channel enhanced attention, propagation and aggregation.

[0051] Deep learning technology is relatively effective for detection. With enough data, deep neural networks can be used to better model acoustic features. With the development of speaker verification tasks and inspired by the x-vector framework, ECAPA-TDNN (Emphasized Channel Attention, Propagation and Aggregation in Time-Delayed Neural Network Based Speaker Verification) achieved state-of-the-art performance on the Voxceleb (dataset) benchmark. At the same time, ResNet (residual network) is also an important work in speaker verification.

[0052] Figure 3 The basic blocks of ResNet and ECAPATDNN are shown. ResNet uses 2D features as input and processes them using 2D Convolutional Neural Network (CNN) layers. The mean and standard deviation are used to collect frame-level information. They are then concatenated together and propagated through the embedding layer. Meanwhile, ECAPA-TDNN utilizes a 1D Res2Net module with effective skip connections. Then, the “Squeeze and Excitation Block” (SE Block) explicitly models the interdependencies of the channels. Finally, “Channel-Dependent Frame Attention Pooling” and hierarchical features are adopted to exploit the global properties of the recorded user’s speech audio. The deep speaker model is trained to identify the speakers in the training set. Finally, the embedding before the softmax layer is taken as the speaker embedding, which is the second embedding vector.

[0053] For step S14, the classification model is finally classified by a support vector machine (SVM), whose input is an embedding based on speech feature pooling and a speaker embedding based on deep learning.

[0054] For each sentence, the input to the classification model is an embedding. The model is then trained to determine whether the embedding vector is positive or negative for the detection of a difficult airway. First, the input first embedding vector combined with the second embedding vector is mapped to the feature space using a nonlinear mapping based on the kernel function of the support vector machine. The kernel function then generates features by measuring the similarity between points in the feature space. Finally, a hyperplane is estimated in which the feature space is separated by markers, resulting in the detection of a difficult airway.

[0055] It can be seen from this implementation that difficult airway detection is achieved based on voice technology. The problems of cumbersomeness, high professional requirements and inaccuracy of traditional methods are solved. In addition, the proposed speaker embedding-based method shows the ability to identify difficult airways. Speaker embedding well characterizes the physical structure of the airway. More importantly, due to the convenient transmission of voice signals, users only need to use edge devices for detection, which greatly facilitates user detection.

[0056] The method was experimentally demonstrated using a dataset of 1,189 native Mandarin speakers who were anesthetized from December 28, 2020 to September 16, 2021. Exclusion criteria included recent upper respiratory abnormalities, history of language disorders, etc. They were asked to read 10 sentences in a quiet environment. These 16-bit handheld recorders with a sampling frequency of 44.1kHz were used to record the participants' speech. The 10 sentences were selected based on the coverage of toneless pinyin. To evaluate the performance of the method, 201 speakers, including 107 positive samples and 94 negative samples, were excluded from the test set. The training set was then cross-validated tenfold. The CL (Cormack-Lehane, laryngoscopy) grading describes the visibility of the vocal cords during laryngoscopy, ranging from 1 (full view of the vocal cords) to 4 (perineum not visible). At the same time, the CL scores of all participants were obtained using direct laryngoscopy. According to the airway difficulty score, users were divided into airway difficulty (CL 3-4) and non-airway difficulty (CL 1-2).

[0057] The dataset is aligned and segmented. Long recordings are segmented by an energy-based voice activity detection system. Hypotheses and session time tag outputs are then generated through a chain model and compared with the reference text. In addition, the reference text with the smallest edit distance is used as the final transcript of the segment.

[0058] On the basis of preparing the training dataset, a speaker verification dataset is also prepared. For the deep speaker system of this method, audio data from the VoxCeleb 2 dataset is used, which is collected from interview videos uploaded to YouTube. For training, the development part of the VoxCeleb 2 dataset is used, which contains 5994 speakers and 1092009 sentences. All recordings and speaker identities are from celebrities. Most of the recording scenes are in relatively quiet spaces.

[0059] The data of the support vector machine for classification was trained to classify the speech features using labels based on CL scores. In order to evaluate the generalization performance of the algorithm, the hyperparameter selection with ten-fold cross validation was used. Then the optimal hyperparameters were selected based on the ten-fold cross validation to train 988 speakers. For hyperparameter optimization, the candidate kernels were sigmoid kernel, rational basis function kernel, linear kernel and polynomial kernel. The regularization parameter C∈{1, 10, 100, 1000} and the kernel coefficient γ∈{0.01, 0.001, 0.0001} were grid searched.

[0060] Original speech features, the sampling frequency of the user's speech audio is reduced from 44.1kHz to 16.0kHz. They are framed based on an interval of 10ms and a window length of 25ms, with an overlap of 15ms between frames. A Hamming window is then used on each frame, and the Kaldi energy-based VAD (Voice Activity Detection) is used to select the utterance frame. The method then transforms each frame using various audio feature extraction methods, including Mel frequency cepstral coefficients (MFCC), filter banks (Fbank), LPC (linear predictive coding), and resonance peaks. Except for the above-mentioned resonance peaks, all feature dimensions are set to 40. The first to fourth resonance peaks are extracted. The speech features are then averaged over time and used as the final feature vector.

[0061] The i-vector speaker embedding system is trained following the Kaldi VoxCeleb recipe. MFCC features are extracted from the VoxCeleb 2 dataset. A generic background model is then trained using 2048 Gaussian components. The longest 1,000,000 utterances are then selected to train the i-vector extractor, as short utterances have a negative effect on the extractor.

[0062] For the deep learning speaker model, we train ResNet34 and an enlarged ECAPA-TDNN on the Vox-Celeb 2 dataset. We use a 40-dimensional Fbank with a window size of 25ms and a frame shift of 10ms as input, the same settings as in the original speech pipeline. During training, the audio is randomly split into 300 frames. When evaluating, we use the output of the last layer of ResNet34 and ECAPA-TDNN as the speaker embedding.

[0063] The test results of the above data are as follows. The ROC (area Under Receiver Operating Characteristic, receiver operating characteristic) curve (AUC-defined as the area under the ROC curve and the coordinate axis) is used to evaluate different features and embedding conditions. The highest point of the Youden index is designated as the threshold to obtain accuracy, specificity, and recall (ie, sensitivity). Specificity refers to the true negative rate of all negative samples, and recall refers to the true positive rate of all positive samples. The back end of the sample is averaged by the speaker. The experiment is then repeated 10 times, and the average AUC, accuracy, specificity, and recall results are reported.

[0064] Systems based on raw speech features, such as Figure 4 Results for the pipeline based on raw speech features are shown. Speech features are extracted from the raw waveform and averaged over time into a single embedding for each speech. From an AUC perspective, the performance of LPC, MFCC, and Fbank is not ideal for traditional spectrum-based features. In fact, a lot of information is lost by directly averaging the features over time. These speech features are based on mathematical transformations of the raw audio signal, so the process of extracting features does not make good use of the knowledge in the data. However, formant frequency achieves better performance than those spectrum-based features. Formant frequency describes the resonance of the vocal tract and is related to the internal structure of the upper airway, including its compliance, shape, and size. Although the performance of formant frequency is not good enough, it still guides our method to find ways to characterize speaker-related features.

[0065] Speaker embedding based systems, such as Figure 5 and Figure 6 Results of different speaker embedding systems for difficult airway detection are shown. Two different deep speaker models are used: ResNet34 and dilated-ECPA-TDNN along with the traditional i-vector speaker model. It is easy to see that the performance of deep speaker embeddings is significantly better than that of i-vector. This is probably because i-vector cannot model the non-speaker background (e.g., noise and recording channel differences) well in the presence of a large amount of data.

[0066] Subsequently, the deep speaker system outperforms the formant frequencies. This suggests that the deep speaker embeddings are better than the formant frequencies in describing acoustic features such as the structure of the speaker’s upper airway. In the deep speaker system, the knowledge learned from big data plays an important role in improving the system performance. Furthermore, ECAPA-TDNN outperforms ResNet-34 in AUC, precision, and recall, except for specificity, indicating that the threshold specified by the Youden index of ResNet-34 has a too high bias in specificity.

[0067] Compared with traditional clinical examination methods, the modified Mallampati test (MMT), upper lip bite test (ULBT) and chin-mental distance (TMD) are three traditional airway detection methods. MMT evaluates the visibility of oropharyngeal structures. For ULBT, the range of mandibular movement is assessed by asking the user to bite the upper lip with the lower incisors. TMD refers to the distance between the uppermost edge of the thyroid cartilage and the chin, measured with the neck extended and the mouth closed. MMT, ULBT and TMD require the user to be present and tested by a doctor using special instruments. Figure 7 The comparison results of all traditional clinical methods with the proposed method are shown. The last two rows are the fusion results. The posterior probabilities are averaged over different fusion models. The fusion of similar systems (ECAPA and ResNet34) shows limited improvement over ECAPA. In contrast, the formant frequency system and the deep speaker system are complementary. In addition, the system based on the final voice technique significantly outperforms all traditional methods in all cases except recall, which clearly shows the performance advantage of the voice technique over traditional methods.

[0068] In summary, this method proposes a method for detecting difficult vocal airway based on speech technology. The proposed method solves the problems of cumbersomeness, high professional requirements and insufficient accuracy of traditional methods. In addition, the proposed method based on speaker embedding shows the ability to identify difficult airway. Speaker embedding well characterizes the physical structure of the airway. More importantly, due to the convenient transmission of speech signals, users only need to use edge devices for detection, which greatly facilitates users.

[0069] like Figure 8 The figure is a schematic diagram of the structure of a difficult airway detection system provided by an embodiment of the present invention. The system can execute the difficult airway detection method described in any of the above embodiments and is configured in a terminal.

[0070] A difficult airway detection system 10 provided in this embodiment includes: a speech processing program module 11 , a first embedding determination program module 12 , a second embedding determination program module 13 and a detection program module 14 .

[0071] Among them, the speech processing program module 11 is used to extract acoustic features and detect speech activity of the user's speech audio; the first embedding determination program module 12 is used to perform average pooling processing on the extracted acoustic features to obtain a first embedding vector; the second embedding determination program module 13 is used to input the speech frame after activity detection into the deep speaker model of the speaker for characterizing the difficult airway, and obtain a second embedding vector characterizing the difficult airway; the detection program module 14 is used to input the first embedding vector and the second embedding vector into a support vector machine to detect the difficult airway.

[0072] Furthermore, the detection program module is used to:

[0073] Based on the kernel function of the support vector machine, the first embedding vector and the second embedding vector are mapped to the feature space by nonlinear mapping to estimate the hyperplane;

[0074] The user's speech audio is classified by marking the separation feature space to determine a difficult airway detection result.

[0075] The embodiment of the present invention further provides a non-volatile computer storage medium, the computer storage medium stores computer executable instructions, and the computer executable instructions can execute the difficult airway detection method in any of the above method embodiments;

[0076] As an implementation mode, the non-volatile computer storage medium of the present invention stores computer executable instructions, and the computer executable instructions are configured as follows:

[0077] Perform acoustic feature extraction and voice activity detection on the user's voice audio;

[0078] The extracted acoustic features are average-pooled to obtain a first embedding vector;

[0079] Inputting the utterance frame after activity detection into a deep speaker model for representing a speaker with difficult airway to obtain a second embedding vector representing the difficult airway;

[0080] The first embedding vector is combined with the second embedding vector to be input into a support vector machine to detect difficult airway.

[0081] As a non-volatile computer-readable storage medium, it can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the method in the embodiment of the present invention. One or more program instructions are stored in the non-volatile computer-readable storage medium, and when executed by the processor, the difficult airway detection method in any of the above method embodiments is executed.

[0082] Fig. 9 FIG. 1 is a schematic diagram of the hardware structure of an electronic device for a difficult airway detection method provided by another embodiment of the present application, such as Fig. 9 As shown, the device includes:

[0083] One or more processors 910 and memory 920, Fig. 9 A processor 910 is taken as an example. The device for the difficult airway detection method may further include: an input device 930 and an output device 940 .

[0084] The processor 910, the memory 920, the input device 930 and the output device 940 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0085] The memory 920, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the difficult airway detection method in the embodiment of the present application. The processor 910 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 920, that is, implementing the difficult airway detection method in the above method embodiment.

[0086] The memory 920 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data, etc. In addition, the memory 920 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 920 may optionally include a memory remotely arranged relative to the processor 910, and these remote memories may be connected to the mobile device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0087] The input device 930 can receive input digital or character information. The output device 940 can include a display device such as a display screen.

[0088] The one or more modules are stored in the memory 920, and when executed by the one or more processors 910, the difficult airway detection method in any of the above method embodiments is executed.

[0089] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0090] The non-volatile computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the non-volatile computer-readable storage medium may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0091] An embodiment of the present invention also provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the difficult airway detection method of any embodiment of the present invention.

[0092] The electronic device of the embodiment of the present application exists in various forms, including but not limited to:

[0093] (1) Mobile communication equipment: This type of equipment is characterized by having mobile communication functions and its main purpose is to provide voice and data communications. This type of terminal includes: smart phones, multimedia phones, functional phones, and low-end phones.

[0094] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access features. These terminals include: PDAs, MIDs, and UMPC devices, such as tablet computers.

[0095] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0096] (4) Other electronic devices with data processing functions.

[0097] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include" and "comprise" include not only those elements, but also other elements not explicitly listed, or also include elements inherent to such processes, methods, articles or equipment. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or equipment that includes the elements.

[0098] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0099] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting a difficult airway, comprising: Perform acoustic feature extraction and voice activity detection on the user's voice audio; The extracted acoustic features are average-pooled to obtain a first embedding vector; Inputting the utterance frame after activity detection into a deep speaker model for representing a speaker with difficult airway or a speaker model based on a traditional machine learning method to obtain a second embedding vector representing the difficult airway; The first embedding vector is combined with the second embedding vector to be input into a support vector machine to detect difficult airway.

2. The method according to claim 1, wherein: The deep speaker model is constructed by a deep neural network based on the acoustic features of the speaker, including: The residual network constructed by the two-dimensional convolutional neural network is used to collect frame-level information, and The time-delay neural network constructed by one-dimensional convolutional neural network module, residual enhanced one-dimensional convolutional neural network module, and squeeze and excitation network module performs speaker verification tasks based on channel enhanced attention, propagation and aggregation.

3. The method according to claim 2, wherein: The speaker model based on the traditional machine learning method includes: The unsupervised modeling Gaussian mixture model-general background model is used to determine the supervector of the user's speech audio to distinguish the airway type of the user.

4. The method according to claim 1, wherein: The step of inputting the first embedding vector and the second embedding vector into a support vector machine comprises: Based on the kernel function of the support vector machine, the first embedding vector and the second embedding vector are mapped to the feature space by nonlinear mapping to estimate the hyperplane; The user's speech audio is classified by marking the separation feature space to determine a difficult airway detection result.

5. A difficult airway detection system, comprising: A speech processing program module is used to extract acoustic features and detect speech activity of the user's speech audio; A first embedding determination program module, used for performing average pooling processing on the extracted acoustic features to obtain a first embedding vector; A second embedding determination program module is used to input the utterance frame after activity detection into a deep speaker model for representing a speaker with difficult airway or a speaker model based on a traditional machine learning method to obtain a second embedding vector representing the difficult airway; A detection program module is used to input the first embedding vector and the second embedding vector into a support vector machine to detect difficult airway.

6. The system according to claim 5, wherein: The deep speaker model is constructed by a deep neural network based on the acoustic features of the speaker, including: The residual network constructed by the two-dimensional convolutional neural network is used to collect frame-level information, and The time-delay neural network constructed by one-dimensional convolutional neural network module, residual enhanced one-dimensional convolutional neural network module, and squeeze and excitation network module performs speaker verification tasks based on channel enhanced attention, propagation and aggregation.

7. The system according to claim 6, wherein: The speaker model based on the traditional machine learning method includes: The unsupervised modeling Gaussian mixture model-general background model is used to determine the supervector of the user's speech audio to distinguish the airway type of the user.

8. The system according to claim 5, wherein: The detection program module is used for: Based on the kernel function of the support vector machine, the first embedding vector and the second embedding vector are mapped to the feature space by nonlinear mapping to estimate the hyperplane; The user's speech audio is classified by marking the separation feature space to determine a difficult airway detection result.

9. An electronic device, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method described in any one of claims 1 to 4.

10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 4 are implemented.

Images