Laryngeal mask displacement monitoring method and system

Abstract

The application discloses a laryngeal mask displacement monitoring method, comprising the following steps: S1, continuously collecting video images of a patient's airway during laryngeal mask catheterization, and extracting deep semantic features of each frame of image; S2, taking a current frame at the time when the laryngeal mask is placed in place as a standard image and extracting deep semantic features thereof; constructing a laryngeal mask displacement image feature space; S3, calculating the feature distance between the deep semantic features of the standard image and each feature vector in the laryngeal mask displacement image feature space, and calculating a laryngeal mask displacement judgment threshold adaptive to the patient; S4, calculating the feature distance between the deep semantic features of each frame of image after the laryngeal mask is placed in place and the deep semantic features of the standard image, and outputting an alarm information of laryngeal mask displacement when the feature distance is greater than the laryngeal mask displacement judgment threshold. The application can automatically identify the laryngeal mask displacement state and issue an alarm when an abnormality is detected, thereby improving the active safety of airway management and reducing the incidence of related complications.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and specifically to a method and system for monitoring laryngeal mask displacement. Background Technology

[0002] A video laryngeal mask airway (LMA) is a crucial airway management device in thoracic lung surgery, used to provide safe and effective respiratory support to patients during general anesthesia. Compared to a traditional LMA, it integrates a high-definition camera, enabling visualization of the patient's internal airway structure during insertion and use, significantly improving the safety and convenience of airway management.

[0003] However, several risks remain in the clinical application of video laryngeal masks. On the one hand, changes in patient position, head and neck movements, or abnormal pressure in the laryngeal mask cuff during surgery may cause the mask to shift or even partially dislodge, increasing the risk of hypoxia and respiratory failure. On the other hand, if secretions such as sputum in the patient's airway are not suctioned and cleared in time, they may obstruct the airway or increase the probability of reflux, aspiration, and bacterial pneumonia. Although current video laryngeal masks can provide real-time video images to help doctors observe the position of the mask and the condition of secretions, laryngeal mask shift is a high-risk but low-incidence event, and the shift process is sudden and brief. In actual clinical applications, anesthesiologists need to monitor multiple vital signs and depth of anesthesia simultaneously, making it difficult to continuously and attentively observe the laryngeal mask video.

[0004] Traditional methods typically detect anomalies based on preset key point locations, and are highly sensitive to individual differences in patient physiology, catheter placement angle, and image quality (such as blurriness or abnormal lighting). Figure 2 The images show laryngeal cavities of four patients after laryngeal mask airway insertion. It is assumed that three corner points near the patients' vocal cords are defined as preset key points, such as... Figure 2 Points A, B, and C in the image of patient 1 are shown. However, clinical images of the other three patients reveal that the visibility and stability of key points cannot be guaranteed under different clinical scenarios. For example, in patient 2, the vocal cords, located in the dark, could not be observed due to the insertion angle and poor lighting conditions in the laryngeal cavity; in patient 3, vocal cord closure caused variations in the appearance and structure of key points; and in patient 4, thickening of the laryngeal cavity resulted in the absence of vocal key points, making them impossible to observe. As can be seen from the above description, existing technologies relying on preset key points are easily interfered with in complex scenarios, resulting in insufficient monitoring reliability.

[0005] Therefore, due to the influence of many factors such as individual differences in patients' physiological structure, deviations in the selection of laryngeal mask size, and differences in doctors' insertion techniques, the images of different patients after the laryngeal mask is properly inserted show significant differences, lacking unified characteristics and judgment standards, which increases the identification burden for inexperienced clinicians. Summary of the Invention

[0006] In order to overcome the problems in the prior art, the present invention aims to provide a laryngeal mask displacement monitoring method and system that can automatically identify the laryngeal mask displacement status and issue an alarm when an abnormality is detected, thereby improving the active safety of airway management and reducing the incidence of related complications.

[0007] To achieve the above objectives, the present invention provides a method for monitoring laryngeal mask displacement, characterized by comprising the following steps:

[0008] S1. During the laryngeal mask airway insertion process, continuously acquire video images of the patient's airway at a preset frame rate, extract the deep semantic features of each frame image and cache them until the laryngeal mask is inserted in place.

[0009] S2: Extract the current frame at the moment the laryngeal mask is inserted as the standard image and extract its deep semantic features. At the same time, based on the cached deep semantic features of the entire insertion process, construct the laryngeal mask displacement image feature space corresponding to the patient's own physiological structure.

[0010] S3: Calculate the feature distance between the deep semantic features of the standard image and each feature vector in the feature space of the laryngeal mask displacement image, and calculate the laryngeal mask displacement judgment threshold adapted to the patient based on all feature distances;

[0011] S4: Continuously acquire images after the laryngeal mask is inserted and extract its depth semantic features. Calculate the feature distance between the depth semantic features of each frame of the image after the laryngeal mask is inserted and the depth semantic features of the standard image. When the feature distance is greater than the laryngeal mask displacement judgment threshold, output the laryngeal mask displacement alarm information.

[0012] Furthermore, step S1 also includes a tube placement initiation determination step:

[0013] After extracting deep semantic features from each frame of the captured image, the deep semantic features are input into the image classification model, and the output is a probability vector of the image belonging to a preset category. The category index corresponding to the maximum probability is then taken.

[0014] When the category index is equal to the preset in vitro category index, it is determined that the tube placement process has not started and no feature caching is performed; when the category index is greater than the in vitro category index, it is determined that the tube placement process has started and the depth semantic features of each frame image are continuously cached until a confirmation signal indicating that the tube placement is in place is received.

[0015] Furthermore, the image classification model is composed of fully connected layers and softmax layers connected sequentially. The calculation process of the probability vector output by the deep semantic features through the image classification model is as follows:

[0016] ;

[0017] Where y is a probability vector of length equal to the total number of categories C, denoted as And the sum of the probabilities of all categories is 1. The input is a deep semantic feature vector. This is the weight matrix of the fully connected layer; For bias terms of fully connected layers;

[0018] The formula for calculating the category index of an image is:

[0019] ;

[0020] in, The category index corresponding to the image, with a value ranging from 0 to... Integer; the first element Indicates that the input image belongs to the first... The probability of a class.

[0021] Furthermore, the total number of the categories Among them, category index 0 corresponds to in vitro images, and category indices 1 to 6 correspond to oral cavity, epiglottis, epiglottis, glottis, arytenoid cartilage, and sputum, respectively.

[0022] Furthermore, in step S2, the laryngeal mask displacement image feature space is constructed by keyframe extraction, specifically including: inputting a feature set of all cached deep semantic features of the insertion process into the keyframe extraction model, and outputting the importance score corresponding to the deep semantic features of each frame in the set; filtering out deep semantic features with an importance score greater than a preset score threshold, and constructing the laryngeal mask displacement image feature space by all the filtered deep semantic features.

[0023] Furthermore, the keyframe extraction model adopts the CSTA model, whose backbone network is a convolutional neural network.

[0024] Furthermore, the deep semantic features are extracted using a convolutional neural network, which comprises multiple cascaded network layers. Each network layer consists of a convolutional layer, a normalization layer, and an activation function connected sequentially. For the i-th network layer, the input feature matrix is... This layer outputs a feature matrix. The calculation process is as follows:

[0025] ;

[0026] in, For the first Convolution operations in network layers;

[0027] For the first Batch normalization operations in network layers;

[0028] For the first Activation functions of network layers;

[0029] No. Output feature matrix of network layer As the first Input feature matrix of network layer The output of the last layer of the convolutional neural network is the deep semantic feature vector.

[0030] Furthermore, the formula for calculating the feature distance is:

[0031] ;

[0032] in, Representing the deep semantic features of a standard image With deep semantic feature vectors The feature distance between them, with a value range of 0 to 1; Representing the deep semantic features of a standard image With deep semantic feature vectors dot product operation; Representing the deep semantic features of a standard image The length of the mold, Represents deep semantic feature vectors The length of the module.

[0033] Furthermore, in step S3, the calculation formula for the laryngeal mask displacement judgment threshold is as follows:

[0034] ;

[0035] Where T is the adaptively obtained laryngeal mask displacement judgment threshold; The mean of the feature distances between the standard image depth semantic features and all feature vectors in the feature space of the laryngeal mask displacement image; The standard deviation of the feature distance between the deep semantic features of the standard image and all feature vectors in the feature space of the laryngeal mask shift image is given. This is a preset scaling factor.

[0036] This application also provides a laryngeal mask displacement monitoring system, characterized in that it includes an image acquisition module, a depth feature extraction module, an image classification module, a keyframe extraction module, a feature distance calculation module, and an interaction and output module;

[0037] The image acquisition module is used to acquire and output video images of the patient's airway at a preset frame rate.

[0038] The deep feature extraction module is used to receive the input image, extract and output the deep semantic features corresponding to the image;

[0039] The image classification module is used to receive the deep semantic features output by the deep feature extraction module, perform category determination on the input image and output the category index to determine whether the tube placement process has started.

[0040] The keyframe extraction module is used to receive all the deep semantic features cached during the tube placement process to construct a laryngeal mask displacement image feature space that adapts to the patient's own physiological structure.

[0041] The feature distance calculation module is used to calculate the feature distance between two deep semantic feature vectors and to determine the threshold.

[0042] The interaction and output module is used to output video images acquired in real time by the image acquisition module, provide a confirmation button for the laryngeal mask placement, and, during the real-time monitoring phase, when the feature distance between the depth semantic features of each frame of the image after the laryngeal mask placement and the depth semantic features of the standard image is greater than the laryngeal mask displacement judgment threshold, output alarm information for laryngeal mask displacement.

[0043] This invention uses images of the patient's own laryngeal cavity collected during the laryngeal mask airway (LMA) insertion process to simulate and construct an image feature space of LMA displacement that reflects the individual physiological characteristics of the patient. Based on this, the judgment threshold of LMA displacement is dynamically calculated, and the parameters of the LMA displacement monitoring method are set for each patient in a personalized manner, eliminating the influence of individual physiological differences of patients and the operational variables of doctors. At the same time, the current frame at the moment of LMA insertion is captured as the standard image and its deep semantic features are extracted, eliminating the need for a large number of real displacement annotation samples, effectively improving the accuracy, flexibility and adaptability of the monitoring method. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the laryngeal mask displacement monitoring method of the present invention;

[0045] Figure 2 This is a comparison of laryngeal cavity images of different patients after the laryngeal mask airway has been properly inserted in a complex clinical setting in the background art.

[0046] Figure 3 Compare the feature distance histograms between the standard image, the original tube placement image before keyframe extraction, and the keyframe image after keyframe extraction;

[0047] Figure 4 The study analyzed minute changes in the patient's laryngeal cavity images and the distance histogram of depth semantic features under stable laryngeal mask position.

[0048] Figure 5 These are the results of clinical trials of the monitoring method described in this invention;

[0049] Figure 6This is a schematic diagram of the structure of a laryngeal mask displacement monitoring system according to the present invention. Detailed Implementation

[0050] Example 1

[0051] This embodiment provides a method for monitoring laryngeal mask airway displacement, such as Figure 1 As shown; S1: During laryngeal mask airway placement, video images of the patient's airway are continuously acquired at a preset frame rate, and the deep semantic features of each frame are extracted and cached.

[0052] Specifically, after the system is powered on, the monitoring program automatically starts and continuously acquires the current frame image data at a certain frame rate (e.g., 30 frames per second, or 30fps). The image data This is an RGB three-channel color image, where each pixel has a value ranging from 0 to 255, and the corresponding matrix form is... The acquired image data is displayed on the terminal in real time and input into the depth feature extraction module for feature calculation.

[0053] The deep feature extraction module can be constructed using classic convolutional neural networks such as ResNet, EfficientNet, and MobileNetV2. In this embodiment, the lightweight MobileNetV2 convolutional neural network is used. The network consists of multiple cascaded network layers, each consisting of a convolutional layer (Conv), a normalization layer (BN), and an activation function (Act). For the first... There are 1 network layer, and the input features are denoted as . The first Output feature matrix of each network layer The calculation process is expressed by the formula as follows:

[0054] ;

[0055] in For the first Depth-separable convolution operations are performed on layers. For the first Batch normalization operations of layers, For the first The activation function of the layer.

[0056] No. Output feature matrix of network layer Will be directly used as the first Input feature matrix of network layer ,when When = 1, the input to the first layer of the convolutional neural network is the original image data IM; after multiple convolution operations, the output length of the last layer of the network is . The row vectors are the deep semantic features corresponding to that frame image. The deep semantic features It can represent high-level semantic information of the image as a whole.

[0057] The extracted deep semantic features A synchronous input image classification module is used to determine whether the tube placement process has officially begun. This image classification module consists of a fully connected layer and a Softmax layer connected sequentially, and incorporates deep semantic features. First, the data passes through a fully connected layer, then is input into a Softmax layer, and finally outputs a probability vector. The calculation process is expressed by the formula as follows:

[0058] ;

[0059] in, This is the weight matrix of the fully connected layer. For bias terms of fully connected layers; For a length equal to the total number of categories The probability vector, denoted as The total number of categories in this embodiment Therefore, the output probability vector y is denoted as All elements take values ​​between 0 and 1, and the sum of the probabilities of all categories is 1.

[0060] The seven categories preset in this embodiment are as follows: Category index 0 corresponds to "in vitro images," and category indices 1 to 6 correspond to six in vivo images: "oral cavity," "valle epiglottis," "epiglottis," "glottis," "arytenoid cartilage," and "sputum," respectively. These categories are defined using the formula... Calculate the category index corresponding to the current image. ,when When =0, the current image is determined to be an external image, and the laryngeal mask airway insertion process has not yet started; the system continues to acquire images, extract features, and classify them. When the value is >0, it is determined that the laryngeal mask airway has been inserted into the patient's body, and the insertion process officially begins. The system then analyzes the deep semantic features corresponding to each frame of the image. The deep semantic feature set of the images in the tube placement process is cached sequentially. In, the set for OK A two-dimensional matrix of columns, where The number of cached features increases continuously during the insertion process. The feature caching process will continue until the doctor confirms that the laryngeal mask airway is in place and clicks the confirmation button on the display screen, indicating that the laryngeal mask airway is in place. After the system receives the confirmation signal, the image acquisition and feature caching phases of the insertion process terminate.

[0061] Upon receiving the confirmation signal from the doctor that the catheter has been placed in place, proceed to step S2.

[0062] Step S2: After receiving the confirmation signal that the laryngeal mask is in place, the current frame is captured as the standard image and its deep semantic features are extracted. Based on the cached deep semantic features of the entire insertion process, a laryngeal mask displacement image feature space corresponding to the patient's own physiological structure is constructed.

[0063] Specifically, the image taken at the moment the laryngeal mask airway is fully inserted is used as the standard image for laryngeal mask airway insertion. , standard image The input depth feature extraction module extracts the depth semantic features of the standard image through the same convolutional neural network operation process as in step S1. The deep semantic features of the standard image The data is cached and used as a comparison benchmark in the real-time monitoring stage of subsequent step S3.

[0064] After the placement process is completed, the cached set of deep semantic features of the entire placement process will be stored. The input is a keyframe extraction module. The construction method of the keyframe extraction module and the calculation process of the deep semantic feature set within this module are similar to the CSTA model in the paper "CSTA: CNN-based Spatiotemporal Attention for Video Summarization". The backbone network of the CSTA model can be constructed using classic convolutional neural networks such as ResNet, EfficientNet, and MobileNetV2, which is existing technology. The backbone network of the model in this application's keyframe extraction module has been pre-trained on the ImageNet public dataset and fine-tuned using clinical laryngeal mask airway insertion video data, which will not be elaborated further here. After receiving the deep semantic feature set F, the keyframe extraction module scores the importance of each deep semantic feature in the set and outputs an importance score vector of length M. , where the m-th element Corresponding set The m-th deep semantic feature Importance score The value ranges from 0 to 1. The higher the score, the stronger the content difference and representativeness of the frame feature.

[0065] After obtaining the importance scores of all features, the scores are then determined according to the preset importance score threshold. In this embodiment, features are filtered. The value is set to 0.45, and can be adjusted between 0.35 and 0.7 according to clinical needs. Deep semantic features with an importance score greater than 0.45 are selected from the original set to form an important deep semantic feature set. The important deep semantic feature set This refers to the laryngeal mask displacement image feature space adapted to the patient's own physiological structure, and the set of important deep semantic features. for OK A two-dimensional matrix of columns, where .

[0066] These important deep semantic features cover all the anatomical structures that the laryngeal mask airway passes through during its entire journey from the oral cavity to the glottis. They can not only fully simulate the image semantic features when the laryngeal mask airway is displaced in different directions and to different degrees, but also naturally carry the individual differences in the physiological structure of the patient's own laryngeal cavity. At the same time, they eliminate the homogeneous data redundancy caused by the pauses and repeated operations during the insertion process, making the feature distribution in the feature space more uniform. This provides an unbiased data basis for the subsequent displacement judgment threshold calculation, avoiding inaccurate threshold calculation due to uneven data distribution.

[0067] Figure 3 Histogram statistics of the distribution of deep semantic features in images of laryngeal mask airway insertion procedures in different patients before and after keyframe extraction are presented. Figure 3 As can be seen, after keyframe extraction, the distribution histogram of the depth semantic features of the keyframe image is more uniform, indicating that keyframe extraction significantly eliminates data distribution bias, ensuring that the judgment threshold calculated subsequently is more reasonable, improving the adaptive ability of the judgment threshold to individual differences of patients, and improving the accuracy of the monitoring method.

[0068] S3: Calculate the feature distance between the deep semantic features of the standard image and each feature vector in the feature space of the laryngeal mask displacement image, and calculate the laryngeal mask displacement judgment threshold adapted to the patient based on the statistics of all feature distances.

[0069] After completing the extraction of deep semantic features from standard images and the construction of the feature space for laryngeal mask displacement images, the deep semantic features of standard images are calculated one by one. With feature space The feature distance between the deep semantic features of each keyframe is calculated using the following formula: ,in Represents two deep semantic feature vectors and The distance between them The value range is 0 to 1; This represents the dot product operation of two vectors. and Do not represent vectors and The modulus length, × represents the multiplication operation; the feature distance Essentially, it is a feature difference measure based on cosine similarity transformation. The larger the value, the greater the difference in semantic content between the two images, and the higher the degree of laryngeal mask position shift.

[0070] After calculating all feature distances, calculate the mean of all distance values. with standard deviation Then through the formula The final laryngeal mask displacement judgment threshold is calculated. ,in As a preset scaling factor, in this embodiment Set to 0.5, the value can be adjusted within the range of 0.2 to 0.6 according to clinical sensitivity requirements.

[0071] The threshold for judging laryngeal mask displacement It is calculated entirely based on the image features of the patient's own airway during the current intubation process. It can accurately adapt to the differences in the anatomical structure of the patient's laryngeal cavity, avoid the problem of misjudgment and missed judgment between different patients with fixed thresholds, and at the same time, it does not need to rely on a large amount of labeled displacement sample data to complete the model training, which greatly reduces the data labeling cost.

[0072] S4: Continuously acquire images after the laryngeal mask is inserted and extract its depth semantic features. Calculate the feature distance between the depth semantic features of each frame of the image after the laryngeal mask is inserted and the depth semantic features of the standard image. When the feature distance is greater than the laryngeal mask displacement judgment threshold, output the laryngeal mask displacement alarm information.

[0073] Specifically, the threshold is determined. After the calculation is completed, it automatically enters the real-time monitoring phase, which continues to run throughout the entire surgical anesthesia period. During the real-time monitoring phase, the image acquisition module continues to acquire images of the laryngeal mask airway in place at a frame rate of 30fps. Each captured image is input into the depth feature extraction module to obtain the depth semantic features of the image after the laryngeal mask is in place. The calculation method is exactly the same as step S1 above; then, the feature distance calculation formula is exactly the same as that used in the feature distance calculation module in step S3 to calculate the depth semantic features of the image after the laryngeal mask is placed in place. Compared with standard deep semantic features Distance between , obtained The value is also within the 0-1 range. The value obtained in real-time calculation... Compared with the pre-calculated laryngeal mask displacement judgment threshold When comparing, < When the current laryngeal mask is determined to be in a normal position and has not shifted, the next frame image is acquired and the above calculation is repeated; when > When the system detects that the laryngeal mask airway has shifted, it immediately issues an alarm. Specifically, the terminal's built-in buzzer emits a "beep beep" sound, and a pop-up window displaying the text "Laryngeal mask airway shift" appears on the screen, reminding medical staff to check and adjust the laryngeal mask airway position in a timely manner.

[0074] Furthermore, the significant changes in visual field caused by substantial retention of secretions in the airway also lead to significant alterations in the overall semantic features of the image, causing the feature distance to exceed a threshold. Therefore, this method can simultaneously monitor and alarm for secretion retention without requiring an additional detection module, further enhancing the safety of clinical airway management. This embodiment possesses excellent edge deployment capabilities and clinical applicability.

[0075] This invention uses a deep feature extraction module to extract deep semantic features from images to reflect the overall semantic information of the image. These features have spatial redundancy characteristics, and their global statistical distribution remains stable even if the image has noise, changes in lighting, or changes in local details. Figure 4 This paper presents standard images of four patients after laryngeal mask airway (LMA) insertion, as described in the background section, and various possible intraoperative image changes under a stable state where the LMA has not shifted. It also displays a histogram of the depth semantic feature distance between the current frame image and the standard image with the LMA in place, under the stable state where the LMA has not shifted. It can be seen that even without LMA displacement, the patient's laryngeal cavity image still exhibits some degree of tissue deformation due to respiratory movements and vocal cord closure. In addition, the image shows minor changes such as illumination fluctuations and bubbling of secretions. However, the depth semantic feature distance between the current frame image and the standard image consistently remains within a small range of 0.02 to 0.06. This indicates that depth semantic features focus more on the overall high-level semantic information of the image rather than changes in local image details and are less sensitive to local interference. Therefore, using overall image depth semantic features instead of local keypoint detection significantly improves the method's anti-interference capability, ensures the stability of the monitoring method in complex clinical scenarios, and reduces the risk of false alarms.

[0076] Figure 5The figures show some monitoring results of the method described in this invention during actual clinical trials. The figures illustrate a standard image of the laryngeal mask airway (LMA) placement as set by the physician, as well as instances of LMA displacement or secretion retention detected by the monitoring method. As can be seen from the figures, the monitoring system and method of this invention can respond promptly to LMA displacement or secretion retention, outputting alarm information to remind the physician to handle the situation. Furthermore, it can achieve accurate and effective monitoring even in clinical settings with multiple patients exhibiting significant differences in physiological structure, demonstrating good adaptability to individual patient differences.

[0077] Existing technologies monitor laryngeal mask movement by sequentially executing multiple modules, including feature extraction, region detection, and key point detection. This results in long computational paths, high latency, difficulty in achieving real-time response, and a tendency to introduce cumulative errors. This invention simplifies the computational process, optimizing the real-time monitoring phase into a concise "feature extraction - feature difference calculation" model, significantly shortening the computational path and reducing latency and errors.

[0078] Table 1 shows the test results of the monitoring system and method described in this invention on the Rockchip RK3568 edge device development board. The NPU on this development board was used for model inference calculations, and the NPU computing power was 1 TFLOPs. The table summarizes the memory usage, computational load, and inference time of the model used in the real-time monitoring phase.

[0079]

[0080] As shown in Table 1, on the resource-constrained lightweight edge device RK3568, the monitoring method's model has a memory footprint of only 4.56MB, a computational load of 1.305 GFLOPs, and an inference time of only 77ms. The monitoring algorithm can detect abnormalities within 0.1s of laryngeal mask displacement or secretion retention, outputting audible and text alarm information, meeting the requirements for real-time intraoperative monitoring, and demonstrating feasibility for deployment on resource-constrained devices.

[0081] In summary, this invention outperforms existing technologies in terms of adaptability to individual patient differences, data dependence and annotation costs, method resistance to interference, and computational efficiency, providing a more economical and reliable solution for monitoring laryngeal mask displacement and secretion retention.

[0082] Example 2

[0083] This embodiment provides a laryngeal mask displacement monitoring system, including an image acquisition module, a depth feature extraction module, an image classification module, a keyframe extraction module, a feature distance calculation module, and an interaction and output module.

[0084] The image acquisition module is used to acquire and output video images of the patient's airway at a preset frame rate.

[0085] The deep feature extraction module is used to receive the input image, extract and output the deep semantic features corresponding to the image.

[0086] The image classification module is used to receive the deep semantic features output by the deep feature extraction module, perform category determination on the input image and output the category index to determine whether the tube placement process has started.

[0087] The keyframe extraction module is used to receive all the deep semantic features cached during the tube placement process to construct a laryngeal mask displacement image feature space that adapts to the patient's own physiological structure.

[0088] The feature distance calculation module is used to calculate the feature distance between two deep semantic feature vectors and to determine the threshold.

[0089] The interaction and output module is used to output video images acquired in real time by the image acquisition module, provide a confirmation button for the laryngeal mask placement, and, during the real-time monitoring phase, when the feature distance between the depth semantic features of each frame of the image after the laryngeal mask placement and the depth semantic features of the standard image is greater than the laryngeal mask displacement judgment threshold, output alarm information for laryngeal mask displacement.

[0090] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for monitoring laryngeal mask airway displacement, characterized in that, Includes the following steps: S1: During the laryngeal mask airway insertion process, video images of the patient's airway are continuously acquired at a preset frame rate, and the deep semantic features of each frame image are extracted and cached until the laryngeal mask is in place. S2: Extract the current frame at the moment the laryngeal mask is inserted as the standard image and extract its deep semantic features. At the same time, based on the cached deep semantic features of the entire insertion process, construct the laryngeal mask displacement image feature space corresponding to the patient's own physiological structure. S3: Calculate the feature distance between the deep semantic features of the standard image and each feature vector in the feature space of the laryngeal mask displacement image, and calculate the laryngeal mask displacement judgment threshold adapted to the patient based on all feature distances; S4: Continuously acquire images after the laryngeal mask is inserted and extract its depth semantic features. Calculate the feature distance between the depth semantic features of each frame of the image after the laryngeal mask is inserted and the depth semantic features of the standard image. When the feature distance is greater than the laryngeal mask displacement judgment threshold, output the laryngeal mask displacement alarm information.

2. The laryngeal mask displacement monitoring method according to claim 1, characterized in that, Step S1 further includes a tube placement initiation determination step: After extracting deep semantic features from each frame of the captured image, the deep semantic features are input into the image classification model, and the output is a probability vector of the image belonging to a preset category. The category index corresponding to the maximum probability is then taken. When the category index equals the preset in vitro category index, it is determined that the tube placement process has not started and feature caching is not performed; When the category index is greater than the in vitro category index, the tube placement process is determined to begin, and the deep semantic features of each frame image are continuously cached until a confirmation signal indicating that the tube placement is in place is received.

3. The laryngeal mask displacement monitoring method according to claim 2, characterized in that, The image classification model consists of fully connected layers and softmax layers connected sequentially. The calculation process of the probability vector output by the deep semantic features through the image classification model is as follows: ； Where y is a probability vector of length equal to the total number of categories C, denoted as And the sum of the probabilities of all categories is 1. The input is a deep semantic feature vector. This is the weight matrix of the fully connected layer; For bias terms of fully connected layers; The formula for calculating the category index of an image is: ； in, The category index corresponding to the image, with a value ranging from 0 to... Integer; the first element Indicates that the input image belongs to the first... The probability of a class.

4. The laryngeal mask displacement monitoring method according to claim 3, characterized in that, The total number of categories Among them, category index 0 corresponds to in vitro images, and category indices 1 to 6 correspond to oral cavity, epiglottis, epiglottis, glottis, arytenoid cartilage, and sputum, respectively.

5. The laryngeal mask displacement monitoring method according to claim 1, characterized in that, In step S2, the laryngeal mask displacement image feature space is constructed by keyframe extraction. Specifically, this includes: inputting a feature set of all cached deep semantic features of the insertion process into the keyframe extraction model, and outputting the importance score corresponding to the deep semantic features of each frame in the set; filtering out deep semantic features with an importance score greater than a preset score threshold, and constructing the laryngeal mask displacement image feature space from all the filtered deep semantic features.

6. The laryngeal mask displacement monitoring method according to claim 5, characterized in that, The keyframe extraction model adopts the CSTA model, whose backbone network is a convolutional neural network.

7. The laryngeal mask displacement monitoring method according to claim 1, characterized in that, The deep semantic features are extracted using a convolutional neural network, which contains multiple cascaded network layers. Each network layer consists of a convolutional layer, a normalization layer, and an activation function connected sequentially. For the i-th network layer, the input feature matrix is... This layer outputs a feature matrix. The calculation process is as follows: ； in, For the first Convolution operations in network layers; For the first Batch normalization operations in network layers; For the first Activation functions of network layers; No. Output feature matrix of network layer As the first Input feature matrix of network layer The output of the last layer of the convolutional neural network is the deep semantic feature vector.

8. The laryngeal mask displacement monitoring method according to claim 1, characterized in that, The formula for calculating the feature distance is: ； in, Representing the deep semantic features of a standard image With deep semantic feature vectors The feature distance between them, with a value range of 0 to 1; Representing the deep semantic features of a standard image With deep semantic feature vectors dot product operation; Representing the deep semantic features of a standard image The length of the mold, Represents deep semantic feature vectors The length of the module.

9. The laryngeal mask displacement monitoring method according to claim 1, characterized in that, In step S3, the formula for calculating the laryngeal mask displacement judgment threshold is: ； Where T is the adaptively obtained laryngeal mask displacement judgment threshold; The mean of the feature distances between the standard image depth semantic features and all feature vectors in the feature space of the laryngeal mask displacement image; The standard deviation of the feature distance between the deep semantic features of the standard image and all feature vectors in the feature space of the laryngeal mask shift image is given. This is a preset scaling factor.

10. A laryngeal mask displacement monitoring system, characterized in that, It includes an image acquisition module, a depth feature extraction module, an image classification module, a keyframe extraction module, a feature distance calculation module, and an interaction and output module; The image acquisition module is used to acquire and output video images of the patient's airway at a preset frame rate; The deep feature extraction module is used to receive the input image, extract and output the deep semantic features corresponding to the image; The image classification module is used to receive the deep semantic features output by the deep feature extraction module, perform category determination on the input image and output the category index to determine whether the tube placement process has started. The keyframe extraction module is used to receive all the deep semantic features cached during the tube placement process to construct a laryngeal mask displacement image feature space that adapts to the patient's own physiological structure. The feature distance calculation module is used to calculate the feature distance between two deep semantic feature vectors and to determine the threshold. The interaction and output module is used to output video images acquired in real time by the image acquisition module, provide a confirmation button for the laryngeal mask placement, and, during the real-time monitoring phase, when the feature distance between the depth semantic features of each frame of the image after the laryngeal mask placement and the depth semantic features of the standard image is greater than the laryngeal mask displacement judgment threshold, output alarm information for laryngeal mask displacement.

Images