Laryngeal mask ventilation monitoring method and device

Abstract

The present invention discloses a laryngeal mask ventilation monitoring method and device, comprising real-time collection of video image information during laryngeal mask ventilation; extraction of object appearance features of a single-frame image in the video image information; extraction of pixel optical flow features of multiple frames in the video image information; performing region detection and key point detection on the object appearance features, and generating predicted key point positions; performing optical flow prediction on the pixel optical flow features, and generating motion prediction information; combining and analyzing the predicted key point positions and the motion prediction information to obtain optical flow information of the key point positions: the laryngeal mask ventilation monitoring method and device of the present invention can monitor the ventilation process of the laryngeal mask during surgery in real time through visualization technology and synchronous image analysis technology, including risks such as whether the laryngeal mask is displaced and whether the respiratory rate is abnormal, and promptly remind anesthesiologists to check and deal with existing problems, ensure safety, and prevent adverse events in airway management.

Description

Technical Field

[0001] The present invention relates to the field of intelligent medical devices, and in particular to a laryngeal mask ventilation monitoring method and device. Background Art

[0002] At present, visual intubation technology has been widely used, including the insertion of endotracheal tube or laryngeal mask under direct vision of a video laryngoscope, ensuring the successful implementation of mechanical ventilation.

[0003] However, changes in the patient's body position and slight swallowing movements in the throat during surgery may cause changes in the position of the laryngeal mask, resulting in increased intragastric pressure, insufficient ventilation, and even suffocation. These problems are often difficult to detect in time through conventional ventilator parameters, posing risks to intraoperative airway management.

[0004] To ensure the stability and safety of laryngeal mask ventilation, it is necessary to continuously monitor the position and movement of the glottis using computer vision methods based on the video captured by the visual laryngeal mask. This can promptly detect changes in the mask's position or irregular breathing, trigger an alarm, and alert the doctor to conduct a prompt inspection, thereby ensuring the safety of laryngeal mask ventilation. The successful application of deep convolutional neural networks in various computer vision tasks lays a solid foundation for this patent to ensure the safety of visual monitoring of laryngeal mask ventilation technology.

[0005] In view of the problems existing in the prior art, it is of great significance to provide a laryngeal mask ventilation monitoring method and device. Summary of the Invention

[0006] In order to solve the above problems, the present invention provides a laryngeal mask ventilation monitoring method and equipment.

[0007] To achieve the above-mentioned object, a laryngeal mask ventilation monitoring method of the present invention comprises the following steps: real-time acquisition of video image information during laryngeal mask ventilation; extraction of object appearance features of a single-frame image in the video image information; extraction of pixel optical flow features of multiple frames in the video image information; performing region detection and key point detection on the object appearance features and generating predicted key point positions; performing optical flow prediction on the pixel optical flow features and generating motion prediction information; and combining and analyzing the predicted key point positions and the motion prediction information to obtain optical flow information of the key point positions.

[0008] Furthermore, after extracting the object appearance features of a single frame image in the video image information, the method further includes the steps of: extracting appearance features of target objects of different scales in the object appearance features based on the object appearance features to obtain multi-scale object appearance features;

[0009] Further, performing region detection and key point detection on the surface features of the object and generating predicted key point positions specifically comprises: performing region detection and key point detection on the surface features of the multi-scale object and generating predicted key point positions;

[0010] Furthermore, after performing region detection and key point detection on the surface features of the object and generating predicted key point positions, the method further includes the steps of: performing abnormality detection on laryngeal mask ventilation based on the predicted key point positions to determine whether abnormal displacement of the laryngeal mask occurs and whether abnormal leakage or reflux occurs;

[0011] Furthermore, after combining and analyzing the predicted key point positions and the motion prediction information to obtain the optical flow information of the key point positions, the method further includes the steps of: performing quantitative analysis on laryngeal mask ventilation based on the optical flow information of the key point positions;

[0012] The present invention also provides a laryngeal mask ventilation monitoring device, comprising an image acquisition module, an object appearance feature extraction module, a pixel optical flow feature extraction module, a detection module, an optical flow prediction module and an analysis module; the image acquisition module is used to acquire video image information during laryngeal mask ventilation in real time; the object appearance feature extraction module is used to extract object appearance features of a single frame image in the video image information in real time; the pixel optical flow feature extraction module is used to extract pixel optical flow features of multiple frames in the video image information in real time; the detection module is used to perform region detection and key point detection on the object appearance features and generate predicted key point positions; the optical flow prediction module is used to perform optical flow prediction on the pixel optical flow features and generate motion prediction information; the analysis module is used to combine and analyze the predicted key point positions and the motion prediction information to obtain optical flow information of the key point positions;

[0013] Furthermore, the object appearance feature extraction module can be further used to extract appearance features of target objects of different scales in the object appearance features based on the object appearance features, thereby obtaining multi-scale object appearance features; the detection module can also be used to perform region detection and key point detection on the multi-scale object appearance features, and generate predicted key point positions;

[0014] Furthermore, the system further includes an abnormality detection module, which is used to perform abnormality detection on the laryngeal mask ventilation according to the predicted key point positions, and determine whether the laryngeal mask has abnormal displacement and whether leakage or reflux has occurred;

[0015] Furthermore, a quantitative analysis module is included, and the quantitative analysis module is specifically used to perform quantitative analysis on laryngeal mask ventilation based on the optical flow information of the key point positions;

[0016] Furthermore, the object appearance feature extraction module specifically extracts the object appearance features of a single frame image in the video image information in real time through a convolutional neural network; and based on the object appearance features, extracts the appearance features of target objects of different scales in the object appearance features through image pyramid technology to obtain multi-scale object appearance features; the pixel optical flow feature extraction module specifically extracts the pixel optical flow features of multiple frames in the video image information in real time through a recurrent neural network.

[0017] The laryngeal mask ventilation monitoring method and device of the present invention uses visualization technology to complete the laryngeal mask placement and then continues to turn on the visualization device, continuously shoots during the operation, and synchronously analyzes the laryngeal mask displacement and laryngopharyngeal activity through video, thereby achieving real-time monitoring and management of the laryngeal mask ventilation process during the operation; the laryngeal mask ventilation monitoring method and device described in the present invention can continuously monitor the glottis position and activity with the help of computer vision methods based on the video captured by the visual laryngeal mask, promptly detect emergency situations such as changes in the position of the laryngeal mask or irregular breathing of the patient, issue an alarm, and remind the doctor to check in time, thereby ensuring the safety of laryngeal mask ventilation. The successful application of deep convolutional neural networks in various computer vision tasks has laid a solid foundation for this patent to ensure the safety of visual monitoring of laryngeal mask ventilation technology.

[0018] It also fully utilizes video information, providing qualitative and quantitative analysis of laryngeal ventilation technology from both spatial and temporal dimensions. The laryngeal mask ventilation monitoring method and device described in this invention inputs multi-dimensional information into deep convolutional neural networks with different structures to learn the surface features of objects in single-frame images and the optical flow features of pixels in multi-frame images, thereby achieving the goal of real-time positioning of the laryngeal mask and detecting the patient's respiratory status.

[0019] Conventional laryngeal mask ventilation techniques and methods in existing technologies are blind and can only be adjusted through ventilator parameters. The specific position of the laryngeal mask cannot be determined, and changes in the position of the mask during ventilation cannot be monitored. Through visualization technology and synchronized image analysis technology, the ventilation process of the laryngeal mask during surgery can be monitored in real time, including whether the laryngeal mask is displaced, whether the respiratory rate is abnormal, and whether there is a risk of air leakage or reflux. This can promptly alert the anesthesiologist to check and address any problems, ensuring the safety of mechanical ventilation and preventing adverse events during laryngeal mask ventilation management. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of the first process of the laryngeal mask ventilation monitoring method of the present invention;

[0021] Figure 2 Schematic diagram of a second process of the laryngeal mask ventilation monitoring method of the present invention;

[0022] Figure 3This is a schematic structural diagram of the laryngeal mask ventilation monitoring device of the present invention. DETAILED DESCRIPTION

[0023] The structure and working principle of the present invention will be further described below with reference to the accompanying drawings.

[0024] like Figure 1 As shown, Figure 1 FIG1 is a first flow chart of the laryngeal mask ventilation monitoring method of the present invention. The laryngeal mask ventilation monitoring method of the present invention specifically comprises the following steps:

[0025] S1 collects video image information during laryngeal mask ventilation in real time; specifically, collects continuous frame image information during laryngeal mask ventilation;

[0026] S2 extracts the object appearance features of a single frame image in the video image information; in a preferred embodiment of the present invention, specifically extracts the object appearance features of a single frame image in the video image information in real time through a convolutional neural network, such as ResNet;

[0027] S3 extracts pixel optical flow features of multiple frames of images in the video image information; in a preferred embodiment of the present invention, specifically extracts pixel optical flow features of multiple frames of images in the video image information in real time through a recurrent neural network, LSTM, etc.;

[0028] S41 performs region detection and key point detection on the surface features of the object, and generates predicted key point positions. In a preferred embodiment of the present invention, it specifically utilizes a convolutional neural network (ResNet) to decode the surface features of the object, thereby achieving region detection and key point detection of an object of interest in a single-frame image. The object of interest can be specifically set according to the needs of the operator, including large and small bubbles caused by air leakage or reflux. The key points are specifically preset definition points in the glottis area of the patient, and the preset definition points can be vocal cords, small angular nodules, and wedge-shaped nodules.

[0029] S42 performs abnormality detection on the laryngeal mask ventilation based on the predicted key point position to determine whether the laryngeal mask has abnormal displacement and whether abnormal leakage or reflux occurs. In a preferred embodiment of the present invention, the abnormality detection specifically determines whether the laryngeal mask has abnormal left-right or front-back displacement at the position of the glottis during ventilation, and whether abnormal leakage or reflux occurs, thereby ensuring the safety of laryngeal mask ventilation.

[0030] S5 performs optical flow prediction on the pixel optical flow features and generates motion prediction information. In a preferred embodiment of the present invention, based on the primary image depth features, VGG features or ResNet features, a recurrent neural network, LSTM, etc. is used to capture the correlation information between different video frames. Based on this correlation information, an optical flow network, FlowNet, etc. is used to achieve optical flow prediction between frames in the video image, that is, pixel-level motion information prediction.

[0031] S6 combines and analyzes the predicted key point positions and the motion prediction information to obtain optical flow information of the key point positions.

[0032] S7 performs a quantitative analysis of the laryngeal mask ventilation based on the optical flow information at the key point positions. In a preferred embodiment of the present invention, the quantitative analysis specifically calculates and analyzes respiratory rate based on operating information over a preset period of time, thereby improving the practicality of the laryngeal mask ventilation monitoring method and device.

[0033] like Figure 2 As shown, Figure 2 FIG2 is a second flow chart of the laryngeal mask ventilation monitoring method according to the present invention. The laryngeal mask ventilation monitoring method according to the present invention further includes the following steps:

[0034] S1 collects video image information during laryngeal mask ventilation in real time;

[0035] S21′ extracts the object appearance features of a single frame image in the video image information;

[0036] S22′ further extracts appearance features of target objects of different scales in the object appearance features based on the object appearance features to obtain multi-scale object appearance features. In a preferred embodiment of the present invention, specifically, image pyramid technology, i.e., FPN, is introduced on the basis of primary image depth features, VGG features or ResNet features, so as to extract powerful representation features of target objects of different scales in a single frame image.

[0037] S3 extracts pixel optical flow features of multiple frames of images in the video image information;

[0038] S41 ′ performs region detection and key point detection on the multi-scale object appearance features, and generates predicted key point positions.

[0039] S42′ performs abnormal detection on the laryngeal mask ventilation according to the predicted key point position, and determines whether the laryngeal mask has abnormal displacement and whether leakage or reflux occurs;

[0040] S5 performs optical flow prediction on the pixel optical flow features and generates motion prediction information;

[0041] S6 combines and analyzes the predicted key point position and the motion prediction information to obtain optical flow information of the key point position;

[0042] S7 performs quantitative analysis on the laryngeal mask ventilation based on the optical flow information of the key point positions.

[0043] like Figure 3 As shown, Figure 3 Schematic diagram of the structure of the laryngeal mask ventilation monitoring device of the present invention, which includes an image acquisition module 1, an object appearance feature extraction module 2, a pixel optical flow feature extraction module 3, a detection module 5, an optical flow prediction module 4, an analysis module 7, an anomaly detection module 6 and a quantitative analysis module 8; the image acquisition module 1 is used to collect video image information during laryngeal mask ventilation in real time;

[0044] The object appearance feature extraction module 2 is used to extract the object appearance features of the single frame image in the video image information in real time. In a preferred embodiment of the present invention, the object appearance feature extraction module 2 extracts the object appearance features of the single frame image in the video image information in real time through a convolutional neural network, namely ResNet. At the same time, it can also extract powerful representation features of target objects of different scales in the single frame image through image pyramid technology, namely FPN, based on the primary image depth features, VGG features or ResNet features, to obtain multi-scale object appearance features.

[0045] The pixel optical flow feature extraction module 3 is used to extract the pixel optical flow features of multiple frames of the video image information in real time; in a preferred embodiment of the present invention, the pixel optical flow feature extraction module 3 extracts the pixel optical flow features of multiple frames of the video image information in real time through a recurrent neural network, i.e., LSTM;

[0046] The detection module 5 is used to perform region detection and key point detection on the surface features of the object, and generate predicted key point positions; in a preferred embodiment of the present invention, the detection module 5 can also be used to perform region detection and key point detection on the surface features of the multi-scale object, and generate predicted key point positions;

[0047] The optical flow prediction module 4 is used to perform optical flow prediction on the pixel optical flow features and generate motion prediction information;

[0048] The analysis module 7 is used to combine and analyze the predicted key point position and the motion prediction information to obtain the optical flow information of the key point position; in a preferred embodiment of the present invention, the optical flow prediction module uses a recurrent neural network, LSTM, etc. to capture the correlation information between different video frames based on the primary image depth features, VGG features or ResNet features, and then uses an optical flow network, FlowNet, etc. on the basis of this correlation information to realize the optical flow prediction between frames in the video image, that is, pixel-level motion information prediction.

[0049] The anomaly detection module 6 is used to detect anomalies in laryngeal mask ventilation based on the predicted key point positions, and determine whether the laryngeal mask has abnormal displacement and whether leakage or reflux has occurred. It also includes a quantitative analysis module 8, which is specifically used to quantitatively analyze laryngeal mask ventilation based on the optical flow information of the key point positions. The object appearance feature extraction module 2 is specifically used to extract the object appearance features of a single frame image in the video image information in real time through a convolutional neural network; and based on the object appearance features, the image pyramid technology is used to extract the appearance features of target objects of different scales in the object appearance features to obtain multi-scale object appearance features. The pixel optical flow feature extraction module 3 is specifically used to extract the pixel optical flow features of multiple frames in the video image information in real time through a recurrent neural network. In the prior art, minimally invasive laparoscopic surgery often requires the patient to be in a head-down position. However, changes in body position during surgery and increased abdominal pressure when the head is in a head-down position can cause the laryngeal mask to shift and increase the risk of reflux and aspiration. In a preferred embodiment of the present invention, the laryngeal mask ventilation monitoring method and device can load an image analysis program into an existing visualization device, perform synchronous image processing, and provide real-time reminders of changes in the laryngeal mask position and other possible airway problems, thereby helping anesthesiologists to promptly detect and deal with airway problems and ensure intraoperative ventilation safety.

[0050] The above is only an illustrative description of the present invention. Those skilled in the art should know that various improvements can be made to the present invention without departing from the working principle of the present invention, which all fall within the scope of protection of the present invention.

Claims

1. A laryngeal mask ventilation monitoring method, characterized in that: The method comprises the following steps: Real-time collection of video image information during laryngeal mask ventilation; Extracting object appearance features of a single frame image in the video image information; Extracting pixel optical flow features of multiple frames of images in the video image information; Performing region detection and key point detection on the surface features of the object, and generating predicted key point positions; According to the predicted key point positions, abnormal detection is performed on the laryngeal mask ventilation to determine whether abnormal displacement of the laryngeal mask occurs and whether abnormal leakage or reflux occurs; Performing optical flow prediction on the pixel optical flow features and generating motion prediction information; Combining and analyzing the predicted key point positions and the motion prediction information to obtain optical flow information of the key point positions; The laryngeal mask ventilation is quantitatively analyzed based on the optical flow information of the key point positions.

2. The laryngeal mask ventilation monitoring method according to claim 1, wherein: After extracting the object appearance features of the single-frame image in the video image information, the method further includes the following steps: Based on the object appearance features, appearance features of target objects of different scales in the object appearance features are further extracted to obtain multi-scale object appearance features.

3. The laryngeal mask ventilation monitoring method according to claim 2, wherein: Performing region detection and key point detection on the surface features of the object and generating predicted key point positions specifically includes: performing region detection and key point detection on the surface features of the multi-scale object and generating predicted key point positions.

4. A laryngeal mask ventilation monitoring device, characterized in that: It includes image acquisition module, object appearance feature extraction module, pixel optical flow feature extraction module, detection module, anomaly detection module, optical flow prediction module and analysis module, and quantitative analysis module; The image acquisition module is used to collect video image information during laryngeal mask ventilation in real time; The object appearance feature extraction module is used to extract the object appearance features of a single frame image in the video image information in real time; The pixel optical flow feature extraction module is used to extract the pixel optical flow features of multiple frames of images in the video image information in real time; The detection module is used to perform region detection and key point detection on the surface features of the object and generate predicted key point positions; The abnormality detection module is used to perform abnormality detection on the laryngeal mask ventilation according to the predicted key point position, and determine whether the laryngeal mask has abnormal displacement and whether leakage or reflux abnormality occurs; The optical flow prediction module is used to perform optical flow prediction on the pixel optical flow features and generate motion prediction information; The analysis module is used to combine and analyze the predicted key point position and the motion prediction information to obtain optical flow information of the key point position; The quantitative analysis module is specifically used to perform quantitative analysis on laryngeal mask ventilation based on the optical flow information of the key point positions.

5. The laryngeal mask ventilation monitoring device according to claim 4, characterized in that: The object appearance feature extraction module can also be used to further extract appearance features of target objects of different scales in the object appearance features based on the object appearance features, so as to obtain multi-scale object appearance features; The detection module can also be used to perform region detection and key point detection on the multi-scale object surface features and generate predicted key point positions.

6. The laryngeal mask ventilation monitoring device according to claim 5, characterized in that: The object appearance feature extraction module specifically extracts the object appearance features of a single frame image in the video image information in real time through a convolutional neural network; and extracts appearance features of target objects of different scales in the object appearance features through image pyramid technology based on the object appearance features to obtain multi-scale object appearance features; The pixel optical flow feature extraction module specifically extracts the pixel optical flow features of multiple frames of images in the video image information in real time through a recurrent neural network.

Images