Artificial airway detection method and system based on multi-sensor fusion
Through multi-sensor fusion technology, the artificial airway is monitored in real time, and image recognition and multi-index monitoring are used to solve the problem of timely detection of catheter position changes in traditional methods, achieving more efficient catheter status monitoring and early warning.
Patent Information
- Application Number
- CN202510633918.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-16
- Publication Date
- 2025-08-19
AI Technical Summary
Traditional artificial airway detection methods are difficult to achieve real-time monitoring, especially when the patient is restless or the position of the body is moved, the changes in the catheter position are difficult to be discovered in time, resulting in lagging early warning measures.
Using multi-sensor fusion technology, through image recognition and multi-index monitoring, the detection profile of the artificial airway is extracted in real time, the parameter change rate and blood oxygen change rate are calculated, and it is mapped into a color-filled detection profile, and the preset threshold is used to initiate alarm measures.
It improves the accuracy and timeliness of artificial airway detection, reduces the work intensity of medical staff, and allows medical staff to more intuitively identify abnormal trends and catheter position changes.
Smart Images
Figure CN120501412A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical care monitoring, and in particular to an artificial airway detection method and system based on multi-sensor fusion. Background Art
[0002] Artificial airways are a critical life-support tool for maintaining ventilation in critically ill patients. Traditional fixation methods rely on physical restraints such as tape and straps, making it difficult to dynamically detect changes in the catheter's position. This poses a risk of hidden catheter displacement if the adhesive tape loses its stickiness due to patient agitation, body movement, or contamination by secretions.
[0003] Traditionally, catheter displacement status has been indirectly inferred through manual inspections by nursing staff or physiological indicators such as blood oxygen levels. This requires frequent checks by medical staff to ensure catheter fixation. However, due to workload constraints, real-time monitoring of catheter displacement is difficult, and coverage of all potential risk periods is limited. Furthermore, manual observation is limited by subjective experience, making it difficult to fully detect even minor dislocations or partial slippages, resulting in delayed early warning measures for catheter anomalies. Summary of the Invention
[0004] The purpose of the present invention is to address the technical problem in the prior art that it is difficult to meet real-time requirements through manual inspection and observation of artificial airways. The artificial airway detection method and system based on multi-sensor fusion provided in the embodiments of the present application realize a multi-dimensional detection method based on artificial airway image recognition and multi-index monitoring, which can perform real-time detection of artificial airways. The monitoring data can be displayed in real time to the monitoring terminal of the nurse station through data transmission, so that medical staff can reduce their workload and monitor the patient's artificial airway usage status in a timely manner.
[0005] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions: Artificial airway detection method based on multi-sensor fusion, including: Extracting the detection outline of the artificial airway from the patient's upper body image and displaying the detection outline in real time on the monitoring device; Acquiring a parameter sequence of the artificial airway during the displacement process and calculating the parameter change rate; acquiring a blood oxygen sequence of the patient during the displacement process and calculating the blood oxygen change rate; mapping the parameter change rate and the blood oxygen change rate to a first color, and filling the detection contour with the first color; The alarm threshold is preset. When the parameter change rate and / or blood oxygen change rate exceeds the alarm threshold, the alarm measure is activated.
[0006] As a preferred technical solution of this application, extracting the detection contour of the artificial airway includes: The upper body image is preprocessed and input into the deep learning segmentation network to extract the artificial airway area. The edge feature points are extracted from the artificial airway area through key point extraction and edge detection algorithms, and the detection contour is obtained after curve fitting of the edge feature points.
[0007] As a preferred technical solution of the present application, the method further includes: presetting an offset threshold and a first threshold; calculating an offset distance based on the coordinates of the detected contour in the adjacent image; and calculating a parameter change rate and / or a blood oxygen change rate when the offset distance is greater than the offset threshold; If the parameter change rate and / or blood oxygen change rate is less than the first threshold in a plurality of consecutive upper body images, the offset distance is used as a new offset threshold; the detected contour is used as a reference contour, and the reference contour is also displayed in real time.
[0008] As a preferred technical solution of the present application, if the parameter change rate and / or the blood oxygen change rate is greater than a first threshold in a plurality of consecutive upper body images, the offset change rate of a plurality of offset distances is calculated; Calculate a first similarity between the offset change rate and the parameter change rate; calculate a second similarity between the offset change rate and the blood oxygen change rate; preset a similarity threshold, and when both the first similarity and the second similarity are greater than the similarity threshold, input the offset change rate and the parameter change rate into a correlation function to predict the blood oxygen change rate; if the predicted blood oxygen change rate is greater than the alarm threshold, initiate an alarm measure.
[0009] As a preferred technical solution of this application, the method for constructing the correlation function includes: Based on the nonlinear dynamic regression model, the first formula is constructed with the blood oxygen change rate as the dependent variable, and the offset change rate and parameter change rate as the independent variables; the second formula is constructed with the parameter change rate as the dependent variable and the offset change rate as the independent variable; the first formula and the second formula are combined to obtain the correlation function.
[0010] As a preferred technical solution of this application, mapping the parameter change rate and the blood oxygen change rate to the first color includes: The parameter change rate and the blood oxygen change rate are normalized; a color space is preset, and the normalized result is used as the color channel of the color space; and the colors corresponding to the color channels are fused to obtain a first color.
[0011] As a preferred technical solution of the present application, a time window is preset, the offset process of the artificial airway in the time window is detected, and a first number of parameter sequences and / or blood oxygen sequences are obtained; the detected contour is divided into a first number of color blocks along the direction of the artificial airway; and the first color is cyclically filled into multiple color blocks in chronological order.
[0012] As a preferred technical solution of this application, the method of fusing the first color includes: ; Among them, C is the fused color value; T is the color value of the parameter change rate or blood oxygen change rate; w is the normalized result of the parameter change rate or blood oxygen change rate.
[0013] As the preferred technical solution of this application, the offset change rate of multiple offset distances is calculated; the RGB color space is preset, and the blood oxygen change rate, parameter change rate and offset change rate are mapped to corresponding RGB channels based on the importance, and the RGB channels are combined to obtain the first color.
[0014] This application also provides an artificial airway detection system based on multi-sensor fusion, including: An image processing module is used to extract the detection outline of the artificial airway from the image of the patient's upper body and display the detection outline in real time on the monitoring device; The parameter acquisition module is used to acquire the parameter sequence of the artificial airway during the excursion process and calculate the parameter change rate; obtain the patient's blood oxygen sequence during the excursion process and calculate the blood oxygen change rate; a color fusion module, configured to map the parameter change rate and the blood oxygen change rate into a first color and fill the detection contour with the first color; The early warning module is used to preset the alarm threshold. When the parameter change rate and / or blood oxygen change rate exceeds the alarm threshold, the alarm measures are activated.
[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. By combining a key point extraction algorithm with a traditional edge detection algorithm, the contour features of the artificial airway are extracted to form a detection outline, reducing the visual focus of the artificial airway and improving the observation of subtle deviations. Computer image recognition can detect feature changes that are difficult for the human eye to detect and issue warnings based on preset thresholds, which helps improve the accuracy and timeliness of artificial airway detection.
[0016] 2. Since the detection of artificial airway and patient blood oxygen saturation is numerical, it is difficult to intuitively represent the real-time status of the patient using the artificial airway. Therefore, in this application, the parameter change rate and the blood oxygen change rate are mapped to a first color through a preset formula, and the two colors are merged to fill the detection outline. Color changes make it easier for medical staff to quickly identify abnormal trends, so that while monitoring the position deviation of the artificial airway, medical staff can also intuitively observe whether the position deviation has an impact on the patient, or monitor whether there is an abnormality in the part of the ventilation tube inserted into the patient's trachea through changes in airway parameters or changes in the patient's blood oxygen.
[0017] 3. By taking the normalized results of the parameter change rate and the blood oxygen change rate as a channel in the color space and presetting the values of the other two channels, the first color can achieve single-channel change, and the indicator monitoring of the artificial airway can be converted to single-channel color monitoring. This helps medical staff to intuitively identify the parameter changes of the artificial airway caused by offset by detecting the color change of the contour.
[0018] 4. By mapping multiple state changes of the artificial airway to corresponding first colors and filling multiple first colors into the detection contour at the same time, the state change trajectory of the artificial airway is intuitively displayed by visualization means, allowing medical staff to quickly judge the stability and displacement trend of the catheter, reducing the inconvenience caused by numerical monitoring.
[0019] 5. By mapping the blood oxygen change rate to the R channel, the parameter change rate to the G channel, and the offset change rate to the B channel, and setting a second weight for different channels based on the importance of the three indicators, the importance of the three indicators is reflected by the color values of different channels. The color corresponding to the more important indicator accounts for a larger proportion and the color change is more obvious, reducing the monitoring burden caused by information dispersion and improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 Schematic diagram of the process of artificial airway detection method; Figure 2 This is a schematic diagram of the artificial airway detection system; Figure 3 Schematic diagram of the process of extracting detection contours from images of the patient's upper body; Figure 4 Schematic diagram of the process of fusing the first color in the HSV color space based on the blood oxygen change rate and the parameter change rate; Figure 5 The figure is a flow chart of fusing the first color in the RGB color space based on the blood oxygen change rate, parameter change rate, and offset change rate. DETAILED DESCRIPTION
[0021] To make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them.
[0022] Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely represents some embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.
[0023] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features and technical solutions therein may be combined with each other.
[0024] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.
[0025] In the description of the present invention, it should be noted that the terms "upper" and "lower" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or the orientations or positional relationships in which the inventive product is typically placed when in use, or the orientations or positional relationships commonly understood by those skilled in the art. Such terms are intended solely to facilitate the description of the present invention and simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first" and "second" and the like are used solely for distinction and should not be construed as indicating or implying relative importance.
[0026] Example 1: For example Figure 1-Figure 5 As shown, the artificial airway detection method based on multi-sensor fusion provided in this embodiment includes: Extracting the detection outline of the artificial airway from the patient's upper body image and displaying the detection outline in real time on the monitoring device; Acquiring a parameter sequence of the artificial airway during the displacement process and calculating the parameter change rate; acquiring a blood oxygen sequence of the patient during the displacement process and calculating the blood oxygen change rate; mapping the parameter change rate and the blood oxygen change rate to a first color, and filling the detection contour with the first color; The alarm threshold is preset. When the parameter change rate and / or blood oxygen change rate exceeds the alarm threshold, the alarm measure is activated.
[0027] Specifically, when the medical staff completes the installation of the artificial airway, they use a high-definition camera placed above the bed to collect images of the patient's upper body while lying in bed, including facial images, neck images, and chest images.
[0028] Furthermore, the upper body image is preprocessed, and the preprocessed image is input into the deep learning segmentation network to extract the artificial airway area. The edge feature points are extracted from the artificial airway area through the key point extraction and edge detection algorithm, and the edge feature points are curve fitted to obtain the detection contour.
[0029] Specifically, for example Figure 3As shown in the figure, during the image acquisition phase, a high-resolution camera is mounted above and facing the bed, capturing images of the patient's upper body. During the preprocessing phase, random noise introduced by sensor noise or environmental interference is removed using methods such as Gaussian filtering and median filtering, completing noise suppression on the raw upper body images. Edge sharpening is then performed to enhance edge feature information in the image, providing more distinct edge structure for feature extraction.
[0030] The preprocessed upper body image is fed into the deep learning segmentation module, which segments the artificial airway area from the original image to further reduce interference during feature extraction. This module can be constructed using a U-Net or convolutional neural network. Based on an edge segmentation algorithm, the convolutional layer extracts edge feature information, such as significant grayscale or color changes at the edge. After pre-training with historical data, these edge features are detected to accurately segment the artificial airway area from the rest of the patient's face and upper body, effectively marking the artificial airway area.
[0031] After the artificial airway area is segmented, the contour features of the artificial airway are extracted by combining a key point extraction algorithm with a traditional edge detection algorithm. The key point extraction algorithm includes SIFT, ORB or SURF algorithms, which detect key pixel points with high contrast and obvious texture features in the image of the artificial airway area. The edge detection algorithm includes the Canny algorithm or the Sobel algorithm, which can obtain continuous and coherent edge contours in the image. Since a single method may produce detection blind spots due to insufficient local features or changes in illumination, two methods are used to extract contour features in this embodiment. The key point extraction algorithm provides significant local information, and the edge detection algorithm provides the overall contour trend of the artificial airway. The two methods complement each other to improve the reliability of artificial airway contour detection.
[0032] The key pixels and edge contours are superimposed on the original image, and a coordinate system is established based on the original image. The coordinates of the key pixels and edge contours are calculated, and the key pixels and edge contours are integrated through spatial distance matching. The key pixels closest to the edge contours are screened out to ensure that the key pixels have a high correlation with the actual contour of the artificial airway. For locally missing edge information, an interpolation algorithm is used to fill it in. Then, a curve fitting algorithm is used to perform continuous curve fitting on the key pixels and edge contours to obtain the detection contour of the artificial airway. Curve fitting methods include spline curve fitting and Bezier curve fitting.
[0033] After curve fitting is complete, the resulting detection contour is transmitted in real time via a graphics processing module to the monitoring device display screens, including those in the ward and nurse station monitoring terminals. The detection contour is superimposed on an image of the patient's upper body and then displayed in real time on the monitoring device, allowing medical staff to observe abnormalities such as artificial airway displacement or detachment. Image feature extraction using artificial intelligence algorithms such as a deep learning segmentation module, key point extraction, and edge detection can capture characteristic changes that are difficult for the human eye to detect, thereby improving the accuracy and timeliness of artificial airway detection.
[0034] Methods for establishing an artificial airway include oral intubation, nasotracheal intubation, and tracheotomy intubation. The ventilation tube is fixed with medical tape, sheaths, etc. to reduce the movement of the ventilation tube. However, since the ventilation tube needs to be inserted into the patient's trachea, the insertion part of the ventilation tube may shift or twist after the patient turns over or moves his head, causing upper airway obstruction and a decrease in the patient's blood oxygen saturation. Therefore, when performing artificial airway testing, it is also necessary to combine the patient's physiological indicators with testing.
[0035] The pressure sensor, gas sensor, etc. of the monitor connected to the artificial airway collects various parameters of the artificial airway, including airway pressure, ventilation volume, and end-tidal carbon dioxide partial pressure, and forms a parameter sequence in chronological order. The difference between at least two parameters in the same parameter sequence and the ratio of the time difference are calculated to obtain the parameter change rate. The patient's blood oxygen saturation is collected through a medical monitor, and a blood oxygen sequence is formed in chronological order. The blood oxygen change rate is calculated by the difference between at least two parameters in the blood oxygen sequence and the ratio of the time difference. An alarm threshold is preset to monitor fluctuations in parameters related to the artificial airway and the patient's blood oxygen saturation. When the parameter change rate or blood oxygen change rate detected in real time exceeds the alarm threshold, the system automatically triggers alarm measures, including issuing audio, visual, and other alarm signals, and transmitting abnormal status information to medical staff in real time for timely intervention.
[0036] Since the detection of artificial airway and patient blood oxygen saturation is numerical, it is difficult to intuitively represent the real-time status of the patient using the artificial airway. Therefore, in this application, the parameter change rate and the blood oxygen change rate are mapped to a first color through a preset formula, and the two colors are merged to fill the detection outline. The color change makes it easier for medical staff to quickly identify abnormal trends, so that medical staff can monitor whether the position deviation of the artificial airway affects the patient while monitoring the position deviation, or monitor whether there is any abnormality in the part of the ventilation tube inserted into the patient's trachea through changes in airway parameters or changes in the patient's blood oxygen.
[0037] Furthermore, an offset threshold and a first threshold are preset; an offset distance is calculated based on the coordinates of the detected contour in the adjacent image; when the offset distance is greater than the offset threshold, a parameter change rate and / or a blood oxygen change rate is calculated; If the parameter change rate and / or blood oxygen change rate is less than the first threshold in a plurality of consecutive upper body images, the offset distance is used as a new offset threshold; the detected contour is used as a reference contour, and the reference contour is also displayed in real time.
[0038] Specifically, the present application calculates the offset distance between the current position of the artificial airway and the position in the previous image frame using the coordinates of the detected contour in consecutive image frames. If the offset distance is greater than an offset threshold, the patient's blood oxygen change rate and / or the artificial airway parameter change rate are further calculated.
[0039] When the artificial airway is offset, if the offset distance is less than the offset threshold, the impact of the offset on the patient can be ignored, thereby reducing the monitoring intensity of medical staff. When the offset distance is greater than the offset threshold, and the parameter change rate and / or blood oxygen change rate in multiple consecutive frames of images are less than the first threshold, it means that the offset of the catheter is relatively stable and the impact on the patient is still negligible. At this time, the offset distance can be used as a new offset threshold to optimize the detection accuracy. The artificial airway contour detected in this state is used as a reference contour and continuously compared with the detected contours at subsequent moments in real time. By dynamically adjusting the offset threshold, the sensitivity of the detection system is optimized, reducing false alarms caused by slight limb changes, coughing and other occasional movements of the patient, and improving the detection accuracy and reliability.
[0040] Further, if the parameter change rate and / or the blood oxygen change rate is greater than a first threshold in a plurality of consecutive upper body images, calculating the offset change rate of a plurality of offset distances; Calculate a first similarity between the offset change rate and the parameter change rate; calculate a second similarity between the offset change rate and the blood oxygen change rate; preset a similarity threshold, and when both the first similarity and the second similarity are greater than the similarity threshold, input the offset change rate and the parameter change rate into a correlation function to predict the blood oxygen change rate; if the predicted blood oxygen change rate is greater than the alarm threshold, initiate an alarm measure.
[0041] Specifically, when the offset distance is greater than the offset threshold, and the parameter change rate and / or blood oxygen change rate in multiple consecutive image frames is greater than a first threshold, it indicates that the catheter offset is affecting the patient. For example, excessive catheter offset can cause the catheter to bend, reducing the cross-sectional area of the ventilation channel at the bend and reducing ventilation. In this case, there is a correlation between catheter offset and blood oxygen saturation and artificial airway monitoring parameters. Furthermore, blood oxygen saturation is directly affected by the artificial airway, and the parameters and indicators of artificial airway monitoring are also correlated with blood oxygen saturation.
[0042] Therefore, in this embodiment, a first similarity between the offset change rate and the parameter change rate, and a second similarity between the offset change rate and the blood oxygen change rate are calculated; a similarity threshold is preset, and when both are higher than the similarity threshold, it is determined that the offset change rate is correlated with the parameter change rate and the blood oxygen change rate. In this embodiment, this correlation feature is captured by constructing a correlation function. The specific construction method includes: obtaining historical monitoring data that satisfies the first similarity and the second similarity that are both higher than the similarity threshold. Based on a nonlinear dynamic regression model, a first formula is constructed with the blood oxygen change rate as the dependent variable and the offset change rate and the parameter change rate as independent variables; a second formula is constructed with the parameter change rate as the dependent variable and the offset change rate as the independent variable; the first formula and the second formula are combined to obtain a correlation function. Based on the historical monitoring data, the correlation function is solved to obtain the corresponding correlation function model.
[0043] After oxygen from an artificial airway enters the patient's lungs, it must be transported to the cells through the bloodstream. Therefore, there is a lag between the blood oxygen index and the deviation and parameter changes of the artificial airway. By constructing a correlation function between the deviation change rate, parameter change rate, and blood oxygen change rate, the system can promptly predict the blood oxygen change rate after detecting artificial airway deviation. If the predicted result indicates that the blood oxygen change rate exceeds the warning threshold, the system immediately initiates an alarm, prompting medical staff to intervene promptly. This approach helps improve the sensitivity and timeliness of artificial airway detection, thereby accelerating the ability to warn of dangerous situations and reducing the risks caused by monitoring delays.
[0044] Furthermore, mapping the parameter change rate and the blood oxygen change rate to a first color includes: The parameter change rate and the blood oxygen change rate are normalized; a color space is preset, and the normalized result is used as the color channel of the color space; and the colors corresponding to the color channels are fused to obtain a first color.
[0045] Specifically, the parameter change rate and the blood oxygen change rate are normalized and converted into a numerical range [0, 1]. Multiple parameters of the same device can be normalized separately, and then the average value is calculated. In this embodiment, the color space includes HSV space or RGB space. Taking HSV space as an example, since the H channel is an angle range [0 o , 360 o ], the S channel and V channel are in the range [0, 1]. Therefore, the normalized result can be directly used as the S channel or V channel; but if the normalized result is to be used as the H channel, the normalized result and 360 need to be calculated. o The product result is used as the H channel. Preferably, for example Figure 4 As shown, in this embodiment, the normalized result is used as the S channel, the V channel is preset to the maximum brightness value of 1, and the H channel selects deep red 0 oBy dynamically changing the parameter change rate and the blood oxygen change rate, the S channel changes dynamically. The color range in the HSV space is [gray, dark red], corresponding to the S channel range of [0, 1]. Taking the RGB space as an example, the values of the R, G, and B channels are all in the range of [0, 255]. The product of the normalized result and 255 can be used as the R, G, or B channel, while presetting the values of the other two channels.
[0046] In this embodiment, the method of fusing the colors of the parameter change rate and the blood oxygen change rate into a first color includes: calculating the color values mapped by the two indicators using a preset formula to obtain a fused color value: ; Where C is the fused color value; T is the color value of the parameter change rate or blood oxygen change rate; and w is the normalized result of the parameter change rate or blood oxygen change rate. Preferably, a first weight can be assigned to the parameter change rate and blood oxygen change rate based on their importance, thereby highlighting the color of the more important indicator. For example, in this embodiment, blood oxygen saturation directly reflects the patient's blood oxygen content. Therefore, the first weight assigned to blood oxygen saturation is greater than the first weight assigned to the parameter change rate, and the sum of the first weights is set to 1, so that the color mapped to the more important indicator has a larger proportion in the first color.
[0047] By taking the normalized results of the parameter change rate and the blood oxygen change rate as a channel in the color space and presetting the values of the other two channels, the first color can achieve single-channel change, and the indicator monitoring of the artificial airway can be converted into single-channel color monitoring. This helps medical staff to intuitively identify the parameter changes of the artificial airway caused by offset by detecting the color changes of the contour.
[0048] Furthermore, a time window is preset, and the offset process of the artificial airway in the time window is detected to obtain a first number of parameter sequences and / or blood oxygen sequences; the detection contour is divided into a first number of color blocks along the direction of the artificial airway; and the first color is cyclically filled into multiple color blocks in chronological order.
[0049] Specifically, the portion of the detected contour that overlaps with the artificial airway tube is segmented, and this portion is evenly divided into a first number of color blocks along the direction of the artificial airway. With the patient's head moving toward the chest as the filling direction, the first color is filled starting from the first color block in the head direction based on the time sequence corresponding to the parameter sequence or blood oxygen sequence. If the first color block already has color, the color is moved one color block toward the chest, and the first color is then filled into the first color block, and this cycle repeats. Over time, the color blocks at earlier time points are gradually covered by the new first color, forming a color-changing trajectory that can reflect the state changes of the artificial airway during the displacement process in real time.
[0050] By mapping multiple state changes of the artificial airway into corresponding first colors and filling multiple first colors into the detection contour at the same time, the state change trajectory of the artificial airway is intuitively displayed by visualization means, allowing medical staff to quickly judge the stability and displacement trend of the catheter, reducing the inconvenience caused by numerical monitoring.
[0051] Furthermore, the offset change rates of multiple offset distances are calculated; an RGB color space is preset, and the blood oxygen change rate, parameter change rate, and offset change rate are mapped to corresponding RGB channels based on their importance, and the RGB channels are combined to obtain a first color.
[0052] Specifically, the importance of the blood oxygen change rate, parameter change rate, and offset change rate can be scored by multiple medical staff based on their actual work experience and the degree of attention paid to the indicators. The average values of the blood oxygen change rate, parameter change rate, and offset change rate are calculated respectively, and the average values are summed up. The ratio of the average value and the summation result is used as the second weight of the blood oxygen change rate, parameter change rate, and offset change rate. In this embodiment, the importance of the blood oxygen change rate is greater than that of the parameter change rate, and the importance of the parameter change rate is greater than that of the offset change rate. Therefore, preferably, for example Figure 5 As shown, in this embodiment, the blood oxygen change rate is mapped to the R channel, the parameter change rate is mapped to the G channel, and the offset change rate is mapped to the B channel. The formula for integrating the blood oxygen change rate, parameter change rate, and offset change rate into the first color is: ; Among them, C is the fused color value; R is the red channel value, G is the green channel value, and B is the blue channel value; w is the normalized result of the blood oxygen change rate, parameter change rate, or offset change rate; S is the second weight of the blood oxygen change rate, parameter change rate, or offset change rate, S1 is greater than S2, S2 is greater than S3, and the sum of S1, S2, and S3 is 1.
[0053] By mapping the blood oxygen change rate to the R channel, the parameter change rate to the G channel, and the offset change rate to the B channel, and setting a second weight for different channels based on the importance of the three indicators, the importance of the three indicators is reflected by the color values of different channels. The colors corresponding to the more important indicators account for a larger proportion and the color changes are more obvious, reducing the monitoring burden caused by information dispersion and improving work efficiency.
[0054] Embodiment 2: This embodiment provides an artificial airway detection system, device, and storage medium based on multi-sensor fusion.
[0055] For example Figure 2 As shown, the artificial airway detection system based on multi-sensor fusion includes: An image processing module is used to extract the detection outline of the artificial airway from the image of the patient's upper body and display the detection outline in real time on the monitoring device; The parameter acquisition module is used to acquire the parameter sequence of the artificial airway during the excursion process and calculate the parameter change rate; obtain the patient's blood oxygen sequence during the excursion process and calculate the blood oxygen change rate; a color fusion module, configured to map the parameter change rate and the blood oxygen change rate into a first color and fill the detection contour with the first color; The early warning module is used to preset the alarm threshold. When the parameter change rate and / or blood oxygen change rate exceeds the alarm threshold, the alarm measures are activated.
[0056] An artificial airway detection device based on multi-sensor fusion, the device comprising: sensors for collecting various data related to the artificial airway detection process; at least one processor capable of processing various data related to the artificial airway detection collected by the sensors; at least one memory for storing various data related to the artificial airway collected by the sensors and at least one program; when the at least one program is executed by the at least one processor, the at least one processor implements the artificial airway detection method based on multi-sensor fusion; the processor and memory can be connected via a bus or other means to store the at least one program; the memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include high-speed random access memory and can also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory can optionally include a memory remotely located relative to the processor, and these remote memories can be connected to the device via a network. Examples of the aforementioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. When the at least one program is executed by the at least one processor, the at least one processor implements the artificial airway detection method based on multi-sensor fusion.
[0057] A computer storage medium stores a program executable by a processor, wherein the program executable by the processor is used to implement the artificial airway detection method based on multi-sensor fusion when executed by the processor.
[0058] Those skilled in the art will appreciate that all or some of the steps and systems in the method disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those skilled in the art, the term computer storage media is included in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data) and is volatile and non-volatile, removable, and non-removable. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage, or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0059] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described in the present invention. Although this specification has described the present invention in detail with reference to the above embodiments, the present invention is not limited to the above specific implementation methods. Therefore, any modification or equivalent replacement of the present invention; and all technical solutions and improvements thereof that do not depart from the spirit and scope of the invention are included in the scope of the claims of the present invention.
Claims
1. An artificial airway detection method based on multi-sensor fusion, characterized in that: include: Extracting the detection outline of the artificial airway from the patient's upper body image and displaying the detection outline in real time on the monitoring device; Acquiring a parameter sequence of the artificial airway during the displacement process and calculating the parameter change rate; acquiring a blood oxygen sequence of the patient during the displacement process and calculating the blood oxygen change rate; mapping the parameter change rate and the blood oxygen change rate to a first color, and filling the detection contour with the first color; The alarm threshold is preset. When the parameter change rate and / or blood oxygen change rate exceeds the alarm threshold, the alarm measure is activated.
2. The artificial airway detection method based on multi-sensor fusion according to claim 1, characterized in that: Extract the detection contour of the artificial airway, including: The upper body image is preprocessed and input into the deep learning segmentation network to extract the artificial airway area. The edge feature points are extracted from the artificial airway area through key point extraction and edge detection algorithms, and the detection contour is obtained after curve fitting of the edge feature points.
3. The artificial airway detection method based on multi-sensor fusion according to claim 1, characterized in that: Also includes: Presetting an offset threshold and a first threshold; Calculating an offset distance based on the coordinates of the detected contour in the adjacent image; calculating a parameter change rate and / or a blood oxygen change rate when the offset distance is greater than an offset threshold; If the parameter change rate and / or blood oxygen change rate is less than the first threshold in a plurality of consecutive upper body images, the offset distance is used as a new offset threshold; the detected contour is used as a reference contour, and the reference contour is also displayed in real time.
4. The artificial airway detection method based on multi-sensor fusion according to claim 3, characterized in that: If the parameter change rate and / or the blood oxygen change rate is greater than a first threshold in a plurality of consecutive upper body images, calculating the offset change rate of a plurality of offset distances; Calculating a first similarity between the offset change rate and the parameter change rate; calculating a second similarity between the offset change rate and the blood oxygen change rate; A similarity threshold is preset. When both the first similarity and the second similarity are greater than the similarity threshold, the offset change rate and the parameter change rate are input into the correlation function to predict the blood oxygen change rate; if the predicted blood oxygen change rate is greater than the alarm threshold, the alarm measure is initiated.
5. The artificial airway detection method based on multi-sensor fusion according to claim 4, characterized in that: The methods for constructing the correlation function include: Based on the nonlinear dynamic regression model, the first formula is constructed with the blood oxygen change rate as the dependent variable, and the offset change rate and parameter change rate as the independent variables; the second formula is constructed with the parameter change rate as the dependent variable and the offset change rate as the independent variable; the first formula and the second formula are combined to obtain the correlation function.
6. The artificial airway detection method based on multi-sensor fusion according to claim 1, characterized in that: Mapping the parameter change rate and blood oxygen change rate to the first color includes: The parameter change rate and the blood oxygen change rate are normalized; a color space is preset, and the normalized result is used as the color channel of the color space; and the colors corresponding to the color channels are fused to obtain a first color.
7. The artificial airway detection method based on multi-sensor fusion according to claim 6, characterized in that: Preset a time window, detect the deviation process of the artificial airway in the time window, and obtain a first number of a parameter sequence and / or a blood oxygen sequence; Dividing the detection contour into a first number of color blocks along the artificial airway direction; The first color is cyclically filled into the plurality of color blocks in chronological order.
8. The artificial airway detection method based on multi-sensor fusion according to any one of claims 6 to 7, characterized in that: Methods for blending the first color include: ； Among them, C is the fused color value; T is the color value of the parameter change rate or blood oxygen change rate; w is the normalized result of the parameter change rate or blood oxygen change rate.
9. The artificial airway detection method based on multi-sensor fusion according to any one of claims 6 to 7, characterized in that: Calculate the rate of change of offset for multiple offset distances; The RGB color space is preset, and the blood oxygen change rate, parameter change rate, and offset change rate are mapped to corresponding RGB channels based on their importance. The first color is obtained by combining the RGB channels.
10. The artificial airway detection system based on multi-sensor fusion is characterized by: include: An image processing module is used to extract the detection outline of the artificial airway from the image of the patient's upper body and display the detection outline in real time on the monitoring device; The parameter acquisition module is used to acquire the parameter sequence of the artificial airway during the excursion process and calculate the parameter change rate; obtain the patient's blood oxygen sequence during the excursion process and calculate the blood oxygen change rate; a color fusion module, configured to map the parameter change rate and the blood oxygen change rate into a first color and fill the detection contour with the first color; The early warning module is used to preset the alarm threshold. When the parameter change rate and / or blood oxygen change rate exceeds the alarm threshold, the alarm measures are activated.