Breathing circuit abnormity monitoring method and system based on machine vision

Abstract

The invention relates to a breathing circuit abnormity monitoring method and system based on machine vision. The method comprises the following steps: firstly, collecting a waveform image of a current frame respirator through machine vision, transmitting the waveform image to computing control equipment, performing waveform region extraction and correction on the image by the computing control equipment to obtain a processed image, classifying the processed image, and matching a classification result with a preset abnormal waveform classification library to obtain an abnormal waveform classification result; and if matching succeeds, alarming and storing abnormal data, otherwise, returning to collect the next frame of image. Compared with the prior art, the method has the advantages of universality, real-time performance, accuracy and the like.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical devices and auxiliary diagnostics, and in particular to a method and system for monitoring respiratory circuit abnormalities based on machine vision. Background Technology

[0002] A ventilator is a medical device that assists or replaces a patient's spontaneous breathing, and is widely used in the treatment of patients with impaired or failed respiratory function. It provides airflow mechanically to help patients complete gas exchange. Modern ventilators integrate waveform monitoring functions, which can display parameters such as respiratory pressure, flow rate, and volume in real time, providing dynamic data for clinical treatment and care.

[0003] In clinical applications, common abnormalities in the breathing circuit include:

[0004] Breathing circuit leakage: This causes the amount of gas inhaled or exhaled by the patient to be different from the preset value.

[0005] Accumulation of airway secretions: may cause airway obstruction and reduce ventilation efficiency.

[0006] Condensation buildup can cause abnormal airway humidity and even damage equipment.

[0007] Double triggering and ineffective triggering: Patient-ventilator dysregulation leads to poor ventilation.

[0008] Premature switching: Switching the ventilator prematurely during the respiratory cycle affects ventilation stability.

[0009] Although modern ventilators have alarm functions for abnormalities such as circuit leaks and high pressure, the setting of these alarms often relies on the experience of medical staff, is subjective, and has a high rate of false alarms, affecting clinical work efficiency. More importantly, existing alarms cannot comprehensively cover all abnormal situations, especially problems such as fluid accumulation, secretion buildup, and premature switching.

[0010] Existing methods for acquiring and analyzing ventilator waveform data mainly include:

[0011] Waveform acquisition based on data interface: Data is exported to a server through the ventilator's data output interface (such as RS232), and then the waveform is analyzed in time series or converted into images for processing. However, this method relies on the communication protocol of a specific manufacturer, lacks universality for different models and manufacturers of ventilators, and may be affected by electromagnetic interference, data loss, etc.

[0012] Post-hoc analysis of captured waveform images: Some studies use cameras to capture waveform images of ventilators, followed by image processing and analysis. However, this method is usually limited to post-hoc analysis and cannot achieve real-time monitoring.

[0013] Traditional machine learning-based waveform anomaly identification methods, such as Support Vector Machines (SVM) and Random Forests, require manual extraction of waveform features to identify anomalies like double triggering and invalid triggering. The accuracy of these methods depends heavily on feature extraction and is susceptible to gradient vanishing and overfitting issues.

[0014] Waveform recognition based on deep learning: Deep learning methods such as improved YOLOv5s, Unet, and convolutional neural networks (CNN) perform well in waveform recognition, but they are mostly based on post-captured images for analysis, lacking real-time performance and versatility.

[0015] In general, existing technologies generally struggle to simultaneously achieve device versatility, real-time anomaly warning, and comprehensive anomaly identification.

[0016] Furthermore, machine vision-based methods inevitably suffer from image tilt and perspective effects. For example, while a ventilator waveform classification and recognition method based on improved YOLO object detection (publication number CN119445547A) uses machine vision for ventilator waveform classification and recognition, it does not consider waveform image distortion. Simultaneously, since most existing technologies are based on raw data, there is a lack of research to correct waveform images, affecting the accurate extraction of waveform features. Specifically, tilt can lead to deviations in the measurement of waveform parameters such as period and amplitude (e.g., tilted waveforms may be misjudged as having increased amplitude or longer periods); perspective effects can cause local stretching or compression of waveform images (e.g., waveforms at the screen edges may be distorted), thus interfering with the recognition of key features such as waveform inflection points and slopes. These distortions directly reduce the accuracy of anomaly detection algorithms, leading to an increase in false positives or false negatives, making it difficult to meet the high-precision requirements of clinical monitoring of respiratory circuit anomalies. Summary of the Invention

[0017] The purpose of this invention is to overcome the shortcomings of the prior art by providing a machine vision-based method and system for monitoring respiratory circuit anomalies.

[0018] The objective of this invention can be achieved through the following technical solutions:

[0019] According to one aspect of the present invention, a method for monitoring respiratory circuit anomalies based on machine vision is provided, the method comprising the following steps:

[0020] S1. Acquire the waveform image of the ventilator in the current frame based on machine vision and transmit it to the computing and control device;

[0021] S2. The waveform image is processed by the computing and control equipment, specifically including waveform region extraction and waveform image correction, so as to obtain the processed waveform image;

[0022] S3. Based on the processed waveform image, perform waveform image classification to obtain the classification result of the current waveform image;

[0023] S4. Match the classification result of the current waveform image with the preset abnormal waveform classification library; if the match is successful, an abnormal waveform alarm is triggered and the abnormal waveform data is saved; otherwise, return to step S1 and continue to acquire the next frame of waveform image.

[0024] In S2, the specific steps for waveform region extraction include: based on the YOLOv5-seg model, performing instance segmentation on the acquired ventilator waveform image to locate the pixel range of the waveform display area in the image, and generating a binary mask containing the waveform region.

[0025] In S2, the specific steps for waveform image correction include:

[0026] S21. Convert the binarized mask into an external polygon;

[0027] S22. Perform polygon approximation processing on the circumscribed polygon to obtain the simplified circumscribed polygon;

[0028] S23. Convert the simplified circumscribed polygon into a binary image and fill its interior to obtain a filled binary image.

[0029] S24. Based on the filled binary image, the corner detection algorithm is used to identify the vertices of the circumscribed polygon, that is, the boundary corners of the corresponding waveform display area.

[0030] S25. Adjust the vertex order of the detected vertices to obtain the vertices after the order adjustment;

[0031] S26. Establish a perspective mapping relationship between the vertices after the order adjustment and the vertices of the preset standard waveform region shape, calculate the perspective transformation matrix, and perform perspective transformation on the original waveform image based on the perspective transformation matrix to obtain the corrected waveform region.

[0032] In S21, the specific steps for converting a binary mask into a bounding polygon include: finding the contour of the mask, using OpenCV's findContours function to extract the contour from the binary mask; the contour is a continuous boundary with the same pixel value; filtering by the contour area to select the one with the largest area as the contour that best matches the target region, and outputting this contour as the bounding polygon.

[0033] The polygon approximation process in S22 is based on the perimeter of the circumscribed polygon. The specific steps include: using a binary search method to dynamically adjust the approximation accuracy until the number of vertices is ≤8 and the area retention rate is ≥95%.

[0034] The specific steps for adjusting the vertex order in S25 include: using a straight line with an inclination angle of -45°, starting from the upper left corner of the image, translating downwards to the right, and recording the first vertex where the line intersects as the upper left vertex. Then, calculating the angle of each vertex relative to the geometric center of the polygon, and sorting them counterclockwise from the upper left vertex according to the angle.

[0035] When performing waveform image classification in S3, the YOLOv8 object detection model is used after training, lightweight processing, and format conversion.

[0036] The specific training process of the YOLOv8 object detection model includes:

[0037] S31. Collect waveform images of the ventilator under various typical operating conditions, and label each image with the waveform category and corresponding abnormal risk level; expand the sample size of preset rare abnormal waveforms through generative adversarial network; apply environmental interference to all waveform images, such as monitor screen reflection, cable obstruction or different light intensity, to reproduce real clinical environment scenarios, and finally construct a clinically oriented waveform image dataset.

[0038] S32. Train the YOLOv8 object detection model based on the waveform image dataset. Assign different weights to each training sample according to the labeled anomaly risk level, forcing the model to learn high-risk features first. During the training process, take 5 consecutive frames of waveform images as the input sequence and use the Transformer attention mechanism to capture the waveform evolution pattern between frames in order to solve the problem of progressive anomalies that are difficult to identify in single-frame images.

[0039] The lightweighting process includes: simplifying the trained YOLOv8 object detection model based on channel pruning and knowledge distillation to obtain a lightweight model;

[0040] The format conversion specifically involves exporting the lightweight model to the ONNX format.

[0041] According to another aspect of the present invention, a machine vision-based respiratory circuit anomaly monitoring system is provided. The system includes an image acquisition device and a computing control device. The computing control device includes a waveform image processing and analysis module and an early warning module, wherein:

[0042] The image acquisition device is connected to the computing and control device. The image acquisition device is used to acquire waveform images of the ventilator in real time based on machine vision and transmit them to the computing and control device.

[0043] In the computing control device, the waveform image processing and analysis module includes an image processing submodule and an image classification submodule. The image processing submodule performs waveform region extraction and waveform image correction on the real-time acquired waveform image and outputs the processed waveform image. The image classification submodule performs waveform image classification detection based on the processed waveform image and outputs the classification result of the current waveform.

[0044] In the computing control device, the early warning module receives the current waveform classification result and matches it with the preset abnormal waveform classification library; if the match is successful, an early warning is issued and the abnormal waveform data is saved.

[0045] The computing and control equipment also includes a display module; this module develops an interactive interface based on the PySide6 framework to realize waveform data visualization, operation control, and human-computer interaction functions. Its interface includes a waveform extraction and recognition result display area, an abnormal alarm information area, a recognition result record table, a parameter setting page, and a detection method setting menu, among which:

[0046] The waveform extraction and recognition result display area displays the original waveform image, the corrected waveform region, and the classification result of the current waveform in real time;

[0047] The abnormal alarm information area displays the abnormal waveform type, occurrence time and risk level in the form of hierarchical pop-up windows, and simultaneously highlights the related waveform segments.

[0048] The identification result record table records all detection results in a time series, including statistical data of normal waveforms and details of abnormal events, and supports filtering and querying by time and type;

[0049] The parameter settings page is used to set the anomaly detection confidence threshold, alarm sensitivity, image acquisition frame rate, and data storage cycle;

[0050] The detection method settings menu allows you to switch the detection method to real-time detection mode or timed sampling mode.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. This invention acquires waveform images of the ventilator in the current frame based on machine vision, and performs image segmentation, correction, and classification. Finally, it matches the abnormal waveform classification library and triggers an alarm. This eliminates the need for continuous manual observation of the ventilator waveform, automatically identifying anomalies and triggering alarms, significantly improving the real-time performance of anomaly detection and reducing the risk of missed detections due to human error. Furthermore, this invention not only directly acquires waveform images of the ventilator based on machine vision, demonstrating good versatility for different ventilator models, but also, through image segmentation and perspective transformation-based correction methods for waveform regions in complex environments, it can adapt to various shooting angles and environments, correcting distortion and tilt caused by shooting, thus improving the accuracy of anomaly identification. This method combines versatility, real-time performance, and accuracy.

[0053] 2. In this invention, by using the YOLOv5-seg model for instance segmentation to extract waveform regions, the pixel range of the waveform display area in the image can be accurately located. The generated binarized mask can effectively isolate the waveform region from background interference, providing a high-purity target region for subsequent waveform correction and classification, and improving the accuracy of subsequent processing.

[0054] 3. In this invention, waveform image correction is achieved through steps such as circumscribed polygon transformation, approximation, corner detection, and perspective transformation. This effectively eliminates waveform distortion caused by shooting angle deviation and equipment tilt, resulting in a regular geometric shape and high standardization of the corrected waveform region. This provides a consistent input format for subsequent waveform classification, significantly improves the adaptability of the classification model to different shooting scenarios, and further enhances the accuracy of subsequent abnormal waveform recognition.

[0055] 4. In this invention, a YOLOv8 model that has been trained, lightweighted, and format-converted is used for waveform classification. This ensures classification accuracy while reducing computational complexity through lightweighting. Combined with the cross-platform compatibility of the ONNX format, it can detect common waveforms in the clinical environment, including normal waveforms, waveforms with accumulated fluid or secretions, waveforms with leaks in the breathing circuit, double-triggered waveforms, invalid-triggered waveforms, and prematurely converted waveforms. In clinical applications, this can reduce the workload of medical staff, improve the ventilation effect of ventilators, and promptly alert to and resolve abnormalities in the breathing circuit. Furthermore, the model can run efficiently on resource-constrained devices such as industrial control computers, meeting the computational power requirements for real-time monitoring in clinical scenarios.

[0056] 5. In this invention, a monitoring system is constructed that integrates image acquisition equipment and computing control equipment. By combining the precise processing of the waveform image processing and analysis module with the timely response of the early warning module, fully automated monitoring of respiratory circuit abnormalities can be achieved, reducing the cost of manual intervention and improving the reliability and timeliness of clinical monitoring. Furthermore, the display module, through a structured interface layout (including functional areas for result display, alarm information, record query, and parameter settings), intuitively presents monitoring data and abnormal information, facilitating rapid access to key information by medical staff. It also supports parameter adjustment and detection mode switching, enhancing the system's user-friendliness and enabling medical staff to respond efficiently to abnormalities, thus improving the convenience of clinical operations. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of the steps of the machine vision-based respiratory circuit anomaly monitoring method in this invention;

[0058] Figure 2 This is a flowchart of waveform image correction in this invention;

[0059] Figure 3 This is a diagram of the machine vision-based respiratory circuit anomaly monitoring system architecture in this invention.

[0060] Figure 4a This is a schematic diagram of the main page of the display module in the embodiment;

[0061] Figure 4b This is a schematic diagram of the historical alarm list information in the embodiment;

[0062] Figure 4c This is a schematic diagram of the parameter setting page in the embodiment;

[0063] In the figure, 1 is the waveform image of the ventilator in the current frame; 2 is the processed waveform image; 21 is the processed volume waveform image; 22 is the processed flow rate waveform image; 3 is the alarm display area; 4 is the model and inference time display area; 5 is the zoom bar; 6 is the sidebar; 7 is the start button; 8 is the sidebar introduction; 9 is the historical alarm list; 10 is the model modification settings area; 11 is the intersection-over-union (IoU) setting area; 12 is the confidence level setting area; 13 is the shooting interval setting area; 14 is the save area; and 15 is the theme skin changing area. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0065] Example 1

[0066] In this embodiment, a machine vision-based method for monitoring respiratory circuit anomalies is employed, and the method steps are as follows: Figure 1 As shown, it specifically includes:

[0067] S1. Acquire the waveform image of the ventilator in the current frame based on machine vision and transmit it to the computing and control device;

[0068] S2. The waveform image is processed by the computing and control equipment, specifically including waveform region extraction and waveform image correction, so as to obtain the processed waveform image;

[0069] S3. Based on the processed waveform image, perform waveform image classification to obtain the classification result of the current waveform image;

[0070] S4. Match the classification result of the current waveform image with the preset abnormal waveform classification library; if the match is successful, an abnormal waveform alarm is triggered and the abnormal waveform data is saved; otherwise, return to step S1 and continue to acquire the next frame of waveform image.

[0071] In S2, the specific steps for waveform region extraction include: based on the YOLOv5-seg model, performing instance segmentation on the acquired ventilator waveform image to locate the pixel range of the waveform display area in the image, and generating a binary mask containing the waveform region.

[0072] In S2, the waveform image correction process is as follows: Figure 2 As shown, the specific steps include:

[0073] S21. Convert the binarized mask into an external polygon;

[0074] S22. Perform polygon approximation processing on the circumscribed polygon to obtain the simplified circumscribed polygon;

[0075] S23. Convert the simplified circumscribed polygon into a binary image and fill its interior to obtain a filled binary image.

[0076] S24. Based on the filled binary image, the corner detection algorithm is used to identify the vertices of the circumscribed polygon, that is, the boundary corners of the corresponding waveform display area.

[0077] S25. Adjust the vertex order of the detected vertices to obtain the vertices after the order adjustment;

[0078] S26. Establish a perspective mapping relationship between the vertices after the order adjustment and the vertices of the preset standard waveform region shape, calculate the perspective transformation matrix, and perform perspective transformation on the original waveform image based on the perspective transformation matrix to obtain the corrected waveform region.

[0079] In S21, the specific steps for converting a binary mask into a bounding polygon include: finding the contour of the mask, using OpenCV's findContours function to extract the contour from the binary mask; the contour is a continuous boundary with the same pixel value; filtering by the contour area to select the one with the largest area as the contour that best matches the target region, and outputting this contour as the bounding polygon.

[0080] The polygon approximation process in S22 is based on the perimeter of the circumscribed polygon. The specific steps include: using a binary search method to dynamically adjust the approximation accuracy until the number of vertices is ≤8 and the area retention rate is ≥95%.

[0081] The specific steps for adjusting the vertex order in S25 include: using a straight line with an inclination angle of -45°, starting from the upper left corner of the image, translating downwards to the right, and recording the first vertex where the line intersects as the upper left vertex. Then, calculating the angle of each vertex relative to the geometric center of the polygon, and sorting them counterclockwise from the upper left vertex according to the angle.

[0082] When performing waveform image classification in S3, the YOLOv8 object detection model is used after training, lightweight processing, and format conversion.

[0083] The specific training process of the YOLOv8 object detection model includes:

[0084] S31. Collect waveform images of the ventilator under various typical operating conditions, and label each image with the waveform category and corresponding abnormal risk level; expand the sample size of preset rare abnormal waveforms through generative adversarial network; apply environmental interference to all waveform images, such as monitor screen reflection, cable obstruction or different light intensity, to reproduce real clinical environment scenarios, and finally construct a clinically oriented waveform image dataset.

[0085] S32. Train the YOLOv8 object detection model based on the waveform image dataset. Assign different weights to each training sample according to the labeled anomaly risk level, forcing the model to learn high-risk features first. During the training process, take 5 consecutive frames of waveform images as the input sequence and use the Transformer attention mechanism to capture the waveform evolution pattern between frames in order to solve the problem of progressive anomalies that are difficult to identify in single-frame images.

[0086] The lightweighting process includes: simplifying the trained YOLOv8 object detection model based on channel pruning and knowledge distillation to obtain a lightweight model;

[0087] The format conversion specifically involves exporting the lightweight model to the ONNX format.

[0088] This method directly acquires waveform images of ventilators based on machine vision, and processes and analyzes the images, showing good versatility for different models of ventilators;

[0089] This method proposes a correction method for waveform region images in complex environments based on image segmentation and perspective transformation, which can adapt to various shooting angles and environments to correct distortion and tilt caused by shooting.

[0090] This method designs a hierarchical algorithm structure to process and identify waveforms separately, systematically completing the process from raw data to identification of waveform anomalies;

[0091] This method can detect common waveforms in the clinical environment, including normal waveforms, waveforms of fluid or secretion accumulation, waveforms of air leakage in the breathing circuit, waveforms of double triggering, waveforms of invalid triggering, and waveforms of premature switching. Its application in the clinical environment can reduce the workload of medical staff, improve the ventilation effect of ventilators, and promptly remind and resolve abnormalities in the breathing circuit.

[0092] This method has the following advantages:

[0093] Versatility: Compatible with different models and manufacturers of ventilators.

[0094] Real-time performance: Enables real-time monitoring of waveforms and detection of abnormalities in clinical settings.

[0095] Accuracy: It can not only accurately identify human-ventilator asynchrony (PVA), such as double triggering and invalid triggering, but also accurately identify abnormalities such as water accumulation, secretion accumulation, air leakage and premature switching.

[0096] Example 2

[0097] In this embodiment, a machine vision-based respiratory circuit anomaly monitoring system is used, and the system structure is as follows: Figure 3 As shown, it includes an image acquisition device and a computing and control device. The computing and control device includes a waveform image processing and analysis module and an early warning module, wherein:

[0098] The image acquisition device is connected to the computing and control device. The image acquisition device is used to acquire waveform images of the ventilator in real time based on machine vision and transmit them to the computing and control device.

[0099] In the computing control device, the waveform image processing and analysis module includes an image processing submodule and an image classification submodule. The image processing submodule performs waveform region extraction and waveform image correction on the real-time acquired waveform image and outputs the processed waveform image. The image classification submodule performs waveform image classification detection based on the processed waveform image and outputs the classification result of the current waveform.

[0100] In the computing control device, the early warning module receives the current waveform classification result and matches it with the preset abnormal waveform classification library; if the match is successful, an early warning is issued and the abnormal waveform data is saved.

[0101] The computing and control equipment also includes a display module; this module uses an interactive interface developed based on the PySide6 framework to realize waveform data visualization, operation control, and human-computer interaction functions, as shown in the image below. Figure 4a , 4b As shown in 4c, it includes a waveform extraction and recognition result display area, an abnormal alarm information area, a recognition result record table, a parameter setting page, and a detection method setting menu, among which:

[0102] The waveform extraction and recognition result display area displays the original waveform image, the corrected waveform region, and the classification result of the current waveform in real time;

[0103] The abnormal alarm information area displays the abnormal waveform type, occurrence time and risk level in the form of hierarchical pop-up windows, and simultaneously highlights the related waveform segments.

[0104] The identification result record table records all detection results in a time series, including statistical data of normal waveforms and details of abnormal events, and supports filtering and querying by time and type;

[0105] The parameter settings page is used to set the anomaly detection confidence threshold, alarm sensitivity, image acquisition frame rate, and data storage cycle;

[0106] The detection method settings menu allows you to switch the detection method to real-time detection mode or timed sampling mode.

[0107] Figure 4a , 4b In 4c, 1 is the waveform image of the ventilator in the current frame; 2 is the processed waveform image; 21 is the processed volume waveform image; 22 is the processed flow rate waveform image; 3 is the alarm display area; 4 is the model and inference time display area; 5 is the zoom bar; 6 is the sidebar; 7 is the start button; 8 is the sidebar introduction; 9 is the historical alarm list; 10 is the model modification settings area; 11 is the intersection-over-union (IoU) setting area; 12 is the confidence level setting area; 13 is the shooting interval setting area; 14 is the save area; and 15 is the theme skin changing area.

[0108] In this embodiment, the image acquisition device specifically employs an autofocus camera to capture waveform images under different lighting and angle conditions in the clinical environment. These images are then transmitted via wired connection to a computing and control device for subsequent processing and detection. The different lighting conditions refer to the fact that the ventilator screen in the clinical environment may appear darker, normal, or reflective due to the influence of outdoor and indoor light intensity and direction. The different angle conditions refer to the fact that, to adapt to the complex clinical environment and facilitate operation by medical staff, the image acquisition device can be positioned at different angles to the ventilator screen and can capture images from different directions. The waveform images of the ventilator in the clinical environment are captured and transmitted via wired connection to the computing and control device.

[0109] The computing and control equipment includes a waveform image processing and analysis module that undertakes data processing, control and management tasks, an interface module that receives data from the image acquisition device, and a display device module for visualization and human-computer interaction. Specifically, it can be an industrial computer or an embedded device with an external touch screen.

[0110] The waveform image processing and analysis module is deployed on the computing and control equipment. The waveform image processing and analysis module includes an image processing submodule and an image classification submodule.

[0111] The image processing submodule performs waveform region extraction and waveform image correction on the real-time acquired waveform images;

[0112] The waveform region extraction is based on image segmentation using YOLOv5-seg. Based on image segmentation, the waveform region is separated from the background to obtain a binary mask of the waveform region. Perspective transformation is then used to correct the waveform image, which can be expressed by the following formula:

[0113]

[0114] Where, x i ,y i X represents the coordinates of the vertex of the waveform region in the image. i ,Y i Let be the coordinates of the rectangular region of the transformed standard waveform, and H be the third-order perspective transformation matrix between the two surfaces, i = 1, 2, 3, 4.

[0115] Waveform image correction, such as Figure 2 As shown, it includes: converting the segmented binarized mask into a circumscribed polygon, polygon approximation, region filling, corner detection, vertex order adjustment, and perspective transformation;

[0116] The process of converting a mask to its circumscribed polygon involves finding the mask's contour using OpenCV's `findContours` function to extract the contour from the binary mask. A contour is defined as a series of consecutive boundaries with the same pixel value (255). The contour with the largest area is selected as the one that best matches the target region, ensuring that the target contour is a valid waveform region.

[0117] Polygon approximation is performed based on the perimeter of the contour. The polygon approximation is based on the perimeter of the contour and further simplifies the circumscribed polygon. The approximation accuracy is dynamically adjusted using a binary search method (epsilon) until the following conditions are met: number of vertices ≤ 8, area retention rate ≥ 95%.

[0118] Region filling involves converting the simplified bounding polygon into a binary image and then filling it.

[0119] Corner detection is based on detecting vertices of waveform regions in a filled binary image;

[0120] The method for adjusting the vertex order is to use an inclination angle of -45 degrees. 0 The straight line is translated from the top left corner of the image, and the first point where they intersect is recorded as the top left corner vertex. Then the angle of each vertex relative to the center is calculated, and the lines are sorted counterclockwise from the top left corner vertex according to the angle.

[0121] Perspective transformation is based on establishing a perspective mapping relationship between the vertices after the order adjustment and the vertices of the standard shape waveform region, calculating the perspective transformation matrix, and transforming the waveform region in the image to obtain the corrected waveform region.

[0122] In this embodiment, the image classification submodule is based on the YOLOv8 object detection model. After training, the model is pruned and knowledge distilled to reduce computational complexity while maintaining accuracy. The model is then exported in ONNX format for operation in resource-constrained environments such as industrial control computers.

[0123] The display module's interface includes a waveform extraction and recognition result display area, abnormal waveform alarm information, recognition result record table page, parameter setting page, and detection mode setting menu. Based on PySide6, the interface is loaded and displays, controls, displays page animations, and enables human-computer interaction to display waveform image processing and detection results, record and save abnormal waveform data, generate abnormal waveform alarms, adjust relevant parameters, and select detection modes.

[0124] In summary, this system, through the collaboration of image acquisition and computing control devices, combined with the precise processing of the waveform image processing and analysis module and the timely response of the early warning module, constructs a complete monitoring closed loop of "acquisition-processing-identification-alarm." Each module has clearly defined functions and works efficiently together, enabling fully automated monitoring of respiratory circuit abnormalities, reducing manual intervention costs, and improving the reliability and timeliness of clinical monitoring. Meanwhile, the display module, through a structured interface layout (including result display, alarm information, record query, parameter settings, and other functional areas), intuitively presents monitoring data and abnormal information, facilitating rapid access to key information for medical staff. It also supports parameter adjustment and detection mode switching, enhancing the system's user-friendliness and enabling medical staff to respond efficiently to abnormalities, thus improving the convenience of clinical operations.

[0125] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for monitoring respiratory circuit anomalies based on machine vision, characterized in that, The method steps include: S1. Acquire the waveform image of the ventilator in the current frame based on machine vision and transmit it to the computing and control device; S2. The waveform image is processed by the computing and control equipment, specifically including waveform region extraction and waveform image correction, so as to obtain the processed waveform image; S3. Based on the processed waveform image, perform waveform image classification to obtain the classification result of the current waveform image; S4. Match the classification result of the current waveform image with the preset abnormal waveform classification library; if the match is successful, an abnormal waveform alarm is triggered and the abnormal waveform data is saved; otherwise, return to step S1 and continue to acquire the next frame of waveform image.

2. The method for monitoring respiratory circuit anomalies based on machine vision according to claim 1, characterized in that, In step S2, the specific steps for waveform region extraction include: performing instance segmentation on the acquired ventilator waveform image based on the YOLOv5-seg model to locate the pixel range of the waveform display area in the image and generate a binary mask containing the waveform region.

3. The method for monitoring respiratory circuit anomalies based on machine vision according to claim 1, characterized in that, In step S2, the specific steps for waveform image correction include: S21. Convert the binarized mask into an external polygon; S22. Perform polygon approximation processing on the circumscribed polygon to obtain the simplified circumscribed polygon; S23. Convert the simplified circumscribed polygon into a binary image and fill its interior to obtain a filled binary image. S24. Based on the filled binary image, the corner detection algorithm is used to identify the vertices of the circumscribed polygon, that is, the boundary corners of the corresponding waveform display area. S25. Adjust the vertex order of the detected vertices to obtain the vertices after the order adjustment; S26. Establish a perspective mapping relationship between the vertices after the order adjustment and the vertices of the preset standard waveform region shape, calculate the perspective transformation matrix, and perform perspective transformation on the original waveform image based on the perspective transformation matrix to obtain the corrected waveform region.

4. The method for monitoring respiratory circuit anomalies based on machine vision according to claim 3, characterized in that, In step S21, the specific steps for converting a binary mask into a bounding polygon include: finding the contour of the mask, using the OpenCV findContours function to extract the contour from the binary mask; the contour is a continuous boundary with the same pixel value; filtering by the contour area to select the one with the largest area as the contour that best fits the target region, and outputting the contour as a bounding polygon.

5. The method for monitoring respiratory circuit anomalies based on machine vision according to claim 3, characterized in that, The polygon approximation process in S22 is based on the perimeter of the circumscribed polygon. The specific steps include: using a binary search method to dynamically adjust the approximation accuracy until the number of vertices is ≤8 and the area retention rate is ≥95%.

6. The method for monitoring respiratory circuit anomalies based on machine vision according to claim 3, characterized in that, The specific steps for adjusting the vertex order in S25 include: using a straight line with an inclination angle of -45°, starting from the upper left corner of the image, translating downwards to the right, recording the first vertex where the line intersects as the upper left corner vertex, then calculating the angle of each vertex relative to the geometric center of the polygon, and sorting them counterclockwise from the upper left corner vertex according to the angle.

7. The method for monitoring respiratory circuit anomalies based on machine vision according to claim 1, characterized in that, When performing waveform image classification in S3, a YOLOv8 object detection model that has been trained, lightweighted, and format-converted is used.

8. The method for monitoring respiratory circuit anomalies based on machine vision according to claim 7, characterized in that, The specific training process of the YOLOv8 object detection model includes: S31. Collect waveform images of the ventilator under various typical operating conditions, and label each image with the waveform category and corresponding abnormal risk level; expand the sample size of the preset rare abnormal waveforms through generative adversarial network; apply environmental interference to all waveform images to reproduce real clinical environment scenarios, and finally construct a clinically oriented waveform image dataset. S32. Train the YOLOv8 object detection model based on the waveform image dataset. Assign different weights to each training sample according to the labeled anomaly risk level, forcing the model to learn high-risk features first. During the training process, take 5 consecutive frames of waveform images as the input sequence and use the Transformer attention mechanism to capture the waveform evolution pattern between frames in order to solve the problem of progressive anomalies that are difficult to identify in single-frame images. The lightweighting process includes: simplifying the trained YOLOv8 object detection model based on channel pruning and knowledge distillation to obtain a lightweight model; The format conversion specifically refers to exporting the lightweight model to the ONNX format.

9. A machine vision-based respiratory circuit anomaly monitoring system, characterized in that, The system includes image acquisition equipment and computing and control equipment. The computing and control equipment includes a waveform image processing and analysis module and an early warning module, wherein: The image acquisition device is connected to the computing and control device. The image acquisition device is used to acquire waveform images of the ventilator in real time based on machine vision and transmit them to the computing and control device. In the computing control device, the waveform image processing and analysis module includes an image processing submodule and an image classification submodule; the image processing submodule performs waveform region extraction and waveform image correction on the real-time acquired waveform image, and outputs the processed waveform image; the image classification submodule performs waveform image classification detection based on the processed waveform image, and outputs the classification result of the current waveform; In the computing control device, the early warning module receives the current waveform classification result and matches it with the preset abnormal waveform classification library; if the match is successful, an early warning is issued and the abnormal waveform data is saved.

10. A machine vision-based respiratory circuit anomaly monitoring system according to claim 9, characterized in that, The aforementioned computing and control device also includes a display module; this module develops an interactive interface based on the PySide6 framework to realize waveform data visualization, operation control, and human-computer interaction functions. Its interface includes a waveform extraction and recognition result display area, an abnormal alarm information area, a recognition result record table, a parameter setting page, and a detection method setting menu, wherein: The waveform extraction and recognition result display area displays the original waveform image, the corrected waveform region, and the classification result of the current waveform in real time; The abnormal alarm information area displays the abnormal waveform type, occurrence time and risk level in the form of hierarchical pop-up windows, and simultaneously highlights the related waveform segments. The identification result record table records all detection results in a time series, including statistical data of normal waveforms and details of abnormal events, and supports filtering and querying by time and type; The parameter settings page is used to set the anomaly detection confidence threshold, alarm sensitivity, image acquisition frame rate, and data storage cycle; The detection method settings menu allows you to switch the detection method to real-time detection mode or timed sampling mode.

Images