Respiration monitoring method, device, equipment and storage medium based on machine vision

Abstract

The present application relates to the field of machine vision technology, and discloses a respiratory monitoring method, device, equipment and storage medium based on machine vision. The method comprises: collecting respiratory motion image data through an image sensor; performing motion recognition and segment selection to obtain a motion state image sequence and a rest state image sequence; calculating an initial ROI area image; performing image preprocessing to obtain a target ROI area image, and performing time-varying feature extraction to obtain a time-varying feature set; performing respiratory rate analysis to obtain a respiratory rate distribution curve; performing respiratory rate abnormality detection to obtain a respiratory rate abnormality detection result; matching a respiratory rate abnormality processing strategy from a respiratory rate abnormality processing list through a respiratory rate abnormality detection result to obtain a target respiratory rate abnormality processing strategy and performing a respiratory monitoring alarm. The present application uses machine vision technology to improve the accuracy of respiratory monitoring.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a machine vision-based respiratory monitoring method, device, equipment and storage medium. Background Art

[0002] As one of the basic parameters for measuring human vital signs, respiratory rate monitoring is of great significance for early diagnosis of respiratory diseases, heart diseases, and sleep disorders. However, traditional respiratory monitoring methods, such as chest straps, masks, or other sensors that directly contact the skin, often cause discomfort to patients, affecting compliance with long-term monitoring. In certain environments, such as intensive care units, these devices can interfere with patients' rest or conflict with other medical equipment.

[0003] In addition, many current respiratory monitoring systems often require professional operation and monitoring, which limits their application in home and telemedicine environments. The complexity and cost of these systems have also become obstacles to promotion and popularization. More importantly, traditional methods often have difficulty in monitoring multiple patients at the same time, and lack intelligence and automation in data processing and abnormal alarms, which greatly increases the workload of medical professionals. Summary of the invention

[0004] The present application provides a machine vision-based respiratory monitoring method, apparatus, device and storage medium, which are used to improve the accuracy of respiratory monitoring by using machine vision technology.

[0005] In a first aspect, the present application provides a respiratory monitoring method based on machine vision, and the respiratory monitoring method based on machine vision comprises:

[0006] The target user's respiratory motion image is collected by a preset image sensor to obtain respiratory motion image data;

[0007] Performing motion recognition and segment selection on the target user through the respiratory motion image data to obtain a motion state image sequence and a rest state image sequence of the target user;

[0008] Calculate the respiratory motion ROI region of the target user based on the motion state image sequence and the rest state image sequence to obtain an initial ROI region image of the target user;

[0009] Performing image preprocessing on the initial ROI area image of the target user to obtain a target ROI area image of the target user, and performing time-varying feature extraction on the target ROI area image to obtain a time-varying feature set of the target ROI area image;

[0010] Performing a breathing rate analysis on the target user using the time-varying feature set to obtain a breathing rate distribution curve;

[0011] Based on the respiratory rate distribution curve, performing respiratory rate abnormality detection on the target user to obtain a respiratory rate abnormality detection result of the target user;

[0012] The abnormal breathing rate detection result is used to match the abnormal breathing rate processing strategy from a preset abnormal breathing rate processing list to obtain a target abnormal breathing rate processing strategy, and the target user is subjected to a respiratory monitoring alarm through the target abnormal breathing rate processing strategy.

[0013] In a second aspect, the present application provides a respiratory monitoring device based on machine vision, and the respiratory monitoring device based on machine vision includes:

[0014] The acquisition module is used to acquire the respiratory motion image of the target user through a preset image sensor to obtain the respiratory motion image data;

[0015] A recognition module, used for performing motion recognition and segment selection on the target user through the respiratory motion image data, and obtaining a motion state image sequence and a rest state image sequence of the target user;

[0016] A calculation module, configured to calculate a respiratory motion ROI region of the target user based on the motion state image sequence and the rest state image sequence to obtain an initial ROI region image of the target user;

[0017] An extraction module is used to perform image preprocessing on the initial ROI area image of the target user to obtain a target ROI area image of the target user, and perform time-varying feature extraction on the target ROI area image to obtain a time-varying feature set of the target ROI area image;

[0018] An analysis module, configured to analyze the breathing rate of the target user using the time-varying feature set to obtain a breathing rate distribution curve;

[0019] A detection module, configured to perform abnormal breathing rate detection on the target user based on the breathing rate distribution curve, and obtain an abnormal breathing rate detection result of the target user;

[0020] A matching module is used to match the abnormal breathing rate detection result with a preset abnormal breathing rate processing list to obtain a target abnormal breathing rate processing strategy and to perform respiratory monitoring and alarm on the target user through the target abnormal breathing rate processing strategy.

[0021] The third aspect of the present application provides a machine vision-based respiratory monitoring device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the machine vision-based respiratory monitoring device performs the above-mentioned machine vision-based respiratory monitoring method.

[0022] The fourth aspect of the present application provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned machine vision-based respiratory monitoring method.

[0023] In the technical solution provided by the present application, the respiratory motion image data is collected by an image sensor, and there is no need to directly contact the user. Therefore, it is a non-invasive monitoring method that will not cause discomfort to the user. Through motion recognition and segment selection, the method can automatically distinguish the user's motion state and rest state, without the user manually inputting information, to achieve automated monitoring. Through the Kalman filter parameter matching of the motion state image and the rest state image, the accurate calculation of the respiratory motion ROI area is achieved, and the accuracy of monitoring is improved. Time-varying feature extraction of the target ROI area image helps to capture the dynamic changes of respiratory motion, further improving the accuracy and sensitivity of monitoring. Through respiratory rate distribution curve analysis and anomaly detection, the user's respiratory rate can be monitored in real time, abnormal conditions can be identified in time, and early warnings can be provided. By matching the preset respiratory rate abnormality handling strategy, the method can provide personalized handling suggestions according to the situation of different users, increasing the practicality and applicability of monitoring. Generate abnormal respiratory rate alarm information, which can immediately notify the monitoring terminal, so that the monitoring personnel can take timely action, and improve the real-time nature of monitoring. Therefore, the present application adopts machine vision technology to improve the accuracy of respiratory monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0025] Figure 1 A schematic diagram of an embodiment of a respiratory monitoring method based on machine vision in an embodiment of the present application;

[0026] Figure 2 Schematic diagram of an embodiment of a respiratory monitoring device based on machine vision in an embodiment of the present application. DETAILED DESCRIPTION

[0027] Embodiments of the present application provide a method, device, equipment and storage medium for respiratory monitoring based on machine vision. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In the embodiment of the present application, an embodiment of the respiratory monitoring method based on machine vision includes:

[0029] Step S101, collecting respiratory motion images of a target user through a preset image sensor to obtain respiratory motion image data;

[0030] It is understandable that the execution subject of the present application can be a machine vision-based respiratory monitoring device, or a terminal or a server, which is not specifically limited here. The present application embodiment is described by taking the server as the execution subject as an example.

[0031] Specifically, select a suitable image sensor, usually a high-resolution camera to capture subtle breathing movements. When selecting a camera, consider its capture speed and resolution to ensure that every detail of the breathing process can be clearly and continuously recorded. Calibrate the camera to adapt to different lighting conditions and background environments. Use specific image processing algorithms, such as white balance adjustment and contrast enhancement, to obtain high-quality images in various environments. In order to improve the accuracy of image data, dynamic range adjustment is used to ensure that the image can still clearly show breathing movements when overexposed or underexposed. In the actual image acquisition process, write or configure special software to control the camera's shooting parameters, such as exposure time, frame rate, and focal length.

[0032] Step S102, performing motion recognition and segment selection on the target user through the breathing motion image data, and obtaining a motion state image sequence and a rest state image sequence of the target user;

[0033] Specifically, the target user in the respiratory motion image data is identified and information is annotated. The user position and posture are identified in the image data through target detection algorithms such as convolutional neural networks (CNN). These algorithms can process a large amount of image data and accurately annotate the outline and key features of the target user under various background and lighting conditions, thereby generating object annotated image data. Subsequently, the object annotated image data is classified into object action states, and different action states of the user, such as motion state and rest state, are distinguished through machine learning models such as support vector machines (SVM) or deep learning classification networks. These classification models can identify slight changes in human posture in the image by learning a large amount of labeled training data, and judge its action state based on this, and generate object action state classification results. Based on the action state classification results, the state change trend of the object annotated image data is analyzed. The user's action trend is analyzed by tracking the state changes in continuous image frames. For example, by comparing the changes in the user's posture in continuous frames, the law and frequency of respiratory movement are analyzed. Algorithms such as dynamic time warping (DTW) are used to evaluate and quantify the trend of state changes, and multiple state change trends are obtained therefrom. By calculating the quantitative indicators of state changes in each trend, such as change speed, amplitude, etc., the change amplitude of each state change trend is extracted. The degree of change of the action state is quantified for subsequent analysis and processing. According to the parameter change amplitude corresponding to each state change trend, the object annotated image data is segmented. According to the size and characteristics of the change amplitude, the image sequences that best represent the user's motion state and rest state are selected. For example, when the change amplitude is large, it means that the user is doing more intense exercise, while a small change amplitude means that the user is in a resting state. Finally, the target user's motion state image sequence and rest state image sequence are obtained.

[0034] Step S103, calculating the respiratory motion ROI region of the target user based on the motion state image sequence and the rest state image sequence to obtain an initial ROI region image of the target user;

[0035] Specifically, Kalman filter parameters are matched on the motion state image sequence and the rest state image sequence to accurately predict and track the user's respiratory movement in the image sequence. Kalman filter is an effective prediction and correction algorithm that can provide the best estimation results in the presence of noise. By applying Kalman filter technology, dynamic change information related to respiratory movement is extracted from image data, thereby obtaining a set of Kalman filter parameter sets. Dynamic ROI tracking is performed on the Kalman filter parameter set to continuously monitor and update the region of interest in the image to ensure that these regions are always focused on the user's respiratory movement. The implementation of dynamic ROI tracking usually relies on image processing technology and motion tracking algorithms, which can identify and track specific parts of the user's body, such as the chest and abdomen, in continuous image frames. The movement changes of these parts are directly related to breathing. The position and range of respiratory movement are continuously tracked to obtain the dynamic ROI tracking results of the target user. The pixel change rate of the motion state image sequence and the rest state image sequence is calculated based on the dynamic ROI tracking results. The changes of pixels in the ROI area over time are analyzed, and these changes are related to the user's breathing depth, frequency, etc. By calculating the pixel change rate, the characteristics of respiratory movement are captured and quantified. Based on the pixel change rate, ROI data is extracted from the motion state image sequence and the rest state image sequence. The image area that best represents the respiratory movement is extracted from the entire image sequence, that is, the initial ROI area image. These image areas contain key information about the user's respiratory movement, such as the amplitude, frequency, and rhythm of the movement.

[0036] Step S104: performing image preprocessing on the initial ROI area image of the target user to obtain a target ROI area image of the target user, and performing time-varying feature extraction on the target ROI area image to obtain a time-varying feature set of the target ROI area image;

[0037] Specifically, the initial ROI area image is denoised and contrast enhanced to improve image quality and reduce errors in subsequent processing. Image denoising is used to remove random noise in the image, which is caused by environmental factors or imperfections of the sensor itself. Denoising can use a variety of techniques, such as median filtering, Gaussian filtering, etc., which can effectively reduce noise and maintain the main features of the image. Contrast enhancement is to improve the visibility of respiratory movement in the image, making the details of respiratory movement more obvious. This can usually be achieved by adjusting the brightness and contrast parameters of the image, or using more complex techniques such as histogram equalization. Thus, a clear and contrast-enhanced target ROI area image is obtained. Subsequently, the mean and variance of the pixel intensity of the target ROI area image are calculated to quantify the pixel intensity changes in the image. The mean and variance are commonly used metrics in statistics, providing useful information about the distribution of image pixel intensities. The mean reflects the average level of image pixel intensities, while the variance reflects the degree of change of these intensity values. The calculation results are mapped to time-varying features to convert the statistical characteristics of pixel intensities into time-varying features reflecting the characteristics of respiratory movement. Time-varying feature mapping involves combining statistical parameters of pixel intensity with time series analysis to capture the time-varying characteristics of respiratory motion. The signal in the time domain is converted to the frequency domain through Fourier transform or wavelet transform, thereby revealing the dynamic characteristics of the signal. The initial feature set is subjected to feature weight analysis and feature normalization to improve the effectiveness and consistency of each feature in the feature set. Feature weight analysis is to identify which features are more important for representing respiratory motion, while feature normalization is to eliminate the dimensional effects between different features and ensure that they are compared on the same scale. Finally, a time-varying feature set is obtained.

[0038] Step S105, analyzing the breathing rate of the target user through the time-varying feature set to obtain a breathing rate distribution curve;

[0039] Specifically, the target user's breathing signal is reconstructed through a time-varying feature set, and a signal reflecting the real breathing pattern is recovered from the extracted features. By comprehensively using time series analysis and signal processing methods, the time-varying features are converted into time series data that can characterize the breathing cycle and intensity, and a more intuitive and analyzable breathing signal is extracted from the original image data to form a target breathing signal set. Subsequently, the target breathing signal set is subjected to frequency domain conversion to better analyze the frequency characteristics of the breathing signal. Frequency domain conversion usually uses methods such as Fourier transform or wavelet transform to convert the time series signal into frequency space and reveal the frequency distribution characteristics in the signal. By performing frequency domain analysis on the breathing signal, different frequency components are identified, which helps to analyze the user's breathing pattern and frequency. The breathing rate component is extracted from the breathing frequency domain data set, and the frequency components related to the breathing rate are identified from the frequency domain data. The frequency range related to normal breathing is extracted using frequency filters or spectrum analysis techniques. The signal components related to breathing are separated, and other non-breathing related noise or interference are excluded to obtain a more pure and representative breathing rate component data set. The breathing rate component data set is subjected to breathing rate analysis to calculate the specific breathing frequency and its change characteristics. Statistical methods and algorithms are used to analyze the time series characteristics of the breathing rate components, such as the average breathing rate, the variability of the breathing rate, etc. These calculation results provide quantitative information about the breathing pattern of the target user. The breathing rate calculation results are converted into curves to generate a breathing rate distribution curve. The numerical data of the breathing rate is converted into a visual curve chart, so that the changing trend and pattern of the breathing rate can be intuitively displayed.

[0040] Step S106: Based on the breathing rate distribution curve, perform breathing rate abnormality detection on the target user to obtain a breathing rate abnormality detection result of the target user;

[0041] Specifically, the curve feature points of the respiratory rate distribution curve are extracted, and key feature points are identified from the respiratory rate distribution curve. These feature points reflect important changes in the breathing pattern, such as acceleration, deceleration or periodic fluctuations of breathing. For example, a differential algorithm or edge detection technology is used to identify key turning points or extreme points on the curve. The curve curvature is calculated for each curve feature point to quantify the degree of curvature of the curve at each feature point. The curvature is calculated using the first-order derivative and the second-order derivative of the curve to obtain the curvature data of each feature point. The calculation of the curvature data not only reflects the speed of the respiratory rate change, but also reveals the sudden change or irregularity of the breathing pattern. The curvature data of each curve feature point is used to identify the curve change feature, and the abnormal breathing pattern is identified from the curvature data. By analyzing the curvature change of the curve, the irregularity of the respiratory rate change, such as too fast or too slow breathing frequency, and the incoherence or discontinuity of breathing, is identified. Curve change feature recognition usually requires a combination of statistical analysis and pattern recognition technology, such as using threshold judgment or machine learning algorithms to identify abnormal changes compared with normal breathing patterns. The curve change characteristics of each curve feature point are tested for respiratory rate anomalies, and the previous analysis results are combined to determine whether there is a respiratory abnormality. Respiratory rate anomaly detection involves comparing the curve change characteristics with the preset normal breathing range to determine whether the breathing pattern exceeds the normal physiological range. For example, the abnormality of the respiratory rate is determined based on statistical analysis of historical data or a predictive model based on machine learning. Through comprehensive analysis and calculation, the target user's respiratory rate anomaly detection result is finally obtained. This result can not only indicate whether there is a respiratory abnormality, but also provide important information about the type and severity of the abnormality.

[0042] Step S107, matching the abnormal breathing rate detection result with the abnormal breathing rate processing strategy from the preset abnormal breathing rate processing list, obtaining the target abnormal breathing rate processing strategy, and performing respiratory monitoring and alarming on the target user through the target abnormal breathing rate processing strategy.

[0043] Specifically, the abnormal respiratory rate detection results are matched from the preset abnormal respiratory rate processing list to ensure that the appropriate processing strategy can be selected according to the specific abnormality type and severity. The abnormal respiratory rate processing list contains a series of predefined processing strategies, each of which is associated with a specific abnormal respiratory type and condition. Through the intelligent matching algorithm, the abnormal detection results are analyzed, and the most suitable strategy for the current situation is selected from the processing list, such as reminding the user to relax when the respiratory rate is detected to be too fast, or sending an emergency medical assistance request when apnea is found. Subsequently, based on the selected target abnormal respiratory rate processing strategy, abnormal respiratory rate alarm information is generated. Including the preparation of specific alarm information content, such as abnormality type, recommended treatment measures, urgency, etc. The preset monitoring terminal is used to monitor the target user's respiratory alarm information according to the abnormal respiratory rate alarm information. The monitoring terminal can be in various forms, such as mobile applications, smart watches, bedside monitoring devices, etc. In the process of alarm transmission, ensure the accurate transmission and immediate response of information, especially in emergency situations, where rapid response is crucial to the safety of users. In addition, the monitoring terminal should also have the ability to link with medical service providers or emergency response systems so that further medical intervention measures can be quickly initiated when necessary.

[0044] In the embodiment of the present application, the respiratory motion image data is collected by the image sensor, and there is no need to directly contact the user, so it is a non-invasive monitoring method and will not cause discomfort to the user. Through motion recognition and segment selection, the method can automatically distinguish the user's motion state and rest state, without the user manually inputting information, and realize automatic monitoring. Through the Kalman filter parameter matching of the motion state image and the rest state image, the accurate calculation of the respiratory motion ROI area is realized, and the accuracy of monitoring is improved. Time-varying feature extraction of the target ROI area image helps to capture the dynamic changes of respiratory motion, further improving the accuracy and sensitivity of monitoring. Through respiratory rate distribution curve analysis and abnormality detection, the user's respiratory rate can be monitored in real time, abnormal conditions can be identified in time, and early warnings can be provided. By matching the preset respiratory rate abnormality handling strategy, the method can provide personalized processing suggestions according to the situation of different users, increasing the practicality and applicability of monitoring. Generate respiratory rate abnormality alarm information, which can immediately notify the monitoring terminal, so that the monitoring personnel can take action in time, and improve the real-time nature of monitoring. Therefore, the present application adopts machine vision technology to improve the accuracy of respiratory monitoring.

[0045] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0046] (1) Performing object recognition and information labeling on the target user in the respiratory motion image data to obtain object labeling image data;

[0047] (2) classifying the object action state of the object labeled image data to obtain the object action state classification result;

[0048] (3) Based on the object action state classification results, the state change trend of the object annotated image data is analyzed to obtain multiple state change trends;

[0049] (4) Extract the change amplitude of each state change trend respectively to obtain the state change amplitude corresponding to each state change trend;

[0050] (5) Based on the parameter change amplitude corresponding to each state change trend, segment selection is performed on the object annotated image data to obtain the target user's motion state image sequence and rest state image sequence.

[0051] Specifically, the target user in the respiratory motion image data is identified and information is annotated, and computer vision technology, such as convolutional neural network (CNN) in deep learning, is used to accurately locate and identify the user in the image. The network learns to recognize the shape and features of the human body in the image through training, and then identifies the position and posture of the target user in the respiratory motion image data to achieve accurate object annotation. For example, key areas such as the user's head, chest and abdomen are identified and annotated in the image data. The object action state classification is performed on the object annotated image data to distinguish the different motion states of the user. The user's posture changes are analyzed to determine whether he is in motion or rest. Classification models of machine learning or deep learning, such as support vector machines (SVM) or recurrent neural networks (RNN), are used to classify according to different characteristics of the user's posture. For example, if the user's chest and abdomen areas show a large range of motion in consecutive image frames, the system will classify it as a motion state; on the contrary, if the movement amplitude of these areas is small, the system will classify it as a resting state. Subsequently, based on the object action state classification results, the state change trend analysis of the object annotated image data is performed. The trend of the user's action state over time is analyzed and understood, so as to capture the characteristics of the respiratory motion. Use time series analysis methods, such as dynamic time warping (DTW) or long short-term memory network (LSTM), to track the changes in user status and analyze its changing trends. For example, the system will find that the user's breathing movement gradually increases over a period of time and then gradually weakens. This periodic change trend can be identified and recorded. Further, by calculating the pixel changes in the annotated area in the image sequence, the change amplitude of each state change trend is extracted to quantify the degree of change in the user's action state. For example, the pixel changes in the user's chest area in consecutive frames are measured, and the size of these changes can be used as the change amplitude of the breathing movement. By analyzing the parameter change amplitude corresponding to each state change trend, segment selection is performed to obtain the target user's motion state image sequence and rest state image sequence. Select the image segment that best represents the user's breathing movement according to the size of the change amplitude. For example, select those images that show a large chest movement amplitude as representatives of the motion state, and select those images with almost no chest movement as representatives of the rest state.

[0052] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0053] (1) Perform Kalman filter parameter matching on the motion state image sequence and the rest state image sequence to obtain the corresponding Kalman filter parameter set;

[0054] (2) Dynamically track the Kalman filter parameter set to obtain the dynamic ROI tracking result of the target user;

[0055] (3) Calculate the pixel change rate of the moving state image sequence and the rest state image sequence based on the dynamic ROI tracking results;

[0056] (4) Based on the pixel change rate, ROI data is extracted from the motion state image sequence and the rest state image sequence to obtain the initial ROI area image of the target user.

[0057] Specifically, Kalman filtering is an optimization algorithm that is widely used in the fields of signal processing and data fusion, and is suitable for system state estimation with noise and uncertainty. The Kalman filter is used to process the motion state and rest state image sequences to estimate and track the respiratory motion of the target user. Kalman filter parameter matching involves adjusting the filter parameters, such as process noise covariance and measurement noise covariance, according to the characteristics of the image sequence, such as motion speed and direction, to more accurately predict and track the user's motion state, thereby generating a set of Kalman filter parameter sets optimized for the characteristics of respiratory motion. Dynamic ROI tracking is performed on the Kalman filter parameter set to continuously track the user's respiratory motion, especially focusing on areas closely related to breathing, such as the chest and abdomen. Dynamic ROI tracking adapts to changes in user motion by continuously updating Kalman filter parameters, thereby accurately locating the region of interest in each frame of the image. For example, when the user takes a deep breath, the movement amplitude of the chest and abdomen will increase, and the Kalman filter will adjust its parameters to track this change to ensure that the ROI is always focused on the correct position. Thus, dynamic ROI tracking results in a series of consecutive frames are obtained, which reflect the real-time state of the user's respiratory motion. Based on the dynamic ROI tracking results, the pixel change rate of the motion state image sequence and the rest state image sequence is calculated. The pixel change rate is calculated by comparing the changes in pixel intensity of the ROI area in consecutive frames. For example, the breathing depth and speed are determined by measuring the changes in pixel intensity in the chest area during breathing. Based on the pixel change rate, ROI data is extracted for the motion state and rest state image sequences. The area that best represents the user's breathing movement characteristics is extracted from the entire image sequence. For example, those frames that show significant chest and abdominal movements are selected as representatives of the motion state, while those frames with smaller movements are selected as representatives of the rest state. In this way, the most critical information, that is, the initial ROI area image of the target user, is extracted from a large amount of image data.

[0058] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0059] (1) Perform image denoising and contrast enhancement on the initial ROI area image of the target user to obtain the target ROI area image of the target user;

[0060] (2) Calculate the mean and variance of the pixel intensity of the target ROI area image to obtain the target calculation result;

[0061] (3) Perform time-varying feature mapping on the target calculation results to obtain the initial feature set;

[0062] (4) Perform feature weight analysis and feature normalization on the initial feature set to obtain the time-varying feature set of the target ROI area image.

[0063] Specifically, the image of the initial ROI region is subjected to image denoising and contrast enhancement to improve image quality and ensure the accuracy of subsequent analysis. Image denoising reduces or eliminates random noise in the image, which originates from environmental interference or sensor imperfections during image acquisition. Denoising techniques such as Gaussian filtering and median filtering can effectively remove noise while maintaining the main features of the image. Contrast enhancement is to improve the visual effect of the image and make the area related to respiratory movement more clear and identifiable. Methods such as histogram equalization can enhance the dynamic range of the image and make the details of the ROI region more prominent. The mean and variance of the pixel intensity of the target ROI region image are calculated to quantify the statistical characteristics in the image. The mean provides the average level of the image pixel intensity, which helps to understand the overall brightness of the ROI region; the variance reflects the distribution range of the pixel intensity and can characterize the texture characteristics of the image. For example, when analyzing chest movement, the calculation of the mean and variance can reveal the changes in the brightness and texture of the chest cavity with respiratory movement, which helps to identify different breathing patterns. The calculation results are subjected to time-varying feature mapping to convert the statistical characteristics of the pixel intensity into time series data reflecting the characteristics of respiratory movement. Through Fourier transform or wavelet transform, statistical data is converted into features that can characterize the changes of the image over time. For example, by Fourier transforming the brightness changes in the chest area, frequency domain features reflecting the breathing frequency and rhythm are obtained. The initial feature set is subjected to feature weight analysis and feature normalization. Feature weight analysis is to determine which features are more critical for respiratory monitoring, while feature normalization is to eliminate the dimensional differences between different features to ensure that they can be compared under the same standards. For example, when analyzing breathing patterns, it is found that frequency domain features are more helpful in distinguishing different breathing states than time domain features, so frequency domain features can be given higher weights. The final time-varying feature set will be more accurate and representative, and can fully reflect the respiratory movement characteristics of the target user.

[0064] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0065] (1) Reconstruct the target user's breathing signal through the time-varying feature set and determine the target breathing signal set;

[0066] (2) Perform frequency domain conversion on the target respiratory signal set to obtain a respiratory frequency domain dataset;

[0067] (3) extracting the respiratory rate component of the respiratory frequency domain data set to obtain a respiratory rate component data set of the respiratory frequency domain data set;

[0068] (4) Perform respiratory rate analysis on the respiratory rate component data set to obtain the respiratory rate calculation result;

[0069] (5) Perform curve conversion on the respiratory rate calculation results to obtain the respiratory rate distribution curve.

[0070] Specifically, the target user's breathing signal is reconstructed through a time-varying feature set, and the time-varying features are converted into a signal that can reflect the user's breathing pattern. The time-varying feature set includes a series of features extracted from the image sequence, such as changes in pixel intensity, changes in the size of the motion area, etc. These features together depict the user's breathing dynamics. The reconstructed breathing signal combines these features to form a continuous time series, which can reflect the user's breathing pattern that changes over time. For example, if the chest movement is analyzed, this time series shows that the chest expands and contracts with each breath. The reconstructed target breathing signal set is converted into a frequency domain to analyze the frequency components of the breathing signal. Through Fourier transform, the time series signal is converted into frequency space to reveal the frequency distribution characteristics in the signal. Frequency domain conversion can reveal breathing patterns of different frequency components, such as normal breathing, fast breathing or slow breathing. For example, in general, the normal breathing frequency of an adult is in the range of 12 to 20 times per minute, which will be manifested as a corresponding frequency distribution in the signal after frequency domain conversion. The breathing rate component is extracted from the breathing frequency domain data set. The frequency components directly related to the breathing rate are separated from the frequency domain data. Frequency components reflecting different breathing rates are identified through spectrum analysis technology, for example, frequency components corresponding to normal breathing and other frequency components indicating abnormal breathing (such as breathing too fast or too slow) are identified. Through analysis, a respiratory rate component data set of the respiratory frequency domain data set is obtained. Respiratory rate analysis is performed on the respiratory rate component data set to calculate the specific respiratory frequency and its change characteristics. The time series characteristics of the respiratory rate component are analyzed, such as calculating the average respiratory rate, the variability of the respiratory rate, etc. These calculation results provide quantitative information about the user's breathing pattern and help evaluate the user's respiratory health status. For example, if the analysis results show that the user's breathing rate is significantly higher or lower than the normal range, this indicates a potential problem with the respiratory system. The respiratory rate calculation results are converted into a curve to generate a respiratory rate distribution curve. The numerical data of the respiratory rate is converted into a visual curve chart so that the changing trend and pattern of the respiratory rate can be intuitively displayed.

[0071] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0072] (1) Extracting curve feature points from the respiratory rate distribution curve to obtain multiple curve feature points;

[0073] (2) Calculate the curvature of each curve feature point to obtain the curvature data of each curve feature point;

[0074] (3) Recognize the curve change characteristics of the curvature data of each curve feature point respectively to obtain the curve change characteristics of each curve feature point;

[0075] (4) Perform breathing rate anomaly detection on the curve change characteristics of each curve feature point to obtain the breathing rate anomaly detection result of the target user.

[0076] Specifically, the curve feature points of the respiratory rate distribution curve are extracted to identify points that reflect key changes in the breathing pattern. For example, edge detection technology is used to calculate the first and second derivatives of the curve to determine extreme points or inflection points. These feature points represent important changes in the breathing pattern, such as faster or slower breathing. If the curve rises or falls suddenly, these points represent significant changes in the breathing rate. The curve curvature is calculated for each curve feature point to quantify the degree of curvature of the curve at these feature points. The calculation of curvature can reveal sharp changes in the breathing pattern. For example, a higher curvature indicates rapid breathing changes, such as rapid recovery after apnea, or an irregular breathing pattern. When calculating the curvature, the shape of the curve near the feature point is usually considered, and the curvature value is determined by the first and second derivatives of the curve. Subsequently, the curvature change feature recognition is performed on the curvature data of each curve feature point, and abnormal breathing patterns are identified from the curvature data. By analyzing the change in curvature of the curve, irregularities or abnormal features of the breathing pattern are revealed. For example, if continuous points with sudden changes in curvature are found in the curve, this indicates that an abnormality has occurred in the breathing process, such as apnea or an irregular breathing rhythm. Combine statistical analysis and pattern recognition techniques to identify abnormal changes compared to normal breathing patterns. Perform respiratory rate anomaly detection on the curve change characteristics of each curve feature point to determine whether the user's breathing is abnormal. Compare the curve change characteristics with the predefined normal breathing pattern to determine whether there is an anomaly. Use data-driven methods, such as statistical analysis based on historical data or predictive models based on machine learning, to determine abnormalities in the breathing rate. For example, if the analysis results show that changes in certain curve feature points are beyond the normal breathing pattern range, the system will mark these as abnormalities.

[0077] In a specific embodiment, the process of executing step S107 may specifically include the following steps:

[0078] (1) Matching the abnormal respiratory rate processing strategy from the preset abnormal respiratory rate processing list based on the abnormal respiratory rate detection result to obtain the target abnormal respiratory rate processing strategy;

[0079] (2) Generate abnormal breathing rate alarm information of the target user through the target breathing rate abnormality processing strategy;

[0080] (3) Through the preset monitoring terminal, the target user is given a respiratory monitoring alarm based on the abnormal respiratory rate alarm information.

[0081] Specifically, according to the abnormal respiratory rate detection result, a match is made from the preset abnormal respiratory rate processing list to determine the appropriate processing strategy. This list usually includes a series of predefined response measures, which are set according to different types of respiratory abnormalities (such as too fast, too slow, irregular, etc.). In order to select a suitable strategy, the specific details of the abnormality detection are analyzed, such as the type, severity and duration of the abnormality. Generate a respiratory rate abnormality alarm message according to the selected target respiratory rate abnormality processing strategy. The alarm message should clearly and accurately describe the abnormal situation and provide appropriate suggestions or instructions. The content of the alarm message depends on the selected processing strategy, which includes a description of the abnormality, recommended personal response measures, or notification information containing emergency contacts. The target user is monitored and alarmed for respiratory monitoring according to the abnormal respiratory rate alarm information through a preset monitoring terminal. The monitoring terminal can be a smartphone application, a smart watch, a bedside monitoring device or other networked device. The role of the monitoring terminal is not only to convey the alarm information, but also to guide the user on how to respond to this information. For example, a smartphone application can display specific abnormal information and operation instructions, and a smart watch can vibrate to attract the user's attention and display an emergency message. In an emergency, the monitoring terminal automatically contacts a medical provider or emergency response team and provides information on the user's location and health status.

[0082] The above describes the respiratory monitoring method based on machine vision in the embodiment of the present application. The following describes the respiratory monitoring device based on machine vision in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of a respiratory monitoring device based on machine vision includes:

[0083] The acquisition module 201 is used to acquire the respiratory motion image of the target user through a preset image sensor to obtain the respiratory motion image data;

[0084] The recognition module 202 is used to perform motion recognition and segment selection on the target user through the breathing motion image data to obtain a motion state image sequence and a rest state image sequence of the target user;

[0085] A calculation module 203 is used to calculate the respiratory motion ROI region of the target user based on the motion state image sequence and the rest state image sequence to obtain an initial ROI region image of the target user;

[0086] An extraction module 204 is used to perform image preprocessing on the initial ROI area image of the target user to obtain a target ROI area image of the target user, and perform time-varying feature extraction on the target ROI area image to obtain a time-varying feature set of the target ROI area image;

[0087] An analysis module 205 is used to analyze the breathing rate of the target user by using the time-varying feature set to obtain a breathing rate distribution curve;

[0088] A detection module 206, configured to perform abnormal breathing rate detection on the target user based on the breathing rate distribution curve to obtain an abnormal breathing rate detection result of the target user;

[0089] The matching module 207 is used to match the respiratory rate abnormality processing strategy from the preset respiratory rate abnormality processing list according to the respiratory rate abnormality detection result, obtain the target respiratory rate abnormality processing strategy and perform respiratory monitoring and alarm on the target user according to the target respiratory rate abnormality processing strategy.

[0090] Through the cooperation of the above-mentioned components, the respiratory motion image data is collected through the image sensor, and there is no need to directly contact the user. Therefore, it is a non-invasive monitoring method that will not cause discomfort to the user. Through motion recognition and segment selection, the method can automatically distinguish the user's motion state and rest state, without the user manually inputting information, to achieve automatic monitoring. Through the Kalman filter parameter matching of the motion state image and the rest state image, the accurate calculation of the respiratory motion ROI area is achieved, and the accuracy of monitoring is improved. Time-varying feature extraction of the target ROI area image helps to capture the dynamic changes of respiratory motion, further improving the accuracy and sensitivity of monitoring. Through respiratory rate distribution curve analysis and anomaly detection, the user's respiratory rate can be monitored in real time, abnormal conditions can be identified in time, and early warnings can be provided. By matching the preset respiratory rate abnormality processing strategy, the method can provide personalized processing suggestions according to the situation of different users, increasing the practicality and applicability of monitoring. Generate abnormal respiratory rate alarm information, which can immediately notify the monitoring terminal, so that the monitoring personnel can take timely action, and improve the real-time nature of monitoring. Therefore, this application uses machine vision technology to improve the accuracy of respiratory monitoring.

[0091] The present application also provides a machine vision-based respiratory monitoring device, which includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the machine vision-based respiratory monitoring method in the above-mentioned embodiments.

[0092] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the machine vision-based respiratory monitoring method.

[0093] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0094] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0095] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A respiratory monitoring method based on machine vision, characterized in that: The respiratory monitoring method based on machine vision includes: The target user's respiratory motion image is collected by a preset image sensor to obtain respiratory motion image data; The target user is subjected to action recognition and segment selection through the respiratory motion image data to obtain a motion state image sequence and a rest state image sequence of the target user; specifically comprising: performing object recognition and information labeling on the target user in the respiratory motion image data to obtain object labeling image data; performing object action state classification on the object labeling image data to obtain an object action state classification result; performing state change trend analysis on the object labeling image data through the object action state classification result to obtain multiple state change trends; performing change amplitude extraction on each state change trend respectively to obtain a state change amplitude corresponding to each state change trend; performing segment selection on the object labeling image data through the parameter change amplitude corresponding to each state change trend to obtain a motion state image sequence and a rest state image sequence of the target user; Based on the motion state image sequence and the rest state image sequence, the respiratory motion ROI area of ​​the target user is calculated to obtain the initial ROI area image of the target user; specifically comprising: performing Kalman filter parameter matching on the motion state image sequence and the rest state image sequence to obtain a corresponding Kalman filter parameter set; performing dynamic ROI tracking on the Kalman filter parameter set to obtain a dynamic ROI tracking result of the target user; calculating the pixel change rate of the motion state image sequence and the rest state image sequence according to the dynamic ROI tracking result; performing ROI data extraction on the motion state image sequence and the rest state image sequence according to the pixel change rate to obtain the initial ROI area image of the target user; Performing image preprocessing on the initial ROI area image of the target user to obtain the target ROI area image of the target user, and performing time-varying feature extraction on the target ROI area image to obtain a time-varying feature set of the target ROI area image; specifically comprising: performing image denoising and contrast enhancement on the initial ROI area image of the target user to obtain the target ROI area image of the target user; performing mean and variance calculation of pixel intensity on the target ROI area image to obtain a target calculation result; performing time-varying feature mapping on the target calculation result to obtain an initial feature set; performing feature weight analysis and feature normalization processing on the initial feature set to obtain a time-varying feature set of the target ROI area image; The breathing rate of the target user is analyzed by the time-varying feature set to obtain a breathing rate distribution curve; specifically comprising: reconstructing the breathing signal of the target user by the time-varying feature set to determine a target breathing signal set; performing frequency domain conversion on the target breathing signal set to obtain a breathing frequency domain data set; extracting breathing rate components from the breathing frequency domain data set to obtain a breathing rate component data set of the breathing frequency domain data set; performing breathing rate analysis on the breathing rate component data set to obtain a breathing rate calculation result; performing curve conversion on the breathing rate calculation result to obtain a breathing rate distribution curve; Based on the breathing rate distribution curve, the target user is subjected to breathing rate abnormality detection to obtain the breathing rate abnormality detection result of the target user; specifically comprising: extracting curve feature points of the breathing rate distribution curve to obtain a plurality of curve feature points; calculating the curve curvature of each curve feature point to obtain the curvature data of each curve feature point; identifying the curve change feature of the curvature data of each curve feature point to obtain the curve change feature of each curve feature point; performing breathing rate abnormality detection on the curve change feature of each curve feature point to obtain the breathing rate abnormality detection result of the target user; The method comprises: matching a respiratory rate abnormality processing strategy from a preset respiratory rate abnormality processing list through the respiratory rate abnormality detection result to obtain a target respiratory rate abnormality processing strategy, and performing respiratory monitoring and alarm on the target user through the target respiratory rate abnormality processing strategy; specifically comprising: matching a respiratory rate abnormality processing strategy from a preset respiratory rate abnormality processing list through the respiratory rate abnormality detection result to obtain a target respiratory rate abnormality processing strategy; generating respiratory rate abnormality alarm information of the target user through the target respiratory rate abnormality processing strategy; and performing respiratory monitoring and alarm on the target user according to the respiratory rate abnormality alarm information through a preset monitoring terminal.

2. A respiratory monitoring device based on machine vision, characterized in that: The machine vision-based respiratory monitoring device comprises: The acquisition module is used to acquire the respiratory motion image of the target user through a preset image sensor to obtain the respiratory motion image data; The recognition module is used to perform action recognition and segment selection on the target user through the respiratory motion image data to obtain a motion state image sequence and a rest state image sequence of the target user; specifically comprising: performing object recognition and information labeling on the target user in the respiratory motion image data to obtain object labeling image data; performing object action state classification on the object labeling image data to obtain an object action state classification result; performing state change trend analysis on the object labeling image data based on the object action state classification result to obtain multiple state change trends; performing change amplitude extraction on each state change trend respectively to obtain a state change amplitude corresponding to each state change trend; performing segment selection on the object labeling image data based on the parameter change amplitude corresponding to each state change trend to obtain a motion state image sequence and a rest state image sequence of the target user; A calculation module is used to calculate the respiratory motion ROI area of ​​the target user based on the motion state image sequence and the rest state image sequence to obtain the initial ROI area image of the target user; specifically comprising: performing Kalman filter parameter matching on the motion state image sequence and the rest state image sequence to obtain a corresponding Kalman filter parameter set; performing dynamic ROI tracking on the Kalman filter parameter set to obtain a dynamic ROI tracking result of the target user; calculating the pixel change rate of the motion state image sequence and the rest state image sequence according to the dynamic ROI tracking result; performing ROI data extraction on the motion state image sequence and the rest state image sequence according to the pixel change rate to obtain the initial ROI area image of the target user; The extraction module is used to perform image preprocessing on the initial ROI area image of the target user to obtain the target ROI area image of the target user, and perform time-varying feature extraction on the target ROI area image to obtain the time-varying feature set of the target ROI area image; specifically comprising: performing image denoising and contrast enhancement on the initial ROI area image of the target user to obtain the target ROI area image of the target user; performing mean and variance calculation of pixel intensity on the target ROI area image to obtain a target calculation result; performing time-varying feature mapping on the target calculation result to obtain an initial feature set; performing feature weight analysis and feature normalization processing on the initial feature set to obtain the time-varying feature set of the target ROI area image; The analysis module is used to perform a breathing rate analysis on the target user through the time-varying feature set to obtain a breathing rate distribution curve; specifically comprising: reconstructing a breathing signal of the target user through the time-varying feature set to determine a target breathing signal set; performing a frequency domain conversion on the target breathing signal set to obtain a breathing frequency domain data set; performing a breathing rate component extraction on the breathing frequency domain data set to obtain a breathing rate component data set of the breathing frequency domain data set; performing a breathing rate analysis on the breathing rate component data set to obtain a breathing rate calculation result; performing a curve conversion on the breathing rate calculation result to obtain a breathing rate distribution curve; A detection module is used to perform abnormal breathing rate detection on the target user based on the breathing rate distribution curve to obtain the abnormal breathing rate detection result of the target user; specifically comprising: extracting curve feature points from the breathing rate distribution curve to obtain a plurality of curve feature points; calculating the curve curvature of each curve feature point to obtain the curvature data of each curve feature point; identifying the curve change feature of the curvature data of each curve feature point to obtain the curve change feature of each curve feature point; performing abnormal breathing rate detection on the curve change feature of each curve feature point to obtain the abnormal breathing rate detection result of the target user; A matching module is used to match a respiratory rate abnormality processing strategy from a preset respiratory rate abnormality processing list through the respiratory rate abnormality detection result, obtain a target respiratory rate abnormality processing strategy, and perform respiratory monitoring and alarm on the target user through the target respiratory rate abnormality processing strategy; specifically includes: matching a respiratory rate abnormality processing strategy from a preset respiratory rate abnormality processing list through the respiratory rate abnormality detection result, obtain a target respiratory rate abnormality processing strategy; generating respiratory rate abnormality alarm information of the target user through the target respiratory rate abnormality processing strategy; and performing respiratory monitoring and alarm on the target user according to the respiratory rate abnormality alarm information through a preset monitoring terminal.

3. A respiratory monitoring device based on machine vision, characterized in that: The machine vision-based respiratory monitoring device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor calls the instructions in the memory to enable the machine vision-based respiration monitoring device to perform the machine vision-based respiration monitoring method according to claim 1.

4. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the machine vision-based respiratory monitoring method as claimed in claim 1 is implemented.

Images