Non-contact infant respiration monitoring method, device and equipment and storage medium

Abstract

The invention relates to the technical field of infant sleep nursing, solves the problem that in the prior art, the infant respiration frequency cannot be monitored in a non-contact mode, and provides a non-contact infant respiration monitoring method, device and equipment and a storage medium. The method comprises the following steps: acquiring multiple frames of real-time images in an infant sleep scene; monitoring the sleep state of the real-time image, and identifying a first target image when the infant enters the sleep state; performing image enhancement processing on the first target image, and outputting a second target image after respiratory movement enhancement; feature extraction is carried out on the second target image, and a target area related to infant respiration is recognized; according to the maximum likelihood criterion, the target area is processed, and infant respiration information is determined. The infant respiration is monitored in a non-contact mode, the accuracy of respiration frequency monitoring is improved through image collection, image enhancement processing and other modes, and intrusive monitoring on the infant is not needed.

Description

Technical Field

[0001] The present invention relates to the technical field of infant sleep care, and in particular to a non-contact infant respiration monitoring method, device, equipment and storage medium. Background Art

[0002] Infants and young children are a group of physiologically fragile individuals who are easily affected by respiratory problems. By monitoring the respiratory rate during sleep, early detection of respiratory abnormalities, such as apnea and hypopnea, can help timely intervention and treatment. SIDS refers to the sudden unexplained death of infants while they are sleeping, and respiratory problems are a potential factor in SIDS. By monitoring the respiratory rate of infants and young children during sleep, abnormalities can be detected in time to reduce the risk of SIDS. At the same time, some infants and young children may suffer from respiratory diseases, such as asthma and respiratory distress syndrome. By monitoring the respiratory rate during sleep, the progression of the disease and the effect of treatment can be monitored, and the medical team can be guided to make appropriate treatment decisions. Secondly, respiratory rate is an important indicator for evaluating children's sleep quality. A normal respiratory rate helps maintain a good sleeping atmosphere and promotes the growth and development of infants and young children and the development of the nervous system. In general, the importance of monitoring the respiratory rate of infants and young children during sleep is to detect respiratory abnormalities early, prevent potential risks, improve the health level of infants and young children, and provide effective auxiliary tools for families and medical teams.

[0003] In real life, there are some contact methods for monitoring infants' respiratory rate. Although they can provide accurate physiological parameters to a certain extent, there are also some potential disadvantages and limitations: contact sensors usually need to be in direct contact with infants' bodies, such as through adhesive stickers or straps. This direct contact may cause discomfort to infants, interfere with their normal sleep state, and may even cause them to wake up easily or cry; contact sensors using adhesive stickers or straps may cause irritation to infants' skin and may even cause allergic reactions. For some infants, especially those with sensitive skin, this may be a problem; contact sensors need to be accurately placed at specific locations on the infant's body to ensure effective signal acquisition. However, infants may change their body positions while sleeping, which may cause the sensor position to shift and reduce the accuracy of monitoring; the use of contact sensors requires frequent replacement of patches or cleaning of sensors to ensure their normal operation and prevent infection. This increases the workload of parents and may introduce new hygiene risks; the use of straps or fixed sensors may limit the freedom of infants in or out of the bed. This may affect their activities, and free movement is important for infants in the developmental stage. In summary, although contact sensors can provide accurate physiological monitoring data in some aspects, their limitations include interference with sleep, possible discomfort, skin irritation and allergy risks, location restrictions, etc. These problems may affect the acceptability and practicality of monitoring. Therefore, researchers and engineers are working hard to find more comfortable non-contact infant respiratory rate monitoring methods to improve the convenience of monitoring and the comfort of infants.

[0004] The existing Chinese patent CN116636832A discloses a respiratory rate monitoring method based on video signals, including: collecting human respiratory video, pre-processing the respiratory video and performing face recognition, and preliminarily extracting the chest area according to the head-to-body ratio of the human body; modeling the extracted chest area, and further accurately extracting the respiratory area; filtering the extracted respiratory area and extracting the respiratory movement direction characteristics of the pixels in the respiratory area; performing multi-scale spatial decomposition on the respiratory area image according to the respiratory movement direction feature extraction, extracting the respiratory phase information: filtering the extracted respiratory phase information, and performing peak monitoring to estimate the respiratory rate. The above scheme relies on face recognition technology, which limits its applicability. For example, when a clear infant face image cannot be obtained, and due to the differences in infant physiological characteristics and breathing patterns, the performance of the algorithm fluctuates between infants and non-infants, and the above scheme is not suitable for infant respiratory monitoring.

[0005] Therefore, how to monitor the respiratory rate of infants and young children in a non-contact manner and improve the convenience of monitoring and the comfort of infants is an urgent problem to be solved. Summary of the invention

[0006] In view of this, the present invention provides a non-contact infant respiration monitoring method, device, equipment and storage medium to solve the problem in the prior art that the infant respiration rate cannot be monitored non-contactly.

[0007] The technical solution adopted by the present invention is: In a first aspect, the present invention provides a non-contact infant respiration monitoring method, the method comprising: S1: Acquire real-time infrared video data of a sleeping infant, and decompose the real-time infrared video data into multiple frames of real-time images; S2: monitoring the sleep state of the real-time image, and identifying a first target image in which the infant has entered a sleep state; S3: performing image enhancement processing on the first target image, and outputting a second target image with enhanced respiratory motion; S4: extracting features from the second target image to identify a target area related to infant breathing in the second target image; S5: Processing the target area according to the maximum likelihood criterion to determine the infant's breathing information.

[0008] Preferably, the S2 includes: S21: inputting the real-time image into a pre-trained target detection model to obtain the position of the infant's head; S22: Tracking the position of the infant's head by using a target tracking algorithm, and outputting infant's head tracking frame information; S23: Identifying whether the infant has entered a sleeping state or not according to the infant head tracking frame information; S24: If the infant has entered a sleeping state, acquiring the first target image.

[0009] Preferably, the S23 includes: S231: acquiring center position information of the tracking frame in the real-time image according to the tracking frame information of the infant head; S232: performing average calculation on the center position information of the tracking frame and outputting average position information; S233: determining the movement state of the infant's head in the real-time image based on the center position information and the average position information; S234: If the number of continuous real-time image frames in which the infant's head is in motion is greater than a preset frame number threshold, it is identified that the infant is not in a sleeping state; S235: If the number of continuous real-time image frames in which the infant's head is in motion is less than or equal to a preset frame number threshold, it is recognized that the infant has entered a sleeping state.

[0010] Preferably, S3 includes: S31: performing grayscale conversion on the first target image to obtain a grayscale image; S32: Filter the grayscale image and output a filtered image; S33: Perform Fourier transform on the filtered image to output an initial frequency domain image; S34: enhancing the target frequency related to the infant's breathing in the initial frequency domain image, and outputting the enhanced target frequency domain image; S35: Perform inverse Fourier transform on the target frequency domain image, and output the second target image.

[0011] Preferably, the S4 includes: S41: Acquire continuous preset frames of the second target images, and classify the second target images into multiple image sets; S42: performing pixel-by-pixel averaging processing on the image set, and outputting an average image corresponding to the image set; S43: Calculate the average image difference to obtain a difference image; S44: Determine the target area according to the pixel difference in the difference image and a preset pixel threshold.

[0012] Preferably, the S44 includes: S441: performing binarization processing on the difference image according to the pixel difference and the pixel threshold, and outputting a binarized image; S442: extracting features from the binary image and outputting contour feature information; S443: Acquire target contour area information in the binary image according to the contour feature information; S444: Output the target area according to the target contour area information.

[0013] Preferably, the S5 includes: S51: Counting the number of preset target pixel points in the target area and outputting a numerical sequence; S52: Performing a fast Fourier transform on the numerical sequence to convert the time domain feature information corresponding to the numerical sequence into frequency domain feature information; S53: performing likelihood function modeling processing on the frequency domain feature information to obtain a likelihood function; S54: performing logarithm and derivative processing on the likelihood function to determine the infant's breathing information.

[0014] In a second aspect, the present invention provides a non-contact infant respiratory monitoring device, the device comprising: A real-time image acquisition module is used to acquire real-time infrared video data of a sleeping infant and decompose the real-time infrared video data into multiple frames of real-time images; A sleep state monitoring module, used to monitor the sleep state of the real-time image and identify a first target image in which the infant has entered a sleep state; An image enhancement module, used for performing image enhancement processing on the first target image, and outputting a second target image with enhanced respiratory motion; a target region identification module, performing feature extraction on the second target image, and identifying a target region related to infant breathing in the second target image; The infant breathing information determination module is used to process the target area according to the maximum likelihood criterion to determine the infant breathing information.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect in the above-mentioned embodiment.

[0016] In a fourth aspect, an embodiment of the present invention further provides a storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method of the first aspect in the above-mentioned embodiment is implemented.

[0017] In summary, the beneficial effects of the present invention are as follows: The present invention provides a non-contact infant respiration monitoring method, device, equipment and storage medium, the method comprising: acquiring real-time infrared video data in an infant sleeping scene, decomposing the real-time infrared video data into multiple frames of real-time images; performing sleep state monitoring on the real-time image, identifying a first target image in which the infant has entered a sleep state; performing image enhancement processing on the first target image, and outputting a second target image in which the respiratory movement is enhanced; performing feature extraction on the second target image, and identifying a target area in the second target image related to the infant's respiration; and processing the target area according to the maximum likelihood criterion to determine the infant's respiration information. The present invention uses image processing technology to monitor the sleep state of real-time images, and can accurately identify whether the infant has entered a sleep state, which provides key initial information for subsequent monitoring and processing; image enhancement processing is performed on the first target image to improve image quality and highlight key features corresponding to respiratory movements, which is helpful to more clearly observe and analyze the respiratory movements of infants; through feature extraction technology, target areas related to the infant's breathing are identified, which is helpful to accurately capture the infant's respiratory movements; the target area is processed using the maximum likelihood criterion to provide an accurate estimate of the infant's respiratory information. Maximum likelihood estimation is a statistical method that improves the accuracy of respiratory rate monitoring by optimizing parameter estimates; there is no need to use sensors attached to infants, which avoids invasiveness to infants and provides a more convenient and comfortable monitoring method. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the drawings required for use in the embodiment of the present invention will be briefly introduced below. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work, and these are all within the protection scope of the present invention.

[0019] Figure 1 This is a schematic diagram of the overall working process of the non-contact infant respiration monitoring method in Example 1 of the present invention; Figure 2 This is a schematic diagram of the process of monitoring the sleep state of infants in Example 1 of the present invention; Figure 3 This is a schematic diagram of a process for identifying whether an infant has entered a sleeping state or not in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of a process for performing image enhancement processing on the first target image in Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of a process for identifying a target area related to infant breathing in the second target image in Embodiment 1 of the present invention; Figure 6It is a schematic diagram of a process of determining the target area according to the pixel difference in the difference image and a preset pixel threshold in Embodiment 1 of the present invention; Figure 7 This is a schematic diagram of a process for determining infant breathing information in Example 1 of the present invention; Figure 8 is a structural block diagram of a non-contact infant respiratory monitoring device in Embodiment 3 of the present invention; Fig. 9 It is a schematic diagram of the structure of an electronic device in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described in conjunction with the drawings in the embodiment of the present invention. It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the orientation or position relationship indicated by the terms "center", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such a process, method, article or device. In the absence of further restrictions, the elements defined by the phrase "comprising..." do not exclude the existence of other identical elements in the process, method, article or device comprising the elements. If there is no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other and are all within the protection scope of the present invention.

[0021] Example 1

[0022] See also Figure 1 Embodiment 1 of the present invention discloses a non-contact infant respiration monitoring method, the method comprising: S1: Acquire real-time infrared video data of a sleeping infant, and decompose the real-time infrared video data into multiple frames of real-time images; Specifically, a high-definition infrared camera with a fill light of 940nm wavelength (this is the preferred wavelength, but other wavelengths are also acceptable) is used to capture real-time infrared video data of infants and young children sleeping scenes. The real-time infrared video data includes the real-time situation of the infants and young children in the sleeping scenes. The real-time infrared video data is divided into multiple frames of continuous static images. The static images can be processed separately or used for further analysis, recognition or other applications. This helps to capture the instantaneous situation of infants and young children sleeping scenes so as to understand and process the information in the video in more detail.

[0023] S2: monitoring the sleep state of the real-time image, and identifying a first target image in which the infant has entered a sleep state; Specifically, the real-time image is subjected to sleep state monitoring, and the sleep state monitoring includes: using a pre-trained target monitoring model and a target tracking algorithm to identify the real-time movement state of the infant's head. If it is identified that the infant's head continues to not move, it is identified that the infant has entered a sleep state, and a first target image corresponding to the infant's sleep state is obtained. Using the infant's head movement state to monitor the sleep state is non-invasive. Compared with traditional physiological signal monitoring, such as heart rate monitoring or respiratory monitoring, the analysis of the head movement state is easier to achieve without direct contact with the infant's body. For infants, monitoring of the head movement state is usually more comfortable. They do not need to wear any sensors or equipment, which helps to provide a more natural and comfortable sleeping environment. At the same time, the use of pre-trained target monitoring models and target tracking algorithms can simplify implementation. These models and algorithms are usually able to automatically learn and identify the movement state of the infant's head, reducing the need for complex manual feature engineering.

[0024] In one embodiment, see Figure 2 , said S2 comprises: S21: inputting the real-time image into a pre-trained target detection model to obtain the position of the infant's head; Specifically, the real-time image is input into the pre-trained YoloV8s model to obtain the position of the infant's head. YoloV8s is a target detection model with high accuracy and real-time performance. First, the real-time image is passed to the YoloV8s model for processing. The model is pre-trained to recognize and locate the infant's head. The pre-training process is as follows: During the training process, the model gradually adjusts its parameters to improve the ability to accurately recognize and locate the head by learning the characteristics of the infant's head in the training image, such as shape, color and texture, and the relationship between the infant's head and the surrounding environment. By comparing with the training data marked with the position information of the infant's head, the model continuously optimizes its own weights so that it can accurately recognize and locate the infant's head in the new image. By parsing the model output, the bounding box position of the infant's head is obtained for subsequent analysis. The real-time performance of the YoloV8s model makes this method suitable for application scenarios that require timely response to changes in the infant's sleep state.

[0025] S22: Tracking the position of the infant's head by using a target tracking algorithm, and outputting infant's head tracking frame information; Specifically, the target tracking algorithm is used to track the position of the infant's head and provide real-time and continuous head movement information. In this process, the DeepSort algorithm is used for target tracking. The DeepSort algorithm is a target tracking algorithm based on deep learning. It combines target monitoring and appearance feature embedding, and can track the trajectory of the target between video frames. First, the position of the infant's head is passed to the DeepSort algorithm. The DeepSort algorithm uses a Kalman filter for target tracking. By predicting the next position of the target and matching it with the actual monitoring results, the continuous tracking of the infant's head is achieved. The output results include the tracking frame information of the infant's head, that is, the head position, speed and other information in each video frame. The method combining target monitoring and target tracking can effectively deal with the tiny movements of infants during sleep, provide high-precision head position tracking information, and thus support subsequent sleep status monitoring.

[0026] S23: Identifying whether the infant has entered a sleeping state or not according to the infant head tracking frame information; Specifically, based on the infant's head tracking frame information, the movement of the infant's head tracking frame over a period of time is monitored, and by analyzing the movement speed and acceleration of the head, it is determined whether the head remains still. In the sleeping state, the infant's head is usually relatively stable, while in the awake state, the head will have some slight movements. The changes in the head tracking frame over time are observed. When the infant enters the sleeping state, the movement of the head gradually slows down and the changes in the tracking frame are also stable. On the contrary, when the infant is awake, the head shows more movement, resulting in more obvious changes in the tracking frame.

[0027] In one embodiment, see Figure 3 , the S23 comprises: S231: acquiring center position information of the tracking frame in the real-time image according to the tracking frame information of the infant head; Specifically, according to the infant head tracking frame information, the tracking frames of 300 frames of images are acquired each time, and in each acquired tracking frame, for each tracking frame Track(i), its center coordinates Ci(x, y) are extracted.

[0028] S232: performing average calculation on the center position information of the tracking frame and outputting average position information; Specifically, by calculating the average coordinates Cavg(x, y) of the center coordinates Ci(x, y) of these 300 tracking boxes, we get an overall average position.

[0029] S233: determining the movement state of the infant's head in the real-time image based on the center position information and the average position information; S234: If the number of continuous real-time image frames in which the infant's head is in motion is greater than a preset frame number threshold, it is identified that the infant is not in a sleeping state; S235: If the number of continuous real-time image frames in which the infant's head is in motion is less than or equal to a preset frame number threshold, it is recognized that the infant has entered a sleeping state.

[0030] Specifically, the Euclidean distance Dis(i) between the center coordinates Ci(x, y) of each tracking frame and the average coordinates Cavg(x, y) is calculated. If the Euclidean distance of a certain frame is greater than the preset threshold Thre1, which is set according to specific needs, the frame is judged to indicate that the infant's head is moving. If the number of frames of movement exceeds 50, it is considered that the infant has not yet entered the sleep state, and the next 300 consecutive frames are taken again for judgment. If the number of frames of movement is less than 50, it is considered that the infant has fallen asleep. By judging the average position and movement state of the center coordinates of the infant's head tracking frame, it is assisted in judging whether the infant has entered the sleep state. By combining the real-time monitoring of the head movement and the analysis of the overall movement pattern, the accuracy of the judgment of the infant's sleep state is improved. The setting of the threshold Thre1 and the number of movement frames fully considers the changes in the actual infant's sleep scene and is more scene-applicable.

[0031] S24: If the infant has entered a sleeping state, acquiring the first target image.

[0032] S3: performing image enhancement processing on the first target image, and outputting a second target image with enhanced respiratory motion; Specifically, image enhancement processing is performed on the first target image, and the image enhancement processing includes: converting the first target image into a frequency domain image through fast Fourier transform, performing enhancement processing on the frequency range related to the respiratory frequency, and outputting a second target image with enhanced respiratory movement; image enhancement processing helps to more clearly observe and analyze subtle changes in respiratory movement, and provide stronger support for subsequent respiratory movement monitoring and analysis.

[0033] In one embodiment, see Figure 4 , said S3 comprises: S31: performing grayscale conversion on the first target image to obtain a grayscale image; Specifically, the first target image is converted into a corresponding grayscale image, and the grayscale conversion method includes: average value method: taking the average value of the red, green and blue channel values ​​of each pixel as the grayscale value: Weighted average method: Considering the different brightness contributions of different color channels, the weighted average method is used: The weights are set according to the sensitivity of the human eye to different color channels.

[0034] Brightness method: Use the brightness in the color space as the grayscale value: These methods are all based on the RGB color model, where R, G, and B represent the pixel values ​​of the red, green, and blue channels, respectively. The choice of method depends on the specific needs and application scenarios, as well as the sensitivity to the image brightness information.

[0035] S32: Filter the grayscale image and output a filtered image; Specifically, in order to remove noise from the image, a high-pass filter is applied to each frame of the grayscale image. The main function of the high-pass filter is to emphasize the high-frequency information in the image and suppress the low-frequency information, thereby highlighting the details and edge features in the image; The principle is to remove low-frequency components in the image and retain high-frequency components by reducing the gradient between pixel values. This is effective in removing some low-frequency noise introduced by sensor noise or other environmental factors, and helps to improve the accuracy of subsequent frequency domain analysis.

[0036] S33: Perform Fourier transform on the filtered image to output an initial frequency domain image; Specifically, the frequency domain analysis of each frame of grayscale image is performed by applying fast Fourier transform (FFT), and the formula is as follows: , where F(u,v) is the representation of the filtered image in the frequency domain, indicating the amplitude and phase information of a specific frequency in the frequency domain; f(x,y) is the grayscale distribution of the original filtered image in the spatial domain, which is a two-dimensional function, where x and y represent the horizontal and vertical coordinates in the image, respectively; u and v represent the horizontal and vertical frequencies in the frequency domain, respectively; It is a complex exponential function that describes the phase information in the frequency domain; after Fourier transform, it outputs the initial frequency domain image.

[0037] S34: enhancing the target frequency related to the infant's breathing in the initial frequency domain image, and outputting the enhanced target frequency domain image; Specifically, in the frequency domain image, the amplitude values ​​in the frequency range of 5-30 are amplified to enhance the information related to respiratory movement. This is achieved by using the formula Famplified(u,v) = α × F(u,v), where Famplified represents the amplified complex representation in the frequency domain, α is the amplification factor, and is the complex representation of the original image f(x,y) in the frequency domain, which is the result of Fourier transform and contains the amplitude and phase information of the image at different frequencies. The enhanced target frequency domain image is output.

[0038] S35: Perform inverse Fourier transform on the target frequency domain image, and output the second target image.

[0039] Specifically, by applying the inverse Fourier transform, the processed frequency domain data is converted back to the spatial domain to obtain a new spatial domain image Fnew(x,y). This process produces a series of new images with noise removal and respiratory motion enhancement, which helps to more clearly observe and analyze the breathing-related motion in the video. This frequency domain analysis and processing method helps to highlight the frequency range of interest and improves the visibility and analyticity of the respiratory movements of infants and young children.

[0040] S4: extracting features from the second target image to identify a target area related to infant breathing in the second target image; Specifically, first, image processing techniques such as edge detection, morphological operations or region-based segmentation methods are used to highlight and extract features related to respiratory movements in the image. The feature extraction method includes extracting information such as texture, color, shape, etc. from the image to capture visual features related to respiratory movements. This helps to identify the area of ​​infants' respiratory movements and form a target area with representative features in the image. By accurately extracting the target area related to infants' breathing from the second target image, reliable data is provided for subsequent respiratory movement monitoring and analysis.

[0041] In one embodiment, see Figure 5 , said S4 comprises: S41: Acquire continuous preset frames of the second target images, and classify the second target images into multiple image sets; Specifically, first, the spatial domain image Fnew(x,y) reconstructed every 400 frames is used as the research object. This means that a 20-second video clip is selected to ensure that it contains at least 2 complete breathing processes, and the 400 frames of images are divided into 40 groups, each containing 10 frames of images (equivalent to 0.5 seconds of video).

[0042] S42: performing pixel-by-pixel averaging processing on the image set, and outputting an average image corresponding to the image set; Specifically, the average image Fi is calculated for each group of 10 frames of images, by adding the pixel values ​​of the 10 frames of images at the same position pixel by pixel, and then dividing by 10 to obtain the 10-frame average image.

[0043] S43: Calculate the average image difference to obtain a difference image; Specifically, the difference image between two adjacent frames, ie, F2-F1, F3-F2, ..., F40-F39, is calculated by subtracting the pixel values ​​of the average images of two adjacent frames pixel by pixel.

[0044] S44: Determine the target area according to the pixel difference in the difference image and a preset pixel threshold.

[0045] Specifically, in each difference image, the difference between pixels is judged. If the difference is greater than the set threshold Thre2, the point is set as a white point. Then, the contour of the white point in each difference image is extracted. Finally, the contours extracted from the 39 difference images are compared, and the area C with the same maximum contour is selected, and the minimum bounding rectangle of the area is calculated. The minimum bounding rectangle is defined as the region of interest (ROI), that is, the breathing monitoring area when the baby is sleeping. This process highlights the dynamic changes in the image by averaging and differencing continuous image frames, and then finds the breathing monitoring area when the baby is sleeping by comparing the contours of the difference image. The setting of the threshold Thre2 needs to be adjusted according to the specific situation to adapt to different video data and breathing monitoring requirements.

[0046] In one embodiment, see Figure 6 , the S44 comprises: S441: performing binarization processing on the difference image according to the pixel difference and the pixel threshold, and outputting a binarized image; S442: extracting features from the binary image and outputting contour feature information; S443: Acquire target contour area information in the binary image according to the contour feature information; Specifically, for each binary image, such as Diffk, the maximum contour area in the difference image is calculated, and then the circumscribed rectangle of the area is obtained. The calculation method includes a contour detection algorithm and a region marking algorithm. By calculating the contour and finding the maximum contour, and then obtaining its circumscribed rectangle, the most significant motion area in each difference image is obtained.

[0047] S444: Output the target area according to the target contour area information.

[0048] Specifically, the contour area with the largest circumscribed rectangle area is selected from the 39 binary images, and the area is defined as the area C with the same maximum contour. Then, the minimum circumscribed rectangle of the area is calculated, and the rectangle is the area of ​​interest C. By analyzing the 39 difference images, the area with the most significant changes in the whole process, that is, the area with significant changes in respiratory motion, is found. This process extracts the area most related to the intensity of respiratory motion by calculating the contour, circumscribed rectangle and comparing the area size, providing a targeted area of ​​interest for subsequent respiratory monitoring.

[0049] S5: Processing the target area according to the maximum likelihood criterion to determine the infant's breathing information.

[0050] Specifically, based on the acquired region of interest C, a mathematical model is established, which can describe the changes in respiratory movements. The maximum likelihood criterion is used to estimate the parameters of the model by modeling the probability distribution of the observed data. In this scenario, it is necessary to estimate the parameters related to the infant's respiratory movement, such as the amplitude and frequency of the movement. The maximum likelihood estimation method is used to maximize the likelihood function to determine the values ​​of the model parameters so as to maximize the probability that the model generates observed data. According to the determined model parameters, the infant's respiratory information is extracted, including the frequency and amplitude of the breathing. In general, the process of processing the infant's respiratory information using the maximum likelihood criterion is a process of mathematical modeling and parameter estimation. Through this process, the infant's respiratory information can be more accurately extracted from the image data, providing strong support for further monitoring and analysis.

[0051] In one embodiment, see Figure 7 , the S5 comprises: S51: Counting the number of preset target pixel points in the target area and outputting a numerical sequence; Specifically, the number of white dots in the area of ​​interest C is extracted from the 39 frames of difference images obtained in every 20 seconds of video. For each frame of the difference image, the number of white dots is counted to form a numerical sequence containing 39 points, where each value represents the characteristic intensity of the corresponding frame. The sampling frequency of this sequence is 1 / 0.5=2Hz, corresponding to data collection every 0.5 seconds.

[0052] S52: Performing a fast Fourier transform on the numerical sequence to convert the time domain feature information corresponding to the numerical sequence into frequency domain feature information; Specifically, this numerical sequence is transformed by FFT (Fast Fourier Transform) to convert it from the time domain to the frequency domain. The transformation formula is: Where N is the number of samples (39 here), x[n] is the value of the nth frame in the numerical sequence, and the purpose of this step is to analyze the components of the numerical sequence in the frequency domain to find the main periodic changes, that is, the components of the breathing frequency.

[0053] S53: performing likelihood function modeling processing on the frequency domain feature information to obtain a likelihood function; Specifically, assuming that the distribution of respiratory frequency is approximately normal, the likelihood function is: Where μ(f) is the theoretical mean at frequency f, and σ is the standard deviation; S54: performing logarithm and derivative processing on the likelihood function to determine the infant's breathing information.

[0054] Specifically, the maximum likelihood function is converted to a log-likelihood function, i.e., ln(L(f;x1,x2,...,xn)), the log-likelihood function is differentiated, the derivative is set to 0, and the equation is solved to obtain f, which is the frequency f used to determine the infant's respiratory information. The whole process extracts frequency features from the numerical sequence, and estimates the infant's respiratory frequency through frequency domain analysis and maximum likelihood estimation. This estimation process is based on statistical modeling of the observed data to maximize the likelihood of the respiratory frequency.

[0055] Example 2

[0056] In Example 1, the respiratory frequency characteristic f of infants is obtained as the infant's respiratory information. However, in real life, infants' sleep cycles are shorter than those of adults, and the ratio of deep sleep to shallow sleep is different. For infants, too long nap time will affect the quality of sleep at night, leading to difficulty falling asleep at night and frequent waking up. Therefore, timely monitoring and controlling the sleep stage of infants, especially during the nap period, helps to maintain a normal sleep cycle and improve the quality of sleep at night; at the same time, long-term deep sleep will cause infants to feel tired and sleepy after waking up, and even have emotional instability. Therefore, controlling the duration of deep sleep, especially during the nap period, can avoid this situation and ensure that infants are energetic and emotionally stable after waking up. Therefore, monitoring the shallow and deep sleep stages of infants, especially the control of nap time, is crucial to ensuring the sleep quality and sleep effect of infants.

[0057] To this end, in Embodiment 2 of the present invention, based on Embodiment 1, the non-contact infant respiration monitoring method further includes: The real-time sleep stage of the infant is determined based on the infant's breathing information and a preset breathing frequency interval related to the infant's sleep stage, wherein the sleep stage includes at least a deep sleep stage and a light sleep stage; and the current sleep state of the infant is judged based on the infant's breathing frequency, including deep sleep and light sleep.

[0058] Specifically, by monitoring the breathing rate, it can help determine whether the infant is in a deep sleep state (slower breathing rate) or a shallow sleep state (faster breathing rate). Assume that the normal deep sleep breathing rate of an infant is in the range of 15-20 times / minute, and the shallow sleep breathing rate is in the range of 20-25 times / minute. If the infant's breathing rate is monitored to be 17 times / minute, then according to the preset range, it can be determined that the infant is currently in a deep sleep state.

[0059] According to the time information, determine whether the infant is in the nap stage or the night sleep stage; Specifically, time information is used to determine the current sleep stage, thereby facilitating further analysis. For example, in the first preset time period during the day, it is recognized that the current sleep stage is a nap, while in the second preset time period at night, it is recognized as a night nap. The first preset time period and the second preset time period are set and adaptively adjusted according to actual needs. This distinction is of great significance for analyzing the overall sleep patterns and behaviors of infants and young children, because naps and night naps usually differ in duration, sleep depth, etc.

[0060] If the infant is in the nap stage and the real-time sleep stage is the deep sleep stage, the deep sleep duration is obtained; Specifically, when the infant is detected to be in the nap stage and judged to have entered the deep sleep stage, the duration of deep sleep will be recorded. This process is automatic and updated in real time to ensure the accuracy of the monitoring data. The duration of deep sleep is an important indicator for evaluating the quality of naps and directly affects the physical recovery and growth of infants. Therefore, accurately capturing and recording this duration helps parents or caregivers understand the infant's sleep status and take timely measures to improve it.

[0061] Compare the deep sleep duration with a preset duration threshold, and output a comparison result; Specifically, the actual deep sleep duration of the infant is compared with the preset ideal duration threshold. The duration threshold is set according to factors such as the infant's age and health status. The comparison result can indicate whether the current deep sleep meets the standard. If the deep sleep duration does not reach the preset value, parents or caregivers are prompted to take corresponding measures, such as adjusting the work and rest schedule or improving the sleeping environment, to help the infant get higher quality sleep.

[0062] If the infant is in the late sleep stage, obtain the deep sleep duration and light sleep duration; Specifically, for the late sleep stage, not only the deep sleep time is monitored, but also the light sleep time is recorded. This is because late sleep is usually the main sleep cycle for infants and young children, involving the alternation of deep and light sleep. By obtaining these two types of sleep duration at the same time, the quality of nighttime sleep can be more comprehensively assessed, which helps to identify potential sleep problems, such as insufficient deep sleep or too much light sleep, which may affect the growth and development of infants and young children.

[0063] Determining a sleep cycle ratio according to the deep sleep duration and the light sleep duration; Specifically, the sleep cycle ratio of infants and young children is analyzed, that is, the relative ratio of deep sleep duration to light sleep duration. A normal sleep cycle usually includes a certain proportion of alternating deep and light sleep, which is essential for physical recovery and consolidation of brain function. By calculating this ratio, abnormal sleep patterns, such as insufficient deep sleep or excessive light sleep, are identified, and corresponding feedback is provided to help parents improve the sleep habits of infants and young children.

[0064] A reminder message is issued according to the comparison result and the sleep cycle ratio.

[0065] Specifically, based on the analysis results of the previous steps, especially the comparison between the deep sleep duration and the ideal value and the sleep cycle ratio, reminders will be sent to parents or caregivers. If it is found that the sleep quality of infants and young children is not ideal, such as insufficient deep sleep or excessive shallow sleep, suggestions or prompts will be provided through notifications, alarms, etc. These reminders can help parents understand the sleep status of infants and young children in a timely manner, so as to take necessary intervention measures to ensure that infants and young children get sufficient and high-quality sleep.

[0066] Example 3

[0067] See also Figure 8 Embodiment 3 of the present invention further provides a non-contact infant respiratory monitoring device, the device comprising: A real-time image acquisition module is used to acquire real-time infrared video data of a sleeping infant and decompose the real-time infrared video data into multiple frames of real-time images; A sleep state monitoring module, used to monitor the sleep state of the real-time image and identify a first target image in which the infant has entered a sleep state; An image enhancement module, used for performing image enhancement processing on the first target image, and outputting a second target image with enhanced respiratory motion; a target region identification module, performing feature extraction on the second target image, and identifying a target region related to infant breathing in the second target image; The infant breathing information determination module is used to process the target area according to the maximum likelihood criterion to determine the infant breathing information.

[0068] Specifically, the non-contact infant respiratory monitoring device provided by the embodiment of the present invention includes: a real-time image acquisition module, which is used to acquire real-time infrared video data in an infant sleeping scene, and decompose the real-time infrared video data into multiple frames of real-time images; a sleep state monitoring module, which is used to perform sleep state monitoring on the real-time image, and identify a first target image in which the infant has entered a sleep state; an image enhancement module, which is used to perform image enhancement processing on the first target image, and output a second target image with enhanced respiratory movement; a target area recognition module, which performs feature extraction on the second target image, and identifies a target area related to the infant's respiration in the second target image; an infant respiration information determination module, which is used to process the target area according to the maximum likelihood criterion, and determine the infant's respiration information. The device uses image processing technology to monitor the sleep state of real-time images, and can accurately identify whether the infant has entered a sleep state, which provides key initial information for subsequent monitoring and processing; image enhancement processing is performed on the first target image to improve image quality and highlight key features corresponding to respiratory movements, which is helpful to more clearly observe and analyze the respiratory movements of infants and young children; through feature extraction technology, the target area related to the infant's breathing is identified, which is helpful to accurately capture the infant's respiratory movements; the target area is processed using the maximum likelihood criterion to provide an accurate estimate of the infant's respiratory information. Maximum likelihood estimation is a statistical method that improves the accuracy of respiratory frequency estimation by optimizing parameter estimates; there is no need to use sensors attached to infants and young children, which avoids invasiveness to infants and young children and provides a more convenient and comfortable monitoring method.

[0069] Example 4

[0070] In addition, combined Figure 1 The non-contact infant respiration monitoring method described in Example 1 of the present invention can be implemented by an electronic device. Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in Embodiment 4 of the present invention is shown.

[0071] The electronic device may include a processor and a memory storing computer program instructions.

[0072] Specifically, the processor may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0073] The memory may include a large capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside the data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0074] The processor implements any one of the non-contact infant respiration monitoring methods in the above embodiments by reading and executing computer program instructions stored in the memory.

[0075] In one example, the electronic device may further include a communication interface and a bus. Fig. 9 As shown, the processor, memory, and communication interface are connected via a bus and communicate with each other.

[0076] The communication interface is mainly used to implement communication between modules, devices, units and / or equipment in the embodiments of the present invention.

[0077] The bus includes hardware, software or both, and the components of the device are coupled to each other. For example, but not limitation, the bus may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industrial standard architecture (EISA) bus, a front-side bus (FSB), a hypertransport (HT) interconnection, an industrial standard architecture (ISA) bus, an infinite bandwidth interconnection, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable bus or a combination of two or more of these. Where appropriate, the bus may include one or more buses. Although the embodiments of the present invention describe and illustrate a specific bus, the present invention contemplates any suitable bus or interconnection.

[0078] Example 5

[0079] In addition, in combination with the non-contact infant respiration monitoring method in the above embodiment 1, embodiment 5 of the present invention can also provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the non-contact infant respiration monitoring methods in the above embodiments is implemented.

[0080] In summary, the embodiments of the present invention provide a non-contact infant respiration monitoring method, device, equipment and storage medium.

[0081] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0082] The functional blocks shown in the structural block diagram described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0083] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant location, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0084] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0085] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A non-contact infant respiratory monitoring method, characterized in that: The method comprises: S1: Acquire real-time infrared video data of a sleeping infant, and decompose the real-time infrared video data into multiple frames of real-time images; S2: monitoring the sleep state of the real-time image and identifying a first target image indicating that the infant has entered a sleep state; S3: performing image enhancement processing on the first target image, and outputting a second target image with respiratory motion enhanced; S4: performing feature extraction on the second target image to identify a target area related to infant breathing in the second target image; S5: Processing the target area according to the maximum likelihood criterion to determine the infant's breathing information.

2. The non-contact infant respiratory monitoring method according to claim 1, characterized in that: The S2 includes: S21: Inputting the real-time image into a pre-trained target detection model to obtain the position of the infant's head; S22: Tracking the position of the infant's head using a target tracking algorithm, and outputting infant's head tracking frame information; S23: Identifying whether the infant has entered a sleeping state or not based on the infant head tracking frame information; S24: If the infant has entered a sleeping state, the first target image is acquired.

3. The non-contact infant respiratory monitoring method according to claim 2, characterized in that: The S23 includes: S231: Acquire center position information of the tracking frame in the real-time image based on the infant head tracking frame information; S232: Calculate the average value of the center position information of the tracking frame and output the average position information; S233: Determine the movement state of the infant's head in the real-time image based on the center position information and the average position information; S234: If the number of consecutive real-time image frames of the infant's head in motion is greater than a preset frame number threshold, it is determined that the infant is not in a sleeping state; S235: If the number of continuous real-time image frames in which the infant's head is in motion is less than or equal to a preset frame number threshold, it is recognized that the infant has entered a sleeping state.

4. The non-contact infant respiratory monitoring method according to claim 1, characterized in that: The S3 includes: S31: performing grayscale conversion on the first target image to obtain a grayscale image; S32: Filter the grayscale image and output a filtered image; S33: Perform Fourier transform on the filtered image to output an initial frequency domain image; S34: enhancing the target frequency related to the infant's breathing in the initial frequency domain image, and outputting an enhanced target frequency domain image; S35: Perform inverse Fourier transform on the target frequency domain image to output the second target image.

5. The non-contact infant respiratory monitoring method according to claim 1, characterized in that: The S4 includes: S41: Acquire consecutive preset frames of the second target image, and classify the second target image into multiple image sets; S42: performing pixel-by-pixel averaging processing on the image set, and outputting an average image corresponding to the image set; S43: Calculate the average image difference to obtain a difference image; S44: Determine the target area according to the pixel difference in the difference image and a preset pixel threshold.

6. The non-contact infant respiratory monitoring method according to claim 5, characterized in that: The S44 includes: S441: Binarizing the difference image according to the pixel difference and the pixel threshold, and outputting a binary image; S442: Perform feature extraction on the binary image and output contour feature information; S443: Acquire target contour area information in the binary image based on the contour feature information; S444: Output the target area according to the target contour area information.

7. The non-contact infant respiratory monitoring method according to any one of claims 1 to 6, characterized in that: The S5 includes: S51: Counting the number of preset target pixels in the target area and outputting a numerical sequence; S52: Performing a fast Fourier transform on the numerical sequence to convert the time domain feature information corresponding to the numerical sequence into frequency domain feature information; S53: performing likelihood function modeling processing on the frequency domain feature information to obtain a likelihood function; S54: performing logarithm and derivative processing on the likelihood function to determine the infant's breathing information.

8. A non-contact infant respiratory monitoring device, characterized in that: The device comprises: A real-time image acquisition module is used to acquire real-time infrared video data of a sleeping infant and decompose the real-time infrared video data into multiple frames of real-time images; a sleep state monitoring module, configured to monitor the sleep state of the real-time image and identify a first target image indicating that the infant has entered a sleep state; an image enhancement module, configured to perform image enhancement processing on the first target image and output a second target image with respiratory motion enhanced; a target region identification module, performing feature extraction on the second target image, and identifying a target region related to infant breathing in the second target image; The infant breathing information determination module is used to process the target area according to the maximum likelihood criterion to determine the infant breathing information.

9. An electronic device, characterized in that: include: At least one processor, at least one memory, and computer program instructions stored in the memory, which implement the method according to any one of claims 1 to 7 when the computer program instructions are executed by the processor.

10. A storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images