Newborn respiratory heart rate monitoring method, system, equipment and medium

Abstract

The invention provides a neonatal respiration heart rate monitoring method, system and device and a medium, the method comprises the steps that synchronous collection data of an RVF system is acquired, and the RVF system comprises a radar and a camera device; according to radar equipment data, digital beam forming is carried out in an antenna dimension, and echo beams in different directions are obtained; according to camera equipment data, on the basis of image recognition and camera calibration, the thoracic cavity and the abdominal cavity of the newborn are positioned and converted into the angular positions of the newborn relative to the RVF system; according to the obtained angle position information, matching and selecting corresponding wave beams on the angles of the chest and the abdominal cavity; distance measurement is carried out according to the selected wave beams, and thoracic cavity phase information and abdominal cavity phase information are extracted from echo wave beams corresponding to the distance information; acquiring heart rate frequency according to the thoracic cavity phase information; and obtaining the breathing heart rate frequency according to the abdominal cavity phase information. By independently extracting the abdominal cavity echoes for respiration measurement, a more accurate respiration measurement result can be obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of vital sign monitoring, and in particular to a method, system, equipment and medium for monitoring the breathing and heart rate of a newborn. Background Art

[0002] For vital sign monitoring, bio-radar uses radar technology to detect tiny human movements, such as subtle changes in the body surface caused by breathing and heartbeat, and then non-invasively obtain vital sign information. This technology transmits electromagnetic waves, receives reflected signals, and can detect sub-wavelength level respiratory and heart rate signals through phase demodulation and time-frequency analysis. It is mainly used for daily health monitoring of adult and elderly patients.

[0003] Although the bio-radar solution in the related art performs well in adult applications, it has some limitations or shortcomings in its application in neonatal monitoring because it usually needs to be precisely aimed at the chest area of ​​the subject in order to capture clear vital sign signals:

[0004] Unlike adults, newborns mainly use chest breathing. Due to the underdeveloped intercostal muscles, newborns mainly rely on the diaphragm for abdominal breathing, which significantly reduces the signal-to-noise ratio of the respiratory signal and affects the measurement accuracy. Moreover, in the neonatal monitoring environment, the baby's own limb movements will produce a lot of random motion interference, which may also cause the radar beam to deviate from the predetermined measurement area of ​​the chest cavity, seriously affecting the quality of the radar echo signal, making it difficult to accurately measure the respiratory heart rate. Summary of the invention

[0005] The present invention provides a method, device and medium for monitoring the respiratory heart rate of a newborn, which solves the problem of how to improve the measurement accuracy of the respiratory heart rate monitoring of a newborn in a biological radar solution.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, a method for monitoring the respiratory heart rate of a newborn is provided, comprising:

[0008] Acquiring synchronously collected data of an RVF system, wherein the RVF system includes radar and camera equipment;

[0009] According to the echo signal in the radar equipment data, digital beam forming is performed in the antenna dimension to obtain echo beams in different directions;

[0010] According to the camera device data, based on image recognition and camera calibration, the thoracic and abdominal positions of the newborn are located and converted into their angular positions relative to the RVF system respectively;

[0011] According to the angular position of the thoracic cavity and the abdominal cavity relative to the RVF system, matching and selecting corresponding beams at the angles of the thoracic cavity and the abdominal cavity;

[0012] Performing distance measurement according to the corresponding beams at the selected chest and abdominal angles, and extracting chest phase information and abdominal phase information from the echo beams corresponding to the distance information;

[0013] The heart rate frequency is obtained according to the thoracic phase information; and the respiratory frequency is obtained according to the abdominal phase information.

[0014] In a first possible implementation manner of the first aspect, performing digital beamforming in the antenna dimension according to the echo signal in the radar device data to obtain echo beams in different directions includes:

[0015] Establish radar array flow patterns,

[0016] The azimuth array flow pattern is:

[0017]

[0018] The pitch angle array flow pattern is:

[0019]

[0020] Where d is the distance between antennas, λ is the wavelength of the transmitted signal, N1 and N2 are the numbers of equivalent receiving antennas in azimuth and elevation, respectively, and θ, is a specific angle; j is an imaginary unit;

[0021] Take the value of the product of the radar array flow pattern rotation rank and the radar receiving antenna echo, and perform angle division according to the value,

[0022] The value of the rank of the azimuth and elevation radar array flow pattern multiplied by the radar's receiving antenna echo is:

[0023]

[0024] Among them, X is the data matrix of N2×N1 channels receiving data:

[0025]

[0026] If the polar coordinate angle When there is a target, S will have a larger amplitude.

[0027] In a second possible implementation of the first aspect, the method of locating the chest and abdominal positions of the newborn based on the camera device data, image recognition and camera calibration, and converting them into their angular positions relative to the RVF system, respectively, includes:

[0028] Collect and produce neonatal datasets;

[0029] Use the YOLO network to train based on the neonatal dataset;

[0030] Using the trained YOLO network, the armpits and navels of the newborn images are identified, and the pixel coordinates of the armpits and navels are calculated on the optical images;

[0031] The midpoint of the two armpits is identified as the chest cavity of the newborn; the navel is identified as the abdominal cavity of the newborn, and the chest cavity pixel position and the abdominal cavity pixel position are obtained;

[0032] Calibrate the camera and calculate the camera's intrinsic parameter matrix;

[0033] The thoracic cavity pixel position and the abdominal cavity pixel position are respectively converted into the angular positions of the thoracic cavity and the abdominal cavity relative to the RVF system according to the internal reference matrix.

[0034] In a third possible implementation manner of the first aspect, the performing of ranging according to corresponding beams at the selected angles of the chest and the abdomen, and extracting chest phase information and abdominal phase information respectively from echo beams corresponding to the distance information, includes:

[0035] Performing mixing pulse compression on the corresponding beams at the selected chest and abdomen angles respectively to obtain SFCW signals and achieve ranging;

[0036] The original transmission signal is in the form of:

[0037]

[0038] Where k is the sweep frequency; when the target distance is R(τ), the echo signal is Mix it with the carrier for down-conversion and pulse compression to get the signal:

[0039]

[0040] Where K is the frequency sweep slope, T is the pulse repetition time PRT, c is the speed of light, and the signal at the peak of the pulse pressure spectrum is extracted, that is, The signal at can be obtained by:

[0041]

[0042] The thoracic cavity phase information and the abdominal cavity phase information are extracted according to the signal containing the phase information.

[0043] In a fourth possible implementation manner of the first aspect, obtaining the heart rate frequency according to the thoracic phase information; and obtaining the respiratory frequency according to the abdominal phase information include:

[0044] According to the displacement information, continuous wavelet transform is performed to obtain the time-frequency diagram;

[0045] Performing curve fitting in the time-frequency diagram, and obtaining frequency values ​​of breathing and heart rate according to the curve;

[0046] The obtained curve is subjected to Kalman filtering to obtain accurate frequency values ​​of the breathing and heart rate.

[0047] In a second aspect, a neonatal respiratory heart rate monitoring system is provided, comprising:

[0048] A data acquisition module, used to obtain synchronously collected data of the RVF system, wherein the RVF system includes radar and camera equipment;

[0049] The radar data processing module is used to perform digital beam forming in the antenna dimension according to the echo signal in the radar equipment data to obtain echo beams in different directions;

[0050] A camera data processing module, for locating the chest and abdominal positions of the newborn according to the camera device data, based on image recognition and camera calibration, and converting them into their angular positions relative to the RVF system respectively;

[0051] A beam selection module, used for matching and selecting corresponding beams at the angles of the chest cavity and the abdomen according to the angular positions of the chest cavity and the abdomen relative to the RVF system;

[0052] A phase information extraction module, used for performing distance measurement according to the corresponding beams at the selected chest and abdominal angles, and extracting chest phase information and abdominal phase information from the echo beams corresponding to the distance information;

[0053] The time-frequency analysis module is used to obtain the heart rate frequency according to the thoracic phase information; and obtain the respiratory frequency according to the abdominal phase information.

[0054] In a first possible implementation manner of the second aspect, the camera data processing module is specifically configured to:

[0055] Collect and produce neonatal datasets;

[0056] Use the YOLO network to train based on the neonatal dataset;

[0057] Using the trained YOLO network, the armpits and navels of the newborn images are identified, and the pixel coordinates of the armpits and navels are calculated on the optical images;

[0058] The midpoint of the two armpits is identified as the chest cavity of the newborn; the navel is identified as the abdominal cavity of the newborn, and the chest cavity pixel position and the abdominal cavity pixel position are obtained;

[0059] Calibrate the camera and calculate the camera's intrinsic parameter matrix;

[0060] The thoracic cavity pixel position and the abdominal cavity pixel position are respectively converted into the angular positions of the thoracic cavity and the abdominal cavity relative to the RVF system according to the internal reference matrix.

[0061] In a second possible implementation manner of the second aspect, the phase information extraction module is specifically configured to:

[0062] Performing mixing pulse compression on the corresponding beams at the selected chest and abdomen angles respectively to obtain SFCW signals and achieve ranging;

[0063] The original transmission signal is in the form of:

[0064]

[0065] Where k is the sweep frequency; when the target distance is R(τ), the echo signal is Mix it with the carrier for down-conversion and pulse compression to get the signal:

[0066]

[0067] Where K is the frequency sweep slope, T is the pulse repetition time PRT, c is the speed of light, and the signal at the peak of the pulse pressure spectrum is extracted, that is, The signal at can be obtained by:

[0068]

[0069] The thoracic cavity phase information and the abdominal cavity phase information are extracted according to the signal containing the phase information.

[0070] In a third aspect, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the neonatal respiratory and heart rate monitoring method as described in the first aspect.

[0071] In a fourth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the neonatal respiratory heart rate monitoring method as described in the first aspect are implemented.

[0072] This application aims at the physiological characteristics of abdominal breathing of newborns, which is mainly based on diaphragmatic movement, and proposes a method of using the chest phase of the newborn to measure heart rate and the abdominal phase to measure respiration. Compared with the traditional method of using the chest echo phase to measure respiration and heart rate at the same time, this application significantly improves the signal-to-interference ratio of the respiratory signal by extracting the abdominal echo separately for respiration measurement. At the same time, the narrow beam with higher angular resolution can achieve stronger spatial filtering to meet the clutter filtering requirements in complex clinical application scenarios, thereby obtaining more accurate respiration and heart rate measurement results. Furthermore, this application introduces monocular camera-assisted tracking to enable the system to have adaptive target position tracking capabilities to ensure stable operation when the newborn moves. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A schematic flow chart of a method for monitoring the respiration and heart rate of a newborn provided in an embodiment of the present application;

[0074] Figure 2 A schematic flow chart of another method for monitoring the respiration and heart rate of a newborn provided in an embodiment of the present application;

[0075] Figure 3 A schematic diagram of a neonatal respiration measurement provided in an embodiment of the present application;

[0076] Figure 4 A chest and abdomen measurement effect simulation comparison diagram provided in an embodiment of the present application;

[0077] Figure 5 A schematic diagram of wavelet transformation and curve fitting provided in an embodiment of the present application;

[0078] Figure 6 A heart rate measurement effect diagram provided in an embodiment of the present application;

[0079] Figure 7 A breathing measurement effect provided by an embodiment of the present application;

[0080] Figure 8 An experimental scene diagram provided for an embodiment of the present application;

[0081] Fig. 9 A schematic diagram of a real-time monitoring interface in an experimental scenario provided in an embodiment of the present application;

[0082] Fig.10 A structural block diagram of a neonatal respiratory heart rate monitoring system provided in an embodiment of the present application;

[0083] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0084] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose, the technical solutions in the embodiments of the present application are clearly described. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present application.

[0085] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0086] The description of the method flow in the specification of the present application and the steps of the flowchart in the drawings of the present specification do not have to be strictly executed according to the step numbers, and the method steps can be executed in a different order. Moreover, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0087] The following is a detailed description of the neonatal respiratory heart rate monitoring method, system, device and medium provided in the embodiments of the present application in conjunction with the accompanying drawings and preferred embodiments.

[0088] First, the application scenario of the neonatal respiratory heart rate monitoring method of the embodiment of the present application is described in detail.

[0089] The heart rate measurement methods recommended by the neonatal resuscitation guidelines include arterial pulse palpation, auscultation, pulse oximetry monitor and electrocardiogram assessment. Although they are widely used in clinical applications, they each have limitations, such as inconvenient operation, the risk of infection caused by skin contact, and unstable signals.

[0090] The non-contact respiratory and heart rate measurement technology based on bio-radar has achieved certain results in adult health monitoring due to its advantages such as non-contact and fast response. In terms of technical implementation, the chest fluctuation caused by breathing and heartbeat will phase modulate the radar echo, so the bio-radar can detect the respiratory and heart rate signals at the sub-wavelength level through time-frequency analysis after phase demodulation.

[0091] However, in the application scenario for newborns, there are several defects:

[0092] (1) The abdominal breathing pattern of newborns is not taken into account: 70% of the driving force for human breathing comes from the diaphragm, and the remaining 30% is completed by the secondary respiratory muscle group, the external intercostal muscles. Therefore, most adults, especially adult women, breathe mostly in the form of thoracic breathing, with the rise and fall of the chest cavity being the main feature. However, the intercostal muscles of newborns are very weak, and their breathing relies entirely on the diaphragm, that is, abdominal breathing. The solutions in the related art directly measure respiration and heart rate simultaneously through the echo signal of the chest cavity for newborns, which will result in an extremely low signal-to-interference ratio of the respiratory signal, a significant decrease in measurement accuracy, and cannot meet the measurement accuracy requirements.

[0093] (2) Susceptible to interference from scene motion factors: Current studies have shown that random movements such as the movement of surrounding medical staff and the limb movements of the newborn in the neonatal monitoring environment can cause great interference to the echo phase and even drown out the respiratory and heart rate signals. Due to the insufficient clutter filtering capability of the monitoring system and the lack of a suitable method to filter out random motion interference, it is susceptible to interference from scene motion factors.

[0094] (3) Lack of continuous tracking capability: Compared with adults, newborns have stronger random movements during monitoring and can easily deviate from the direction of radar wave irradiation. For example, the radar beam is set to irradiate the chest and abdomen of a newborn, but after a period of time it irradiates the bed sheet or arm.

[0095] In response to the above technical problems, the present application proposes a method for monitoring the respiratory and heart rate of newborns, aiming to provide a respiratory and heart rate monitoring radar solution specially designed for newborns. It not only takes into account the unique abdominal breathing pattern of newborns, but also enhances the resistance to environmental interference and target tracking performance, thereby achieving more stable and reliable respiratory and heart rate monitoring of newborns; at the same time, by optimizing the radar signal processing algorithm, it provides a new, accurate and convenient respiratory and heart rate monitoring solution for neonatal monitoring.

[0096] See also Figure 1-2 , the present application embodiment provides a method for monitoring the respiratory heart rate of a newborn, such as Figure 1-2 As shown, the method of the embodiment of the present application includes:

[0097] Step S1, obtaining synchronously collected data of the RVF system, wherein the RVF system includes radar and camera equipment.

[0098] In this step S1, the data of the radar device is used to detect the body movement of the newborn caused by breathing and heartbeat by emitting electromagnetic waves and receiving echo signals; the data of the camera device is used to locate the chest and abdominal positions of the newborn based on image recognition.

[0099] During specific implementation, the radar device can be configured as a MIMO radar, and the camera device can be configured as a monocular camera. Exemplarily, the MIMO radar has 16-transmitted 20 narrow beams, and its signal power is less than -10dBm, which is much lower than the radio power of mobile phones or WiFi devices. The maximum angular resolution is 6.7°, the angular measurement accuracy is within 2°, and the operating frequency is 64GHz. There are no strict parameter restrictions for monocular cameras. MIMO radar and monocular cameras together constitute the RVF (Radar-version Fusion) system of this application. In some possible implementations, the radar equipment used can also be replaced with other types of radars such as TI millimeter-wave radar boards, and the performance can be improved by further adding transceiver antennas.

[0100] Step S2, performing digital beam forming in the antenna dimension according to the echo signal in the radar device data to obtain echo beams in different directions.

[0101] Based on the original data of the radar, multiple echo beams corresponding to different directions can be obtained by performing digital beam forming on the radar echo signal in the antenna dimension.

[0102] Specifically, step S2 includes:

[0103] Step S201, establishing radar array flow pattern,

[0104] The azimuth array flow pattern is:

[0105]

[0106] The pitch angle array flow pattern is:

[0107]

[0108] Where d is the distance between antennas, λ is the wavelength of the transmitted signal, N1 and N2 are the numbers of equivalent receiving antennas in azimuth and elevation, respectively, and θ, is a specific angle; j is an imaginary unit.

[0109] Step S202, obtain the value obtained by multiplying the radar array flow pattern by the radar receiving antenna echo, and perform angle division according to the value.

[0110] The value of the rank of the azimuth and elevation radar array flow pattern multiplied by the radar's receiving antenna echo is:

[0111]

[0112] Among them, X is the data matrix of N2×N1 channels receiving data:

[0113]

[0114] If the polar coordinate angle When there is a target, S will have a larger amplitude.

[0115] After the above processing steps, a two-dimensional matrix will be obtained It represents the radar's angle measurement results for targets within the field of view, and each element represents a beam. In other words, the azimuth and elevation angles are artificially divided into different θ. horn.

[0116] Step S3, based on the camera device data, image recognition and camera calibration, locate the chest and abdominal positions of the newborn and convert them into their angular positions relative to the RVF system.

[0117] The step S3 specifically includes:

[0118] Step S301, collect and create a newborn data set. This step can be based on the hospital data to create a newborn data set. The data set should contain a sufficient number of newborn images, where the newborn's armpits and navels need to be clearly shown in the images.

[0119] Step S302: Use the YOLO network to perform training based on the newborn data set. The purpose of the training is to enable the YOLO network to accurately identify the armpits and navels in the newborn images.

[0120] Step S303, using the trained YOLO network to identify the armpits and navel of the newborn image, and calculating the pixel coordinates of the armpits and navel positions on the optical image.

[0121] Step S304, identifying the midpoint of the two armpits as the chest cavity of the newborn; identifying the navel position as the abdominal cavity of the newborn, and obtaining the chest cavity pixel position and the abdominal cavity pixel position.

[0122] Step S305, calibrate the camera and calculate the camera's internal parameter matrix. The calibration method may use the Zhang Zhengyou calibration method, which is a prior art; the internal parameter matrix is ​​a set of camera internal parameters, including the camera's optical properties, such as focal length, optical center position, etc.

[0123] Step S306: Convert the thoracic cavity pixel position and the abdominal cavity pixel position into the angular position of the thoracic cavity and the abdominal cavity relative to the RVF system according to the internal reference matrix, that is, the actual angular position during the implementation process.

[0124] Step S4, matching and selecting corresponding beams at the angles of the chest cavity and the abdomen according to the angular positions of the chest cavity and the abdomen relative to the RVF system.

[0125] In the process of implementing step S4, after locking the body part of the newborn with the camera, the corresponding beam can be directly selected from the two-dimensional matrix S obtained in step S202, so as to achieve accurate spatial filtering. In the above steps S3-S4, a human data set is produced for newborns, and adaptive target position tracking is achieved with the aid of a monocular camera-assisted radar system. Through camera-assisted tracking, the present application can adjust the direction of the radar beam in real time to ensure that even when the newborn moves, the radar beam can accurately illuminate the chest and abdominal cavities, avoiding beam deviations caused by twisting and displacement of the newborn. This enables the present application to have adaptive target position tracking capabilities, can work stably in complex clinical application scenarios, and improve the continuity and reliability of monitoring.

[0126] Step S5, performing distance measurement according to the corresponding beams at the selected chest and abdominal angles, and extracting chest phase information and abdominal phase information from the echo beams corresponding to the distance information.

[0127] The step S5 specifically includes:

[0128] Perform mixing pulse compression on the corresponding beams at the selected chest and abdomen angles to obtain SFCW signals and achieve ranging;

[0129] The original transmission signal is in the form of:

[0130]

[0131] Where k is the sweep frequency; when the target distance is R(τ), the echo signal is Mix it with the carrier for down-conversion and pulse compression to get the signal:

[0132]

[0133] Where K is the frequency sweep slope, T is the pulse repetition time PRT, c is the speed of light, and the signal at the peak of the pulse pressure spectrum is extracted, that is, The signal at can be obtained by:

[0134]

[0135] The chest phase information and the abdominal phase information are extracted from the signal containing the phase information. The phase signal contains the displacement R of the chest and abdomen caused by breathing and heart rate, that is, the phase information of the chest and abdomen movement. Thus, the chest phase of the newborn can be further extracted for heart rate measurement, and the abdominal phase can be extracted for breathing measurement.

[0136] Step S6, obtaining the respiratory frequency according to the thoracic phase information; obtaining the heart rate according to the abdominal phase information.

[0137] The step S6 specifically includes:

[0138] Step S601, performing continuous wavelet transform according to the displacement information to obtain a time-frequency diagram.

[0139] For the extracted displacement information R, the continuous wavelet transform is performed as follows:

[0140]

[0141] Where a is the scale parameter, b is the translation parameter, and is the wavelet basis.

[0142] Step S602: performing curve fitting in the time-frequency diagram, and obtaining the frequency value of the respiratory heart rate according to the curve.

[0143] See also Figure 5 , by fitting a curve to the most continuous and highest intensity area in the time-frequency diagram, the frequency value of the respiratory heart rate can be obtained.

[0144] Furthermore, in order to reduce noise interference, the following steps may be performed:

[0145] Step S603, performing Kalman filtering on the obtained curve to obtain accurate frequency values ​​of breathing and heart rate.

[0146] Based on the above technical solution, this application proposes a method of using the chest phase of the newborn to measure heart rate and the abdominal phase to measure respiration, respectively, targeting the physiological characteristics of abdominal respiration of newborns, which is mainly based on diaphragmatic movement. Compared with the traditional method of using the chest echo phase to measure respiration and heart rate at the same time, this application significantly improves the signal-to-interference ratio of the respiration signal by extracting the abdominal echo separately for respiration measurement. At the same time, the narrow beam with higher angular resolution can achieve stronger spatial filtering to meet the clutter filtering requirements in complex clinical application scenarios, thereby obtaining more accurate respiration and heart rate measurement results. See Figure 3-4 Newborns mainly rely on the diaphragm for abdominal breathing. The method of the present application can better adapt to this physiological characteristic and improve the reliability and stability of the measurement.

[0147] Furthermore, the present application introduces monocular camera-assisted tracking to enable the system to have adaptive target position tracking capabilities, ensuring that it can still work stably when the newborn moves.

[0148] See also Figure 6-7 The test results of the present invention are as follows Figure 6 and Figure 7As shown: the breathing and heart rate measurement results are respectively, compared with the AMF method (a method for measuring the breathing and heart rate of newborns with random motion suppression in the related art known to the inventor, referred to as AMF (adaptive motion artifact filter)), this method has a better breathing and heart rate measurement effect (black is the three-lead electrocardiogram reference value, red is the measurement result of the AMF method, and blue is the measurement result of this application). For example, see Figure 8-9 , this application provides an experimental scenario and a real-time monitoring interface to demonstrate the application and effects of this application.

[0149] See also Fig.10 , corresponding to the above-mentioned neonatal respiration and heart rate monitoring method embodiment, the present application embodiment provides a neonatal respiration and heart rate monitoring system, the system comprising:

[0150] A data acquisition module 1001 is used to acquire synchronously collected data of an RVF system, wherein the RVF system includes radar and camera equipment;

[0151] The radar data processing module 1002 is used to perform digital beam forming in the antenna dimension according to the echo signal in the radar device data to obtain echo beams in different directions;

[0152] The camera data processing module 1003 is used to locate the chest and abdomen of the newborn according to the camera device data based on image recognition and camera calibration, and convert them into their angular positions relative to the RVF system respectively;

[0153] A beam selection module 1004, configured to match and select corresponding beams at the angles of the chest cavity and the abdomen according to the angular positions of the chest cavity and the abdomen relative to the RVF system;

[0154] The phase information extraction module 1005 is used to perform distance measurement according to the corresponding beams at the selected chest and abdominal angles, and extract chest phase information and abdominal phase information from the echo beams corresponding to the distance information respectively;

[0155] The time-frequency analysis module 1006 is used to obtain the heart rate frequency according to the thoracic phase information; and obtain the respiratory frequency according to the abdominal phase information.

[0156] Furthermore, the camera data processing module 1003 is specifically used for:

[0157] Collect and produce neonatal datasets;

[0158] Use the YOLO network to train based on the neonatal dataset;

[0159] Using the trained YOLO network, the armpits and navels of the newborn images are identified, and the pixel coordinates of the armpits and navels are calculated on the optical images;

[0160] The midpoint of the two armpits is identified as the chest cavity of the newborn; the navel is identified as the abdominal cavity of the newborn, and the chest cavity pixel position and the abdominal cavity pixel position are obtained;

[0161] Calibrate the camera and calculate the camera's intrinsic parameter matrix;

[0162] The thoracic cavity pixel position and the abdominal cavity pixel position are respectively converted into the angular positions of the thoracic cavity and the abdominal cavity relative to the RVF system according to the internal reference matrix.

[0163] Furthermore, the phase information extraction module 1005 is specifically used to:

[0164] Performing mixing pulse compression on the corresponding beams at the selected chest and abdomen angles respectively to obtain SFCW signals and achieve ranging;

[0165] The original transmission signal is in the form of:

[0166]

[0167] Where k is the sweep frequency; when the target distance is R(τ), the echo signal is Mix it with the carrier for down-conversion and pulse compression to get the signal:

[0168]

[0169] Where K is the frequency sweep slope, T is the pulse repetition time PRT, c is the speed of light, and the signal at the peak of the pulse pressure spectrum is extracted, that is, The signal at can be obtained by:

[0170]

[0171] The thoracic cavity phase information and the abdominal cavity phase information are extracted according to the signal containing the phase information.

[0172] The above-mentioned neonatal respiratory heart rate monitoring system implements the steps and various processes of the above-mentioned neonatal respiratory heart rate monitoring method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0173] See also Fig.11Corresponding to the above-mentioned embodiment of the neonatal respiratory and heart rate monitoring method, an embodiment of the present application provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the above-mentioned embodiment of the neonatal respiratory and heart rate monitoring method and the various processes of the embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0174] The memory 1009 can be used to store software programs and various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1009 may include a volatile memory or a non-volatile memory, or the memory 1009 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1009 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0175] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1010.

[0176] Corresponding to the above-mentioned embodiment of the neonatal respiratory and heart rate monitoring method, the embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by the processor, the steps of the above-mentioned embodiment of the neonatal respiratory and heart rate monitoring method and the various processes of the embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0177] The processor is the processor in the electronic device described in the above embodiment of the present application. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0178] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be noted that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0179] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0180] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for monitoring the respiratory heart rate of a newborn, characterized in that: include: Acquiring synchronously collected data of an RVF system, wherein the RVF system includes radar and camera equipment; According to the echo signal in the radar equipment data, digital beam forming is performed in the antenna dimension to obtain echo beams in different directions; According to the camera device data, based on image recognition and camera calibration, the thoracic and abdominal positions of the newborn are located and converted into their angular positions relative to the RVF system respectively; According to the angular position of the thoracic cavity and the abdominal cavity relative to the RVF system, matching and selecting corresponding beams at the angles of the thoracic cavity and the abdominal cavity; Performing distance measurement according to the corresponding beams at the selected chest and abdominal angles, and extracting chest phase information and abdominal phase information from the echo beams corresponding to the distance information; The heart rate frequency is obtained according to the thoracic phase information; and the respiratory frequency is obtained according to the abdominal phase information.

2. The method for monitoring the breathing and heart rate of a newborn according to claim 1, characterized in that: The method of performing digital beam forming in the antenna dimension according to the echo signal in the radar device data to obtain echo beams in different directions includes: Establish radar array flow patterns, The azimuth array flow pattern is: The pitch angle array flow pattern is: Where d is the distance between antennas, λ is the wavelength of the transmitted signal, N1 and N2 are the numbers of equivalent receiving antennas in azimuth and elevation respectively, θ, is a specific angle; j is an imaginary unit; Take the value of the product of the radar array flow pattern rotation rank and the radar receiving antenna echo, and perform angle division according to the value, The value of the rank of the azimuth and elevation radar array flow pattern multiplied by the radar's receiving antenna echo is: Among them, X is the data matrix of N2×N1 channels receiving data: If the polar coordinate angle When there is a target, S will have a larger amplitude.

3. The method for monitoring the breathing and heart rate of a newborn according to claim 1, characterized in that: The method of locating the chest and abdomen of the newborn according to the camera device data and converting them into their angular positions relative to the RVF system respectively based on image recognition and camera calibration includes: Collect and produce neonatal datasets; Use the YOLO network to train based on the neonatal dataset; Using the trained YOLO network, the armpits and navels of the newborn images are identified, and the pixel coordinates of the armpits and navels are calculated on the optical images; The midpoint of the two armpits is identified as the chest cavity of the newborn; the navel is identified as the abdominal cavity of the newborn, and the chest cavity pixel position and the abdominal cavity pixel position are obtained; Calibrate the camera and calculate the camera's intrinsic parameter matrix; The thoracic cavity pixel position and the abdominal cavity pixel position are respectively converted into the angular positions of the thoracic cavity and the abdominal cavity relative to the RVF system according to the internal reference matrix.

4. The method for monitoring the respiration and heart rate of a newborn according to claim 1, characterized in that: The step of performing distance measurement according to the corresponding beams at the selected angles of the chest cavity and the abdomen, and extracting chest cavity phase information and abdominal cavity phase information from the echo beams corresponding to the distance information, respectively, includes: Performing mixing pulse compression on the corresponding beams at the selected chest and abdomen angles respectively to obtain SFCW signals and achieve ranging; The original transmission signal is in the form of: Where k is the sweep frequency; when the target distance is R(τ), the echo signal is Mix it with the carrier for down-conversion and pulse compression to get the signal: Where K is the frequency sweep slope, T is the pulse repetition time PRT, c is the speed of light, and the signal at the peak of the pulse pressure spectrum is extracted, that is, The signal at can be obtained by: The thoracic cavity phase information and the abdominal cavity phase information are extracted according to the signal containing the phase information.

5. The method for monitoring the respiration and heart rate of a newborn according to claim 4, characterized in that: The step of obtaining the heart rate frequency according to the thoracic phase information and obtaining the respiratory frequency according to the abdominal phase information includes: According to the displacement information, continuous wavelet transform is performed to obtain the time-frequency diagram; Performing curve fitting in the time-frequency diagram, and obtaining frequency values ​​of breathing and heart rate according to the curve; The obtained curve is subjected to Kalman filtering to obtain accurate frequency values ​​of the breathing and heart rate.

6. A neonatal respiratory heart rate monitoring system, characterized in that: include: A data acquisition module, used to obtain synchronously collected data of the RVF system, wherein the RVF system includes radar and camera equipment; The radar data processing module is used to perform digital beam forming in the antenna dimension according to the echo signal in the radar equipment data to obtain echo beams in different directions; A camera data processing module, for locating the chest and abdominal positions of the newborn according to the camera device data, based on image recognition and camera calibration, and converting them into their angular positions relative to the RVF system respectively; A beam selection module, used for matching and selecting corresponding beams at the angles of the chest cavity and the abdomen according to the angular positions of the chest cavity and the abdomen relative to the RVF system; A phase information extraction module, used for performing distance measurement according to the corresponding beams at the selected chest and abdominal angles, and extracting chest phase information and abdominal phase information from the echo beams corresponding to the distance information; The time-frequency analysis module is used to obtain the heart rate frequency according to the chest phase information; and obtain the respiratory frequency according to the abdominal phase information.

7. The neonatal respiratory heart rate monitoring system according to claim 6, characterized in that: The camera data processing module is specifically used for: Collect and produce neonatal datasets; Use the YOLO network to train based on the neonatal dataset; Using the trained YOLO network, the armpits and navels of the newborn images are identified, and the pixel coordinates of the armpits and navels are calculated on the optical images; The midpoint of the two armpits is identified as the chest cavity of the newborn; the navel is identified as the abdominal cavity of the newborn, and the chest cavity pixel position and the abdominal cavity pixel position are obtained; Calibrate the camera and calculate the camera's intrinsic parameter matrix; The thoracic cavity pixel position and the abdominal cavity pixel position are respectively converted into the angular positions of the thoracic cavity and the abdominal cavity relative to the RVF system according to the internal reference matrix.

8. The neonatal respiratory heart rate monitoring system according to claim 6, characterized in that: The phase information extraction module is specifically used for: Performing mixing pulse compression on the corresponding beams at the selected chest and abdomen angles respectively to obtain SFCW signals and achieve ranging; The original transmission signal is in the form of: Where k is the sweep frequency; when the target distance is R(τ), the echo signal is Mix it with the carrier for down-conversion and pulse compression to get the signal: Where K is the frequency sweep slope, T is the pulse repetition time PRT, c is the speed of light, and the signal at the peak of the pulse pressure spectrum is extracted, that is, The signal at can be obtained by: The thoracic cavity phase information and the abdominal cavity phase information are extracted according to the signal containing the phase information.

9. An electronic device, characterized in that: The electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the neonatal respiratory heart rate monitoring method as claimed in any one of claims 1 to 5.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method for monitoring the respiratory and heart rate of a newborn as claimed in any one of claims 1 to 5 are implemented.

Images