Child stroller sign monitoring and audio and video interaction system based on 60G millimeter waves

Abstract

The invention relates to the technical field of video coding and intelligent monitoring, and particularly discloses a child stroller sign monitoring and audio and video interaction system based on 60G millimeter waves. The system comprises a millimeter wave radar front-end module, a multi-mode vibration sensing and modeling module, a self-adaptive combined noise elimination and signal separation module, a physical sign parameter resolving and state evaluation module and an audio and video interaction and intelligent early warning module. By constructing a vibration noise reference model and performing two-stage signal separation processing, physiological signals such as heart rate and respiratory rate of children can be accurately extracted from strong noise, and high-precision physical sign monitoring and intelligent early warning are realized.

Description

Technical Field

[0001] This invention belongs to the field of video coding and intelligent monitoring technology, specifically relating to a child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave. Background Technology

[0002] In the fields of intelligent monitoring and the Internet of Things (IoT), non-contact, continuous vital sign monitoring of special groups such as infants and young children has become an important research direction. Its core lies in acquiring vital sign data in real time through sensor technology and combining it with communication technology to achieve remote interaction and early warning. Among these technologies, non-contact vital sign sensing using radio frequency signals has attracted widespread attention due to its wearable, seamless, and convenient characteristics.

[0003] Millimeter-wave radar-based vital sign monitoring technology enables remote detection of vital signs such as heart rate and respiration by analyzing the phase changes of radar echoes caused by subtle human movements. The basic principle of this technology is that the electromagnetic wave signals emitted by radar, after being reflected by parts of the human body such as the chest cavity, undergo regular phase modulation in accordance with the body's periodic physiological activities. Physiological parameters can be extracted by demodulating and analyzing the echo signals.

[0004] In existing technologies, integrating millimeter-wave radar into mobile platforms such as strollers for dynamic monitoring faces significant technical challenges. During stroller movement, vibrations caused by uneven road surfaces introduce strong Doppler frequency shift noise. This noise highly overlaps with weak physiological signals in the frequency domain, leading to a sharp deterioration in the signal-to-noise ratio of the radar echo signal.

[0005] Traditional signal filtering algorithms struggle to separate motion artifacts caused by vibration from genuine physiological signals in complex motion scenarios in real time and with high accuracy. This leads to a significant decrease in the detection accuracy of key vital signs such as heart rate, failing to meet the needs of continuous and reliable health monitoring. Therefore, effectively suppressing motion interference and improving the robustness and accuracy of millimeter-wave radar vital sign monitoring in dynamic mobile environments has become an urgent technical challenge. Summary of the Invention

[0006] The purpose of this invention is to provide a child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave, so as to solve the technical problem that the accuracy of vital sign monitoring of millimeter wave radar integrated into child strollers in the prior art is drastically reduced in dynamic moving environment due to severe superposition of vehicle vibration noise and physiological micro-motion signals.

[0007] This invention provides a child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave, comprising:

[0008] The millimeter-wave radar front-end module is integrated inside the stroller body. It is used to transmit continuous wave radar signals in the 60GHz band and receive echo signals reflected by the child's body, and output raw baseband signals containing mixed information of physiological micro-movements and stroller vibrations.

[0009] The multimodal vibration sensing and modeling module is used to collect and quantify the multidimensional vibration state of the vehicle body in real time, and to build a vibration noise reference model that is of the same origin as the radar signal.

[0010] The adaptive joint noise cancellation and signal separation module is used to receive the raw baseband signal from the millimeter-wave radar front-end module and the vibration noise reference model from the multimodal vibration sensing and modeling module. By performing joint time-frequency domain analysis and adaptive filtering, it separates the pure physiological micro-motion signal component from the raw baseband signal.

[0011] The vital signs parameter calculation and status assessment module is used to perform time-frequency analysis and feature extraction on the isolated pure physiological micro-motion signals, calculate the real-time heart rate and respiratory rate values, and assess the child's vital signs health status based on a preset threshold model, generating status assessment results.

[0012] The audio-visual interaction and intelligent early warning module integrates a camera and a speaker. Based on the status assessment results output by the vital signs parameter calculation and status assessment module, it executes corresponding audio-visual interaction strategies or triggers graded early warning signals, and uploads the data to the cloud monitoring platform.

[0013] Furthermore, the millimeter-wave radar front-end module adopts a MIMO antenna array structure with one transmitter and two receivers. The antenna array is encapsulated inside the vehicle body backrest, and its beam direction is precisely calibrated and focused on the chest and back area of ​​the child. The module includes a 60GHz radio frequency transceiver chip, a phase-locked loop frequency synthesizer, and a baseband analog front-end. The transmission power of the continuous wave radar signal is set to 10dBm and the bandwidth is 4GHz to meet the high sensitivity detection requirements for micron-level displacement, while ensuring compliance with electromagnetic radiation safety standards.

[0014] Furthermore, the multimodal vibration sensing and modeling module includes a high-precision six-axis inertial measurement unit, a micro-strain sensor array installed at the vehicle body shock absorber suspension, and a vibration noise modeling processor;

[0015] A high-precision six-axis inertial measurement unit measures the linear acceleration and angular velocity of the vehicle body in three-dimensional space in real time at a sampling rate of 1000Hz; a micro-strain sensor array is used to sense the deformation stress of the vehicle body structure caused by bumps.

[0016] The vibration and noise modeling processor is used to fuse inertial measurement data and strain data. Through a preset vibration transfer function, it calculates the equivalent phase modulation sequence and Doppler frequency shift spectrum of vehicle body vibration under the observation perspective of millimeter-wave radar, thereby generating a vibration and noise reference model that is highly correlated with the vibration and noise components in the radar echo.

[0017] Furthermore, the adaptive joint noise cancellation and signal separation module includes a signal alignment unit, a reference noise reconstruction unit, and an adaptive filter bank;

[0018] The signal alignment unit is used to perform precise time synchronization and sampling rate matching between the input raw baseband signal and the vibration noise reference model, ensuring that the two are strictly aligned in the time domain.

[0019] The reference noise reconstruction unit, based on the aligned vibration noise reference model, generates an estimated signal that can approximate the actual vibration noise in the radar signal in the time and frequency domain through a nonlinear transformer.

[0020] The adaptive filter bank employs an improved normalized least mean square algorithm. The core of this algorithm is to dynamically calculate the cross-correlation matrix and autocorrelation matrix between the estimated signal and the original baseband signal, and update the weight coefficients of the filter in real time. This allows the estimated vibration noise component to be subtracted from the original baseband signal to the maximum extent, and the output residual signal is regarded as the initially separated physiological micro-motion signal.

[0021] Furthermore, the adaptive joint noise cancellation and signal separation module also includes a blind source separation post-processing unit;

[0022] This unit performs independent component analysis on the preliminary physiological micro-motion signal output after adaptive filtering; specifically, it constructs an observation vector from multiple channels of the preliminary physiological micro-motion signal and assumes that it is a linear mixture of statistically independent source signals.

[0023] By iteratively optimizing the separation matrix, the mutual information between the components of the output signal is minimized, thereby further separating the residual interference related to vibration and the independent source signals corresponding to heartbeat and respiration, respectively, and finally outputting high-fidelity heartbeat source signals and respiration source signals.

[0024] Furthermore, the vital sign parameter calculation and state assessment module includes a time-frequency analysis unit, a peak detection and tracking unit, and a state decision unit; the time-frequency analysis unit performs continuous wavelet transform on the input heartbeat source signal and respiratory source signal respectively to generate the time-frequency spectrum of the signal;

[0025] The peak detection and tracking unit uses a dynamic programming-based optimal path search algorithm to track the dominant frequency ridge line within the frequency band corresponding to heartbeat and respiration on the time-spectrum graph, and calculates the continuous heartbeat interval sequence and respiratory cycle sequence based on the instantaneous frequency of the ridge line.

[0026] The state decision-maker has a built-in sliding time window to calculate the average, standard deviation, and variation trend of heart rate and respiratory rate over the past 30 seconds, and compares these statistical characteristics with a dynamic threshold model established based on physiological data of children of different ages.

[0027] An abnormal status identifier is generated when the statistical feature exceeds the threshold range for three consecutive detection cycles; otherwise, a normal status identifier is generated.

[0028] Furthermore, the audio-visual interaction and intelligent early warning module includes an integrated camera, array microphone, speaker, interaction logic controller, and wireless communication unit;

[0029] An integrated camera is used to capture video of a child's facial expressions and surrounding environment during non-monitoring periods or when interactive commands are received;

[0030] An array microphone is used to collect ambient sounds and the cries of children; the interactive logic controller executes a preset strategy based on the received state evaluation results;

[0031] When the status assessment result is normal, the controller can respond to the request from the remote monitoring terminal and start the audio and video call function;

[0032] When a child is detected crying but their vital signs are normal, the controller can automatically play a preset soothing audio.

[0033] When the status assessment result is abnormal, the controller will, according to the level of abnormality, play a local warning sound through the speaker, push a warning message to the bound mobile monitoring terminal, and finally upload the complete abnormal data packet and video recordings of the preceding and following time periods to the cloud monitoring platform through the wireless communication unit.

[0034] Furthermore, the system operates under a multi-level power management framework, which includes a continuous monitoring mode, an intermittent scanning mode, and a standby mode;

[0035] The continuous monitoring mode is activated when the trolley is stationary or moving at a constant and smooth speed, and all modules operate at full power.

[0036] The intermittent scanning mode automatically switches to this mode when the system detects continuous and severe vibration. The vital signs parameter calculation and status assessment module reduces the data update rate to 1 time / s. The audio and video interaction and intelligent early warning module turns off the camera to save power.

[0037] The standby mode is activated when there are no signs of life from the child inside the stroller for more than 5 minutes, and only the low-power periodic scanning function of the millimeter-wave radar front-end module is maintained.

[0038] Furthermore, the wireless communication unit adopts a dual-mode design, integrating a low-power Bluetooth and a fourth-generation mobile communication technology chip;

[0039] Bluetooth Low Energy is used to maintain a constant connection with the guardian's mobile device, transmitting real-time vital sign summary data and receiving control commands;

[0040] Fourth-generation mobile communication technology chips are used to establish a secure transmission link directly with the cloud server when there is no Bluetooth connection or when a large amount of data needs to be uploaded, ensuring reliable data upload and remote access.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] 1. This invention, by introducing a multimodal vibration sensing and modeling module, for the first time constructs a precisely quantified vehicle vibration and noise reference model at the system level, which is identical to radar observations. This design transforms the complex motion interference, which is difficult to handle and superimposed in radar echoes, into known or partially known reference quantities that can be independently observed and mathematically modeled, laying a crucial physical foundation for subsequent signal separation. Compared to traditional solutions that attempt to blindly separate noise from a single radar signal using algorithms, this invention changes the problem paradigm, shifting from blind estimation to precise cancellation with reference assistance, greatly improving the targeting and effectiveness of noise suppression.

[0043] 2. Based on the aforementioned vibration noise reference model, this invention designs an adaptive joint noise cancellation and signal separation module, which innovatively employs a two-stage cascaded processing architecture of adaptive filtering for initial noise reduction and blind source separation for fine extraction. The adaptive filter can quickly track and cancel the main vibration noise components that are strongly correlated with the reference model. Subsequently, the blind source separation post-processing unit, as a second line of defense, can further separate nonlinear interference components that may remain after the initial filtering and are not fully correlated with the reference model, and finally separate the heartbeat and respiratory signals from each other. This cascaded processing mechanism realizes a progressive extraction process from strong background noise to weak physiological signals, ensuring extremely high robustness and accuracy of vital sign signal extraction in complex dynamic environments.

[0044] 3. This invention constructs a complete closed-loop system from physical perception and signal processing to intelligent interaction. The vital sign parameter calculation and status assessment module not only provides raw data but also outputs clinically valuable health status assessment results through time-frequency analysis and intelligent judgment. The audio-visual interaction and intelligent early warning module executes a tiered response strategy from daily interaction to emergency warning based on these assessment results. This system-level integration allows this invention to transcend the scope of a single monitoring tool, becoming a comprehensive childcare solution integrating real-time monitoring, early warning, and remote communication, significantly enhancing the practical value and safety assurance capabilities of intelligent child strollers. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the overall technical solution architecture of the child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave proposed in this invention;

[0046] Figure 2 This is a schematic diagram of the core principle framework of the adaptive joint noise cancellation and signal separation module in this invention;

[0047] Figure 3 This is a logical flowchart of the multimodal vibration sensing and modeling module in this invention;

[0048] Figure 4 This is a flowchart of the logic flow of the vital signs parameter calculation and status assessment module in this invention.

[0049] Figure 5 This is a schematic diagram of the multi-level interaction relationship and data flow of the audio-visual interaction and intelligent early warning module in this invention. Detailed Implementation

[0050] This invention provides a child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave.

[0051] Please refer to the attached document. Figures 1 to 5 The system is physically integrated into the stroller's body structure, forming a complete closed-loop technology system from physical layer perception, signal layer processing to application layer interaction.

[0052] The system aims to solve the core technical problem of multi-dimensional vibrations of children's strollers caused by factors such as road bumps, changes in pushing speed, and user operation in real-world usage scenarios. These vibrations and noises, along with the physiological micro-motion signals generated by the child's weak chest wall undulations and heartbeat, are severely superimposed in the millimeter-wave radar echo, causing the monitoring accuracy of traditional single radar solutions to fail.

[0053] The implementation of this system introduces independent multi-dimensional vehicle vibration perception and modeling, generates a noise reference with the same source as radar observation, and adopts an advanced two-level cascaded signal processing architecture to achieve high-fidelity extraction of physiological micro-motion signals from a strong noise background, and finally completes vital sign calculation, state assessment and intelligent response.

[0054] The entire system's hardware deployment uses the stroller's body as a platform.

[0055] The millimeter-wave radar front-end module is precisely integrated into the sandwich structure inside the backrest of the vehicle body. Its antenna beam direction is strictly calibrated before leaving the factory to ensure that, under standard usage posture, the radar beam energy is concentrated to cover the chest and back area of ​​the child sitting or lying in the stroller.

[0056] The core sensors of the multimodal vibration sensing and modeling module, namely the high-precision six-axis inertial measurement unit and the micro-strain sensor array, are fixedly installed at the center of the vehicle chassis and on the shock-absorbing suspension components connecting the wheels and the frame, respectively, to directly sense the overall kinematic state and structural deformation of the vehicle body.

[0057] The core processors and logic controllers of the adaptive joint noise cancellation and signal separation module, the vital sign parameter calculation and status assessment module, and the audio-visual interaction and intelligent early warning module are all integrated into a main control box with waterproof and shockproof characteristics. This main control box is usually installed near the storage basket under the trolley and is connected to the above-mentioned sensors and execution units through internal wiring harnesses.

[0058] The integrated camera and array microphone of the audio-visual interaction and intelligent early warning module are installed at the front of the support rod of the stroller's sunshade, facing the child's face; the speaker is installed near the handrail of the stroller.

[0059] The antenna of the wireless communication unit is integrated within the main control box housing. The system is powered by a rechargeable lithium battery pack built into the trolley and regulated by the aforementioned multi-level power management framework.

[0060] The specific implementation of the millimeter-wave radar front-end module is as follows.

[0061] This module adopts a multi-input multi-output antenna array structure with one transmitter and two receivers.

[0062] The antenna array consists of three patch antenna elements, one of which is a transmitting antenna and the other two are receiving antennas, arranged in a one-dimensional linear pattern.

[0063] The antenna array and its associated radio frequency circuits are encapsulated together in a metal shielded cavity, which is then embedded in the filling material of the stroller backrest, leaving only a radiation window facing the child. The window is made of a specific engineering plastic that is transparent to 60GHz electromagnetic waves.

[0064] This integration method ensures both effective radar beam radiation and reception, while also achieving physical concealment and protection of the module. The core radio frequency chip of the module is an integrated transceiver chip that supports the 60GHz frequency band.

[0065] The chip integrates key circuits such as a voltage-controlled oscillator, a power amplifier, a low-noise amplifier, a mixer, and an intermediate frequency amplifier.

[0066] The phase-locked loop frequency synthesizer provides a highly stable frequency reference for the voltage-controlled oscillator, ensuring that the generated continuous wave radar signal has a pure frequency and extremely low phase noise.

[0067] In terms of specific operating parameters, the center frequency of the continuous wave radar signal radiated by the transmitting antenna is 60 GHz. The signal bandwidth is configured to 4 GHz, which is achieved by linear frequency modulation of a voltage-controlled oscillator controlled by a phase-locked loop frequency synthesizer or by using other broadband modulation methods.

[0068] The 4GHz bandwidth provides the system with extremely high distance resolution, enabling it to distinguish echoes from different depths of reflection points on the surface of a child's body, and theoretically possessing sub-millimeter level detection sensitivity for micrometer-level displacement changes.

[0069] The power of the transmitted signal is precisely set to 10dBm by the gain control of the power amplifier inside the chip.

[0070] This power value represents an engineering balance between meeting the signal-to-noise ratio requirements for effective detection of minor chest wall movements in children at distances of logarithmic meters and strictly adhering to international and regional electromagnetic radiation exposure safety standards.

[0071] After the signal is radiated by the transmitting antenna, it is reflected when it encounters the child's body, clothing, and other objects inside the stroller.

[0072] Two receiving antennas receive these reflected echoes respectively.

[0073] The received radio frequency echo signal is first amplified by a low-noise amplifier inside the chip to suppress the noise effects of subsequent circuits.

[0074] It is then mixed with the local oscillator signal from the transmission channel and down-converted to obtain a baseband signal containing Doppler frequency shift and phase modulation information;

[0075] The baseband signal is then conditioned by a programmable gain intermediate frequency amplifier and finally converted into a digital signal by an analog-to-digital converter, which is the original baseband signal containing mixed information of physiological micro-motion and vehicle vibration.

[0076] The two receiving channels provide initial spatial diversity capability, which helps to distinguish signal components from different directions in subsequent processing.

[0077] The specific implementation of the multimodal vibration sensing and modeling module is as follows.

[0078] The core task of this module is to quantify the vibration state of the vehicle body itself directly from a physical level, independent of the radar system, and to construct a mathematical model that can equivalently describe the interference of the vibration on the radar echo at the signal level, namely the vibration noise reference model.

[0079] Please refer to the attached document. Figure 3 The module consists of three main parts: a high-precision six-axis inertial measurement unit, a micro-strain sensor array, and a vibration and noise modeling processor.

[0080] The high-precision six-axis inertial measurement unit is a chip that integrates a three-axis microelectromechanical system accelerometer and a three-axis microelectromechanical system gyroscope.

[0081] This unit synchronously acquires linear acceleration and angular velocity data of the vehicle body in three-dimensional space at a fixed sampling rate of 1000Hz.

[0082] Acceleration data reflects the intensity and frequency of translational vibrations of the vehicle body in the front-to-back, left-to-right, and up-to-down directions, and the unit is usually m / s². 2 .

[0083] Angular velocity data reflects the intensity and frequency of the vehicle's rotational swaying around these three axes, and is typically expressed in rad / s. These data directly characterize the six-degree-of-freedom motion of the vehicle as a rigid body in space.

[0084] The micro-strain sensor array consists of multiple resistance strain gauges or fiber optic grating sensors, which are attached or embedded in key load-bearing and easily deformable parts of the stroller in a specific topology, especially near the shock-absorbing springs or shock-absorbing rubber sleeves connecting the wheels and the frame.

[0085] When the vehicle travels over a bumpy road, the impact force is transmitted through the wheels to the suspension system, causing the shock absorbers to compress and rebound, which in turn leads to slight elastic deformation of the local structure of the vehicle.

[0086] Micro-strain sensors can detect the surface strain of materials caused by this deformation and convert it into changes in resistance or light wavelength. After processing by conditioning circuitry, they output a voltage signal that is proportional to the deformation.

[0087] This signal reflects the changes in internal stress caused by non-rigid vibration of the vehicle body structure, and these changes are closely related to the high-frequency vibration modes of the vehicle body.

[0088] The vibration and noise modeling processor receives a six-dimensional motion data stream from a high-precision six-axis inertial measurement unit and a multi-channel strain data stream from a micro-strain sensor array.

[0089] Its primary task is to perform timestamp alignment and sampling rate normalization on these heterogeneous data to ensure that all data are strictly synchronized on the timeline. Subsequently, the processor executes the algorithm for constructing the vibration noise reference model.

[0090] The physical basis of this algorithm is a pre-defined vibration transfer function.

[0091] This transfer function was obtained through calibration experiments before the system left the factory:

[0092] In a laboratory environment, multidimensional vibrations of different frequencies and amplitudes are simulated using a vibration table. Data from a high-precision six-axis inertial measurement unit and a micro-strain sensor array are recorded simultaneously, along with pure vibration noise echo data collected by a millimeter-wave radar front-end module when there are no inanimate targets.

[0093] By using a systematic identification method, a mapping model is established from vehicle motion and strain data to radar echo phase noise or Doppler spectrum, i.e., vibration transfer function.

[0094] In actual operation, the vibration and noise modeling processor inputs the vehicle body acceleration, angular velocity and strain data collected at the current moment into the vibration transfer function in real time.

[0095] The function calculates and outputs either the predicted phase modulation sequence or the predicted Doppler shift power spectral density.

[0096] The output of this prediction is the vibration noise reference model.

[0097] Essentially, it is a mathematical estimate of the interference component in the echo signal received by millimeter-wave radar under the current vehicle vibration state, which is purely caused by vehicle vibration.

[0098] Since the data source of this model is the same as the physical disturbance source observed by radar, it is highly correlated with the vibration and noise components in the actual radar echo, providing key prior reference information for subsequent noise cancellation.

[0099] The adaptive joint noise cancellation and signal separation module is the core of the signal processing in this system. Please refer to the appendix. Figure 2 This module receives two inputs:

[0100] One path is the raw baseband signal from the millimeter-wave radar front-end module, and the other path is the vibration noise reference model from the multimodal vibration sensing and modeling module.

[0101] The module contains a signal alignment unit, a reference noise reconstruction unit, an adaptive filter bank, and a blind source separation post-processing unit, which work in series.

[0102] The signal alignment unit is responsible for solving the synchronization problem between input signals.

[0103] Although each module uses a unified system clock source, due to differences in signal path delay, filter group delay, and data processing time, there may be a time delay difference of several sampling points between the original baseband signal and the vibration and noise reference model when they arrive at the unit.

[0104] This unit calculates the cross-correlation function of the two signals, finds the point with the largest cross-correlation value, thereby determining the precise time delay between them, and performs corresponding digital delay compensation on one of the signals. Furthermore, if the sampling rates of the two signals differ due to different front-end designs, this unit also uses interpolation or decimation algorithms to match the sampling rates, ensuring that data points correspond one-to-one in subsequent processing.

[0105] The reference noise reconstruction unit receives the vibration noise reference model after time synchronization.

[0106] Since the vibration noise reference model is a linear or nonlinear transformation output of the vibration transfer function onto the vehicle motion data, it may only be a simplified or partial description of the actual vibration noise in the radar echo.

[0107] Therefore, this unit further processes the input reference model through a nonlinear transformer.

[0108] This nonlinear transformer can be a pre-trained neural network model or a set of empirical nonlinear functions.

[0109] The aim is to make the reconstructed estimated signal more closely approximate the actual vibration and noise components in the original radar baseband signal in terms of time-domain waveform, frequency-domain characteristics, and even higher-order statistical properties.

[0110] The signal output by this unit is called the vibration noise estimation signal.

[0111] The adaptive filter bank is a key component for initial noise cancellation. It employs an improved normalized least mean square algorithm.

[0112] The core idea of ​​this algorithm is:

[0113] The vibration noise estimation signal output by the reference noise reconstruction unit is used as the reference input of the filter, and the original baseband signal output by the millimeter-wave radar front-end module is used as the desired response.

[0114] The filter dynamically adjusts its weight coefficients to make its output as similar as possible to the vibration noise estimation signal. Then, the output of this filter is subtracted from the original baseband signal, and the resulting difference signal is the error signal, which is the initially separated physiological micro-motion signal.

[0115] The improved normalized least mean square algorithm updates the filter weight vector in real time in the following way.

[0116] Suppose the vibration noise estimation signal is at time 10:00. The vector formed is The filter weight vector is The original baseband signal is Then the filter output Error signal The weight update formula is:

[0117] ;

[0118] in, Indicates transpose, step factor Adaptive adjustment is performed based on the cross-correlation matrix between the vibration noise estimation signal and the original baseband signal, as well as the short-term estimate of the autocorrelation matrix of the vibration noise estimation signal. It is a very small positive number, used to avoid the denominator approaching 0, which would lead to unstable calculations.

[0119] The improvement lies in that the algorithm calculates the cross-correlation matrix between the vibration noise estimation signal and the original baseband signal in real time, as well as the short-term estimate of the autocorrelation matrix of the vibration noise estimation signal, and uses this matrix information to adjust the step size factor. Adaptive adjustments are made: when the signal is stable, a larger step size is used to accelerate convergence; when the signal changes abruptly, the step size is reduced to maintain stability.

[0120] Through this continuous adaptive adjustment, the filter can track the slow changes in the statistical characteristics of vehicle vibration noise, thereby effectively subtracting the main vibration noise components that are strongly correlated with the reference model from the original baseband signal and outputting preliminary physiological micro-motion signals.

[0121] This preliminary signal may still contain residual vibration interference that is not fully correlated with the reference model, as well as a mixture of heartbeat and respiratory signals.

[0122] The blind source separation post-processing unit receives the preliminary physiological micro-motion signal output by the adaptive filter bank.

[0123] This signal typically contains data from two receiving channels, forming a two-dimensional observation vector.

[0124] The task of this unit is to further separate the independent source components in these observed signals, particularly to separate potentially residual vibration-related nonlinear interferences, as well as heartbeat and respiratory source signals. The unit employs an independent component analysis algorithm to achieve this goal.

[0125] The specific process is as follows: assuming the observed two-dimensional signal vector It consists of several statistically independent source signals Linear mixture, i.e. ,in It is an unknown mixture matrix. The goal of independent component analysis is to find the separation matrix. This makes the output Each component Statistically independent as much as possible between them, thus That is, the source signal The estimate.

[0126] The algorithm iteratively optimizes the separation matrix. This is achieved by minimizing the mutual information between the components of the output signal, or equivalently maximizing its non-Gaussianity. Commonly used optimization methods include fixed-point algorithms based on maximizing negative entropy.

[0127] In each iteration, the algorithm updates the separation matrix. And calculate the new output. Through multiple iterations, the algorithm converges when the independence metric of the output signal components reaches its extreme value.

[0128] At this point, the output It typically contains three main components: the component corresponds to residual, vibration-related broadband disturbances, which have a dispersed energy distribution and are temporally correlated with vehicle vibration events;

[0129] The other two components correspond to the distinctly periodic heartbeat signal and the lower-frequency respiratory signal, respectively.

[0130] The blind source separation post-processing unit sends the separated heartbeat source signal and respiratory source signal as the final high-fidelity output to the next module.

[0131] The vital signs parameter calculation and status assessment module is responsible for extracting specific vital sign values ​​from pure physiological source signals and assessing health status. Please refer to the appendix. Figure 4 This module includes a time-frequency analysis unit, a peak detection and tracking unit, and a state decision unit.

[0132] The time-frequency analysis unit performs time-frequency transformation on the input heartbeat source signal and respiratory source signal respectively to reveal the variation of their frequency components over time.

[0133] This implementation uses continuous wavelet transform because it provides a good balance between time and frequency resolution, making it particularly suitable for analyzing non-stationary physiological signals.

[0134] For the heartbeat source signal, wavelet basis functions with a center frequency between 1Hz and 3Hz were selected for analysis, which covers the possible heart rate range of infants and young children.

[0135] For respiratory source signals, wavelet basis functions with center frequencies between 0.2 Hz and 1 Hz are selected.

[0136] By performing continuous wavelet transforms on the signals, time-spectrum diagrams, also known as scale diagrams, are generated for the two signals. On the time-spectrum diagram, the signal intensity is represented by the intensity of the color, with the horizontal axis representing time and the vertical axis representing the scale or corresponding frequency.

[0137] The peak detection and tracking unit operates on the generated time-spectrum graph. For the heartbeat time-spectrum graph, the algorithm finds the frequency point with the strongest energy in each time slice within the range corresponding to the heart rate frequency band. However, due to noise or instantaneous fluctuations in the signal, these strongest frequency points may jump between different time slices.

[0138] To obtain a smooth and continuous heart rate curve, this unit applies an optimal path search algorithm based on dynamic programming. This algorithm treats the time-spectrum graph as a grid, with each grid point having an energy value. Starting from the first time slice, the algorithm searches for a path that traverses the entire time axis, maximizing the sum of the energy of all grid points along the path, while simultaneously ensuring the continuity of heart rate physiological changes along the frequency axis.

[0139] The optimal path ultimately found is the heart rate ridge. Based on the instantaneous frequency corresponding to the ridge... It can calculate the instantaneous heart rate value. The time interval between adjacent heartbeat cycles, i.e., the inter-heartbeat sequence, can also be extracted from the zero-crossing points or feature points of the ridge.

[0140] For the respiratory spectrogram, the exact same dynamic programming path search algorithm is used to trace the dominant respiratory ridge within the respiratory band and calculate the instantaneous respiratory rate. And respiratory cycle sequence, This is the instantaneous frequency corresponding to the dominant respiratory ridge.

[0141] The status decision unit receives real-time heart rate and respiratory rate data streams from the peak detection and tracking unit.

[0142] Its internal maintenance sliding time window is set to a length of 30 seconds.

[0143] For each heart rate and respiratory rate data point within the window, the decision maker calculates a series of statistical characteristics, including the mean, standard deviation, and trend of variation over the past 30 seconds.

[0144] The trend of variation can be quantified by calculating the root mean square of the difference between adjacent periods or by performing a linear fit.

[0145] These statistical characteristics reflect the short-term stability and regularity of children's physical signs.

[0146] Meanwhile, the state decision maker has a built-in dynamic threshold model.

[0147] The model is not a fixed value, but is pre-established based on big data analysis of physiological data of infants and young children at different ages.

[0148] For example, for infants aged 0 to 3 months, the normal resting heart rate range may be between 120 and 160 beats per minute, and the respiratory rate between 30 and 60 breaths per minute; while for toddlers over 1 year old, the range is different.

[0149] Threshold models not only define the upper and lower limits of heart rate and respiratory rate, but may also include the normal range for heart rate variability and respiratory variability.

[0150] The state decision-maker compares the statistical features calculated in real time with the dynamic threshold model corresponding to the child's current age in months.

[0151] The comparison strategy employs a multi-cycle continuous decision-making mechanism. The system sets a basic detection cycle, for example, 10 seconds.

[0152] The decision-maker performs feature extraction and threshold comparison on the data within the sliding window every 10 seconds.

[0153] The decision-maker determines that a certain vital sign is in an abnormal state only when the feature value of a certain vital sign parameter exceeds its corresponding threshold range for three consecutive detection cycles, and generates an abnormal state identifier containing the abnormality type, abnormality level and timestamp.

[0154] Otherwise, the system generates a normal status indicator. This mechanism effectively avoids false alarms caused by momentary interference or brief activity by children.

[0155] The audio / video interaction and intelligent early warning module is the terminal for this system to interact with users and the external environment. Please refer to the appendix. Figure 5 The module includes an integrated camera, array microphone, speaker, interactive logic controller, and wireless communication unit.

[0156] The integrated camera uses a low-light, wide dynamic range image sensor, paired with an autofocus lens. It primarily operates in two modes:

[0157] Firstly, during periods of non-continuous vital sign monitoring, such as when the system is in intermittent scanning or standby mode, images can be captured periodically for recording.

[0158] Secondly, upon receiving an interactive command, video stream acquisition is initiated. The array microphone consists of two or more microphone elements, utilizing beamforming technology to enhance the ability to pick up sounds from the direction of the child's mouth, while suppressing ambient background noise, thus clearly capturing sounds such as the child's crying and babbling.

[0159] The interaction logic controller is the brain of the module. It receives state evaluation results from the vital sign parameter calculation and state assessment module and drives other components to work according to a preset multi-level strategy. The strategy is gradient-based:

[0160] Level 1: Normal interaction.

[0161] When the status assessment result is normal, the controller is in standby mode.

[0162] At this time, it can respond to requests from the remote monitoring application software via the wireless communication unit and initiate two-way audio and video call functionality.

[0163] The camera and array microphone are turned on to collect local audio and video streams, which are then encoded and sent to the monitoring terminal via the wireless communication unit.

[0164] At the same time, it receives audio streams from the monitoring terminal and plays them through a speaker to enable remote monitoring and interaction.

[0165] The second level is intelligent soothing.

[0166] When the array microphone detects an acoustic pattern that matches the characteristics of an infant's crying, and the status assessment result output by the vital signs parameter calculation and status assessment module is normal, the interactive logic controller infers that the child may be crying due to unhealthy reasons such as hunger or drowsiness.

[0167] At this point, the controller automatically selects a preset soothing audio from local storage, such as white noise, soft music, or a recording of the mother, and plays it in a loop through the speaker to try to calm the child.

[0168] Level 3, graded early warning.

[0169] When the status assessment result is abnormal, the controller initiates a graded early warning process based on the level of abnormality.

[0170] The anomaly level is determined by the status decision unit based on the degree and persistence of the deviation of the vital signs from the threshold. Primary anomalies may trigger local alerts:

[0171] The controller controls the speaker to play a soft but continuous alert tone to alert nearby caregivers. If the anomaly persists or escalates to a medium-level anomaly, the controller immediately pushes a warning notification containing a brief information about the anomaly to the paired caregiver's mobile terminal via the Bluetooth Low Energy link of the wireless communication unit.

[0172] If an advanced anomaly occurs or the Bluetooth connection is unavailable, the controller activates the fourth-generation mobile communication technology chip of the wireless communication unit to establish a secure transmission link with the cloud monitoring platform. The data packet, which includes a complete anomaly identifier, high-fidelity heart rate and respiratory signal segments, trends of vital signs parameters 30 seconds before and after the anomaly, and possibly short video clips recorded simultaneously, is encrypted and uploaded to the cloud.

[0173] The cloud platform can then send alerts to multiple emergency contacts.

[0174] The wireless communication unit employs a dual-mode design to balance power consumption and reliability. A Bluetooth Low Energy chip maintains a persistent connection with a dedicated application on the caregiver's smartphone. This connection is primarily used for transmitting periodic vital sign summary data.

[0175] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0176] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave, characterized in that, include: The millimeter-wave radar front-end module is integrated inside the stroller body. It is used to transmit continuous wave radar signals in the 60GHz band and receive echo signals reflected by the child's body, and output raw baseband signals containing mixed information of physiological micro-movements and stroller vibrations. The multimodal vibration sensing and modeling module is used to collect and quantify the multidimensional vibration state of the vehicle body in real time, and to build a vibration noise reference model that is of the same origin as the radar signal. The adaptive joint noise cancellation and signal separation module is used to receive the raw baseband signal from the millimeter-wave radar front-end module and the vibration noise reference model from the multimodal vibration sensing and modeling module. By performing joint time-frequency domain analysis and adaptive filtering, it separates the pure physiological micro-motion signal component from the raw baseband signal. The vital signs parameter calculation and status assessment module is used to perform time-frequency analysis and feature extraction on the isolated pure physiological micro-motion signals, calculate the real-time heart rate and respiratory rate values, and assess the child's vital signs health status based on a preset threshold model, generating status assessment results. The audio-visual interaction and intelligent early warning module integrates a camera and a speaker. Based on the status assessment results output by the vital signs parameter calculation and status assessment module, it executes corresponding audio-visual interaction strategies or triggers graded early warning signals, and uploads the data to the cloud monitoring platform.

2. The child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave as described in claim 1, characterized in that, The millimeter-wave radar front-end module adopts a MIMO antenna array structure with one transmitter and two receivers. The antenna array is encapsulated inside the backrest of the vehicle body, and its beam direction is precisely calibrated and focused on the chest and back area of ​​the child. The millimeter-wave radar front-end module includes a 60GHz radio frequency transceiver chip, a phase-locked loop frequency synthesizer, and a baseband analog front-end.

3. The child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave as described in claim 2, characterized in that, The multimodal vibration sensing and modeling module includes a high-precision six-axis inertial measurement unit, a micro-strain sensor array installed on the vehicle body's shock absorption suspension, and a vibration noise modeling processor. The high-precision six-axis inertial measurement unit measures the linear acceleration and angular velocity of the vehicle body in three-dimensional space in real time at a sampling rate of 1000Hz. The micro-strain sensor array is used to sense the deformation stress of the vehicle body structure caused by bumps; the vibration noise modeling processor is used to fuse inertial measurement data and strain data, and calculate the equivalent phase modulation sequence and Doppler frequency shift spectrum of the vehicle body vibration under the observation view of millimeter-wave radar through a preset vibration transfer function, thereby generating a vibration noise reference model.

4. The child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave as described in claim 3, characterized in that, The adaptive joint noise cancellation and signal separation module includes a signal alignment unit, a reference noise reconstruction unit, and an adaptive filter bank. The signal alignment unit is used to perform precise time synchronization and sampling rate matching between the input raw baseband signal and the vibration noise reference model; The reference noise reconstruction unit, based on the aligned vibration noise reference model, generates an estimated signal that can approximate the actual vibration noise in the radar signal in the time and frequency domain through a nonlinear transformer. The adaptive filter bank employs an improved normalized least mean square algorithm. The core of this algorithm is to dynamically calculate the cross-correlation matrix and autocorrelation matrix between the estimated signal and the original baseband signal, and update the weight coefficients of the filter in real time. This allows the estimated vibration noise component to be subtracted from the original baseband signal, and the preliminarily separated physiological micro-motion signal to be output.

5. The child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave according to claim 4, characterized in that, The adaptive joint noise cancellation and signal separation module also includes a blind source separation post-processing unit; The blind source separation post-processing unit performs independent component analysis on the preliminary physiological micro-motion signal output by the adaptive filter bank; The specific process involves constructing an observation vector from multiple channels of preliminary physiological micro-motion signals, and assuming that it is a linear mixture of statistically independent source signals. By iteratively optimizing the separation matrix, the mutual information between the components of the output signal is minimized, thereby further separating out any residual interference that may remain and the independent source signals corresponding to heartbeat and respiration, respectively, and finally outputting high-fidelity heartbeat source signals and respiration source signals.

6. The child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave according to claim 5, characterized in that, The vital signs parameter calculation and status assessment module includes a time-frequency analysis unit, a peak detection and tracking unit, and a status decision unit; the time-frequency analysis unit performs continuous wavelet transform on the input heartbeat source signal and respiratory source signal respectively to generate the time-frequency spectrum of the signal; The peak detection and tracking unit applies an optimal path search algorithm based on dynamic programming to track the dominant frequency ridge line within the frequency band corresponding to heartbeat and respiration on the time-spectrum graph, and calculates the continuous heartbeat interval sequence and respiratory cycle sequence based on the instantaneous frequency of the ridge line. The state decision-maker has a built-in sliding time window, which is used to calculate the average, standard deviation and variation trend of heart rate and respiratory rate over the past time, and compares these statistical characteristics with a dynamic threshold model established based on physiological data of children of different ages. An abnormal status identifier is generated when the statistical feature exceeds the threshold range for three consecutive detection cycles; otherwise, a normal status identifier is generated.

7. The child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave as described in claim 6, characterized in that, The audio-visual interaction and intelligent early warning module includes an integrated camera, array microphone, speaker, interaction logic controller, and wireless communication unit. The integrated camera is used to capture video of the child's facial expressions and surrounding environment during non-monitoring periods or when an interactive command is received; The array microphones are used to collect ambient sounds and the cries of children; the interactive logic controller executes a preset strategy based on the received state evaluation results; When the status assessment result is normal, the controller responds to the request from the remote monitoring terminal and starts the audio and video call function; When a child's crying is detected and their vital signs are normal, the controller automatically plays a preset soothing audio. When the status assessment result is abnormal, the controller will, according to the level of abnormality, play a local warning sound through the speaker, push a warning message to the bound mobile monitoring terminal, and upload the complete abnormal data packet and video recordings of the preceding and following time periods to the cloud monitoring platform through the wireless communication unit.

8. The child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave according to claim 7, characterized in that, The system operates under a multi-level power management framework, which includes a continuous monitoring mode, an intermittent scanning mode, and a standby mode. The continuous monitoring mode is activated when the trolley is stationary or moving at a constant speed and smooth motion, and all modules operate at full power. The intermittent scanning mode automatically switches to this mode when the system detects continuous and severe vibration. The vital signs parameter calculation and status assessment module reduces the data update rate to 1 time / s, and the audio and video interaction and intelligent early warning module turns off the camera. The standby mode is activated when there are no signs of life from the child inside the stroller for more than 5 minutes, and only maintains the low-power periodic scanning function of the millimeter-wave radar front-end module.

9. A child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave as described in claim 8, characterized in that, The wireless communication unit adopts a dual-mode design, integrating a low-power Bluetooth and a fourth-generation mobile communication technology chip. The Bluetooth Low Energy is used to maintain a constant connection with the guardian's mobile terminal, transmit real-time vital sign summary data and receive control commands; The fourth-generation mobile communication technology chip is used to establish a secure transmission link directly with the cloud server when there is no Bluetooth connection or when a large amount of data needs to be uploaded.

10. A child stroller vital sign monitoring and audio-visual interaction system based on 60G millimeter wave as described in claim 9, characterized in that, The weight update process of the improved normalized least mean square algorithm is as follows: Suppose the vibration noise estimation signal is at time 10:

00. The vector formed is The filter weight vector is The original baseband signal is Then the filter output Error signal The weight update formula is: ； in, Indicates transpose, step factor Adaptive adjustment is performed based on the cross-correlation matrix between the vibration noise estimation signal and the original baseband signal, as well as the short-term estimate of the autocorrelation matrix of the vibration noise estimation signal. It is a very small positive number, used to avoid the denominator approaching 0, which would lead to unstable calculations.

Images