Self-adaptive sleep assisting method, system and equipment based on multi-modal perception and medium

Abstract

The invention relates to a self-adaptive sleep assisting method, system and device based on multi-mode perception and a medium. The method comprises the steps of performing seasonal correction based on latitude and longitude coordinates, constructing a dynamic rhythm curve, and obtaining user portrait data and real-time multi-source data; performing Kalman filtering fusion on the respiratory frequency signal and the vibration spectrum to generate a fused sleep staging result, performing classification processing on the original audio data, and generating a noise type label and duration; on the basis of the dynamic rhythm curve and a preset adjustment rate, calculating an upward light real-time parameter, and on the basis of the fused sleep staging result, the noise type label and the user portrait data, generating a downward light real-time parameter; and inputting the real-time parameters of the upper light and the real-time parameters of the lower light into a light field coupling model to calculate a light field distribution target value, and generating a light source driving instruction. According to the method, through multi-modal data perception, rhythm curve construction and light field cooperative regulation and control, the sleep staging precision, the light intervention suitability and the sleep assisting effect can be improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent lighting, and particularly relates to a self-adaptive sleep assistance method, system, device and medium based on multi-modal perception. BACKGROUND

[0002] Although the existing sleep assistance lighting system and related patents (such as CN112954895A) attempt to optimize sleep quality through lighting regulation, there are still obvious deficiencies, which are difficult to meet the actual needs. On the one hand, the traditional sleep assistance lighting scheme generally has the problems of single perception dimension and insufficient sleep staging accuracy. For example, the CN112954895A patent only relies on an accelerometer to detect body motion signals, which cannot capture deep physiological indicators such as heart rate variability and respiratory rhythm disorder, which can easily lead to misjudgment such as incorrect judgment of shallow sleep turning over and wake-up actions, so that the subsequent light intervention lacks accurate physiological basis and cannot adapt to the actual needs of different sleep stages. On the other hand, the traditional sleep assistance lighting scheme can only realize timed light adjustment, and does not establish coupling logic with the external environment. For example, the scheme does not correlate environmental noise interference with sleep, cannot perform targeted light adjustment for noise interference, and does not automatically generate a circadian rhythm curve based on latitude and longitude, but requires the user to manually set the biological clock, which has poor adaptability and is difficult to meet the rhythm synchronization needs of different regions and seasons. SUMMARY

[0003] Therefore, it is necessary to provide a self-adaptive sleep assistance method, system, device and medium based on multi-modal perception to improve the accuracy of sleep staging and the adaptability of light field regulation to meet the individual needs of users.

[0004] In a first aspect, the application provides a self-adaptive sleep assistance method based on multi-modal perception, comprising:

[0005] Obtaining latitude and longitude coordinates, performing seasonal correction processing based on the latitude and longitude coordinates, constructing a dynamic rhythm curve, and obtaining user portrait data and real-time multi-source data, the real-time multi-source data including physiological signals, vibration spectrum and original audio data, wherein the physiological signals include respiratory frequency signals and heart rate variability signals;

[0006] Performing Kalman filter fusion processing on the respiratory frequency signals and the vibration spectrum to generate a fusion sleep staging result, and performing classification processing on the original audio data to generate a noise type label and a duration;

[0007] Based on the dynamic rhythm curve and a preset adjustment rate, real-time parameters of the upper light are calculated, and based on the fusion sleep staging result, the noise type label and the user portrait data, the real-time parameters of the lower light are calculated through a preset physiological-light effect mapping model to generate the real-time parameters of the lower light;

[0008] The real-time parameters of the up-beam and down-beam are input into the optical field coupling model to calculate the target value of the optical field distribution. Based on the target value of the optical field distribution, a light source driving command is generated. The light source driving command is used to control the light source and construct an adaptive optical field for sleep assistance.

[0009] In one embodiment, latitude and longitude coordinates are obtained, seasonal correction processing is performed based on the latitude and longitude coordinates, and a dynamic rhythm curve is constructed, including:

[0010] The latitude and longitude coordinates are obtained through the GPS positioning module, and the reference sunrise and sunset times for the current region are determined based on the latitude and longitude coordinates.

[0011] Obtain the current season information, perform seasonal offset determination based on the current season information, and obtain the adjusted sunrise time and adjusted sunset time;

[0012] Based on the adjusted sunrise time, adjusted sunset time, and preset time period division rules, multiple time intervals are determined, and corresponding target color temperature and target luminous flux are configured for each time interval. The time intervals include the morning wake-up transition interval, the daytime stable interval, the evening transition interval, and the nighttime deep sleep interval.

[0013] Based on the target color temperature, target luminous flux, and preset adjustment rate, a rhythm curve fitting process is performed to generate a dynamic rhythm curve.

[0014] The adjusted sunrise and sunset times are obtained through the following steps:

[0015] When the current season information is winter, the benchmark sunrise time is delayed by a first preset duration, and the benchmark sunset time is advanced by a second preset duration, so as to obtain the corresponding adjusted sunrise time and adjusted sunset time respectively.

[0016] When the current season information is summer, the benchmark sunrise time is advanced by a second preset duration, and the benchmark sunset time is delayed by a first preset duration, to obtain the corresponding adjusted sunrise and sunset times.

[0017] In one embodiment, the respiratory rate signal and vibration spectrum are fused using Kalman filtering to generate a fused sleep staging result, including:

[0018] The vibration spectrum is processed to extract the dominant frequency, thus obtaining the vibration dominant frequency.

[0019] The dominant vibration frequency is compared with a preset first frequency threshold and a preset second frequency threshold, respectively. A sleep period marker is constructed based on the comparison results, wherein the preset first frequency threshold is greater than the preset second frequency threshold. The sleep period marker is obtained through the following steps:

[0020] When the comparison result shows that the vibration dominant frequency is greater than the preset first frequency threshold, a light sleep stage marker is generated and used as a sleep stage marker.

[0021] When the vibration frequency is less than the preset second frequency threshold, a deep sleep stage marker is generated and used as a sleep stage marker.

[0022] When the vibration dominant frequency is between a preset first frequency threshold and a preset second frequency threshold, a transitional sleep period marker is generated and used as a sleep period marker.

[0023] The standard deviation of heart rate variability signal is calculated and processed to obtain the standard deviation of heart rate variability. The corresponding baseline standard deviation of heart rate variability and the weighting coefficient of sleep stage are calculated based on user profile data.

[0024] Based on the deviation rate between the standard deviation of heart rate variability and the benchmark standard deviation of heart rate variability, a weighted calculation is performed using the sleep stage weighting coefficient to generate the physiological fluctuation coefficient.

[0025] Initialize the state vector, error covariance matrix, and observation noise covariance matrix of the Kalman filter. The state vector consists of sleep stage state parameters, which include the probability of light sleep, deep sleep, and transitional sleep. The observation noise covariance matrix is ​​calibrated based on the classification error rate of the sleep stage labels.

[0026] Sleep phase markers are used as observation vectors, and physiological fluctuation coefficients are used as state transition weights. The observation vectors and state transition weights are input into a preset system dynamic model for prediction processing to generate state prediction values ​​and prediction error covariance. The state prediction values ​​include the probability of predicting light sleep, the probability of predicting deep sleep, and the probability of predicting transitional sleep.

[0027] The Kalman gain is calculated based on the prediction error covariance and the observation noise covariance matrix.

[0028] Based on Kalman gain, state prediction value and observation vector, observation update processing is performed to correct the state prediction value and prediction error covariance, and the current sleep stage state estimate and updated error covariance are obtained.

[0029] Spectral analysis was performed on the heart rate variability signal to extract the proportion of high-frequency components, and the REM phase determination index was calculated by combining it with the respiratory rate signal.

[0030] Based on the sleep stage and REM stage determination index corresponding to the maximum probability value in the current sleep stage state estimate, a fusion sleep stage result is generated. The fusion sleep stage result includes the transitional sleep stage result, light sleep stage result, deep sleep stage result, and REM sleep stage result of non-rapid eye movement sleep.

[0031] In one embodiment, based on the fusion of sleep staging results, noise type labels, and user profile data, downlight parameters are calculated using a preset physiological-optical effect mapping model to generate real-time downlight parameters, including:

[0032] Based on the results of the fusion sleep staging, the current sleep stage and the corresponding time interval are determined. Based on the time interval, the heart rate variability signal and respiratory rate signal corresponding to the time interval are called to calculate the corresponding changes in heart rate variability and respiratory rate, respectively.

[0033] The changes in heart rate variability and respiratory rate are input into a preset physiological-light effect mapping model for priority coefficient calculation, thereby generating light effect regulation priority coefficients.

[0034] The noise type labels are classified and identified to determine the noise category corresponding to the noise type label. The noise categories include continuous ambient noise, snoring, and sudden noise.

[0035] When the noise type label is continuous ambient noise, the upward light parameter correction amount is generated based on the loudness and duration of the continuous ambient noise. When the noise type label is snoring, red light pulse parameters are generated, including pulse frequency and pulse intensity.

[0036] When the noise type is labeled as burst noise, the instantaneous adjustment parameters of the downlight are generated based on the peak sound pressure level and duration of the burst noise. The instantaneous adjustment parameters of the downlight include the brightness drop amplitude and recovery rate.

[0037] Personalized adjustment parameters are extracted from user profile data. These parameters include target blue light percentage, target adjustment rate, and target brightness range. The target blue light percentage is configured based on the user's age group, and the target adjustment rate is configured based on the user's sleep type.

[0038] Based on the light effect adjustment priority coefficient, the up-sunlight parameter correction amount, the red light pulse parameter, the down-sunlight instantaneous adjustment parameter, and the personalized control parameter, the down-sunlight real-time parameter is generated.

[0039] In one embodiment, the real-time parameters of the upward-facing beam and the real-time parameters of the downward-facing beam are input into the optical field coupling model to calculate the target value of the optical field distribution, including:

[0040] Obtain the preset horizontal angle between the human eye and the light source, perform light perception intensity calculation based on the preset horizontal angle, and generate a light perception intensity coefficient;

[0041] The current color temperature value is extracted from the real-time parameters of the upward-sunlight, and the light field coordination coefficient is calculated based on the current color temperature value, the light perception intensity coefficient, and the preset weighting coefficient.

[0042] Based on the optical field coordination coefficient, the luminous flux value in the real-time parameters of the up-sunlight and the brightness value in the real-time parameters of the down-sunlight, the illuminance of the desktop target and the illuminance of the ground target are generated.

[0043] The target illuminance on the desktop, the target illuminance on the ground, and the current color temperature value are integrated to generate the target value of the light field distribution.

[0044] In one embodiment, the method further includes:

[0045] Real-time monitoring of fusion sleep staging results, and extraction of body movement frequency and respiratory parameters based on the original vibration spectrum and respiratory frequency signals corresponding to the fusion sleep staging results. The body movement frequency is calculated based on the frequency of change of the dominant frequency of the original vibration spectrum, and the respiratory parameters include respiratory frequency fluctuation value and apnea duration.

[0046] When the body movement frequency is greater than the first preset frequency threshold or the respiratory rate fluctuation value is greater than the first preset fluctuation threshold, and the corresponding first abnormal duration exceeds the first preset duration, a first-level intervention is performed. The brightness and color temperature values ​​in the downlight real-time parameters are reduced according to the preset first-level rules to obtain the first optimized downlight real-time parameters. The light source driving command is updated based on the first optimized downlight real-time parameters to obtain the first optimized light source driving command.

[0047] When the body movement frequency is greater than the second preset frequency threshold and the respiratory rate fluctuation value is greater than the second preset fluctuation threshold, and the corresponding second abnormal duration exceeds the second preset duration, a secondary intervention process is performed to generate a mild abnormality reminder message, push the mild abnormality reminder message to the associated terminal, and reduce the brightness value and color temperature value in the downlight real-time parameters according to the preset secondary rules to generate a second optimized downlight real-time parameter. Based on the second optimized downlight real-time parameter, the light source driving command is updated to obtain the second optimized light source driving command.

[0048] When the body movement frequency is greater than the third preset frequency threshold and the duration of apnea exceeds the preset apnea duration, and the corresponding third abnormal duration exceeds the third preset duration, a three-level intervention is performed to generate an emergency abnormality reminder message, push the emergency abnormality reminder message to the associated terminal, adjust the red light pulse control parameter in the downlight real-time parameter, generate the third optimized downlight real-time parameter, update the light source driving command based on the third optimized downlight real-time parameter, and obtain the third optimized light source driving command.

[0049] Among them, the first preset frequency threshold is less than the second preset frequency threshold, the second preset frequency threshold is less than the third preset frequency threshold, the first preset duration is less than the second preset duration, and the second preset duration is less than the third preset duration.

[0050] In one embodiment, the optical field coordination coefficient is calculated using the following formula:

[0051]

[0052] in, The optical field coordination coefficient, This is the color temperature weighting coefficient. The sleep stage adaptation coefficient. This is the current color temperature value. For spatial perception weighting coefficients, The light perception intensity coefficient. For user age, This is the preset horizontal angle.

[0053] Secondly, this application also provides an adaptive sleep assistance system based on multimodal perception, comprising:

[0054] The data acquisition and processing module is used to acquire latitude and longitude coordinates, perform seasonal correction processing based on latitude and longitude coordinates, construct dynamic rhythm curves, and acquire user profile data and real-time multi-source data. The real-time multi-source data includes physiological signals, vibration spectrum and raw audio data, among which physiological signals include respiratory rate signals and heart rate variability signals.

[0055] The data fusion and analysis module is used to perform Kalman filtering fusion processing on respiratory frequency signals and vibration spectra to generate fused sleep staging results, classify raw audio data, and generate noise type labels and durations.

[0056] The parameter calculation and mapping module is used to calculate the real-time parameters of uplight based on the dynamic rhythm curve and the preset adjustment rate, and to calculate the downlight parameters based on the fusion of sleep stage results, noise type labels and user profile data through the preset physiological-photodynamic effect mapping model to generate the real-time parameters of downlight.

[0057] The light field control module is used to input the real-time parameters of the upward and downward light into the light field coupling model to calculate the target value of the light field distribution, and generate light source driving instructions based on the target value of the light field distribution. The light source driving instructions are used to control the light source and construct an adaptive light field for sleep assistance.

[0058] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the first aspect.

[0059] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the first aspect.

[0060] The aforementioned adaptive sleep assistance method, system, device, and medium based on multimodal perception first constructs a dynamic circadian rhythm curve by acquiring latitude and longitude coordinates and performing seasonal corrections, while simultaneously acquiring real-time multi-source data from the user, providing a circadian rhythm benchmark and multi-dimensional data support for subsequent light regulation. Secondly, it uses Kalman filtering to fuse respiratory frequency signals and vibration spectra to generate sleep staging results, and classifies the original audio to generate noise labels and durations, effectively solving the problem of misjudgment of sleep staging caused by traditional single-motion signal perception, and clearly identifying the type of noise interference, providing precise physiological and environmental basis for light intervention. Furthermore, based on the dynamic circadian rhythm curve, it calculates upward light parameters, combines sleep staging, noise labels, and user profiles, and generates downward light parameters through a physiological-light effect mapping model, achieving differentiated adaptation of upward and downward light. This not only synchronizes with natural rhythms but also allows for precise regulation based on individual and environmental differences. Finally, the up and down light parameters are input into the light field coupling model to calculate the target value of the light field distribution and generate light source driving instructions. This can build an adaptive light field that meets sleep needs, reduce visual discomfort caused by sudden changes in light, significantly improve the accuracy and adaptability of sleep assistance, effectively optimize users' sleep quality, and meet the personalized sleep needs of different regions, seasons and individuals. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 A flowchart of an adaptive sleep assistance method based on multimodal perception is provided as an exemplary embodiment of the present invention;

[0063] Figure 2 A flowchart illustrating a method for obtaining a target value of an optical field distribution, provided as an exemplary embodiment of the present invention;

[0064] Figure 3 This is a schematic diagram of an adaptive sleep assistance system based on multimodal perception, provided as an exemplary embodiment of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] In one embodiment, such as Figure 1As shown, an adaptive sleep assistance method based on multimodal perception is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0067] S101: Obtain latitude and longitude coordinates, perform seasonal correction based on latitude and longitude coordinates, construct dynamic rhythm curves, and obtain user profile data and real-time multi-source data. The real-time multi-source data includes physiological signals, vibration spectrum and raw audio data, among which physiological signals include respiratory rate signals and heart rate variability signals.

[0068] Specifically, latitude and longitude coordinates can be obtained through a GPS module. However, in scenarios with weak or obstructed GPS signals, location data from the user's mobile phone can be automatically received via Bluetooth as redundant input to ensure the reliability of latitude and longitude acquisition. Subsequently, sunrise and sunset times can be adjusted based on the user's location latitude and longitude and the current season. For example, in winter (December to February), sunrise is delayed by 15 minutes and sunset by 10 minutes, eliminating the need for users to manually set their biological clock. Based on these corrected sunrise and sunset times, a 24-hour dynamic circadian rhythm curve can be constructed, detailing the changes in color temperature and luminous flux from late night to sunrise, noon, sunset, and back to night. Furthermore, user profile data and real-time multi-source data can be acquired. User profile data can include personalized parameters such as user age, sleep type (e.g., normal sleep, insomnia, elderly-specific), target blue light percentage, and light intensity modulation rate. Real-time multi-source data can be synchronously collected through a multimodal sensing matrix, with physiological signals acquired by millimeter-wave radar, enabling precise extraction of respiratory rate and heart rate variability signals. The vibration spectrum can be acquired using a 2×4 array of piezoelectric sensors, which can be fixed to the chest (2), abdomen (4), and lumbar (2) areas of the mattress, respectively. With a sampling rate of 100Hz, these sensors can detect bed vibrations within the range of 0.1-20Hz to reflect the user's body movement. Raw audio data can be acquired using a dual-microphone array with a microphone spacing of 30mm and a sampling rate of 48kHz. This allows the capture of acoustic information such as ambient noise and the user's snoring, providing data support for subsequent noise adaptation and sleep disturbance assessment.

[0069] S102: Perform Kalman filtering fusion processing on the respiratory rate signal and vibration spectrum to generate fused sleep staging results, classify the original audio data, and generate noise type labels and durations.

[0070] Specifically, Kalman filtering is a recursive filter that can estimate noisy signals, thereby improving their accuracy and reliability. By fusing respiratory frequency signals with vibration spectra using Kalman filtering, more accurate fused sleep staging results can be generated. Furthermore, by extracting 12-dimensional audio features from raw audio data using the MFCC algorithm, and classifying these features, different types of noise, such as snoring and traffic noise, can be identified. In addition, calculating the duration of noise can provide a basis for subsequent noise intervention.

[0071] S103: Based on the dynamic rhythm curve and preset adjustment rate, calculate the real-time parameters of the uplight, and based on the fusion of sleep stage results, noise type labels and user profile data, calculate the downlight parameters through a preset physiological-light effect mapping model to generate the real-time parameters of the downlight.

[0072] Specifically, based on a 24-hour dynamic circadian rhythm curve and preset adjustment rates, real-time uplight parameters can be calculated, enabling precise simulation of natural light variations and adjustment of the user's circadian rhythm. For example, from 30 minutes before sunrise to 1 hour after sunrise, the color temperature can linearly increase from 2700K to 5000K, and the luminous flux can linearly increase from 80lm to 1200lm, with adjustment rates of 50K / min and 100lm / min, respectively. By integrating sleep stage results, noise type labels, and user profile data, and combining this with a physiological-light effect mapping model to calculate downlight parameters, personalized sleep assistance can be provided based on the user's physiological state and environmental noise, significantly improving sleep quality.

[0073] S104: Input the real-time parameters of the up-beam and down-beam into the optical field coupling model to calculate the target value of the optical field distribution, and generate a light source driving command based on the target value of the optical field distribution. The light source driving command is used to control the light source and construct an adaptive optical field for sleep assistance.

[0074] Specifically, the light field coupling model, by considering factors such as color temperature, illuminance, and spatial distribution, defines a light field coordination coefficient to achieve coordination between upward and downward light, thereby optimizing the light field distribution and ensuring the comfort and effectiveness of the lighting environment. Through this light field coupling model, real-time parameters of the upward and downward light can be comprehensively calculated to generate target values ​​for the light field distribution, and corresponding light source driving commands can be generated to control the light source. Illustratively, this command can include control signals such as the on / off state of the light source, color temperature, brightness, and red light pulse parameters, which can be transmitted to a distributed light field controller to drive the corresponding upward rhythm module and downward sleep aid module to execute parameters, constructing an adaptive light field for sleep assistance.

[0075] The above method first constructs a dynamic circadian rhythm curve by acquiring latitude and longitude coordinates and performing seasonal correction, simultaneously acquiring user profile data and real-time multi-source data to provide a data benchmark for subsequent regulation. Second, it uses Kalman filtering to fuse respiratory rate signals and vibration spectra to generate a fused sleep staging result, which not only makes sleep staging more accurate but also clarifies the type of noise interference based on the original audio data. Furthermore, it calculates real-time upward light parameters based on the dynamic circadian rhythm curve, and combines the fused sleep staging result, noise type labels, and user profile data to generate real-time downward light parameters through a preset physiological-light effect mapping model, achieving deep adaptation of light parameters to circadian rhythms, sleep stages, noise environment, and user needs. Finally, it inputs the real-time upward and downward light parameters into a light field coupling model to calculate the target value of the light field distribution and generates light source driving commands to regulate the light source, constructing an adaptive light field for sleep assistance. This not only achieves intelligent regulation of the light field but also improves the personalization and effectiveness of sleep assistance.

[0076] In one embodiment, latitude and longitude coordinates are obtained, seasonal correction is performed based on the latitude and longitude coordinates, and a dynamic rhythm curve is constructed, including:

[0077] The latitude and longitude coordinates are obtained through the GPS positioning module, and the reference sunrise and sunset times for the current region are determined based on the latitude and longitude coordinates.

[0078] Obtain the current season information, perform seasonal offset determination based on the current season information, and obtain the adjusted sunrise time and adjusted sunset time;

[0079] Based on the adjusted sunrise time, adjusted sunset time, and preset time period division rules, multiple time intervals are determined, and corresponding target color temperature and target luminous flux are configured for each time interval. The time intervals include the morning wake-up transition interval, the daytime stable interval, the evening transition interval, and the nighttime deep sleep interval.

[0080] Based on the target color temperature, target luminous flux, and preset adjustment rate, a rhythm curve fitting process is performed to generate a dynamic rhythm curve.

[0081] The adjusted sunrise and sunset times are obtained through the following steps:

[0082] When the current season information is winter, the benchmark sunrise time is delayed by a first preset duration, and the benchmark sunset time is advanced by a second preset duration, so as to obtain the corresponding adjusted sunrise time and adjusted sunset time respectively.

[0083] When the current season information is summer, the benchmark sunrise time is advanced by a second preset duration, and the benchmark sunset time is delayed by a first preset duration, to obtain the corresponding adjusted sunrise and sunset times.

[0084] Specifically, the latitude value determines the latitudinal distance of the subsolar point, and the longitude value determines the time difference between local time and standard time. Substituting these values ​​into the sunrise / sunset angle formula, the baseline sunrise time (the moment the upper edge of the sun is tangent to the horizon) and baseline sunset time (the moment the lower edge of the sun is tangent to the horizon) for the current date can be calculated, providing an initial rhythmic reference for subsequent seasonal adjustments. Furthermore, the current month can be read from the local real-time clock to determine the corresponding season. If the user manually updates seasonal information (e.g., seasonal shifts in special climatic regions), the user-inputted values ​​can be prioritized, balancing the flexibility of automatic recognition with manual adaptation. Due to the Earth's revolution causing the movement of the subsolar point (e.g., the sun is directly overhead in the Southern Hemisphere during winter, resulting in shorter days and longer nights in the Northern Hemisphere), it is necessary to determine the seasonal shift based on the current seasonal information. The first preset duration can be set to 15 minutes and the second preset duration can be set to 10 minutes. In winter, the sunrise is delayed by 15 minutes and the sunset is advanced by 10 minutes, so that the light environment is consistent with the user's perception of delayed dawn and advanced dusk. In summer, the sunrise is advanced by 10 minutes and the sunset is delayed by 15 minutes, which can match the user's physiological adaptation needs for earlier dawn and later dusk.

[0085] Specifically, multiple time intervals are determined based on the adjusted sunrise and sunset times and preset time period division rules. These rules are based on the correlation between human physiological rhythms and natural light. For example, the morning transition interval can be set from 30 minutes before sunrise to 1 hour after sunrise, during which the body transitions from sleep to wakefulness, simulating the gradual change of dawn light. The daytime stable interval is set from 1 hour after sunrise to 1 hour before sunset, during which the body is in a state of wakefulness and activity, maintaining high color temperature and high luminous flux to promote vitality. The evening transition interval is set from 1 hour before sunset to 30 minutes after sunset, during which the body needs to transition from activity to relaxation, simulating the decay of warm sunset light. The nighttime deep sleep interval is set from 30 minutes after sunset to 30 minutes before sunrise the next day, maintaining low color temperature and low luminous flux to avoid interference. Furthermore, target color temperature and target luminous flux can be configured for each time interval by combining the characteristics of the light source hardware and the human photobiological effect. For example, in the morning wake-up transition zone, the target color temperature linearly increases from 2700K to 5000K, and the luminous flux linearly increases from 80lm to 1200lm. This utilizes warm, low-color-temperature light to awaken the biological clock, while the gradually increasing color temperature inhibits melatonin secretion. In the daytime stable zone, the target color temperature is fixed at 5000K, and the luminous flux is fixed at 1200lm, meeting the indoor wake-up lighting needs. In the evening transition zone, the target color temperature linearly decreases from 5000K to 3200K, and the luminous flux linearly decreases from 1200lm to 300lm. This uses warm light to promote the conversion of serotonin to melatonin, guiding relaxation. In the nighttime deep sleep zone, the target color temperature is fixed at 2700K, and the luminous flux is fixed at 50-80lm, simulating a moonlight environment.

[0086] Specifically, the preset adjustment rate can be set to a color temperature of 50K / min and a luminous flux of 100lm / min. Then, a linear interpolation algorithm can be used, with time as the X-axis and color temperature / luminous flux as the Y-axis, using the target parameters for each time interval as interpolation nodes. A continuous 24-hour dynamic curve is generated through linear interpolation. For example, in the morning-wake transition interval, from 2700K / 80lm 30 minutes before sunrise to 5000K / 1200lm 1 hour after sunrise, linear fitting is performed at a rate of 50K / min and 100lm / min, generating one intermediate parameter point every minute to ensure the curve has no abrupt changes. In the deep sleep interval at night, because the target parameters are fixed, the fitted curve is a horizontal straight line, maintaining a stable light environment. The final generated dynamic rhythm curve can be directly used as a reference for upward light regulation.

[0087] In one embodiment, the respiratory rate signal and vibration spectrum are fused using Kalman filtering to generate a fused sleep staging result, including:

[0088] The vibration spectrum is processed to extract the dominant frequency, thus obtaining the vibration dominant frequency.

[0089] The dominant vibration frequency is compared with a preset first frequency threshold and a preset second frequency threshold, respectively. A sleep period marker is constructed based on the comparison results, wherein the preset first frequency threshold is greater than the preset second frequency threshold. The sleep period marker is obtained through the following steps:

[0090] When the vibration frequency is greater than the preset first frequency threshold, a light sleep stage marker is generated and used as the sleep stage marker; when the vibration frequency is less than the preset second frequency threshold, a deep sleep stage marker is generated and used as the sleep stage marker; when the vibration frequency is between the preset first frequency threshold and the preset second frequency threshold, a transitional sleep stage marker is generated and used as the sleep stage marker.

[0091] The heart rate variability signal is processed by standard deviation calculation to obtain the heart rate variability standard deviation. The corresponding baseline heart rate variability standard deviation and sleep stage weight coefficient are calculated based on user profile data. Based on the deviation rate between the heart rate variability standard deviation and the baseline heart rate variability standard deviation, combined with the sleep stage weight coefficient, a weighted calculation is performed to generate the physiological fluctuation coefficient.

[0092] Initialize the state vector, error covariance matrix, and observation noise covariance matrix of the Kalman filter. The state vector consists of sleep stage state parameters, which include the probability of light sleep, deep sleep, and transitional sleep. The observation noise covariance matrix is ​​calibrated based on the classification error rate of the sleep stage labels.

[0093] Sleep phase markers are used as observation vectors, and physiological fluctuation coefficients are used as state transition weights. The observation vectors and state transition weights are input into a preset system dynamic model for prediction processing to generate state prediction values ​​and prediction error covariance. The state prediction values ​​include the probability of predicting light sleep, the probability of predicting deep sleep, and the probability of predicting transitional sleep.

[0094] The Kalman gain is calculated based on the prediction error covariance and the observation noise covariance matrix.

[0095] Based on Kalman gain, state prediction value and observation vector, observation update processing is performed to correct the state prediction value and prediction error covariance, and the current sleep stage state estimate and updated error covariance are obtained.

[0096] Spectral analysis was performed on the heart rate variability signal to extract the proportion of high-frequency components, and the REM phase determination index was calculated by combining it with the respiratory rate signal.

[0097] Based on the sleep stage and REM stage determination index corresponding to the maximum probability value in the current sleep stage state estimate, a fusion sleep stage result is generated. The fusion sleep stage result includes the transitional sleep stage result, light sleep stage result, deep sleep stage result, and REM sleep stage result of non-rapid eye movement sleep.

[0098] Specifically, by performing frequency domain analysis on the vibration spectrum using Fast Fourier Transform, the frequency component with the highest energy can be extracted as the dominant vibration frequency. This dominant frequency directly reflects the intensity of body movement. During light sleep, body movement is frequent, and vibration energy is concentrated in the high-frequency range. During deep sleep, body movement is weak, and energy is concentrated in the low-frequency range. Transitional sleep lies between the two. The preset first frequency threshold can be 5Hz, and the preset second frequency threshold can be 2Hz. When the dominant vibration frequency is >5Hz, the number of body movements is >3 times / minute, which can be considered as meeting the characteristics of light sleep. When the dominant vibration frequency is <2Hz, the number of body movements is <1 time / minute, which can be considered as meeting the characteristics of deep sleep. When the dominant vibration frequency is between 2-5Hz, the number of body movements is 1-3 times / minute, which corresponds to the transitional sleep period. Through the above process, sleep period markers can be generated.

[0099] Specifically, after heart rate variability (HRV) signals are acquired by millimeter-wave radar, the FMCW algorithm can be used to separate human signals from interference, and the standard deviation can be calculated based on the RR interval. Corresponding parameters can be extracted from the user profile database, namely the baseline HRV standard deviation for the corresponding user age group and the sleep stage weighting coefficient. This coefficient can be set according to sleep depth; a higher weight indicates a greater impact of HRV fluctuations on sleep stages. By calculating the ratio of the HRV standard deviation to the baseline HRV standard deviation and multiplying it by the sleep stage weighting coefficient, the physiological fluctuation coefficient can be obtained. This coefficient reflects the deviation of the current physiological state from the baseline and can be used as a state transition weight for Kalman filtering, avoiding misjudgment based on a single vibration signal.

[0100] Indicatively, when initializing the Kalman filter parameters, the state vector can be a 3D vector, with initial values ​​set based on the pre-sleep state. The initial value of the error covariance matrix is ​​set as a diagonal matrix, based on the factory noise parameters of the piezoelectric sensor and radar. The observation noise covariance matrix is ​​calibrated based on the classification error rate of sleep stage markers; that is, if the misclassification rate for light sleep stage markers is 8%, for deep sleep stage 6%, and for transitional sleep stage 10%, then this matrix is ​​set to diag(0.08, 0.06, 0.10). Subsequently, during prediction and update processing, the preset system dynamic model expression can be... ,in Let F be the predicted state value, and F be the state transition matrix. This is the state estimate from the previous time step. Let be the process excitation vector. Based on the prediction error covariance and observation noise covariance matrices, the Kalman gain can be calculated. This Kalman gain can be used to correct the predicted state value and prediction error covariance, yielding the current sleep stage state estimate and the updated error covariance. For example, if the predicted light sleep probability is 0.6 and the observation vector is the light sleep stage marker, the updated light sleep probability can be increased to 0.75, making the result more closely reflect the actual physiological state.

[0101] Specifically, when performing spectral analysis on heart rate variability signals, the proportion of high-frequency components in the 0.15-0.4Hz range can be extracted. Combined with respiratory rate signals, a weighted summation is used to calculate the REM (Rapid Eye Movement) stage determination index. When this index is greater than 0.7, it can be determined as a REM stage. Subsequently, the basal sleep stage is determined based on the maximum probability value in the current state estimate. If this index is greater than 0.7, the basal sleep stage can be corrected to a REM stage, ultimately generating the fusion sleep staging result.

[0102] In one embodiment, based on the fusion of sleep staging results, noise type labels, and user profile data, downlight parameters are calculated and processed using a preset physiological-optical effect mapping model to generate real-time downlight parameters, including:

[0103] Based on the results of the fusion sleep staging, the current sleep stage and the corresponding time interval are determined. Based on the time interval, the heart rate variability signal and respiratory rate signal corresponding to the time interval are called to calculate the corresponding changes in heart rate variability and respiratory rate, respectively.

[0104] The changes in heart rate variability and respiratory rate are input into a preset physiological-light effect mapping model for priority coefficient calculation, thereby generating light effect regulation priority coefficients.

[0105] The noise type labels are classified and identified to determine the noise category corresponding to the noise type label. The noise categories include continuous ambient noise, snoring, and sudden noise.

[0106] When the noise type label is continuous ambient noise, the upward light parameter correction amount is generated based on the loudness and duration of the continuous ambient noise. When the noise type label is snoring, red light pulse parameters are generated, including pulse frequency and pulse intensity.

[0107] When the noise type is labeled as burst noise, the instantaneous adjustment parameters of the downlight are generated based on the peak sound pressure level and duration of the burst noise. The instantaneous adjustment parameters of the downlight include the brightness drop amplitude and recovery rate.

[0108] Personalized adjustment parameters are extracted from user profile data. These parameters include target blue light percentage, target adjustment rate, and target brightness range. The target blue light percentage is configured based on the user's age group, and the target adjustment rate is configured based on the user's sleep type.

[0109] Based on the light effect adjustment priority coefficient, the up-sunlight parameter correction amount, the red light pulse parameter, the down-sunlight instantaneous adjustment parameter, and the personalized control parameter, the down-sunlight real-time parameter is generated.

[0110] Specifically, after determining the current sleep stage based on the fusion sleep staging results, the system can automatically match the corresponding time interval for that stage. For example, a single episode of light sleep typically lasts 20-40 minutes. Based on this time interval, it can retrieve the concurrently stored heart rate variability (HRV) and respiratory rate (RR) signals. Using the HRV signal, the difference between the average standard deviation of HRV in the current time interval and the average value corresponding to the previous sleep stage can be used to calculate the HRV change. Furthermore, by taking the average respiratory rate of the current and previous stages, the RR change can be calculated. Both indicators reflect the stability of the current sleep state and provide a physiological basis for subsequent light effect priority determination. When inputting the HRV and RR changes into a preset physiological-light effect mapping model to calculate the light effect adjustment priority coefficient, the model expression can be... ,in This is the priority coefficient for adjusting light efficacy. This represents the change in heart rate variability. This represents the change in respiratory rate. If the current sleep stage is light sleep, the calculated value is... The higher the value, the more the current sleep state requires increased light intervention intensity.

[0111] Furthermore, when classifying and identifying noise type labels, the labels are derived from the original audio data. 12-dimensional audio features are extracted using the MFCC algorithm, and then classified. The criteria for continuous environmental noise (such as traffic noise or air conditioning noise) are a loudness of 45-60 dB, a wide frequency spectrum, and a duration greater than 30 seconds. Snoring is defined as a loudness of 60-80 dB, a dominant frequency of 200-500 Hz, and a duration greater than 5 seconds. Sudden noises, such as doorbell sounds, are defined as a loudness greater than 60 dB, a transient pulse signal, and a duration less than 3 seconds. Corresponding parameters can then be generated for different noise categories. Since continuous environmental noise continuously stimulates the brain through the auditory nerve, this interference can be masked by correcting the uplight parameters. Specifically, a +200K warm tone shift can be added to the current uplight color temperature. Warmer light makes it easier for people to ignore environmental noise, and the luminous flux is increased by 20%, thus reducing the perceived intensity of continuous noise. Snoring is often accompanied by disrupted breathing rhythms. Red light has a weak inhibitory effect on melatonin and can alleviate breathing disorders through photobiological effects. Therefore, a 660nm red light pulse can be triggered by directing the light downwards. The pulse frequency can be set to 0.5Hz, duty cycle 50%, and light intensity 50-80cd / m² to reduce the frequency of snoring. Sudden noises can easily cause momentary awakenings. This can be addressed by rapidly reducing stimulation through instantaneous adjustment of the downward-directed light. The brightness drop can be set to 30% of the current brightness (to avoid sudden stimulation from strong light), and the recovery rate can be set to 50lm / min (slowly restoring to the original brightness) to reduce the awakening rate caused by sudden noises.

[0112] When extracting personalized adjustment parameters from user profile data, the core parameters can be configured based on the physiological characteristics of specific populations. For example, regarding the target blue light percentage, since the function of the macular region of the retina deteriorates in elderly users, 460-480nm blue light can easily cause visual fatigue, so it can be reduced to 5% (8% for ordinary users). Regarding the target adjustment rate, for users with insomnia who have difficulty falling asleep, the transition time of the downward light color temperature reduction can be extended, such as setting it to 4000K→2700K / 60min (40K / min for ordinary users), to avoid nerve excitation caused by a sudden drop in color temperature. Regarding the target brightness range, since elderly users have reduced sensitivity to light intensity, it can be set to 30-100cd / m² (20-80cd / m² for ordinary users), ensuring clear vision without stimulation.

[0113] When generating real-time downlight parameters based on all the above parameters, the light effect adjustment priority coefficient can be used as the core weight (0.6), the personalized control parameters as the basic weight (0.25), and the noise-related parameters as the dynamic weight (0.15) to obtain the corresponding real-time downlight parameters. These real-time downlight parameters are transmitted to the downlight sleep aid module of the distributed light field controller, which can drive the light source to perform adjustments.

[0114] In one embodiment, such as Figure 2 As shown, the real-time parameters of the upward-facing beam and the downward-facing beam are input into the optical field coupling model to calculate the target value of the optical field distribution, including:

[0115] S201: Obtain the preset horizontal angle between the human eye and the light source, perform light perception intensity calculation based on the preset horizontal angle, and generate a light perception intensity coefficient;

[0116] S202: Extract the current color temperature value from the real-time parameters of the upward-sunlight, and calculate the light field coordination coefficient based on the current color temperature value, light perception intensity coefficient and preset weight coefficient;

[0117] S203: Based on the optical field coordination coefficient, the luminous flux value in the real-time parameters of the up-sunlight and the brightness value in the real-time parameters of the down-sunlight, generate the illuminance of the desktop target and the illuminance of the ground target;

[0118] S204: Integrate the target illuminance on the desktop, the target illuminance on the ground, and the current color temperature value to generate the target value of the light field distribution.

[0119] Specifically, a spatial distribution array of light sensors (installed 0.8m high at the head of the bed) combined with the installation coordinates of light sources (uplighting from above and below, respectively, installed in the center of the ceiling and above the head of the bed) can be used to pre-calibrate the horizontal angle between the human eye and the light source at different positions on the bed, i.e., the preset horizontal angle between the human eye and the light source. Based on this preset horizontal angle, the following can be used: , This is a preset coefficient, with a value of 0.02. The light perception intensity coefficient is calculated based on the preset horizontal angle. The current color temperature value is extracted from the real-time parameters of the upward-facing light. The light field coordination coefficient can be calculated using the following formula based on the current color temperature value, the light perception intensity coefficient, and the preset weighting coefficient:

[0120]

[0121] in, The optical field coordination coefficient, This is the color temperature weighting coefficient. The sleep stage adaptation coefficient. This is the current color temperature value. For spatial perception weighting coefficients, The light perception intensity coefficient. For user age, This is the preset horizontal angle.

[0122] The light field coordination coefficient value calculated by the above formula can reflect the intensity of the coordination demand of the light field at different times and different human eye angles, providing an adjustment benchmark for subsequent illuminance calculations.

[0123] Based on the optical field coordination coefficient, the luminous flux value in the real-time parameters of the up-beam, and the luminance value in the real-time parameters of the down-beam, the illuminance of the desktop target and the illuminance of the ground target can be generated. The ground target illuminance can be the sum of the product of the up-beam luminous flux and the optical field coordination coefficient, calibrated by the light source's luminous efficacy and spatial reflection characteristics. The desktop target illuminance can be calculated using the following formula:

[0124]

[0125] in, For desktop target illuminance, For the corresponding weight description, The brightness of the downward-facing light. is the optical field coordination coefficient.

[0126] Finally, by integrating the target illuminance on the desktop, the target illuminance on the ground, and the current color temperature value, a target value for the light field distribution can be generated. This target value includes the ground illuminance, the desktop illuminance, and the color temperature, and can be directly used as the basis for controlling the light source.

[0127] In one embodiment, the method further includes:

[0128] Real-time monitoring of fusion sleep staging results, and extraction of body movement frequency and respiratory parameters based on the original vibration spectrum and respiratory frequency signals corresponding to the fusion sleep staging results. The body movement frequency is calculated based on the frequency of change of the dominant frequency of the original vibration spectrum, and the respiratory parameters include respiratory frequency fluctuation value and apnea duration.

[0129] When the body movement frequency is greater than the first preset frequency threshold or the respiratory rate fluctuation value is greater than the first preset fluctuation threshold, and the corresponding first abnormal duration exceeds the first preset duration, a first-level intervention is performed. The brightness and color temperature values ​​in the downlight real-time parameters are reduced according to the preset first-level rules to obtain the first optimized downlight real-time parameters. The light source driving command is updated based on the first optimized downlight real-time parameters to obtain the first optimized light source driving command.

[0130] When the body movement frequency is greater than the second preset frequency threshold and the respiratory rate fluctuation value is greater than the second preset fluctuation threshold, and the corresponding second abnormal duration exceeds the second preset duration, a secondary intervention process is performed to generate a mild abnormality reminder message, push the mild abnormality reminder message to the associated terminal, and reduce the brightness value and color temperature value in the downlight real-time parameters according to the preset secondary rules to generate a second optimized downlight real-time parameter. Based on the second optimized downlight real-time parameter, the light source driving command is updated to obtain the second optimized light source driving command.

[0131] When the body movement frequency is greater than the third preset frequency threshold and the duration of apnea exceeds the preset apnea duration, and the corresponding third abnormal duration exceeds the third preset duration, a three-level intervention is performed to generate an emergency abnormality reminder message, push the emergency abnormality reminder message to the associated terminal, adjust the red light pulse control parameter in the downlight real-time parameter, generate the third optimized downlight real-time parameter, update the light source driving command based on the third optimized downlight real-time parameter, and obtain the third optimized light source driving command.

[0132] Among them, the first preset frequency threshold is less than the second preset frequency threshold, the second preset frequency threshold is less than the third preset frequency threshold, the first preset duration is less than the second preset duration, and the second preset duration is less than the third preset duration.

[0133] Specifically, when extracting body movement frequencies, the calculation can be based on the frequency of dominant frequency changes in the original vibration spectrum. The piezoelectric sensor can collect vibration signals at a sampling rate of 100Hz, and the dominant frequency can be analyzed every 10 seconds using FFT to count the number of dominant frequency changes within one minute (one body movement corresponds to a complete change in dominant frequency from low to high and back to low). When extracting respiratory parameters, the respiratory rate fluctuation value can be the difference between the maximum and minimum respiratory rate within the current minute. The duration of apnea can be determined by the duration of "disappearance of respiratory signal lasting >10 seconds" detected by radar.

[0134] Specifically, during Level 1 intervention, the first preset frequency threshold can be set to 4 times / minute, the first preset fluctuation threshold to 1.5Hz, and the first preset duration to 3 minutes, corresponding to mild sleep instability, i.e., frequent body movements but not reaching the level of awakening, and slightly large fluctuations in respiratory rate but no risk of pause. Subsequently, according to the preset Level 1 rules, based on the photobiological effect of "warm light and low brightness inhibiting cortical excitation," warm light can promote melatonin secretion, and low brightness reduces visual stimulation. The brightness value in the real-time parameters of the downlight can be reduced by 20% (e.g., from 80cd / m² to 64cd / m²) and the color temperature value can be reduced by 500K (e.g., from 3200K to 2700K), generating the first optimized real-time parameters of the downlight and updating the light source driving instructions.

[0135] Specifically, during secondary intervention, the second preset frequency threshold can be set to 6 times / minute, the second preset fluctuation threshold to 2Hz, and the second preset duration to 5 minutes, corresponding to a moderate sleep disturbance state, where body movement and respiratory fluctuations are close to the wakefulness threshold. Subsequently, a mild abnormality alert can be generated, such as a notification that the user's sleep is moderately unstable, and pushed to a related terminal (mobile app or smart speaker) to remind the user's family or caregivers to pay attention. Simultaneously, according to preset secondary rules, the downlight brightness can be reduced by 30% and the color temperature by 800K to further enhance the sleep-aiding effect of warm light and low brightness, generating second optimized downlight real-time parameters and updating corresponding instructions.

[0136] Furthermore, during the three-level intervention, the third preset frequency threshold can be set to 8 times / minute, the preset pause duration to 15 seconds, and the third preset duration to 8 minutes, corresponding to severe sleep abnormalities, i.e., extremely frequent body movements accompanied by apnea. An emergency alert message is generated if the user is at risk of apnea and pushed to the associated terminal. Simultaneously, the red light pulse control parameters in the downlight real-time parameters can be adjusted, such as setting the pulse frequency to 0.8Hz, the duty cycle to 50%, and increasing the light intensity to 100 cd / m², to stimulate the carotid body chemoreceptors through photochemical action, enhancing respiratory drive and alleviating apnea tendency. Finally, the third optimized downlight real-time parameters can be generated and the corresponding instructions updated to improve sleep safety.

[0137] Based on the same inventive concept, such as Figure 3 As shown, this application also provides a multimodal perception-based adaptive sleep assistance system 300 for implementing the aforementioned multimodal perception-based adaptive sleep assistance method. The solution provided by this system is similar to the implementation scheme described in the above method. Therefore, the specific limitations of one or more embodiments of a multimodal perception-based adaptive sleep assistance system provided below can be found in the limitations of the various method embodiments above, and will not be repeated here. The system includes:

[0138] The data acquisition and processing module 301 is used to acquire latitude and longitude coordinates, perform seasonal correction processing based on latitude and longitude coordinates, construct dynamic rhythm curves, and acquire user profile data and real-time multi-source data. The real-time multi-source data includes physiological signals, vibration spectrum and raw audio data, among which physiological signals include respiratory rate signals and heart rate variability signals.

[0139] The data fusion and analysis module 302 is used to perform Kalman filtering fusion processing on the respiratory frequency signal and vibration spectrum to generate fused sleep staging results, classify the original audio data, and generate noise type labels and durations.

[0140] The parameter calculation and mapping module 303 is used to calculate the real-time parameters of the uplight based on the dynamic rhythm curve and the preset adjustment rate, and to calculate the downlight parameters based on the fusion of sleep stage results, noise type labels and user profile data through the preset physiological-light effect mapping model to generate the real-time parameters of the downlight.

[0141] The light field control module 304 is used to input the real-time parameters of the up-illuminated light and the real-time parameters of the down-illuminated light into the light field coupling model to calculate the target value of the light field distribution, and generate a light source driving command based on the target value of the light field distribution. The light source driving command is used to control the light source and construct an adaptive light field for sleep assistance.

[0142] In one exemplary embodiment, the present invention also provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the adaptive sleep assistance method based on multimodal perception proposed in this application. A multi-core processor is preferred to improve the parallel processing capability of the system. The memory provides sufficient temporary storage space to support program execution and data processing. The memory capacity should be large enough to accommodate large amounts of data and computational tasks.

[0143] In one exemplary embodiment, the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of an adaptive sleep assistance method based on multimodal perception according to this application. The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drive (SSD), or optical disk, etc.

[0144] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A multi-modal perception based adaptive sleep assistance method, characterized in that, The method comprises: Obtaining latitude and longitude coordinates, performing seasonal correction processing based on the latitude and longitude coordinates, constructing a dynamic rhythm curve, and obtaining user portrait data and real-time multi-source data, wherein the real-time multi-source data includes physiological signals, vibration frequency spectrum and original audio data, and the physiological signals include respiratory frequency signals and heart rate variation signals; Performing Kalman filter fusion processing on the respiratory frequency signals and the vibration frequency spectrum to generate a fusion sleep staging result, and performing classification processing on the original audio data to generate noise type labels and duration; Based on the dynamic rhythm curve and a preset adjustment rate, the real-time parameters of the upper light are calculated, and based on the fusion sleep staging result, the noise type labels and the user portrait data, the real-time parameters of the lower light are calculated through a preset physiological-light effect mapping model to generate the real-time parameters of the lower light; The real-time parameters of the upper light and the real-time parameters of the lower light are input into a light field coupling model to calculate a light field distribution target value, and based on the light field distribution target value, a light source driving instruction is generated, which is used to regulate the light source and construct an adaptive light field for sleep assistance.

2. The method of claim 1, wherein, The method comprises: Obtaining the latitude and longitude coordinates through a GPS positioning module, and determining the reference sunrise time and the reference sunset time of the current region based on the latitude and longitude coordinates; Obtaining current season information, determining the seasonal offset based on the current season information, and obtaining the adjusted sunrise time and the adjusted sunset time; Based on the adjusted sunrise time, the adjusted sunset time and a preset time interval division rule, a plurality of time intervals are determined, and corresponding target color temperature and target luminous flux are configured for each time interval, wherein the time intervals include a morning wake-up transition interval, a daytime stable interval, an evening transition interval and a night deep sleep interval; Based on the target color temperature, the target luminous flux and the preset adjustment rate, rhythm curve fitting processing is performed to generate the dynamic rhythm curve; Wherein the adjusted sunrise time and the adjusted sunset time are obtained by the following steps: When the current season information is winter, the reference sunrise time is delayed by a first preset time length, and the reference sunset time is advanced by a second preset time length, respectively to obtain the corresponding adjusted sunrise time and adjusted sunset time; When the current season information is summer, the reference sunrise time is advanced by the second preset time length, and the reference sunset time is delayed by the first preset time length, respectively to obtain the corresponding adjusted sunrise time and adjusted sunset time.

3. The method of claim 1, wherein, The method comprises: Performing main frequency extraction processing on the vibration frequency spectrum to obtain the vibration main frequency; comparing the vibration main frequency with a preset first frequency threshold and a preset second frequency threshold respectively, and constructing a sleep period label according to a comparison result, wherein the preset first frequency threshold is greater than the preset second frequency threshold, and the sleep period label is obtained by the following steps: when the comparison result is that the vibration main frequency is greater than the preset first frequency threshold, generating a light sleep period label, and taking the light sleep period label as the sleep period label; when the vibration main frequency is less than the preset second frequency threshold, generating a deep sleep period label, and taking the deep sleep period label as the sleep period label; when the vibration main frequency is between the preset first frequency threshold and the preset second frequency threshold, generating a transition sleep period label, and taking the transition sleep period label as the sleep period label; performing standard deviation calculation processing on the heart rate variation signal to obtain a heart rate variation standard deviation, and calculating a corresponding reference heart rate variation standard deviation and a sleep stage weight coefficient according to the user portrait data; based on a deviation rate of the heart rate variation standard deviation and the reference heart rate variation standard deviation, and combining the sleep stage weight coefficient to perform weighted calculation processing, a physiological fluctuation coefficient is generated; initializing a state vector, an error covariance matrix and an observation noise covariance matrix of a Kalman filter, wherein the state vector is a sleep staging state parameter, the sleep staging state parameter includes a light sleep period probability, a deep sleep period probability and a transition sleep period probability, and the observation noise covariance matrix is obtained based on a classification error rate of the sleep period label; taking the sleep period label as an observation vector, and taking the physiological fluctuation coefficient as a state transition weight, inputting the observation vector and the state transition weight into a preset system dynamic model to perform prediction processing, generating a state prediction value and a prediction error covariance, and the state prediction value includes a predicted light sleep period probability, a predicted deep sleep period probability and a predicted transition sleep period probability; based on the prediction error covariance and the observation noise covariance matrix, a Kalman gain is calculated; based on the Kalman gain, the state prediction value and the observation vector, performing observation update processing, correcting the state prediction value and the prediction error covariance, obtaining a current time sleep staging state estimation value and an updated error covariance; performing frequency spectrum analysis on the heart rate variation signal, extracting a high frequency component proportion, and combining the respiratory frequency signal to calculate a REM period determination index; based on a sleep stage corresponding to a probability maximum value in the current time sleep staging state estimation value and the REM period determination index, the fusion sleep staging result is generated, and the fusion sleep staging result includes a transition sleep period result of non-ocular rapid movement sleep, a light sleep period result, a deep sleep period result and an ocular rapid movement sleep period result.

4. The method of claim 1, wherein, based on the fusion sleep staging result, the noise type label and the user portrait data, performing down light parameter calculation processing through a preset physiological-light effect mapping model to generate down light real-time parameters, including: determine a current sleep stage and a time interval corresponding to the current sleep stage based on the fusion sleep staging result, and call a heart rate variability signal and a respiration rate signal corresponding to the time interval based on the time interval, and calculate a heart rate variability change amount and a respiration rate change amount respectively; input the heart rate variability change amount and the respiration rate change amount into the preset physiological-light effect mapping model for priority coefficient calculation processing to generate a light effect adjustment priority coefficient; classify and identify the noise type label to determine a noise category corresponding to the noise type label, the noise category including continuous environmental noise, snoring sound, and burst noise; when the noise type label is continuous environmental noise, generate an uplight parameter correction amount based on the loudness and duration of the continuous environmental noise, and when the noise type label is snoring sound, generate a red light pulse parameter, the red light pulse parameter including a pulse frequency and a pulse light intensity; when the noise type label is burst noise, generate a downlight instantaneous adjustment parameter based on the peak sound pressure level and duration of the burst noise, the downlight instantaneous adjustment parameter including a brightness drop amplitude and a recovery rate; extract individualized regulation parameters from the user portrait data, the individualized regulation parameters including a target blue light proportion, a target adjustment rate, and a target brightness interval, wherein the target blue light proportion is configured based on the user's age group, and the target adjustment rate is configured based on the user's sleep type; generate the downlight real-time parameter based on the light effect adjustment priority coefficient, the uplight parameter correction amount, the red light pulse parameter, the downlight instantaneous adjustment parameter, and the individualized regulation parameters.

5. The method of claim 1, wherein, inputting the uplight real-time parameter and the downlight real-time parameter into a light field coupling model to calculate a light field distribution target value, includes: obtaining a preset horizontal angle between a human eye and the light source, and performing light perception intensity calculation processing based on the preset horizontal angle to generate a light perception intensity coefficient; extracting a current color temperature value from the uplight real-time parameter, and calculating a light field synergy coefficient based on the current color temperature value, the light perception intensity coefficient, and a preset weight coefficient; generating a desktop target illuminance and a ground target illuminance based on the light field synergy coefficient, a luminous flux value in the uplight real-time parameter, and a brightness value in the downlight real-time parameter; integrating the desktop target illuminance, the ground target illuminance, and the current color temperature value to generate the light field distribution target value.

6. The method of claim 1, wherein, The method further includes: monitoring the fusion sleep staging result in real time, and extracting a body movement frequency and a respiration parameter from the original vibration frequency spectrum and the respiration frequency signal corresponding to the fusion sleep staging result, the body movement frequency being calculated based on the main frequency change frequency of the original vibration frequency spectrum, and the respiration parameter including a respiration frequency fluctuation value and a respiratory pause duration; When the body motion frequency is greater than a first preset frequency threshold or the respiratory frequency fluctuation value is greater than a first preset fluctuation threshold, and a corresponding first abnormal duration exceeds a first preset time length, a first-level intervention processing is performed, the brightness value and the color temperature value in the downcast light real-time parameter are reduced according to a preset first-level rule, first optimized downcast light real-time parameters are obtained, and the light source driving instruction is updated based on the first optimized downcast light real-time parameters to obtain first optimized light source driving instructions; When the body motion frequency is greater than a second preset frequency threshold and the respiratory frequency fluctuation value is greater than a second preset fluctuation threshold, and a corresponding second abnormal duration exceeds a second preset time length, a second-level intervention processing is performed, a mild abnormality reminding information is generated, the mild abnormality reminding information is pushed to an associated terminal, and the brightness value and the color temperature value in the downcast light real-time parameter are reduced according to a preset second-level rule to generate second optimized downcast light real-time parameters, the light source driving instruction is updated based on the second optimized downcast light real-time parameters to obtain second optimized light source driving instructions; When the body motion frequency is greater than a third preset frequency threshold and the apnea duration exceeds a preset apnea time length, and a corresponding third abnormal duration exceeds a third preset time length, a third-level intervention processing is performed, an emergency abnormality reminding information is generated, the emergency abnormality reminding information is pushed to the associated terminal, and a red light pulse control parameter in the downcast light real-time parameter is adjusted to generate third optimized downcast light real-time parameters, the light source driving instruction is updated based on the third optimized downcast light real-time parameters to obtain third optimized light source driving instructions. The first preset frequency threshold is less than the second preset frequency threshold, the second preset frequency threshold is less than the third preset frequency threshold, the first preset time length is less than the second preset time length, and the second preset time length is less than the third preset time length.

7. The method of claim 5, wherein, The light field coordination coefficient is calculated by the following formula: wherein, is the light field coordination coefficient, is the color temperature weight coefficient, is the sleep staging adaptation coefficient, is the current color temperature value, is the spatial perception weight coefficient, is the light perception intensity coefficient, is the user age, is the preset horizontal angle.

8. An adaptive sleep assistance system based on multi-modal sensing, the system comprising: The system comprises: A data acquisition and processing module is configured to acquire latitude and longitude coordinates, perform seasonal correction processing based on the latitude and longitude coordinates, construct a dynamic rhythm curve, and acquire user portrait data and real-time multi-source data, wherein the real-time multi-source data comprises physiological signals, vibration frequency spectrum, and original audio data, and the physiological signals comprise a respiratory frequency signal and a heart rate variability signal; A data fusion and analysis module is configured to perform Kalman filter fusion processing on the respiratory frequency signal and the vibration frequency spectrum to generate a fusion sleep staging result, and perform classification processing on the original audio data to generate a noise type label and a duration; A parameter calculation and mapping module is configured to calculate upcast light real-time parameters based on the dynamic rhythm curve and a preset adjustment rate, and perform downcast light parameter calculation processing based on the fusion sleep staging result, the noise type label, and the user portrait data through a preset physiological-light efficiency mapping model to generate downcast light real-time parameters. A light field regulation module is configured to input the upwelling light real-time parameter and the downwelling light real-time parameter into a light field coupling model to calculate a light field distribution target value, and generate a light source driving instruction based on the light field distribution target value, the light source driving instruction being used to regulate a light source to construct an adaptive light field for sleep assistance. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 7.

Images