Snoring sound detection method and apparatus

Abstract

This application discloses a snoring detection method and apparatus. The method includes: acquiring human body induction signals and audio signals; determining the device usage status and audio characteristics based on the human body induction signals and audio signals; selecting corresponding feature combinations based on the human body induction signals, device usage status, and audio characteristics and inputting them into a preset snoring detection model for snoring detection to determine the current snoring state; if the current state is snoring, determining the initial snoring attribution probability based on a preset sound intensity coefficient reference table, sound phase coefficient reference table, and audio characteristics, and performing a weighted summation with the device usage status, and determining the final snoring attribution result based on the weighted summation result. This application effectively solves the technical defects of snoring detection in multi-person cohabitation scenarios, such as the inability to clearly determine snoring attribution, insufficient detection accuracy, and lack of a scientific benchmark reference system and probability calculation logic; and effectively improves the accuracy of detection and attribution judgment.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and smart home technology, and in particular to a snoring detection method and device. Background Technology

[0002] In smart home and smart sleep monitoring scenarios where multiple people live together, simple snoring detection can only determine whether snoring exists, but cannot determine the source of the snoring. This results in sleep monitoring data being mixed with snoring information from different users, making it difficult to achieve accurate health tracking for individuals.

[0003] Currently, snoring detection is mostly based on sound signals, vibration signals, or non-contact sensor signals. Most rely solely on single-dimensional signal features (such as sound intensity or single sensor data) or simple threshold judgments to identify and classify snoring, without forming a complete feature fusion and probability calculation system. Although some solutions attempt to combine multiple features for classification, they lack a unified benchmark reference system and scientific probability estimation methods, resulting in low accuracy in snoring classification and failing to meet the personalized and precise sleep monitoring needs in smart home environments. Summary of the Invention

[0004] Therefore, it is necessary to provide a snoring detection method and apparatus to address the above-mentioned technical problems and solve at least one of the problems existing in the prior art.

[0005] In a first aspect, embodiments of this application provide a snoring detection method, including: Acquire human body sensor signals and audio signals; Based on the human body sensing signals and audio signals, the device's usage status and audio characteristics are determined; Based on the human body sensing signals, device usage status, and audio characteristics, the corresponding feature combinations are selected and input into the preset snoring detection model to detect snoring and determine the current snoring status. If the current state is snoring, the initial snoring probability is determined based on the preset sound intensity coefficient reference table, sound phase coefficient reference table, and audio characteristics. Based on the initial snoring probability and the device usage status, the snoring result is determined.

[0006] In one possible implementation, the audio features include voiceprint features, and the preset snoring detection model is a deep learning model for snoring detection or a snoring judgment model. The step of selecting corresponding feature combinations based on the human body sensing signal, device usage status, and audio features and inputting them into the preset snoring detection model for snoring detection to determine the current snoring state includes: The device usage status, human body sensing signals, and audio signals are input into the snoring detection deep learning model for feature processing and classification to obtain snoring classification results. If the probability value corresponding to the snoring classification result is greater than the preset snoring determination threshold, then the current state is determined to be snoring; otherwise, it is determined to be non-snoring. The voiceprint features and the device usage status are input into the snoring judgment model for analysis and judgment, and the corresponding snoring status is output.

[0007] In one possible implementation, the audio features include sound intensity features and sound phase features. If the current state is snoring, an initial snoring attribution probability is determined based on a preset sound intensity coefficient reference table, a sound phase coefficient reference table, and the audio features. Based on the initial snoring attribution probability and the device usage status, a snoring attribution result is determined, including: Based on the preset sound intensity coefficient reference table, sound phase coefficient reference table, sound intensity characteristics and sound phase characteristics, the initial probability values ​​of snoring belonging to the left and right sides are determined respectively. Based on the device usage status and the initial probability value, the actual probability values ​​of snoring belonging to the left side and snoring belonging to the right side are obtained. If the actual probability value of the snoring sound belonging to the left side is greater than the actual probability value of the snoring sound belonging to the right side, then the snoring sound is determined to belong to the left side; otherwise, the snoring sound is determined to belong to the right side.

[0008] In one possible implementation, the device usage states include an out-of-bed state and a sleep state. The determination of initial probability values ​​for assigning snoring to the left and right sides, based on the preset sound intensity coefficient reference table, sound phase coefficient reference table, sound intensity characteristics, and sound phase characteristics, includes: The preset sound intensity coefficient reference table and sound phase coefficient reference table are fitted respectively to obtain the sound intensity fitting curve and the sound phase fitting curve. Based on the sound intensity fitting curve and the sound intensity characteristics, a first initial probability value for assigning snoring to the left and a second initial probability value for assigning snoring to the right are determined. Based on the sound phase fitting curve and the sound phase characteristics, a third initial probability value for assigning snoring to the left and a fourth initial probability value for assigning snoring to the right are determined. The process of obtaining the actual probability values ​​of snoring belonging to the left and the snoring belonging to the right based on the device usage status and the initial probability value includes: The weighted sum of the in / out bed state, sleep state, first initial probability value and third initial probability value is used to obtain the first actual probability value of snoring belonging to the left side; The weighted sum of the in / out bed state, sleep state, second initial probability value and fourth initial probability value is used to obtain the second actual probability value of snoring belonging to the right side.

[0009] In one possible implementation, after determining the snoring attribution result, the method further includes: Based on the human body sensing signals and audio signals, obtain multidimensional feature data of the current user; Based on the pre-stored user prior information and the current user's multi-dimensional feature data, the identity of the current user is verified. If the current user's identity verification result is abnormal, output an identity verification abnormality prompt message to the current user and obtain user feedback information; Based on the user feedback information, the user prior information is adaptively updated, and the updated user prior information is synchronously stored.

[0010] In one possible implementation, after determining the device usage status and audio characteristics, the method further includes: Determine if the current sound meets the preset calibration conditions; If the preset sound calibration conditions are met, the preset sound intensity coefficient reference table and sound phase coefficient reference table are calibrated respectively to obtain the calibrated sound intensity coefficient and calibrated sound phase coefficient at the corresponding calibration point. The effectiveness of the calibrated sound intensity coefficient and the calibrated sound phase coefficient were evaluated respectively. If the validity assessment result is valid, then the sound intensity coefficient and sound phase coefficient of the corresponding calibration point are updated based on the calibrated sound intensity coefficient and the calibrated sound phase coefficient.

[0011] In one possible implementation, the audio signal includes a left audio signal and a right audio signal. The step of calibrating the preset sound intensity coefficient reference table and sound phase coefficient reference table to obtain the calibrated sound intensity coefficient and calibrated sound phase coefficient at the corresponding calibration points includes: The ratio of the sound intensity of the left audio signal to the sound intensity of the right audio signal is calculated and used as the calibrated sound intensity coefficient. The azimuth angle of the sound source is calculated based on the time delay between the left and right audio signals, and is used as the calibrated sound phase coefficient.

[0012] In one possible implementation, the effectiveness evaluation of the calibrated sound intensity coefficient and the calibrated sound phase coefficient includes: Obtain historical sound intensity reference data and historical sound phase reference data corresponding to the calibration point within a preset time range; Based on the historical sound intensity benchmark data and the historical sound phase benchmark data, the historical sound intensity benchmark mean and the historical sound phase benchmark mean are calculated respectively. Calculate the sound intensity difference between the calibrated sound intensity coefficient and the historical sound intensity benchmark mean, and the sound phase difference between the calibrated sound phase coefficient and the historical sound phase benchmark mean; If the absolute value of the ratio between the sound intensity difference and the historical sound intensity benchmark mean is greater than the preset sound intensity threshold, then the calibrated sound intensity coefficient is invalid; otherwise, the calibrated sound intensity coefficient is valid. If the absolute value of the ratio between the sound phase difference and the historical sound intensity benchmark mean is greater than the preset sound phase threshold, then the calibrated sound phase coefficient is invalid; otherwise, the calibrated sound phase coefficient is valid.

[0013] In one possible implementation, calculating the azimuth angle of the sound source based on the left audio signal and the right audio signal includes: The left and right audio signals are respectively subjected to frequency domain transformation to obtain the left frequency domain signal and the right frequency domain signal; Based on the left-side and right-side frequency domain signals, cross-correlation features are constructed, and the cross-correlation features are weighted. The time delay between the left audio signal and the right audio signal is determined based on the weighted cross-correlation characteristics. The azimuth angle of the sound source is calculated based on the time delay, sound speed, and the distance between the left and right audio acquisition positions.

[0014] Secondly, a snoring detection device is provided, comprising: The information acquisition unit is used to acquire human body sensing signals and audio signals; The status and feature determination unit is used to determine the device usage status and audio features based on the human body sensing signal and the audio signal. The snoring detection unit is used to select corresponding feature combinations based on the human body sensing signals, device usage status and audio characteristics, and input them into a preset snoring detection model to detect snoring and determine the current snoring status. The snoring attribution determination unit is used to determine the initial snoring attribution probability based on a preset sound intensity coefficient reference table, a sound phase coefficient reference table, and audio characteristics if the current snoring state is in progress, and to determine the snoring attribution result based on the initial snoring attribution probability and the device usage status.

[0015] The above-mentioned snoring detection method and apparatus include the following steps: acquiring human body sensing signals and audio signals; determining the device usage status and audio characteristics based on the human body sensing signals and audio signals; selecting corresponding feature combinations based on the human body sensing signals, device usage status, and audio characteristics and inputting them into a preset snoring detection model for snoring detection to determine the current snoring status; if the current state is snoring, determining an initial snoring attribution probability based on a preset sound intensity coefficient reference table, sound phase coefficient reference table, and audio characteristics; and determining the snoring attribution result based on the initial snoring attribution probability and the device usage status. In this embodiment, the technical shortcomings of existing technologies in smart home and smart bedroom scenarios with multiple occupants—that snoring detection can only determine the presence of snoring but cannot clearly identify its attribution, and that the detection accuracy is insufficient and lacks a scientific benchmark reference system and probability calculation logic—are effectively addressed. By acquiring human body sensing signals and audio signals to determine the device's usage status and audio characteristics, a snoring detection model is adopted. This model combines dual feature fusion of sound intensity and sound phase features, fitting analysis of sound intensity coefficient reference tables and sound phase coefficient reference tables, and complete coefficient calibration, effectiveness evaluation mechanisms, and time delay and sound source azimuth angle calculations. By combining the device's usage status with weighted summation, the attribution of snoring is determined. This not only improves the accuracy of snoring detection and attribution judgment, avoiding health misjudgments caused by mixed snoring data, but also adapts to different computing power scenarios, realizing a leap from generalized environmental monitoring to precise individual monitoring in sleep monitoring. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the device end in one embodiment of this application; Figure 2 This is a schematic diagram of the structure of a night light module in one embodiment of this application; Figure 3 This is a schematic diagram of an application environment for a snoring detection system according to one embodiment of this application; Figure 4 This is a flowchart illustrating a snoring detection method in one embodiment of this application; Figure 5 This is a cross-sectional schematic diagram of the calibration point in one embodiment of this application; Figure 6 This is a schematic diagram of the azimuth angle of the sound source in one embodiment of this application; Figure 7This is a schematic diagram of a snoring detection device according to one embodiment of this application; Figure 8 This is a schematic diagram of a computer device according to one embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The snoring detection method provided in this embodiment can be applied to, for example... Figure 1 The application environment shown includes a server S31, a client S32, and a device S33, which can communicate and connect with each other. Device S33 collects the user's audio signals, human body sensing signals, and sleep state signals in real time through audio acquisition sensors, human body sensing sensors, and sleep monitoring sensors, and uploads the collected data to server S31 for storage and analysis. Server S31 identifies and assigns snoring status based on a preset snoring detection model and historical calibration data, and sends the detection results and analysis data to device S33 and client S32. Client S32 displays snoring detection results and sleep reports to the user, receives user feedback, and synchronizes the feedback data to server S31 to update prior information and calibration parameters, achieving dynamic optimization of the detection model and reference coefficient table. Based on the analysis results and control commands sent by server S31, device S33 adaptively adjusts the user's sleep state and sleep environment through a snoring intervention module and an adaptive adjustment module, effectively improving the accuracy and stability of snoring detection and assignment judgment, and enhancing the user's sleep monitoring experience and intervention effect.

[0020] Specifically, server S31 is used to store and distribute user personal information, user device snoring detection parameter information, and user device usage history information; client S32 is used to obtain user information and display snoring detection results to the user. The obtained user information includes personal information such as the user's height, weight, and age, as well as user feedback information, including whether the device is used by the user and whether the snoring detection results are abnormal; the main carrier of device S33 can be a smart bed frame or smart mattress and other smart devices, which mainly consists of five parts: night light module, sleep monitoring module, main control module, snoring intervention module, and adaptive adjustment module.

[0021] The night light module includes an audio acquisition sensor, a human body sensor, a photosensor, a night light, and a microprocessor, used for audio signal acquisition, suspected snoring detection, unilateral human body detection, and night light activation in low ambient light conditions. The sleep monitoring module includes a sleep monitoring sensor, which can be a piezoelectric sensor or an accelerometer, used for functions such as bed exit monitoring and sleep staging. The main control module includes a high-performance microprocessor used for information aggregation and processing between modules and for issuing control commands to different modules. The snoring intervention module is used to adjust the user's sleeping posture or body position after detecting snoring, and can fine-tune the headboard angle through a push rod motor. The adaptive adjustment module is used to adjust the user's sleep environment according to the user's current state, and can realize functions such as mattress firmness adjustment and sleep scene light intensity or color temperature adjustment.

[0022] A simplified structural diagram of the S33 device is shown below. Figure 2 As shown in the figure, S10 is the mattress or bed frame body; S11-S14 are four bed legs, which may be ordinary non-intelligent bed legs, or speaker-type smart bed legs with integrated sensors or other forms of smart bed legs; S15 and S16 are smart night lights located on both sides of the bed.

[0023] A simplified model diagram of the intelligent night light is shown below. Figure 3 As shown, S21 and S22 are fixed structures that can be fixed to the lower part of the left and right sides of the mattress or bed frame using screws or other methods; S23 is a human body sensor, which can be a pyroelectric infrared sensor, used to detect whether there is a person directly in front of the sensor; S24 is a microphone hole, and an audio acquisition sensor is installed below the hole, with a sampling rate of 44kHz; S25 is a photoresistor, mainly used to detect ambient light intensity; S26 is a night light emission window, and the night light is installed below this window, which is made of semi-transparent material; after the smart night light is assembled, the side containing S23, S24, and S25 faces the outside of the mattress, and the side containing S26 faces the ground.

[0024] In one embodiment, such as Figure 4 As shown, a snoring detection method is provided, including the following steps: In step S110, human body sensing signals and audio signals are acquired; Optionally, audio signals can be simultaneously acquired through smart night light modules installed on the left and right sides of the device. The audio acquisition sensors of the smart night light modules on both sides are installed below the microphone holes and use a sampling rate of 44kHz to acquire sound data. This can accurately capture various sound signals around the device, including user snoring and environmental noise, ensuring that the acquired audio data has sufficient sampling accuracy to meet the needs of subsequent sound intensity and phase feature extraction. At the same time, simultaneous acquisition from both sides can realize comparative analysis of dual-channel audio data.

[0025] The human body detection signal can also be acquired through the smart night light modules on both sides. The human body detection sensor S23 uses a pyroelectric infrared sensor, which is installed on the side of the smart night light module facing the outside of the mattress. It uses a sampling rate of 10Hz for real-time detection, which ensures real-time detection while effectively reducing device power consumption and avoiding unnecessary resource waste. The human body detection sensor S23 is mainly used to detect whether there is a human body around the corresponding side (left or right) of the device. Its detection principle is as follows: it detects the heat signal directly in front of the sensor in real time, presets a reasonable heat detection threshold. If the detected heat value exceeds the preset threshold, it is determined that there is a person around the device on that side; if the detected heat value is lower than the preset threshold, it is determined that there is no one around the device on that side.

[0026] In addition, the human body sensing signal can also be acquired by a specific sensor deployed on the device. The specific sensor can be a pressure sensor or a piezoelectric sensor, and its placement can be flexibly selected according to the actual application scenario. It can be installed above the mattress to directly sense whether the user is on the mattress, or it can be installed at the foot of the bed or other locations to indirectly determine the user's on / off state by sensing changes in bed pressure. The main control module receives the data collected by the specific sensor in real time, and after simple data processing, determines the user's on / off state corresponding to the left and right sides of the bed.

[0027] In step S120, the device usage status and audio characteristics are determined based on the human body sensing signal and the audio signal; It should be noted that the device usage status can include both the "on / off bed" status and the "human presence" status. The "on / off bed" status is determined based on the aforementioned human body sensing signals and is used to characterize whether the user is within the monitoring area corresponding to the bed. The "human presence" status can also be determined in conjunction with the human body sensing signals to identify whether there is a user subject capable of snoring around the device. By cross-checking the human presence status with the "on / off bed" status, invalid detection processes in unattended scenarios can be effectively eliminated, providing reliable state constraints for the subsequent triggering and execution of the snoring detection model and the determination of snoring attribution, thereby improving the system's detection accuracy and operating efficiency.

[0028] Meanwhile, the audio feature extraction process can be completed based on the acquired audio signal, specifically including a voiceprint feature array and sound energy intensity features within a preset window time TH3, where TH3 can be set to 0.5 seconds; the voiceprint feature array can be obtained by inputting the audio signal acquired by the audio acquisition sensor into a deep learning neural network model (such as a combination of convolutional neural network (CNN) and long short-term memory network (LSTM)) to complete feature classification, and then extracting the corresponding voiceprint feature array (including but not limited to Mel-frequency cepstral coefficients, Mel-frequency cepstral coefficient differences, nonlinear frequency band energy, and high-frequency energy proportion); the specific calculation method of the sound energy intensity feature is as follows: first, the acquired original audio signal is bandpass filtered, and the passband frequency is set to [500Hz, 4kHz] to filter out irrelevant low-frequency noise and high-frequency interference, then the filtered audio signal within the preset window time TH3 is extracted, and the absolute value of the filtered signal within the window is summed, and finally the absolute value sum is used as the sound energy intensity feature, providing accurate audio feature support for subsequent snoring detection and attribution judgment.

[0029] The steps for obtaining the Mel-frequency cepstral coefficients are as follows: The acquired raw audio signal is pre-emphasized, as shown in the following formula: ; in, t Representing the t The sampling point corresponds to the time. High-frequency attenuation compensation coefficient is used to boost high-frequency components and compensate for signal attenuation (e.g., ), This represents the sample value of the original audio signal at time t. This indicates that the original audio signal is at the 1st... t The sampled value at time -1; The above pre-emphasis processing can compensate for the high-frequency attenuation of the audio signal during transmission / acquisition, increase the relative amplitude of the high-frequency components of the signal, make the spectrum flatter, and thus enhance the recognizability of high-frequency features.

[0030] The pre-emphasized audio signal is segmented into frames, and a short-time window of signal from 40 milliseconds before time t to the current time is extracted. and the window signal Apply a Hamming window to obtain To reduce spectrum leakage. Then for Perform Fourier transform and calculate the amplitude spectrum. Based on the amplitude spectrum Calculate the corresponding power spectrum ,in, N The number of Fourier transform points; then the coefficients of the Mel filter bank. The power spectrum is filtered, and the energy of each filter bank is calculated and its logarithm is taken to obtain the following: , m For Mel filter bank index, M The number of Mel filter banks. Natural logarithm operation.

[0031] Then, feature compression and decorrelation operations are performed using discrete cosine transform to obtain the Mel-frequency cepstral coefficients (MFCC coefficients), as shown below: ； in, L To retain the number of coefficients, its value is, for example, 12. m For Mel spectrum coefficient index, It is a cosine function.

[0032] Finally, using a sliding window with a step size of 20 ms, the above process is repeated to obtain the MFCC coefficients at different times. Simultaneously, based on the MFCC coefficients, the Mel-spectral coefficient difference is calculated using the following formula, specifically expressed as: ; in, i For differential window index, K The difference order / window length determines the time range for difference calculation.

[0033] The Mel-frequency cepstral coefficient difference and the MFCC coefficient are combined to form a voiceprint feature array, providing multi-dimensional audio feature support for subsequent snoring detection and attribution determination.

[0034] It should be noted that the steps for obtaining the nonlinear frequency band energy are similar to those for Mel-Cepstral Coefficients. Specifically, the original audio undergoes pre-emphasis processing, frame segmentation, Hamming windowing, Fourier transform, and power spectrum calculation, consistent with the Mel-Cepstral Coefficients calculation method. Then, after obtaining the power spectrum, the Bark-scale filter bank coefficients can be acquired. To simulate the nonlinear frequency perception characteristics of human hearing; based on the calculated power spectrum and Bark-scale filter bank coefficients. The sub-band energy of each linear frequency band is calculated as follows: ; Finally, the above steps are repeated with a sliding time window of 20ms (milliseconds) to obtain the nonlinear frequency band energy characteristics at different times. There are 24 dimensions of this feature, which can be combined with features such as MFCC to form a multi-dimensional voiceprint feature array, providing richer audio feature support for snoring detection and attribution judgment.

[0035] Furthermore, the proportion of high-frequency energy can be calculated using nonlinear frequency band energy characteristics, as shown in the following formula: ; The numerator is the sum of the energies of the 18th to 24th nonlinear sub-bands (representing the high-frequency energy), and the denominator is the sum of the energies of all 24 nonlinear sub-bands, thus characterizing the proportion of high-frequency energy in the overall frequency band energy.

[0036] In step S130, based on the human body sensing signal, device usage status and audio characteristics, the corresponding feature combination is selected and input into the preset snoring detection model to detect snoring, so as to determine the current snoring status; Optionally, different snoring detection models can be constructed according to actual application scenarios and detection requirements. Corresponding feature combinations can be flexibly selected from the human body sensing signals, device usage status, and audio features, and input into the corresponding snoring detection model for snoring detection. For example, human body sensing signals, device usage status, and audio features can be simultaneously input into the snoring detection model for snoring detection; or only device usage status and audio features can be input into the snoring detection model for classification and prediction. The snoring detection model identifies and classifies snoring based on the input feature combinations, outputs the corresponding detection result, and determines whether the current state is snoring or non-snoring based on the detection result.

[0037] In step S140, if the current state is snoring, the initial snoring attribution probability is determined based on the preset sound intensity coefficient reference table, sound phase coefficient reference table, and audio characteristics. Based on the initial snoring attribution probability and the device usage status, the snoring attribution result is determined.

[0038] Optionally, if snoring is detected, the target area or bed where snoring may occur can be identified based on the distribution of people around and above the mattress during device use. By introducing sound energy intensity and sound phase features from the audio characteristics, and combining them with preset sound intensity and sound phase coefficient reference tables, the initial probabilities of snoring originating from the left and right beds can be calculated. Simultaneously, the voiceprint feature array from the audio characteristics can be matched with preset user prior information, and the voiceprint matching result can be determined through feature correlation coefficients. Finally, the initial probability of snoring origin, device usage status, and voiceprint matching result are weighted, fused, and cross-validated to output the final snoring origin result, clearly indicating whether the snoring originates from the left, right, or other locations. This achieves precise location of the snoring source, providing accurate data support for subsequent snoring intervention, tiered early warning, and user health management.

[0039] It should be noted that both the sound intensity coefficient reference table and the sound phase coefficient reference table are pre-built and stored, and can be dynamically updated in real time according to actual application scenarios and new sample data. The sound intensity coefficient reference table records the ratio of the sound intensity received by the left and right audio sensors when a single sound source is in different fixed positions. This table establishes the correspondence between different sound intensity ratios and the left or right bed position attribution for snoring, clarifying the bed matching coefficient corresponding to different intensity ratios. The sound phase coefficient reference table records the time delay characteristics between the signals received by the left and right audio sensors when a single sound source is in different fixed positions. This table establishes the correspondence between different signal time delays and the left or right bed position attribution for snoring, clarifying the probability of sound source location corresponding to different time delay characteristics, thus assisting in the accurate localization of the snoring source. The sound intensity feature is a quantized value of sound energy obtained by summing the absolute values ​​within a preset window time TH3 (e.g., 0.5 seconds) after bandpass filtering (passband frequency [500Hz, 4kHz]). It is used to characterize the strength of the snoring signal collected by the smart nightlights on both sides. The sound phase feature is a quantized value of the phase difference between the same snoring signal collected by the nightlights on both sides. It is used to assist in locating the sound source and to compensate for the deviation of a single sound intensity feature in locating a sound source at close range.

[0040] The user's prior information consists of pre-collected and stored feature data related to the users bound to the left and right sides. Specifically, this includes voiceprint feature arrays for the left and right bound users in their respective speaking states, voiceprint feature arrays for their snoring states, and physiological parameter feature arrays. The user's speaking and snoring states are obtained by real-time audio signal acquisition from the audio sensors in the left and right side nightlight modules of the device, which are then input into a pre-defined deep learning neural network model for classification and recognition. This deep learning neural network model can be a combination of a convolutional neural network (CNN) and a long short-term memory network (LSTM), or other deep learning models with audio classification capabilities, ensuring the accuracy of state classification.

[0041] Furthermore, the voiceprint feature array is used to accurately distinguish the voice features of different users. The feature types it includes include, but are not limited to, Mel-Cepstral Coefficients, Mel-Cepstral Coefficient Differences, Nonlinear Band Energy, and High-Frequency Energy Ratio. By combining multi-dimensional voiceprint features, the uniqueness of user voice recognition can be effectively improved, providing reliable prior support for subsequent snoring attribution judgment based on voiceprint features and reducing misjudgment problems caused by similar voices.

[0042] The physiological parameter feature array contains the user's relevant physiological data for the most recent day, such as the average heart rate, average respiratory rate, and heart rate variation characteristics in the resting state, as well as the average heart rate and average respiratory rate in the sleep state. The resting state is defined as the duration of no body movement for more than a preset threshold TH1 (e.g., 10 minutes). The heart rate variation characteristics are calculated by the standard deviation of the interval between heart rate peaks within a preset window time TH2 (e.g., 5 minutes). The sleep state is obtained by collecting signals from an array of pressure sensors or piezoelectric sensors placed on the mattress surface, and then analyzing them through a preset sleep stage model, providing prior physiological support for subsequent snoring attribution.

[0043] In this embodiment of the application, a snoring detection method is provided, comprising: acquiring human body sensing signals and audio signals; determining device usage status and audio characteristics based on the human body sensing signals and audio signals; selecting corresponding feature combinations based on the human body sensing signals, device usage status, and audio characteristics and inputting them into a preset snoring detection model for snoring detection to determine the current snoring state; if the current state is snoring, determining an initial snoring attribution probability based on a preset sound intensity coefficient reference table, sound phase coefficient reference table, and audio characteristics; and determining the snoring attribution result based on the initial snoring attribution probability and the device usage status. In this embodiment, the technical shortcomings of existing technologies in multi-person cohabitation scenarios are effectively solved: snoring detection can only determine the presence of snoring but cannot clearly identify the snoring's origin, and it lacks accuracy, a scientific benchmark reference system, and probability calculation logic. By acquiring human body induction signals and audio signals to determine the device's usage status and audio characteristics, a snoring detection model is adopted. This model combines dual feature fusion of sound intensity and sound phase features, fitting analysis of sound intensity coefficient reference tables and sound phase coefficient reference tables, and complete coefficient calibration, effectiveness evaluation mechanisms, and time delay and sound source azimuth angle calculations. Combined with the device's usage status, a weighted sum is performed to determine the snoring's origin. This not only improves the accuracy of snoring detection and origin determination, avoiding health misjudgments caused by mixed snoring data, but also adapts to different computing power scenarios, realizing a leap from generalized environmental monitoring to precise individual monitoring in sleep monitoring.

[0044] In one embodiment of this application, the audio features include voiceprint features, the preset snoring detection model is a snoring detection deep learning model or a snoring judgment model, and the step of detecting snoring based on the preset snoring detection model to determine the snoring state includes: The device usage status, human body sensing signals, and audio signals are input into the snoring detection deep learning model for feature processing and classification to obtain snoring classification results. If the probability value corresponding to the snoring classification result is greater than the preset snoring determination threshold, then the current state is determined to be snoring; otherwise, it is determined to be non-snoring. The voiceprint features and the device usage status are input into the snoring judgment model for analysis and judgment, and the corresponding snoring status is output.

[0045] Optionally, the preset snoring detection model can be either a deep learning model for snoring detection or a snoring judgment model, to adapt to different hardware computing power and application scenarios. The deep learning model for snoring detection can be a multimodal fusion deep learning model, such as a network structure combining a convolutional neural network (CNN) and a long short-term memory network (LSTM). This model can simultaneously perform feature fusion and joint discrimination on multimodal information such as temporal audio signals, human body sensing signals, and device usage status. The specific processing flow can be as follows: the device usage status, human body sensing signals, and audio signals are simultaneously input into the deep learning model for snoring detection. The model sequentially performs feature extraction, feature fusion, and classification on the multimodal data to obtain the corresponding snoring classification result and corresponding probability value. If the probability value is greater than the preset snoring judgment threshold, the current state is determined to be snoring; otherwise, it is determined to be non-snoring.

[0046] The snoring detection model is a lightweight discrimination model based on rule and feature matching, suitable for embedded devices with limited computing power. Its processing flow is as follows: the voiceprint features and device usage status are used as model inputs. The Euclidean distance or cosine correlation coefficient between the real-time voiceprint features and the user's prior voiceprint features is calculated. Combined with the sound energy threshold and the on / off bed status constraints, a joint judgment is made. If the preset conditions are met, it is determined to be a snoring state; otherwise, it is a non-snoring state. This enables flexible detection of snoring states under different architecture models and improves the applicability and robustness of the technical solution of this application.

[0047] In one embodiment of this application, the audio features include sound intensity features and sound phase features. If the current state is snoring, an initial snoring attribution probability is determined based on a preset sound intensity coefficient reference table, a sound phase coefficient reference table, and the audio features. The snoring attribution result is then determined based on the initial snoring attribution probability and the device usage status, including: Based on the preset sound intensity coefficient reference table, sound phase coefficient reference table, sound intensity characteristics and sound phase characteristics, the initial probability values ​​of snoring belonging to the left and right sides are determined respectively. Based on the device usage status and the initial probability value, the actual probability values ​​of snoring belonging to the left side and snoring belonging to the right side are obtained. If the actual probability value of the snoring sound belonging to the left side is greater than the actual probability value of the snoring sound belonging to the right side, then the snoring sound is determined to belong to the left side; otherwise, the snoring sound is determined to belong to the right side.

[0048] Optionally, based on a pre-built and dynamically updated reference table of sound intensity coefficients and a reference table of sound phase coefficients, the sound intensity features and sound phase features acquired and preprocessed in real time are matched using a lookup table to obtain the corresponding attribution coefficients. A pre-defined probability fusion algorithm is then used to weight and calculate the multi-dimensional attribution coefficients to obtain the initial probability value of snoring attribution to the left or right bed based solely on audio features. Then, by combining constraints such as personnel distribution and duration of use in the device's operation, the initial probability value is weighted, corrected, and calibrated to obtain the actual probability value of snoring attribution to the left or right bed. The actual probability values ​​on both sides are compared; if the actual probability value on the left is greater, the snoring is determined to originate from the left bed; otherwise, it is determined to originate from the right bed. Through the joint constraints of multi-dimensional audio features and device usage status, the accuracy and robustness of snoring source localization are effectively improved.

[0049] In one embodiment of this application, determining the initial probability values ​​for assigning snoring to the left and right sides based on the preset sound intensity coefficient reference table, sound phase coefficient reference table, sound intensity features, and sound phase features includes: The preset sound intensity coefficient reference table and sound phase coefficient reference table are fitted respectively to obtain the sound intensity fitting curve and the sound phase fitting curve. Based on the sound intensity fitting curve and the sound intensity characteristics, a first initial probability value for assigning snoring to the left and a second initial probability value for assigning snoring to the right are determined. Based on the sound phase fitting curve and the sound phase characteristics, a third initial probability value for assigning snoring to the left and a fourth initial probability value for assigning snoring to the right are determined.

[0050] Optionally, curve fitting is performed on the sound intensity reference value and sound phase reference value recorded in the preset sound intensity coefficient reference table and sound phase coefficient reference table, respectively. For example, the preset azimuth angle of the four calibration points is used as the horizontal axis and the reference sound intensity ratio corresponding to each calibration point is used as the vertical axis. The fitting method can be linear fitting or nonlinear fitting, thereby obtaining the sound intensity fitting curve and sound phase fitting curve that can characterize the mapping relationship between feature value and belonging probability. Then, the real-time acquired sound intensity features can be substituted into the sound intensity fitting curve to estimate the current sound source azimuth angle and calculate the first initial probability value of the snoring belonging to the left bed and the second initial probability value of the snoring belonging to the right bed. For example, if the estimated current sound source azimuth angle is 'a', then the first initial probability value of the snoring belonging to the left bed is 'a+90° / 180°', and the second initial probability value of the snoring belonging to the right bed can be '1-the first initial probability value'. At the same time, the real-time acquired sound phase features can be substituted into the sound phase fitting curve, and the third initial probability value of the snoring belonging to the left bed and the fourth initial probability value of the snoring belonging to the right bed can be calculated in the same way as above, providing a basis for subsequent multi-feature probability fusion and snoring attribution determination.

[0051] In one embodiment of this application, the device usage state includes an out-of-bed state and a sleep state. The step of obtaining the actual probability values ​​of snoring belonging to the left and snoring belonging to the right based on the device usage state and the initial probability value includes: The weighted sum of the in / out bed state, sleep state, first initial probability value and third initial probability value is used to obtain the first actual probability value of snoring belonging to the left side; The weighted sum of the in / out bed state, sleep state, second initial probability value and fourth initial probability value is used to obtain the second actual probability value of snoring belonging to the right side.

[0052] Optionally, preset weights are assigned to the device usage status (in / out of bed, sleep state), the first initial probability value (initial probability of left-side attribution obtained based on sound intensity characteristics), and the third initial probability value (initial probability of left-side attribution obtained based on sound phase characteristics) of the left-side bed. The weight allocation is set based on the degree of influence of each parameter on the snoring attribution determination. For example, if the left-side bed is in the "in bed + sleep" state, it indicates that there is a user in that bed who is snoring. The sleep state is given a high weight (e.g., 0.3), the in / out of bed state is given a medium weight (e.g., 0.2), and the first and third initial probability values ​​are each given a weight of 0.25. If the left-side bed is in the "out of bed" state, it indicates that there is no user. The in / out of bed state is given a very low weight (e.g., 0), and the weights of the first and third initial probability values ​​are reduced (e.g., 0.2 each). The weight of the sleep state is set to 0, thereby reflecting the constraining effect of the device usage status.

[0053] Then, according to the preset weights mentioned above, the on / off state, sleep state, first initial probability value and third initial probability value of the left bed are weighted and summed. Through weight fusion, the collaborative constraint of multi-dimensional information is realized, and finally the first actual probability value of snoring belonging to the left bed is obtained.

[0054] Using the same weighted logic and weight allocation standard as the left-side bed, the device usage status (in / out of bed, sleep state), the second initial probability value (initial probability of right-side attribution based on sound intensity characteristics), and the fourth initial probability value (initial probability of right-side attribution based on sound phase characteristics) of the right-side bed are weighted and summed to ensure consistency and fairness in the calculation of actual probability values ​​on both sides. This results in a second actual probability value for attributing snoring to the right-side bed. If the first actual probability value is greater than the second actual probability value, the snoring is attributed to the user on the left-side bed; otherwise, the snoring is attributed to the user on the right-side bed.

[0055] By weighted and fused the device usage status with the initial probability values ​​corresponding to the audio features, the system effectively combines user status constraints with sound feature determination, eliminates the possibility of snoring attribution to beds without users, corrects the probability bias caused by single audio feature calculation, and further improves the accuracy and reliability of the actual probability value of snoring attribution.

[0056] In one embodiment of this application, after determining the device usage status and audio characteristics, the method further includes: Determine if the current sound meets the preset calibration conditions; If the preset sound calibration conditions are met, the preset sound intensity coefficient reference table and sound phase coefficient reference table are calibrated respectively to obtain the calibrated sound intensity coefficient and calibrated sound phase coefficient at the corresponding calibration point. The effectiveness of the calibrated sound intensity coefficient and the calibrated sound phase coefficient were evaluated respectively. If the validity assessment result is valid, then the sound intensity coefficient and sound phase coefficient of the corresponding calibration point are updated based on the calibrated sound intensity coefficient and the calibrated sound phase coefficient.

[0057] It should be noted that the preset sound calibration conditions can include two categories: the current usage state of the device and the audio matching state. The current usage state of the device must meet any of the following conditions and the duration must exceed the preset threshold TH4 (e.g., 30 seconds). Taking a mattress as an example, the matching conditions corresponding to the current usage state of the device are as follows: 1. There is someone on only one side of the mattress and no one above the mattress (e.g., the user is not on the bed but is moving around the bed); 2. There is no one on either side of the mattress and only one person above the mattress (e.g., the user is not moving around the bed but is lying on one side of the bed). This is used to identify the user subject who can produce snoring. The matching status of audio signals is further divided into sound energy intensity matching status and voiceprint feature matching status. Sound energy intensity refers to the feature value obtained by taking the absolute value of the filtered signal and summing it within a preset window time TH3 (e.g., 0.5 seconds) after bandpass filtering the original audio signal (passband frequency is typically [500Hz, 4kHz]). This value characterizes the energy level of the current ambient sound and is the basis for determining the presence of valid sounds such as suspected snoring. Voiceprint features refer to a set of multi-dimensional features extracted from the audio signal that can be used to distinguish different user voices, such as Mel-frequency cepstral coefficients and differences, nonlinear frequency band energy characteristics, and the proportion of high-frequency energy. The matching conditions are as follows: if the detected sound energy intensity in the smart nightlights on both sides is greater than a preset value, then the sound energy intensity is determined to meet the preset requirements; at the same time, the voiceprint feature arrays acquired in real time by the smart nightlights on both sides are calculated, and the correlation coefficients are respectively compared with the voiceprint feature arrays in the speaking state in the prior information of the users on both sides, resulting in 4 correlation coefficients and taking the maximum value. If the maximum value is greater than the threshold TH6 (e.g., 0.75) for a duration exceeding the threshold TH7 (e.g., 3 seconds) within the window time threshold TH5 (e.g., 10 seconds), then the voiceprint matching state is determined to meet the preset requirements; when both the device usage state and the audio matching state meet the preset requirements, the operation of calibrating the sound intensity coefficient reference table and the sound phase coefficient reference table is performed so that the latest sound intensity coefficient reference table and the sound phase coefficient reference table can be used to determine the snoring attribution.

[0058] Optionally, when the aforementioned preset sound calibration conditions are met, calibration operations can be performed on the pre-constructed sound intensity coefficient reference table and sound phase coefficient reference table based on the currently acquired raw audio signal. During the calibration process, for each preset calibration point in the reference table, the audio signal is preprocessed through filtering, noise reduction, and feature extraction to calculate the calibrated sound intensity coefficient (i.e., the calibrated ratio of the sound intensity received by the left and right audio sensors at that point) and the calibrated sound phase coefficient (i.e., the calibrated delay of the signals received by the left and right audio sensors at that point), ensuring that the calibration coefficients accurately match the current actual usage scenario. Then, the validity of the calibrated sound intensity coefficient and sound phase coefficient can be evaluated to determine whether the calibration results meet the preset standards and can be used for reference table updates. If the validity assessment result is valid, meaning that both the calibrated sound intensity coefficient and the calibrated sound phase coefficient meet the preset validity standard, then based on this valid calibration coefficient, the original reference values ​​of the corresponding calibration points in the sound intensity coefficient reference table and the sound phase coefficient reference table are updated. The sound intensity coefficient reference value of each calibration point is replaced with the calibrated sound intensity coefficient, and the sound phase coefficient reference value of the corresponding point is replaced with the calibrated sound phase coefficient. If the assessment result is invalid, then the calibration result is discarded, and the reference performance and reference values ​​remain unchanged to avoid invalid calibration affecting the reliability of the reference table. Through the above calibration and update process, deviations in the reference table coefficients caused by environmental changes, equipment wear and tear, etc., can be corrected in real time, ensuring that the sound intensity coefficient reference table and the sound phase coefficient reference table always maintain high accuracy. This provides a reliable reference for subsequent snoring attribution probability calculation and sound source localization, further improving the robustness and adaptability of the entire snoring attribution determination scheme.

[0059] It should be noted that the reference table can be updated by performing a Gaussian filter and weighted summation on multiple sets of historical calibration data for the calibration points. The specific implementation is as follows: First, construct the Gaussian filter template, as shown below: ; in, i The corresponding filter template number, that is, the template number... i One value; Pi; The standard deviation can be 20. e It is a natural constant; N This represents the average template length; for example, a template length of 60. N Take 30; Then, the constructed Gaussian filter template is normalized to obtain the normalized Gaussian coefficients. The details are as follows: ; in, M This is the length of the Gaussian template, which is 60. Then, extract the first half of the Gaussian filter template and multiply all coefficients by 2 to ensure that the sum of the filter coefficients is 1.

[0060] Finally, the historical calibration data of the calibration point is weighted and summed using the normalized Gaussian coefficients. The summation result is used as the latest reference value of the calibration point in the sound intensity coefficient reference table, thereby realizing the dynamic updating of the reference table, ensuring that the reference value can smoothly reflect the changing trend of historical calibration data, and improving the stability and accuracy of the reference table.

[0061] In one embodiment of this application, the audio signal includes a left audio signal and a right audio signal. The step of calibrating the preset sound intensity coefficient reference table and sound phase coefficient reference table to obtain the calibrated sound intensity coefficient and calibrated sound phase coefficient at the corresponding calibration points includes: The ratio of the sound intensity of the left audio signal to the sound intensity of the right audio signal is calculated and used as the calibrated sound intensity coefficient. The azimuth angle of the sound source is calculated based on the left and right audio signals, and used as the calibrated sound phase coefficient.

[0062] Optionally, for the calibration of the sound intensity coefficient, the ratio between the sound intensity of the left audio signal and the sound intensity of the right audio signal is calculated, and this ratio is directly used as the calibrated sound intensity coefficient, thereby quantifying the energy difference between the left and right audio signals at different locations. Secondly, for the calibration of the sound phase coefficient, the signal delay between the left and right audio signals is calculated, and this delay is used as the sound phase coefficient. It should be noted that this delay can be characterized and evaluated using the sound source azimuth angle; that is, the calibration of the sound phase coefficient can be achieved by directly calculating the sound source azimuth angles corresponding to the left and right audio signals. This sound source azimuth angle reflects the spatial position angle of the sound source relative to the sensor. Through the above methods, accurate and reliable calibrated sound intensity coefficients and sound phase coefficients can be obtained at each calibration point, providing an objective and accurate data basis for the subsequent effectiveness evaluation and dynamic updating of the reference table.

[0063] In one embodiment of this application, the effectiveness evaluation of the calibrated sound intensity coefficient and the calibrated sound phase coefficient includes: Obtain historical sound intensity reference data and historical sound phase reference data corresponding to the calibration point within a preset time range; Based on the historical sound intensity benchmark data and the historical sound phase benchmark data, the historical sound intensity benchmark mean and the historical sound phase benchmark mean are calculated respectively. Calculate the sound intensity difference between the calibrated sound intensity coefficient and the historical sound intensity benchmark mean, and the sound phase difference between the calibrated sound phase coefficient and the historical sound phase benchmark mean; If the absolute value of the ratio between the sound intensity difference and the historical sound intensity benchmark mean is greater than the preset sound intensity threshold, then the calibrated sound intensity coefficient is invalid; otherwise, the calibrated sound intensity coefficient is valid. If the absolute value of the ratio between the sound phase difference and the historical sound intensity benchmark mean is greater than the preset sound phase threshold, then the calibrated sound phase coefficient is invalid; otherwise, the calibrated sound phase coefficient is valid.

[0064] Optionally, the current calibration point is determined, and historical sound intensity reference data and historical sound phase reference data corresponding to the current calibration point within a preset time range are obtained. The preset time range can be set according to actual calibration needs (such as historical data of each effective calibration within the corresponding time period from 7 days before the current time to the current time). The historical reference data are the sound intensity coefficients and sound phase coefficients recorded in multiple effective calibrations of the calibration point in the past, ensuring that the reference data is representative and has reference value, and can reflect the normal coefficient fluctuation range of the point.

[0065] Then, based on the acquired historical sound intensity benchmark data, the historical sound intensity benchmark mean is calculated using an arithmetic mean algorithm; similarly, based on the historical sound phase benchmark data, the historical sound phase benchmark mean is calculated. The benchmark mean is used as a reference standard to judge whether the current calibrated coefficient is normal, avoiding evaluation deviation caused by fluctuations in a single historical data.

[0066] Next, the difference in sound intensity between the calibrated sound intensity coefficient and the historical average sound intensity is calculated. The absolute value of the ratio between this difference and the historical average sound intensity is then compared with a preset sound intensity threshold TH8 (e.g., 0.2). If the difference is greater than the preset threshold, it indicates that the calibrated sound intensity coefficient deviates too much from the historical normal level, which may be due to environmental interference, equipment malfunction, or other issues. In this case, the calibrated sound intensity coefficient is deemed invalid. If the difference is less than or equal to the preset threshold, it indicates that the calibrated sound intensity coefficient is within the historical normal fluctuation range. In this case, the calibrated sound intensity coefficient is deemed valid.

[0067] For the validity evaluation of the calibrated sound phase coefficient, the same logic as the sound intensity coefficient evaluation is adopted. Specifically, the difference between the calibrated sound phase coefficient of the current calibration point and the historical average sound phase reference is calculated to obtain the sound phase difference. The absolute value of the ratio between the sound phase difference and the historical average sound phase reference is compared with the preset sound phase threshold. If it is greater than the preset sound phase threshold, the calibrated sound phase coefficient is determined to be invalid. If it is less than or equal to the preset sound phase threshold, the calibrated sound phase coefficient is determined to be valid.

[0068] By using the dual judgment based on historical benchmark averages and preset thresholds, effective calibration coefficients can be accurately screened out, and interference-induced or abnormal calibration data can be eliminated. This provides a reliable guarantee for the dynamic updating of subsequent sound intensity coefficient reference tables and sound phase coefficient reference tables, ensuring that the reference tables always maintain high accuracy and stability.

[0069] For example, calibration points can be as follows Figure 5 As shown, P1 is the location of the left audio sensor, P2 is the location of the right audio sensor, P0 is the midpoint of the line connecting P1 and P2, S51 is the cross-section of the mattress, and P6, P7, P8, and P9 are four preset fixed calibration points, where P6 is the calibration point on the left side of the bed (not on the mattress), P7 is the calibration point on the left side of the bed (on the mattress), P8 is the calibration point on the right side of the bed (on the mattress), and P9 is the calibration point on the right side of the bed (not on the mattress). The specific calibration process for the sound intensity coefficient table is as follows: the calibration point locations are determined based on the distribution of people: if the mattress... If there are no people above the mattress but there are people on the left side of the mattress, then point P6 is selected as the current calibration point; if there are no people on either side of the mattress but there are people on the left side of the mattress above, then point P7 is selected as the current calibration point; if there are no people on either side of the mattress but there are people on the right side of the mattress above, then point P8 is selected as the current calibration point; if there are no people above the mattress but there are people on the right side of the mattress, then point P9 is selected as the current calibration point. This allows for precise selection of calibration points in different scenarios, ensuring the accuracy of sound intensity coefficient calibration and scenario adaptability.

[0070] In one embodiment of this application, calculating the azimuth angle of the sound source based on the left audio signal and the right audio signal includes: The left and right audio signals are respectively subjected to frequency domain transformation to obtain the left frequency domain signal and the right frequency domain signal; Based on the left-side and right-side frequency domain signals, cross-correlation features are constructed, and the cross-correlation features are weighted. The time delay between the left audio signal and the right audio signal is determined based on the weighted cross-correlation characteristics. The azimuth angle of the sound source is calculated based on the time delay, sound speed, and the distance between the left and right audio acquisition positions.

[0071] Optionally, the left and right audio signals are preprocessed (pre-emphasis processing, frame segmentation processing, and Hamming window processing are performed sequentially); Fourier transforms are then performed on the preprocessed left and right audio signals to obtain the left frequency domain signal. and the right-side frequency domain signal The cross-power spectrum is obtained by calculating the conjugate product of the left-side frequency domain signal and the right-side frequency domain signal, as shown below: The cross-power spectrum is then weighted using PHAT to obtain the weighted cross-spectrum, as shown in the following formula: Performing an inverse Fourier transform on the weighted cross-spectrum yields the generalized cross-correlation function, as shown below: The generalized cross-correlation function is cyclically shifted, moving the first sampling point to the center of the time window. The shifted generalized cross-correlation function is then interpolated, and the peak value is detected to find the maximum position point. The x-coordinate of this position point represents the time delay τ between the left and right audio signals. Based on the time delay τ, the sound speed c, and the distance d between the left and right audio sampling positions, the azimuth angle of the sound source is calculated, as shown below: ; in, conj ( ) represents the conjugate operation; | |This is a modulo operation; epsilon To prevent the constant from being zero, the value can be 1e-10; arcsin ( () is the arcsine function; Speed ​​of sound; d This is the distance between the placement positions of the left and right audio sensors. The value should be slightly smaller than the width of the bed. For example, for a 1.5-meter wide bed, the value could be 1.45 meters.

[0072] It should be noted that the calibration, validity evaluation, and updating process for the sound phase coefficient reference table is completely consistent with the corresponding process for the sound intensity coefficient reference table. Furthermore, the construction method and calibration point selection rules for the phase reference coefficient table are also consistent with those for the sound intensity reference coefficient table. Specifically, both types of reference tables share the same four fixed calibration points (P6, P7, P8, P9), differing only in the coefficient calculation dimension: the sound intensity coefficient table is based on the left-right sound intensity ratio, while the sound phase coefficient table is based on the sound source azimuth angle. A standard azimuth angle is predefined for each calibration point: P6 (calibration point on the left side of the bed, not on the mattress), corresponding to a standard azimuth angle of 75°; P7 (calibration point on the left side of the bed, on the mattress), corresponding to a standard azimuth angle of 30°; P8 (calibration point on the right side of the bed, on the mattress), corresponding to a standard azimuth angle of -30°; and P9 (calibration point on the right side of the bed, not on the mattress), corresponding to a standard azimuth angle of -75°. For example, the azimuth angle of the sound source can be as follows: Figure 6 As shown, P1 is the location of the left audio sensor, P2 is the location of the right audio sensor, P0 is the midpoint of the line connecting P1 and P2, and P4 is the location of the sound source. The azimuth angle of the sound source is defined as the angle formed by the line connecting P0 and the location of the sound source p4, and the perpendicular bisector of P0 (the straight line perpendicular to the line connecting P1 and P2), with the counterclockwise direction of this angle being positive. This is used to quantify the azimuth information of the sound source relative to the sensor array.

[0073] In one embodiment of this application, after determining the snoring attribution result, the method further includes: Based on the human body sensing signals and audio signals, obtain multidimensional feature data of the current user; Based on the pre-stored user prior information and the current user's multi-dimensional feature data, the identity of the current user is verified. If the current user's identity verification result is abnormal, output an identity verification abnormality prompt message to the current user and obtain user feedback information; Based on the user feedback information, the user prior information is adaptively updated, and the updated user prior information is synchronously stored.

[0074] Optionally, after determining the snoring attribution result based on the snoring state, device usage state, and audio characteristics, a user identity verification and prior information update process is further executed. First, based on human body sensing signals and audio signals, multidimensional feature data of the current user is extracted and obtained. The multidimensional feature data includes the mean value of the voiceprint feature array in the snoring state, the average heart rate in the resting state, the average respiratory rate in the resting state, the heart rate change characteristics in the resting state, the average heart rate in the sleep state, and the average respiratory rate in the sleep state. The user identity verification process can be executed after the entire night's sleep ends. Taking a user in the left bed as an example, if snoring is detected during sleep, the correlation coefficient between the mean of the voiceprint feature array under the current snoring state and the corresponding voiceprint feature array in the pre-stored user prior information is calculated. When the correlation coefficient is greater than the preset threshold TH9 (e.g., 0.8), physiological feature verification continues; otherwise, the user identity verification result is deemed abnormal. During the physiological feature verification process, the differences between the heart rate, respiratory rate, and other feature parameters under resting and sleep states and the user's prior information are calculated. If the heart rate difference is greater than the preset threshold TH10 (e.g., 0.2 times the prior heart rate value) or the respiratory difference is greater than the preset threshold TH11 (e.g., 0.3 times the prior respiratory value), then the user is deemed abnormal. If the user's identity verification result is abnormal, the user is deemed to have successfully verified their identity. When the user's identity verification result is abnormal, the application will notify the user of the verification error and ask if the sleep data for that night belongs to the user, thereby obtaining user feedback. Based on the user feedback, the user's prior information will be adaptively updated. If the user reports that the abnormal data belongs to the user, the snoring voiceprint features, resting state physiological features, and sleep state physiological features in the user's prior information will be updated using a weighted method of historical values ​​and current values. The updated user prior information will be synchronized to the server for storage, thereby ensuring the accuracy and timeliness of the user's prior information and improving the reliability of subsequent snoring attribution determination and user identification.

[0075] In this embodiment, the technical shortcomings of existing technologies in multi-person cohabitation scenarios are effectively solved: snoring detection can only determine the presence of snoring but cannot clearly identify the snoring's origin, and the detection accuracy is insufficient, lacking a scientific benchmark reference system and probability calculation logic. By acquiring human body sensing signals and audio signals to determine the device's usage status and audio characteristics, a snoring detection model is adopted, combining dual feature fusion of sound intensity and sound phase features, fitting analysis of sound intensity coefficient reference tables and sound phase coefficient reference tables, and a complete coefficient calibration, effectiveness evaluation mechanism, and time delay and sound source azimuth angle calculation. Combined with the device's usage status, a weighted sum is performed to determine the snoring's origin. This not only improves the accuracy of snoring detection and origin judgment, avoiding health misjudgments caused by mixed snoring data, but also adapts to different computing power scenarios, realizing a leap from generalized environmental monitoring to precise individual monitoring of sleep.

[0076] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0077] In one embodiment, a snoring detection device is provided, which corresponds one-to-one with the snoring detection method described in the above embodiments. For example... Figure 7 As shown, the snoring detection device includes an information acquisition unit 10, a status and feature determination unit 20, a snoring detection unit 30, and a snoring attribution determination unit 40. Detailed descriptions of each functional module are as follows: Information acquisition unit 10 is used to acquire human body sensing signals and audio signals; The status and feature determination unit 20 is used to determine the device usage status and audio features based on the human body sensing signal and the audio signal. The snoring detection unit 30 is used to select corresponding feature combinations based on the human body sensing signal, device usage status and audio characteristics, and input them into a preset snoring detection model to detect snoring and determine the current snoring status. The snoring attribution determination unit 40 is used to determine the initial snoring attribution probability based on a preset sound intensity coefficient reference table, sound phase coefficient reference table and audio characteristics if the current snoring state is in progress, and to determine the snoring attribution result based on the initial snoring attribution probability and the device usage status.

[0078] In one embodiment of this application, the audio features include voiceprint features, the preset snoring detection model is a snoring detection deep learning model or a snoring judgment model, and the snoring detection unit 30 is further used for: The device usage status, human body sensing signals, and audio signals are input into the snoring detection deep learning model for feature processing and classification to obtain snoring classification results. If the probability value corresponding to the snoring classification result is greater than the preset snoring determination threshold, then the current state is determined to be snoring; otherwise, it is determined to be non-snoring. The voiceprint features and the device usage status are input into the snoring judgment model for analysis and judgment, and the corresponding snoring status is output.

[0079] In one embodiment of this application, the audio features include sound intensity features and sound phase features, and the snoring attribution determination unit 40 is further configured to: Based on the preset sound intensity coefficient reference table, sound phase coefficient reference table, sound intensity characteristics and sound phase characteristics, the initial probability values ​​of snoring belonging to the left and right sides are determined respectively. Based on the device usage status and the initial probability value, the actual probability values ​​of snoring belonging to the left side and snoring belonging to the right side are obtained. If the actual probability value of the snoring sound belonging to the left side is greater than the actual probability value of the snoring sound belonging to the right side, then the snoring sound is determined to belong to the left side; otherwise, the snoring sound is determined to belong to the right side.

[0080] In one embodiment of this application, the device usage state includes an out-of-bed state and a sleep state, and the snoring attribution determination unit 40 is further used for: The preset sound intensity coefficient reference table and sound phase coefficient reference table are fitted respectively to obtain the sound intensity fitting curve and the sound phase fitting curve. Based on the sound intensity fitting curve and the sound intensity characteristics, a first initial probability value for assigning snoring to the left and a second initial probability value for assigning snoring to the right are determined. Based on the sound phase fitting curve and the sound phase characteristics, a third initial probability value for assigning snoring to the left and a fourth initial probability value for assigning snoring to the right are determined. The weighted sum of the in / out bed state, sleep state, first initial probability value and third initial probability value is used to obtain the first actual probability value of snoring belonging to the left side; The weighted sum of the in / out bed state, sleep state, second initial probability value and fourth initial probability value is used to obtain the second actual probability value of snoring belonging to the right side.

[0081] In one embodiment of this application, the device further includes: an identity verification and user prior information update unit, used for: Based on the human body sensing signals and audio signals, obtain multidimensional feature data of the current user; Based on the pre-stored user prior information and the current user's multi-dimensional feature data, the identity of the current user is verified. If the current user's identity verification result is abnormal, output an identity verification abnormality prompt message to the current user and obtain user feedback information; Based on the user feedback information, the user prior information is adaptively updated, and the updated user prior information is synchronously stored.

[0082] In one embodiment of this application, the apparatus further includes a reference table updating unit, configured to: Determine if the current sound meets the preset calibration conditions; If the preset sound calibration conditions are met, the preset sound intensity coefficient reference table and sound phase coefficient reference table are calibrated respectively to obtain the calibrated sound intensity coefficient and calibrated sound phase coefficient at the corresponding calibration point. The effectiveness of the calibrated sound intensity coefficient and the calibrated sound phase coefficient were evaluated respectively. If the validity assessment result is valid, then the sound intensity coefficient and sound phase coefficient of the corresponding calibration point are updated based on the calibrated sound intensity coefficient and the calibrated sound phase coefficient.

[0083] In one embodiment of this application, the audio signal includes a left audio signal and a right audio signal, and the reference table update unit is further configured to: The ratio of the sound intensity of the left audio signal to the sound intensity of the right audio signal is calculated and used as the calibrated sound intensity coefficient. The azimuth angle of the sound source is calculated based on the left and right audio signals, and used as the calibrated sound phase coefficient.

[0084] In one embodiment of this application, the reference table updating unit is further configured to: Obtain historical sound intensity reference data and historical sound phase reference data corresponding to the calibration point within a preset time range; Based on the historical sound intensity benchmark data and the historical sound phase benchmark data, the historical sound intensity benchmark mean and the historical sound phase benchmark mean are calculated respectively. Calculate the sound intensity difference between the calibrated sound intensity coefficient and the historical sound intensity benchmark mean, and the sound phase difference between the calibrated sound phase coefficient and the historical sound phase benchmark mean; If the absolute value of the ratio between the sound intensity difference and the historical sound intensity benchmark mean is greater than the preset sound intensity threshold, then the calibrated sound intensity coefficient is invalid; otherwise, the calibrated sound intensity coefficient is valid. If the absolute value of the ratio between the sound phase difference and the historical sound intensity benchmark mean is greater than the preset sound phase threshold, then the calibrated sound phase coefficient is invalid; otherwise, the calibrated sound phase coefficient is valid.

[0085] In one embodiment of this application, the reference table updating unit is further configured to: The left and right audio signals are respectively subjected to frequency domain transformation to obtain the left frequency domain signal and the right frequency domain signal; Based on the left-side and right-side frequency domain signals, cross-correlation features are constructed, and the cross-correlation features are weighted. The time delay between the left audio signal and the right audio signal is determined based on the weighted cross-correlation characteristics. The azimuth angle of the sound source is calculated based on the time delay, sound speed, and the distance between the left and right audio acquisition positions.

[0086] In this embodiment, the technical shortcomings of existing technologies in multi-person cohabitation scenarios are effectively solved: snoring detection can only determine the presence of snoring but cannot clearly identify the snoring's origin, and the detection accuracy is insufficient, lacking a scientific benchmark reference system and probability calculation logic. By acquiring human body sensing signals and audio signals to determine the device's usage status and audio characteristics, a snoring detection model is adopted, combining dual feature fusion of sound intensity and sound phase features, fitting analysis of sound intensity coefficient reference tables and sound phase coefficient reference tables, and a complete coefficient calibration, effectiveness evaluation mechanism, and time delay and sound source azimuth angle calculation. Combined with the device's usage status, a weighted sum is performed to determine the snoring's origin. This not only improves the accuracy of snoring detection and origin judgment, avoiding health misjudgments caused by mixed snoring data, but also adapts to different computing power scenarios, realizing a leap from generalized environmental monitoring to precise individual monitoring of sleep.

[0087] Specific limitations regarding snoring detection devices can be found in the limitations of snoring detection methods described above, and will not be repeated here. Each module in the aforementioned snoring detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0088] In one embodiment, a computer device is provided, which may be a terminal device, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, and network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a readable storage medium storing computer-readable instructions. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer-readable instructions implement a snoring detection method. The readable storage medium provided in this embodiment includes both non-volatile and volatile readable storage media.

[0089] In this application embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, it implements the steps of the snoring detection method described above.

[0090] In this embodiment of the application, a readable storage medium is provided, which stores computer-readable instructions. When the computer-readable instructions are executed by a processor, they implement the steps of the snoring detection method described above.

[0091] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, these computer-readable instructions can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0092] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0093] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for detecting snoring, characterized in that, The method includes: Acquire human body sensor signals and audio signals; Based on the human body sensing signals and audio signals, the device usage status and audio characteristics are determined. The device usage status includes the / out-of-bed state and the sleep state. The audio characteristics include sound intensity characteristics and sound phase characteristics. Based on the human body sensing signals, device usage status, and audio characteristics, the corresponding feature combinations are selected and input into the preset snoring detection model to detect snoring and determine the current snoring status. If the current snoring state is in progress, based on the preset sound intensity coefficient reference table, sound phase coefficient reference table, sound intensity characteristics, and sound phase characteristics, the initial probability values ​​of the snoring sound belonging to the left and right sides are determined respectively. The initial probability values ​​of the snoring sound belonging to the left and right sides are weighted and calculated according to the in / out-of-bed state and the sleep state respectively to obtain the actual probability of the left side and the actual probability of the right side. The final snoring sound belonging is determined by comparing the actual probability of the left side and the actual probability of the right side.

2. The snoring detection method according to claim 1, characterized in that, The audio features include voiceprint features. The preset snoring detection model is a deep learning model for snoring detection or a snoring judgment model. The step of selecting corresponding feature combinations based on the human body sensing signals, device usage status, and audio features and inputting them into the preset snoring detection model to detect snoring and determine the current snoring state includes: The device usage status, human body sensing signals, and audio signals are input into the snoring detection deep learning model for feature processing and classification to obtain snoring classification results. If the probability value corresponding to the snoring classification result is greater than the preset snoring determination threshold, then the current state is determined to be snoring; otherwise, it is determined to be non-snoring. The voiceprint features and the device usage status are input into the snoring judgment model for analysis and judgment, and the corresponding snoring status is output.

3. The snoring detection method according to claim 1, characterized in that, Based on a preset sound intensity coefficient reference table, a sound phase coefficient reference table, sound intensity characteristics, and sound phase characteristics, initial probability values ​​for assigning snoring to the left and right sides are determined respectively. These initial probability values ​​are then weighted and calculated based on the on / off-bed state and sleep state to obtain the actual probability of assigning snoring to the left and right sides, including: The preset sound intensity coefficient reference table and sound phase coefficient reference table are fitted respectively to obtain the sound intensity fitting curve and the sound phase fitting curve. Based on the sound intensity fitting curve and the sound intensity characteristics, a first initial probability value for assigning snoring to the left and a second initial probability value for assigning snoring to the right are determined. Based on the sound phase fitting curve and the sound phase characteristics, a third initial probability value for assigning snoring to the left and a fourth initial probability value for assigning snoring to the right are determined. The weighted sum of the in / out bed state, sleep state, first initial probability value and third initial probability value is obtained to obtain the first actual probability value of snoring belonging to the left side; The weighted sum of the in / out bed state, sleep state, second initial probability value and fourth initial probability value is used to obtain the second actual probability value of snoring belonging to the right side.

4. The snoring detection method according to any one of claims 1 to 3, characterized in that, After determining the attribution of snoring sounds, the following steps are also included: Based on the human body sensing signals and audio signals, obtain multidimensional feature data of the current user; Based on the pre-stored user prior information and the current user's multi-dimensional feature data, the identity of the current user is verified. If the current user's identity verification result is abnormal, output an identity verification abnormality prompt message to the current user and obtain user feedback information; Based on the user feedback information, the user prior information is adaptively updated, and the updated user prior information is synchronously stored.

5. The snoring detection method according to any one of claims 1 to 3, characterized in that, After determining the device usage status and audio characteristics, the process also includes: Determine if the current sound calibration conditions are met; If the preset sound calibration conditions are met, the preset sound intensity coefficient reference table and sound phase coefficient reference table are calibrated respectively to obtain the calibrated sound intensity coefficient and calibrated sound phase coefficient at the corresponding calibration point. The effectiveness of the calibrated sound intensity coefficient and the calibrated sound phase coefficient were evaluated respectively. If the validity assessment result is valid, then the sound intensity coefficient and sound phase coefficient of the corresponding calibration point are updated based on the calibrated sound intensity coefficient and the calibrated sound phase coefficient.

6. The snoring detection method according to claim 5, characterized in that, The audio signal includes a left audio signal and a right audio signal. The step of calibrating the preset sound intensity coefficient reference table and sound phase coefficient reference table to obtain the calibrated sound intensity coefficient and calibrated sound phase coefficient at the corresponding calibration points includes: The ratio of the sound intensity of the left audio signal to the sound intensity of the right audio signal is calculated and used as the calibrated sound intensity coefficient. The azimuth angle of the sound source is calculated based on the left and right audio signals, and used as the calibrated sound phase coefficient.

7. The snoring detection method according to claim 5, characterized in that, The effectiveness evaluation of the calibrated sound intensity coefficient and the calibrated sound phase coefficient includes: Obtain historical sound intensity reference data and historical sound phase reference data corresponding to the calibration point within a preset time range; Based on the historical sound intensity benchmark data and the historical sound phase benchmark data, the historical sound intensity benchmark mean and the historical sound phase benchmark mean are calculated respectively. Obtain historical sound intensity reference data and historical sound phase reference data corresponding to the calibration point within a preset time range; Based on the historical sound intensity benchmark data and the historical sound phase benchmark data, the historical sound intensity benchmark mean and the historical sound phase benchmark mean are calculated respectively. Calculate the sound intensity difference between the calibrated sound intensity coefficient and the historical sound intensity benchmark mean, and the sound phase difference between the calibrated sound phase coefficient and the historical sound phase benchmark mean; If the absolute value of the ratio between the sound intensity difference and the historical sound intensity benchmark mean is greater than the preset sound intensity threshold, then the calibrated sound intensity coefficient is invalid; otherwise, the calibrated sound intensity coefficient is valid. If the absolute value of the ratio between the sound phase difference and the historical sound intensity benchmark mean is greater than the preset sound phase threshold, then the calibrated sound phase coefficient is invalid; otherwise, the calibrated sound phase coefficient is valid.

8. The snoring detection method according to claim 6, characterized in that, The calculation of the sound source azimuth angle based on the left audio signal and the right audio signal includes: The left and right audio signals are respectively subjected to frequency domain transformation to obtain the left frequency domain signal and the right frequency domain signal; Based on the left-side and right-side frequency domain signals, cross-correlation features are constructed, and the cross-correlation features are weighted. The time delay between the left audio signal and the right audio signal is determined based on the weighted cross-correlation characteristics. The azimuth angle of the sound source is calculated based on the time delay, sound speed, and the distance between the left and right audio acquisition positions.

9. A snoring detection device, characterized in that, The device includes: The information acquisition unit is used to acquire human body sensing signals and audio signals; The status and feature determination unit is used to determine the device usage status and audio features based on the human body sensing signal and the audio signal. The device usage status includes the / out-of-bed state and the sleep state. The audio features include sound intensity features and sound phase features. The snoring detection unit is used to select corresponding feature combinations based on the human body sensing signal, device usage status and audio characteristics, and input them into a preset snoring detection model to detect snoring and determine the current snoring status. The snoring attribution determination unit is used to determine the initial probability values ​​of snoring attribution to the left and right sides based on a preset sound intensity coefficient reference table, sound phase coefficient reference table, sound intensity characteristics, and sound phase characteristics, if the current snoring state is in progress. The unit then performs a weighted calculation on the initial probability values ​​of snoring attribution to the left and right sides in conjunction with the in-bed / out-of-bed state and the sleep state to obtain the actual probability of the left side and the actual probability of the right side. By comparing the actual probability of the left side and the actual probability of the right side, the final snoring attribution is determined.

Images