Detection method and device for assisting in determining sleep quality level

Abstract

The present invention provides a detection method and device for assisting in determining the sleep quality level. The method comprises: after placing a multimode optical fiber at a predetermined position of a target object, continuously collecting coherent light emitted from the multimode optical fiber, and continuously detecting the heart rate, muscle state, and state of the environment of the target object, to obtain a coherent image sequence, a first heart rate sequence, a first electromyography sequence, and a first environmental feature sequence; performing singular value decomposition on multiple coherent images respectively to obtain a first posture feature sequence and a first motion feature sequence; performing abnormal data detection and correction on multiple first sequences respectively based on an agglomerative hierarchical clustering algorithm; performing data screening on the second sequence according to the correlation between the data at the same time in the second sequence and the second environmental feature sequence; combining and arranging multiple intrinsic modal components corresponding to multiple third sequences, and inputting the obtained multidimensional feature map into a target prediction model to obtain the sleep quality level.

Description

Technical Field

[0001] The present invention relates to the field of wearable sleep detection technology, and more specifically, to a detection method and device for assisting in determining a sleep quality level. Background Art

[0002] According to the survey results, a large number of people have sleep problems. Therefore, continuous quantitative monitoring of sleep in a relatively natural environment is more important for clinical practice of sleep medicine and daily health monitoring.

[0003] Since smart wearable devices have the advantages of multiple parameters, convenience, and no perception, they are widely used in daily sleep monitoring and also occupy an important position in personal consumer electronic products. At the same time, some wearable devices focus on parameters related to sleep continuity, and some wearable devices can be used to detect physiological signals such as movement, breathing, heartbeat, and snoring during sleep, and determine respiratory events such as apnea during sleep based on the above monitoring data.

[0004] In the process of realizing the concept of the present invention, it was found through research that there are at least technical problems in the related technology that wearable devices for monitoring sleep are prone to crosstalk between different parameters, insufficient judgment of human body movements, and interference from the external environment, resulting in inaccurate monitoring of sleep status. Summary of the invention

[0005] In view of this, the present invention provides a detection method and device for assisting in determining the sleep quality level.

[0006] One aspect of the present invention provides a detection method for assisting in determining a sleep quality level, comprising:

[0007] After placing the multimode optical fiber at a predetermined position of the target object, within a preset time period, the coherent light emitted from the multimode optical fiber is continuously collected, and the heart rate, muscle state, and state of the target object's environment are continuously detected to obtain a coherent image sequence, a first heart rate sequence, a first electromyographic sequence, and a first environmental feature sequence; singular value decomposition is performed on a plurality of coherent images included in the coherent image sequence to obtain a first posture feature sequence and a first motion feature sequence; based on an agglomerative hierarchical clustering algorithm, abnormal data detection and correction are performed on the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyographic sequence, and the first environmental feature sequence to obtain a second posture feature sequence, a second motion feature sequence, and a second motion feature sequence. feature sequence, a second heart rate sequence, a second electromyographic sequence and a second environmental feature sequence; for each second sequence in the second posture feature sequence, the second motion feature sequence, the second heart rate sequence and the second electromyographic sequence, calculate the correlation between the data at the same time in the second sequence and the second environmental feature sequence, and screen the second sequence according to the correlation to obtain a third sequence; perform modal decomposition on the multiple third sequences respectively to obtain multiple intrinsic modal components corresponding to the multiple third sequences, wherein the multiple third sequences correspond one to one to the multiple second sequences; combine and arrange the multiple intrinsic modal components to obtain a multidimensional feature map, and input the multidimensional feature map into the target prediction model to obtain the sleep quality level.

[0008] According to an embodiment of the present invention, singular value decomposition is performed on a plurality of coherent images included in a coherent image sequence respectively to obtain a first posture feature sequence and a first motion feature sequence, including: for each coherent image in the coherent image sequence, singular value decomposition is performed on the coherent image to obtain a left singular value matrix, a diagonal matrix and a right singular value matrix; a first posture feature is determined according to the first K row vectors of the left singular value matrix, the diagonal matrix and the right singular value matrix; a first posture feature sequence is obtained according to the first posture feature corresponding to each coherent image in the coherent image sequence; a first motion feature is determined according to the K+1 to nth row vectors of the left singular value matrix, the diagonal matrix and the right singular value matrix, wherein K is a positive integer and n is an integer greater than 1; a first motion feature sequence is obtained according to the first motion feature corresponding to each coherent image in the coherent image sequence.

[0009] According to an embodiment of the present invention, based on an agglomerative hierarchical clustering algorithm, abnormal data detection and correction are performed on the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyographic sequence and the first environmental feature sequence respectively to obtain a second posture feature sequence, a second motion feature sequence, a second heart rate sequence, a second electromyographic sequence and a second environmental feature sequence, including: for each first sequence in the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyographic sequence and the first environmental feature sequence, the characteristic distances between multiple subsequences corresponding to multiple time periods in the first sequence are calculated to obtain multiple characteristic distances; based on the multiple characteristic distances, the multiple subsequences are hierarchically clustered using an agglomerative hierarchical clustering algorithm to determine normal data and abnormal data corresponding to the first sequence; the abnormal data is corrected based on the normal data to obtain a target sequence corresponding to the first sequence, wherein the target sequence is one of the following: a second posture feature sequence, a second motion feature sequence, a second heart rate sequence, a second electromyographic sequence and a second environmental feature sequence.

[0010] According to an embodiment of the present invention, for each first sequence in the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyography sequence and the first environmental feature sequence, the characteristic distances between multiple subsequences corresponding to multiple time periods in the first sequence are calculated to obtain multiple characteristic distances, including: for any two subsequences in the multiple subsequences, the slope difference, the mean difference and the maximum difference of the two subsequences are calculated; according to the slope difference, the mean difference and the maximum difference, the numerical information distance is determined; according to the dynamic time warping algorithm, the target bending cost between the two subsequences is calculated; according to the numerical information distance and the target bending cost, the characteristic distance between the two subsequences is obtained.

[0011] According to an embodiment of the present invention, the target prediction model includes a plurality of identical first network structures and a second network structure, wherein each first network structure includes a feedforward network module and an attention module; a plurality of connotation modal components are combined and arranged to obtain a multidimensional feature map, and the multidimensional feature map is input into the target prediction model to obtain the sleep quality level, comprising: using a plurality of identical first network structures to perform the following iterative processing on the multidimensional feature map to obtain fused features: using the feedforward network module included in the current first network structure to perform feature extraction on the multidimensional feature map to obtain shallow features; using the attention module included in the current first network structure to fuse the shallow features to obtain the current fused features; determining the current fused features as the multidimensional feature map input to the next first network structure; and using the second network structure to process the fused features to obtain the sleep quality level.

[0012] According to an embodiment of the present invention, the attention module includes multiple convolutional layers, a global average pooling layer, a first fully connected layer, a linear rectification function layer, a second fully connected layer, a normalization layer and a feature fusion layer in sequence, wherein the feature fusion layer is used to multiply the convolutional features output by the multiple convolutional layers with the channel weights output by the normalization layer.

[0013] According to an embodiment of the present invention, the number of the multiple convolutional layers is 2.

[0014] According to an embodiment of the present invention, the first environmental feature sequence includes at least one of the following: a first environmental temperature sequence, a first environmental noise sequence, and a first environmental illumination sequence.

[0015] According to an embodiment of the present invention, correcting abnormal data according to normal data to obtain a target sequence corresponding to the first sequence includes: interpolating the sequence position of the first sequence where the abnormal data is located according to the normal data to obtain corrected data; and obtaining the target sequence according to the normal data and the corrected data.

[0016] Another aspect of the present invention provides a detection device for assisting in determining a sleep quality level, comprising:

[0017] A first obtaining module is used to continuously collect coherent light emitted from the multimode optical fiber within a preset time after placing the multimode optical fiber at a predetermined position of the target object, and continuously detect the heart rate, muscle state, and state of the target object's environment to obtain a coherent image sequence, a first heart rate sequence, a first electromyography sequence, and a first environmental feature sequence;

[0018] A second obtaining module is used to perform singular value decomposition on a plurality of coherent images included in the coherent image sequence to obtain a first posture feature sequence and a first motion feature sequence;

[0019] The third obtaining module is used to perform abnormal data detection and correction on the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyographic sequence and the first environment feature sequence respectively based on the agglomerative hierarchical clustering algorithm to obtain the second posture feature sequence, the second motion feature sequence, the second heart rate sequence, the second electromyographic sequence and the second environment feature sequence;

[0020] a fourth obtaining module, for calculating, for each second sequence among the second posture feature sequence, the second motion feature sequence, the second heart rate sequence, and the second electromyography sequence, the correlation between the second sequence and the data at the same time in the second environment feature sequence, and filtering the data of the second sequence according to the correlation to obtain a third sequence;

[0021] A fifth obtaining module is used to perform modal decomposition on the plurality of third sequences respectively to obtain a plurality of connotation modal components corresponding to the plurality of third sequences, wherein the plurality of third sequences correspond to the plurality of second sequences one-to-one;

[0022] The sixth obtaining module is used to combine and arrange multiple connotation modal components to obtain a multi-dimensional feature map, and input the multi-dimensional feature map into the target prediction model to obtain the sleep quality level.

[0023] Another aspect of the present invention provides an electronic device, comprising:

[0024] one or more processors;

[0025] a memory for storing one or more programs,

[0026] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0027] According to an embodiment of the present invention, by placing a multimode optical fiber at a predetermined position of a target object, the coherent light emitted from the multimode optical fiber, the heart rate of the target object, the muscle state, and the state of the environment in which the target object is located are continuously collected, and the noise, light and other information in the environment are obtained at the same time, providing a basis for realizing sleep state monitoring based on multidimensional sensing signals. And because of the use of optical fiber sensing and coherent detection, the influence of the external environment on human body movement and posture is reduced. The multiple coherent images included in the coherent image sequence are singular value decomposed respectively to obtain a first posture feature sequence and a first motion feature sequence. Based on the agglomerative hierarchical clustering algorithm, abnormal data detection and correction are performed on each sequence. The correlation degree is calculated based on the obtained second sequence and the second environmental feature sequence, and the second sequence is further screened, so that the obtained third sequence avoids the influence of abnormal factors and noise signals on the sleep detection results and causes errors. Modal decomposition is performed on multiple third sequences, and multiple connotative modal components are combined and arranged to highlight key features while suppressing relatively unimportant parts, thereby obtaining a multidimensional feature map. The multidimensional feature map is then input into the target prediction model to obtain the sleep quality level, which not only reduces the redundancy in the low-level features from the encoder, but also can effectively shield different interferences, thereby further improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0029] Figure 1 A flow chart of a detection method for assisting in determining a sleep quality level according to an embodiment of the present invention is shown.

[0030] Figure 2 A schematic diagram of a device for collecting physiological sleep parameters of a target object at a predetermined position according to an embodiment of the present invention is shown.

[0031] Figure 3 A flow chart of obtaining a target sequence based on an agglomerative hierarchical clustering algorithm according to an embodiment of the present invention is shown.

[0032] Figure 4 A block diagram of a detection device for assisting in determining a sleep quality level according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0033] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0034] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0035] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0036] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0037] In the embodiments of the present invention, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage and other aspects of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0038] In the embodiment of the present invention, before obtaining or collecting the user's personal information, the user's authorization or consent is obtained.

[0039] The World Health Organization recommends that adults aged 18 to 60 should get more than 7 hours of sleep a day to maintain optimal health. As modern medicine's understanding of sleep disorders gradually increases, sleep monitoring technology is increasingly used in the diagnosis of different diseases. Clinical departments such as neurology, respiratory medicine, and psychiatry have all conducted sleep monitoring research to meet the diagnostic needs of different diseases.

[0040] As an important tool for sleep medicine research, polysomnography (PSG) has an increasingly wider application scope. Patients need to sleep in a special sleep monitoring room in the hospital for one night and connect multiple wires to collect and record bioelectric signals: such as electroencephalogram (EEG), electrooculogram (EOG), electromyogram (EMG) and electrocardiogram (ECG); record physiological signals: such as respiratory airflow, chest and abdominal movement, pulse oxygen saturation and snoring; record external signals: such as transcutaneous carbon dioxide and pressure titration related parameters. The above multiple physiological signals are used to evaluate sleep quality and sleep status, and the doctor interprets the results to analyze the patient's sleep problems and the severity of the problems.

[0041] However, polysomnography requires changing the patient's sleeping environment and habits, which will affect the patient's sleep comfort and cause changes in sleep conditions, resulting in errors between the monitoring parameters or status and natural sleep, which can easily lead to misdiagnosis, missed diagnosis, etc. Therefore, wearable devices are almost ubiquitous in modern life because they can extract action-related features based on ACC (accelerometer) and PPG (photoplethysmography) data to divide the user's action level, and then combine the trend of heart rate changes to determine the user's sleep onset, sleep wake-up time, sleep structure and other data. In addition to conventional sleep monitoring information, some wearable smart devices can also obtain a series of indicators such as geographic location, body temperature, heart rate, skin conductivity, blood oxygen level and emotional assessment to further improve monitoring accuracy.

[0042] Wearable devices have the advantages of a wide variety of monitoring signals and diverse monitoring purposes. Therefore, in the study of sleep and circadian rhythms, it is important to determine appropriate and reliable wearable device monitoring signals. However, there are still some problems in multi-parameter detection, information fusion and discrimination of common wearable devices. For example, wearable devices need to extract motion features to determine whether the wearer is in a sleeping state, and for people who turn over and move more during sleep, the device may record more awake times and shorter sleep time. At the same time, in scenarios such as lying down to read books or watch videos, because the posture is relatively still, and the heart rate and breathing are stable, which are very close to the physical characteristics of the sleeping state, it is easy for the device to misjudge it as a sleeping state, resulting in early sleep time or delayed sleep time. On moving vehicles such as airplanes, high-speed trains, and cars, continuous vibration signals in the environment will interfere with the judgment of the true sleep state, making it difficult for the device to accurately monitor sleep data.

[0043] In view of this, an embodiment of the present invention provides a detection method for assisting in determining the sleep quality level, comprising: after placing a multimode optical fiber at a predetermined position of a target object, continuously collecting coherent light emitted from the multimode optical fiber within a preset time period, and continuously detecting the heart rate, muscle state, and state of the target object's environment of the target object to obtain a coherent image sequence, a first heart rate sequence, a first electromyography sequence, and a first environmental feature sequence; performing singular value decomposition on a plurality of coherent images included in the coherent image sequence to obtain a first posture feature sequence and a first motion feature sequence; performing abnormal data detection on the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyography sequence, and the first environmental feature sequence based on an agglomerative hierarchical clustering algorithm. and corrections to obtain a second posture feature sequence, a second motion feature sequence, a second heart rate sequence, a second electromyography sequence and a second environment feature sequence; for each second sequence in the second posture feature sequence, the second motion feature sequence, the second heart rate sequence and the second electromyography sequence, calculate the correlation between the data at the same time in the second sequence and the second environment feature sequence, and according to the correlation, screen the second sequence to obtain a third sequence; perform modal decomposition on the multiple third sequences respectively to obtain a multiple intrinsic modal components corresponding to the multiple third sequences, wherein the multiple third sequences correspond one to one to the multiple second sequences; combine and arrange the multiple intrinsic modal components to obtain a multidimensional feature map, and input the multidimensional feature map into the target prediction model to obtain the sleep quality level.

[0044] Figure 1 A flow chart of a detection method for assisting in determining a sleep quality level according to an embodiment of the present invention is shown.

[0045] like Figure 1 As shown, the detection method 100 for assisting in determining the sleep quality level includes operations S110 to S160.

[0046] In operation S110, after the multimode optical fiber is placed at a predetermined position of the target object, coherent light emitted from the multimode optical fiber is continuously collected within a preset time period, and the heart rate, muscle state, and state of the target object's environment are continuously detected to obtain a coherent image sequence, a first heart rate sequence, a first electromyography sequence, and a first environmental feature sequence.

[0047] According to an embodiment of the present invention, the predetermined position of the target object may be an arm or a chest. The multimode optical fiber includes an output optical fiber and an input optical fiber. The coherent light source transmits the coherent light to the predetermined position through the input optical fiber. The movement of the target object may cause the multimode optical fiber to deform, thereby changing the coherent light transmission model and the interference state. The output optical fiber transmits the changed coherent light to the CMOS (Complementary Metal Oxide Semiconductor) array for imaging, thereby obtaining a coherent image sequence.

[0048] According to an embodiment of the present invention, the preset duration can be selected according to actual conditions and is not limited here. For example, the preset duration can be 30 minutes, 40 minutes, 1 hour, 2 hours, 3 hours, 4 hours, 5 hours or 8 hours, etc.

[0049] According to an embodiment of the present invention, the heart rate of the target object can be detected by a photodetector and an LED (light-emitting diode) light source, the muscle state can be detected by an electromyographic sensor, and the state of the environment in which the target object is located can be detected by a variety of environmental sensors.

[0050] To better understand the above-mentioned process of collecting the coherent light emitted from the multimode optical fiber and detecting the heart rate, muscle state, and state of the target object's environment, the following will be described. Figure 2 Further elaboration will be given.

[0051] Figure 2 A schematic diagram of a device for collecting physiological sleep parameters of a target object at a predetermined position according to an embodiment of the present invention is shown.

[0052] like Figure 2 As shown, the device 210 includes a coherent light collection module 211 , a heart rate detection module 212 , an electromyography sensor module 213 and an environment sensor module 214 .

[0053] Among them, the coherent light acquisition module 211 is used to continuously collect coherent light emitted from the multimode optical fiber within a preset time period to obtain a coherent image sequence; the heart rate detection module 212 is used to continuously detect the heart rate of the target object within a preset time period to obtain a first heart rate sequence; the electromyography sensor module 213 is used to continuously detect the muscle state of the target object within a preset time period to obtain a first electromyography sequence; the environmental sensor module 214 is used to continuously detect the state of the environment in which the target object is located within a preset time period.

[0054] According to an embodiment of the present invention, the coherent light acquisition module 211 includes an incident optical fiber 211 (1), a coherent light source 211 (2), an output optical fiber 211 (3) and a CMOS array 211 (4), wherein the incident optical fiber 211 (1) is wound around a predetermined position 220 of a target object, and the predetermined position 220 is, for example, a wrist of the target object. The incident optical fiber 211 (1) is used to transmit the coherent light emitted from the coherent light source 211 (2) to the predetermined position 220, and the coherent light is modulated at the predetermined position 220 and then transmitted from the output optical fiber 211 (3) to the CMOS array 211 (4) for imaging. The posture change of the predetermined position 220 will cause the deformation of the incident optical fiber 211 (1), thereby modulating the coherent light in the incident optical fiber 211 (1). The heart rate detection module 212 includes an LED light source 212 (1) and a photodetector 212 (2). The electromyographic sensor module 213 may be an electromyographic sensor. The environmental sensor module 214 includes a temperature sensor 214 ( 1 ), a noise sensor 214 ( 2 ), and a light sensor 214 ( 3 ).

[0055] In operation S120, singular value decomposition is performed on a plurality of coherent images included in the coherent image sequence to obtain a first posture feature sequence and a first motion feature sequence.

[0056] According to an embodiment of the present invention, the first posture feature sequence may be related to a portion of a static state or a slowly changing state in the coherent image, and may include: a lying flat feature, a lying side feature, and the like.

[0057] According to an embodiment of the present invention, the first motion feature sequence may be related to a dynamically changing part in the coherent image, reflecting instantaneous features in the coherent image, and may include: a turning feature and the like.

[0058] According to an embodiment of the present invention, singular value decomposition can analyze and process the coherent image, and separate the first motion feature sequence corresponding to the transient motion signal and the first posture feature sequence corresponding to the static motion signal from the coherent image.

[0059] In operation S130, based on the agglomerative hierarchical clustering algorithm, abnormal data detection and correction are performed on the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyography sequence and the first environmental feature sequence, respectively, to obtain the second posture feature sequence, the second motion feature sequence, the second heart rate sequence, the second electromyography sequence and the second environmental feature sequence.

[0060] According to an embodiment of the present invention, the agglomerative hierarchical clustering algorithm may represent clustering analysis of similar features in various sequences to screen out optimal features from various sequences.

[0061] For example, when some characteristic data in each sequence are divided into multiple subsequences, agglomerative hierarchical clustering can be used to perform multi-level clustering according to the distances between the subsequences, and the minimum distance is used as the clustering criterion.

[0062] According to an embodiment of the present invention, the heart rate parameters, electromyographic parameters, posture change parameters and environmental parameters detected by the above device 210 can be regarded as multi-dimensional time series data. For the errors caused by random noise and abnormal data in the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyographic sequence and the first environmental feature sequence, the abnormal conditions of the feature data can be distinguished and cleaned by extracting features from each sequence to analyze the normal data rules and abnormal data rules implicit therein, and finally complete the abnormal data detection and correction to obtain the second posture feature sequence, the second motion feature sequence, the second heart rate sequence, the second electromyographic sequence and the second environmental feature sequence.

[0063] In operation S140, for each second sequence among the second posture feature sequence, the second motion feature sequence, the second heart rate sequence, and the second electromyography sequence, the correlation between the data of the second sequence and the second environmental feature sequence at the same time is calculated, and based on the correlation, the second sequence is screened to obtain a third sequence.

[0064] According to an embodiment of the present invention, the correlation between the data at each same moment in the second posture feature sequence, the second motion feature sequence, the second heart rate sequence, the second electromyography sequence and the second environment feature sequence is calculated respectively.

[0065] According to an embodiment of the present invention, the absolute value of the difference between the data in the second sequence and the data in the second environmental feature sequence at each same moment can be calculated according to the following formula (1).

[0066] (1)

[0067] Among them, x r Indicates the tth r The data of the fth predetermined index sequence at time and the data in the reference sequence The predetermined indicator sequence includes the normalized second sequence and the second environment feature sequence.

[0068] According to an embodiment of the present disclosure, the reference sequence may include an optimal value corresponding to each of the second sequence and the second environmental characteristic sequence. The optimal value corresponding to the second sequence may be a value representing a value less affected by the environment. The optimal value corresponding to the second environmental characteristic sequence may be a value representing a relatively stable environment.

[0069] According to the embodiment of the present invention, according to the above formula (1), the minimum difference MIN2 between the two levels shown in the following formula (2) and the maximum difference MAX2 between the two levels shown in the following formula (3) can be obtained.

[0070] (2)

[0071] (3)

[0072] Among them, F is the total number of predetermined indicator sequences, f is the sequence number of the predetermined indicator sequence, R is the total number of multiple moments into which the predetermined time length is divided, and r is the moment number.

[0073] According to an embodiment of the present invention, x obtained according to the above formulas (1) to (3) is r , MIN2 and MAX2, it is possible to calculate the relationship between each predetermined index sequence and the reference sequence at the t r Correlation coefficient at time .

[0074] According to an embodiment of the present invention, according to the tth corresponding to each predetermined indicator sequence r Correlation coefficient at time The correlation degree γ between the data at the same time in the second sequence and the second environmental feature sequence can be calculated using the following formula (4).

[0075] (4)

[0076] According to an embodiment of the present invention, r=1, 2, 3, ... R. ρ represents an adjustable coefficient, and the smaller ρ is, the greater the difference between the correlation coefficients is, and ρ is usually set to 0.5.

[0077] According to an embodiment of the present invention, when the reference sequence includes the optimal values ​​corresponding to the second sequence and the second environmental feature sequence, the larger the value of the correlation degree, the less the second sequence is affected by the environment, and the more stable the environment is. The smaller the value of the correlation degree, the greater the impact of the environment on the second sequence, and the more unstable the environment is. According to the correlation degree, the data of the second sequence is arranged and screened, and the data with a correlation value less than a preset correlation threshold in the second sequence is eliminated to obtain a third sequence, which effectively avoids the influence of environmental factors on the actual collected feature data to cause the technical problem of inaccurate subsequent detection results.

[0078] In operation S150, modal decomposition is performed on the plurality of third sequences respectively to obtain a plurality of connotation modal components corresponding to the plurality of third sequences, wherein the plurality of third sequences correspond one-to-one to the plurality of second sequences.

[0079] According to an embodiment of the present invention, each third sequence is determined All the maximum and minimum points of , and the third sequence is fitted by cubic spline interpolation function The upper envelope of and lower envelope .

[0080] According to an embodiment of the present invention, the average signal of the upper envelope and the lower envelope It can be obtained according to the following formula (5).

[0081] (5)

[0082] in, is the time corresponding to the third sequence.

[0083] According to an embodiment of the present invention, using the third sequence Subtract the mean signal , we can get the intermediate signal It can be expressed as the following formula (6).

[0084] (6)

[0085] According to an embodiment of the present invention, judging Whether the following two conditions are met: the absolute difference between the number of extreme points and the number of zero-crossing points in the entire data segment is less than 1 and the average value of the upper envelope and the lower envelope at any time is zero, that is, the upper and lower envelopes are locally symmetrical with respect to the time axis. If the above conditions are met, the current moment is the Intrinsic Mode Functions (IMF) component, denoted by =IMF1. If IMF1 has negative local maximum and positive local minimum, then use the third sequence Subtract the IMF1 component as the new third sequence, and continue to repeat the modal decomposition of the new third sequence according to formula (5) and formula (6). And so on, until the e-th order intrinsic modal component or residual When it is a monotonic function or a constant, the modal decomposition is completed, and finally e intrinsic modal components and a residual are obtained. .

[0086] According to an embodiment of the present invention, the residual It can be expressed as the following formula (7), the sum of e intrinsic modal components It can be expressed as the following formula (8).

[0087] (7)

[0088] (8)

[0089] In operation S160, a plurality of connotative modal components are combined and arranged to obtain a multidimensional feature map, and the multidimensional feature map is input into a target prediction model to obtain a sleep quality level.

[0090] According to an embodiment of the present invention, after a plurality of connotation modes corresponding to each third sequence form a multidimensional initial feature map, the multidimensional initial feature maps corresponding to each of the plurality of third sequences may be concatenated to obtain a multidimensional feature map.

[0091] According to an embodiment of the present invention, in order to more effectively fuse feature data at different levels and suppress irrelevant redundant feature data, the decomposed intrinsic modal components can be arranged and combined and then input into the target prediction model for further self-attention feature extraction.

[0092] According to an embodiment of the present invention, the target prediction model may be an encoder-decoder model. In order to enable the target prediction model to focus more on important feature channels and ignore less important channels, the target prediction model may be adaptively adjusted.

[0093] According to an embodiment of the present invention, sleep quality levels can be divided according to actual needs. For example, level 1 can indicate good sleep quality, which should be maintained, level 2 can indicate average sleep quality, level 3 can indicate poor sleep quality, and level 4 can indicate extremely poor sleep quality. Relevant personnel can further evaluate the sleep quality of the target object according to the sleep quality level obtained by the detection method for assisting in determining the sleep quality level according to an embodiment of the present disclosure.

[0094] According to an embodiment of the present invention, by placing a multimode optical fiber at a predetermined position of a target object, the coherent light emitted from the multimode optical fiber, the heart rate of the target object, the muscle state, and the state of the environment in which the target object is located are continuously collected, and the noise, light and other information in the environment are obtained at the same time, providing a basis for realizing sleep state monitoring based on multidimensional sensing signals. And because of the use of optical fiber sensing and coherent detection, the influence of the external environment on human body movement and posture is reduced. The multiple coherent images included in the coherent image sequence are singular value decomposed respectively to obtain a first posture feature sequence and a first motion feature sequence. Based on the agglomerative hierarchical clustering algorithm, abnormal data detection and correction are performed on each sequence. The correlation degree is calculated based on the obtained second sequence and the second environmental feature sequence, and the second sequence is further screened, so that the obtained third sequence avoids the influence of abnormal factors and noise signals on the sleep detection results and causes errors. Modal decomposition is performed on multiple third sequences, and multiple connotative modal components are combined and arranged to highlight key features while suppressing relatively unimportant parts, thereby obtaining a multidimensional feature map. The multidimensional feature map is then input into the target prediction model to obtain the sleep quality level, which not only reduces the redundancy in the low-level features from the encoder, but also can effectively shield different interferences, thereby further improving the detection accuracy.

[0095] According to an embodiment of the present invention, the first environmental feature sequence includes at least one of the following: a first environmental temperature sequence, a first environmental noise sequence, and a first environmental illumination sequence.

[0096] For example, the first environment feature sequence may include a first environment temperature sequence, a first environment noise sequence, and a first environment illumination sequence. Alternatively, the first environment feature sequence may include a first environment temperature sequence and a first environment noise sequence. Alternatively, the first environment feature sequence may include a first environment temperature sequence and a first environment illumination sequence. Alternatively, the first environment feature sequence may include a first environment noise sequence and a first environment illumination sequence.

[0097] According to an embodiment of the present invention, the first ambient temperature sequence can reflect changes in ambient temperature within a preset time period, the first ambient temperature sequence can reflect changes in ambient noise within a preset time period, and the first ambient lighting sequence can reflect changes in ambient lighting within a preset time period.

[0098] According to an embodiment of the present invention, singular value decomposition is performed on a plurality of coherent images included in a coherent image sequence respectively to obtain a first posture feature sequence and a first motion feature sequence, including: for each coherent image in the coherent image sequence, singular value decomposition is performed on the coherent image to obtain a left singular value matrix, a diagonal matrix and a right singular value matrix; a first posture feature is determined according to the first K row vectors of the left singular value matrix, the diagonal matrix and the right singular value matrix; a first posture feature sequence is obtained according to the first posture feature corresponding to each coherent image in the coherent image sequence; a first motion feature is determined according to the K+1 to nth row vectors of the left singular value matrix, the diagonal matrix and the right singular value matrix, wherein K is a positive integer and n is an integer greater than 1; a first motion feature sequence is obtained according to the first motion feature corresponding to each coherent image in the coherent image sequence.

[0099] According to an embodiment of the present invention, the process of performing singular value decomposition on the coherent image L received by the CMOS array is shown in the following formula (9).

[0100] (9)

[0101] Among them, U represents the left singular value matrix, is the corresponding i-th row vector in the left singular value matrix, S represents a diagonal matrix, is the corresponding i-th row vector in the diagonal matrix, V represents the right singular value matrix, is the corresponding i-th row vector in the right singular value matrix, and n is the total number of row vectors in each matrix.

[0102] According to an embodiment of the present invention, the first posture feature I t It can be expressed as the following formula (10).

[0103] (10)

[0104] According to an embodiment of the present invention, the first motion feature I b It can be expressed as the following formula (11).

[0105] (11)

[0106] According to an embodiment of the present invention, each image in the coherent image sequence is processed as shown in formulas (9) to (11) above, first performing singular value decomposition and then performing feature extraction on the row vector to obtain a first motion feature sequence and a first posture feature sequence.

[0107] According to an embodiment of the present invention, singular value decomposition is performed on each coherent light image in the coherent light image sequence received from the CMOS array, and the corresponding row vector is extracted to construct a first motion feature sequence and a first posture feature sequence, which provides a certain basis for the subsequent analysis of human sleeping posture and movement.

[0108] According to an embodiment of the present invention, based on an agglomerative hierarchical clustering algorithm, abnormal data detection and correction are performed on the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyographic sequence and the first environmental feature sequence, respectively, to obtain a second posture feature sequence, a second motion feature sequence, a second heart rate sequence, a second electromyographic sequence and a second environmental feature sequence, including: for each first sequence in the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyographic sequence and the first environmental feature sequence, the characteristic distances between multiple subsequences corresponding to multiple time periods in the first sequence are calculated to obtain multiple characteristic distances; according to the multiple characteristic distances, the agglomerative hierarchical clustering algorithm is used to perform hierarchical clustering on the multiple subsequences to determine normal data and abnormal data corresponding to the first sequence; the abnormal data is corrected according to the normal data to obtain a target sequence corresponding to the first sequence, wherein the target sequence is one of the following: a second posture feature sequence, a second motion feature sequence, a second heart rate sequence, a second electromyographic sequence and a second environmental feature sequence.

[0109] According to an embodiment of the present invention, each first sequence corresponds to multiple time periods, and the time periods correspond to the subsequences one by one. In order to perform agglomerative hierarchical clustering on the first sequence, it is necessary to calculate the characteristic distance D between each two subsequences. m .

[0110] According to an embodiment of the present invention, for each first sequence in the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyography sequence and the first environmental feature sequence, the characteristic distances between multiple subsequences corresponding to multiple time periods in the first sequence are calculated to obtain multiple characteristic distances, including: for any two subsequences in the multiple subsequences, the slope difference, the mean difference and the maximum difference of the two subsequences are calculated; according to the slope difference, the mean difference and the maximum difference, the numerical information distance is determined; according to the dynamic time warping algorithm, the target bending cost between the two subsequences is calculated; according to the numerical information distance and the target bending cost, the characteristic distance between the two subsequences is obtained.

[0111] According to an embodiment of the present invention, the numerical characteristics can reflect the statistical characteristics of multiple subsequences, and quickly lock subsequences with high similarity for the classification algorithm of sleep quality level based on agglomerative hierarchical clustering. Three indicators, namely slope difference, mean difference and maximum difference, can be selected to characterize the numerical information distance of subsequences.

[0112] According to an embodiment of the present invention, the calculation formula of the slope k can be expressed as the following formula (12).

[0113] (12)

[0114] Among them, λ m is the last data value of the subsequence. λ1 is the first data value of the subsequence, and m is the number of data in the subsequence.

[0115] According to an embodiment of the present invention, the trend characteristics of the subsequence mainly reflect the change trends of multiple subsequences, such as shape characteristics such as rising, falling, and stable. For multiple subsequences corresponding to the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyography sequence, and the first environment feature sequence, in order to achieve one-to-many mapping, the numerical information distance can be determined according to the slope difference, the mean difference, and the maximum difference.

[0116] For example, taking the subsequence P and subsequence Q corresponding to the first sequence as an example, the numerical information distance S1 can be expressed as the following formula (13).

[0117] (13)

[0118] Where P = (p1, p2, ..., p h ), Q = (q1, q2, …, q w ). represents the difference between the means of subsequence P and subsequence Q, represents the difference between the maximum values ​​of subsequence P and subsequence Q, It represents the difference in slope between subsequence P and subsequence Q. a is the weight corresponding to the difference in mean, c is the weight corresponding to the difference in maximum, g is the weight corresponding to the difference in slope, and a+c+g=1 is satisfied. h is the total number of data in subsequence P, and w is the total number of data in subsequence Q.

[0119] According to an embodiment of the present invention, for the above subsequence P and subsequence Q, an h×w matrix network AA can be constructed, and the elements of the matrix network AA are sequentially represented as the distances to the data points in the subsequence P and the subsequence Q, and the set consisting of each group of adjacent elements in the matrix network AA is regarded as a curved path Z, Z={z1, z2, …z y , z j}, and according to the dynamic time warping algorithm, determine the target bending cost, that is, the optimal bending path, from the bending path to minimize the bending cost of subsequence P and subsequence Q. Where y is the data number in the bending path Z, and j is the total number of data in the bending path Z.

[0120] According to an embodiment of the present invention, the target bending cost It can be expressed as the following formula (14).

[0121] (14)

[0122] According to an embodiment of the present invention, the characteristic distance Dm between two subsequences is obtained using the following formula (15) based on the numerical information distance and the target bending cost.

[0123] (15)

[0124] Among them, α is the weight of S1, β is The corresponding weights satisfy α+β=1.

[0125] After calculating the characteristic distance D between each two subsequences m After that, the agglomerative hierarchical clustering algorithm can be used to perform hierarchical clustering on multiple subsequences according to multiple feature distances to screen the optimal features.

[0126] For example, when performing agglomerative hierarchical clustering, we can first obtain multiple feature distances D m Sort in ascending order to get the array d=[D1, D2, D3...D m ]. Where D1 represents the characteristic distance corresponding to the two subsequences with the smallest distance. The two subsequences corresponding to D1 are merged to form a new subsequence. The characteristic distance Dm between the new subsequence and other subsequences is recalculated, and the two subsequences with the smallest characteristic distance are merged. This process is repeated until multiple subsequences are merged into a preset number of target subsequences.

[0127] After obtaining a preset number of target subsequences, the quality of each target subsequence is evaluated. For example, the quality of each target subsequence can be evaluated according to the silhouette coefficient, the square error, the CH index (Calinski-Harabasz Index) or the DB index (Davies-Bouldin Index) to obtain the clustering quality level corresponding to each target subsequence.

[0128] In order to achieve subsequent screening of abnormal data, a clustering quality threshold can be set, and the data corresponding to each target subsequence obtained can be screened according to the clustering quality threshold. When the clustering quality level of the target subsequence is less than the clustering quality threshold, the data corresponding to the target subsequence is determined to be normal data, otherwise it is abnormal data.

[0129] According to an embodiment of the present invention, correcting abnormal data according to normal data may include eliminating abnormal data and filling missing data.

[0130] According to an embodiment of the present invention, correcting abnormal data according to normal data to obtain a target sequence corresponding to the first sequence includes: interpolating the sequence position of the first sequence where the abnormal data is located according to the normal data to obtain corrected data; and obtaining the target sequence according to the normal data and the corrected data.

[0131] According to an embodiment of the present invention, after the abnormal data is eliminated, the missing data can be reconstructed according to the normal data to obtain the corrected data. In order to speed up the reconstruction speed, it can be set to directly use the linear interpolation method to complete the data for the missing sequence with less than 5 consecutive missing data, that is, to estimate the missing data by using multiple known normal data before and after the consecutive missing data.

[0132] For missing sequences with more than 5 consecutive missing data, use the multiple characteristic distances calculated by the first clustering in the above clustering results to determine the subsequence closest to the missing sequence. If the data in the subsequence is normal data, fill the subsequence to the sequence position of the first sequence where the missing sequence is located. If the data in the subsequence is also abnormal data, continue to search for the subsequence that is next closest to the missing sequence until a subsequence containing normal data is found, and then fill the subsequence containing normal data to the sequence position of the first sequence where the missing sequence is located.

[0133] In order to better understand the process of performing abnormal data detection and correction on the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyography sequence and the first environment feature sequence based on the agglomerative hierarchical clustering algorithm as described above, and obtaining the second posture feature sequence, the second motion feature sequence, the second heart rate sequence, the second electromyography sequence and the second environment feature sequence, the following will be based on Figure 3 Further elaboration will be given.

[0134] Figure 3 A flow chart of obtaining a target sequence based on an agglomerative hierarchical clustering algorithm according to an embodiment of the present invention is shown.

[0135] like Figure 3 As shown, the embodiment 300 includes operations S310 to S390.

[0136] In operation S310, each first sequence is divided into a plurality of subsequences.

[0137] In operation S320, characteristic distances between a plurality of subsequences are calculated, and the plurality of characteristic distances are sorted to obtain an array d.

[0138] In operation S330, the minimum distance is used as a clustering criterion to cluster the subsequences corresponding to the minimum characteristic distance in the array d to obtain a new subsequence.

[0139] In operation S340, it is determined whether the number of the new subsequence and the number of the remaining subsequences is less than or equal to a preset number. If so, the process returns to operation S320. If so, the process performs operation S350.

[0140] In operation S350, a preset number of target subsequences are obtained, and a quality evaluation is performed on each target subsequence to determine a target subsequence including normal data and a target subsequence including abnormal data;

[0141] In operation S360, abnormal data is eliminated.

[0142] In operation S370 , it is determined whether the number of consecutive missing data in the missing sequence is greater than 5. If the number of consecutive missing data in the missing sequence is greater than 5, operation S390 is performed; otherwise, if the number of consecutive missing data in the missing sequence is less than or equal to 5, operation S380 is performed.

[0143] In operation S380, linear interpolation is performed on the sequence position of the first sequence where the missing sequence is located according to normal data to obtain a target sequence.

[0144] In operation S390, based on normal data, adjacent sequence interpolation is performed on the sequence position of the first sequence where the missing sequence is located to obtain a target sequence.

[0145] According to an embodiment of the present invention, based on the agglomerative hierarchical clustering algorithm, the abnormal data in the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyography sequence and the first environmental feature sequence are cleaned and supplemented, and the second posture feature sequence, the second motion feature sequence, the second heart rate sequence, the second electromyography sequence and the second environmental feature sequence are obtained by analyzing the patterns of normal data and the patterns of abnormal data, thereby further improving the accuracy of the feature sequence.

[0146] According to an embodiment of the present invention, the target prediction model includes a plurality of identical first network structures and a second network structure, wherein each first network structure includes a feedforward network module and an attention module; a plurality of connotation modal components are combined and arranged to obtain a multidimensional feature map, and the multidimensional feature map is input into the target prediction model to obtain the sleep quality level, comprising: using a plurality of identical first network structures to perform the following iterative processing on the multidimensional feature map to obtain a fused feature: using the feedforward network module included in the current first network structure to perform feature extraction on the multidimensional feature map to obtain an initial feature; using the attention module included in the current first network structure to fuse the initial feature to obtain a current fused feature; determining the current fused feature as a multidimensional feature map input to the next first network structure; and using the second network structure to process the fused feature to obtain the sleep quality level.

[0147] According to an embodiment of the present invention, the feed-forward network module may be a 1×1 convolutional layer, represented as .

[0148] According to an embodiment of the present invention, the attention module may be a multi-head attention module. The attention module includes multiple convolutional layers, a global average pooling layer, a first fully connected layer, a linear rectifier function layer, a second fully connected layer, a normalization layer, and a feature fusion layer in sequence, wherein the feature fusion layer is used to multiply the convolutional features output by the multiple convolutional layers with the channel weights output by the normalization layer, and the number of convolutional layers is 2. The convolutional layer included in the attention module may be a 3×3 convolutional layer, represented as .

[0149] According to an embodiment of the present invention, the convolution layer included in the attention module is used to perform preliminary processing on the shallow features to obtain convolution features, which effectively fuses and refines the features of each layer of the shallow features while maintaining computational efficiency to reduce redundant features therein. 卷积 It can be expressed as the following formula (16).

[0150] (16)

[0151] in, Represents the multi-dimensional feature map output by the num-th first network structure. Concat represents the operation of merging the multi-dimensional feature map output by the num-th first network structure. represents the first 3×3 convolutional layer, Represents the second 3×3 convolutional layer.

[0152] According to an embodiment of the present invention, the global average pooling layer, the first fully connected layer, the linear rectification function layer, the second fully connected layer, the normalization layer and the feature fusion layer can be regarded as a SE (Squeeze Excitation) submodule for further enhancing sleep-related features.

[0153] According to an embodiment of the present invention, the global average pooling layer can perform global average pooling on the convolutional features obtained by the above formula (16) to obtain statistical information corresponding to a certain channel dimension: The statistical information of each channel dimension can be calculated using the following formula (17).

[0154] (17)

[0155] Among them, Hx represents the height of the single channel feature map included in the convolution feature, Wx represents the width of the single channel feature map included in the convolution feature, and sf represents the convolution feature U 卷积 The channel number.

[0156] According to an embodiment of the present disclosure, the linear rectification function layer may be, for example, a ReLU (Rectified Linear Unit) layer. The normalization function may be a Sigmoid function.

[0157] According to an embodiment of the present invention, in order to reduce the computational complexity of the SE submodule and enhance generalization, the first fully connected layer FC1, the linear rectifier function layer , the second fully connected layer FC2 successively performs statistical information Transform and use the normalization function Normalization is performed to determine the channel weight corresponding to each channel. The channel weight can be expressed as the following formula (18).

[0158] (18)

[0159] Among them, the reduction ratios of the first fully connected layer and the second fully connected layer are both 8, which is used to control the capacity and computational cost of the SE submodule.

[0160] According to an embodiment of the present invention, in order to obtain an adaptive recalibration so that the first network structure focuses on relatively important channels and ignores less important channels, the convolution features output by multiple convolution layers and the channel weights output by the normalization layer can be multiplied in sequence according to the following formula (19) to obtain a fusion feature: .

[0161] (19)

[0162] in, Represents channel-by-channel multiplication.

[0163] According to an embodiment of the present invention, an attention module is used to fuse shallow features to obtain current fused features, and the current fused features are determined as multi-dimensional feature maps input to the next first network structure, so that the target prediction model effectively fuses features at different levels and suppresses irrelevant redundant features, effectively shielding different interferences and improving the model's discrimination accuracy.

[0164] Figure 4 A block diagram of a detection device for assisting in determining a sleep quality level according to an embodiment of the present invention is shown.

[0165] like Figure 4 As shown, the apparatus 400 includes a first obtaining module 410 , a second obtaining module 420 , a third obtaining module 430 , a fourth obtaining module 440 , a fifth obtaining module 450 and a sixth obtaining module 460 .

[0166] The first obtaining module 410 is used to continuously collect coherent light emitted from the multimode optical fiber within a preset time period after the multimode optical fiber is placed at a predetermined position of the target object, and continuously detect the heart rate, muscle state, and state of the target object's environment to obtain a coherent image sequence, a first heart rate sequence, a first electromyography sequence, and a first environmental feature sequence.

[0167] The second obtaining module 420 is used to perform singular value decomposition on the multiple coherent images included in the coherent image sequence to obtain a first posture feature sequence and a first motion feature sequence.

[0168] The third obtaining module 430 is used to perform abnormal data detection and correction on the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyography sequence and the first environment feature sequence based on the agglomerative hierarchical clustering algorithm to obtain the second posture feature sequence, the second motion feature sequence, the second heart rate sequence, the second electromyography sequence and the second environment feature sequence.

[0169] The fourth obtaining module 440 is used to calculate the correlation between the second sequence and the data at the same time in the second environment feature sequence for each second sequence in the second posture feature sequence, the second motion feature sequence, the second heart rate sequence, and the second electromyography sequence, and filter the data of the second sequence according to the correlation to obtain the third sequence.

[0170] The fifth obtaining module 450 is used to perform modal decomposition on the multiple third sequences respectively to obtain multiple connotation modal components corresponding to the multiple third sequences, wherein the multiple third sequences correspond to the multiple second sequences one by one.

[0171] The sixth obtaining module 460 is used to combine and arrange multiple connotative modal components to obtain a multi-dimensional feature map, and input the multi-dimensional feature map into the target prediction model to obtain the sleep quality level.

[0172] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a proper combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be executed.

[0173] For example, any multiple of the first obtaining module 410, the second obtaining module 420, the third obtaining module 430, the fourth obtaining module 440, the fifth obtaining module 450, and the sixth obtaining module 460 can be combined in one module / unit / subunit for implementation, or any one of the modules / units / subunits can be split into multiple modules / units / subunits. Alternatively, at least part of the functions of one or more of these modules / units / subunits can be combined with at least part of the functions of other modules / units / subunits and implemented in one module / unit / subunit. According to an embodiment of the present invention, at least one of the first obtaining module 410, the second obtaining module 420, the third obtaining module 430, the fourth obtaining module 440, the fifth obtaining module 450, and the sixth obtaining module 460 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the first obtaining module 410, the second obtaining module 420, the third obtaining module 430, the fourth obtaining module 440, the fifth obtaining module 450, and the sixth obtaining module 460 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0174] It should be noted that the detection device part for assisting in determining the sleep quality level in the embodiment of the present invention corresponds to the detection method part for assisting in determining the sleep quality level in the embodiment of the present invention. The description of the detection device part for assisting in determining the sleep quality level specifically refers to the detection method part for assisting in determining the sleep quality level, which will not be repeated here.

[0175] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All these combinations and / or combinations fall within the scope of the present invention.

[0176] The embodiments of the present invention have been described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment has been described above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. The scope of the present invention is limited by the attached embodiments and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A detection method for assisting in determining a sleep quality level, characterized in that: The detection method comprises: After placing the multimode optical fiber at a predetermined position of the target object, continuously collecting coherent light emitted from the multimode optical fiber within a preset time period, and continuously detecting the heart rate, muscle state, and state of the target object's environment to obtain a coherent image sequence, a first heart rate sequence, a first electromyography sequence, and a first environmental feature sequence; Performing singular value decomposition on a plurality of coherent images included in the coherent image sequence respectively to obtain a first posture feature sequence and a first motion feature sequence; Based on the agglomerative hierarchical clustering algorithm, abnormal data detection and correction are performed on the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyographic sequence and the first environment feature sequence respectively to obtain a second posture feature sequence, a second motion feature sequence, a second heart rate sequence, a second electromyographic sequence and a second environment feature sequence; For each of the second posture feature sequence, the second motion feature sequence, the second heart rate sequence, and the second electromyography sequence, calculate the correlation between the second sequence and the data at the same time in the second environment feature sequence, and perform data screening on the second sequence according to the correlation to obtain a third sequence; Performing modal decomposition on the plurality of third sequences respectively to obtain a plurality of connotation modal components corresponding to the plurality of third sequences, wherein the plurality of third sequences correspond one-to-one to the plurality of second sequences; The multiple connotation modal components are combined and arranged to obtain a multidimensional feature map, and the multidimensional feature map is input into a target prediction model to obtain a sleep quality level.

2. The detection method according to claim 1, characterized in that: The step of performing singular value decomposition on the multiple coherent images included in the coherent image sequence to obtain a first posture feature sequence and a first motion feature sequence comprises: For each coherent image in the coherent image sequence, singular value decomposition is performed on the coherent image to obtain a left singular value matrix, a diagonal matrix and a right singular value matrix; Determine a first posture feature according to the left singular value matrix, the diagonal matrix and the first K row vectors of the right singular value matrix; Obtaining the first posture feature sequence according to the first posture features corresponding to each coherent image in the coherent image sequence; Determine a first motion feature according to the K+1th to nth row vectors of the left singular value matrix, the diagonal matrix, and the right singular value matrix, where K is a positive integer and n is an integer greater than 1; The first motion feature sequence is obtained according to the first motion features corresponding to each coherent image in the coherent image sequence.

3. The detection method according to claim 1 or 2, characterized in that: The agglomerative hierarchical clustering algorithm is based on respectively performing abnormal data detection and correction on the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyographic sequence and the first environment feature sequence to obtain a second posture feature sequence, a second motion feature sequence, a second heart rate sequence, a second electromyographic sequence and a second environment feature sequence, including: For each first sequence among the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyography sequence, and the first environment feature sequence, calculating feature distances between a plurality of subsequences corresponding to a plurality of time periods in the first sequence to obtain a plurality of feature distances; According to the multiple feature distances, the multiple subsequences are hierarchically clustered using an agglomerative hierarchical clustering algorithm to determine normal data and abnormal data corresponding to the first sequence; the abnormal data is corrected according to the normal data to obtain a target sequence corresponding to the first sequence, wherein the target sequence is one of the following: the second posture feature sequence, the second motion feature sequence, the second heart rate sequence, the second electromyography sequence, and the second environmental feature sequence.

4. The detection method according to claim 3, characterized in that: The step of calculating, for each of the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyography sequence, and the first environment feature sequence, feature distances between multiple subsequences corresponding to multiple time periods in the first sequence to obtain multiple feature distances includes: For any two subsequences of the multiple subsequences, calculating the slope difference, mean difference and maximum difference of the two subsequences; Determine the numerical information distance according to the slope difference, the mean difference and the maximum difference; Calculating a target bending cost between the two subsequences according to a dynamic time warping algorithm; A characteristic distance between the two subsequences is obtained according to the numerical information distance and the target bending cost.

5. The detection method according to claim 4, characterized in that: The target prediction model includes a plurality of identical first network structures and a second network structure, wherein each first network structure includes a feedforward network module and an attention module; The multiple connotation modal components are combined and arranged to obtain a multidimensional feature map, and the multidimensional feature map is input into the target prediction model to obtain the sleep quality level including: The multidimensional feature graph is iterated by using the multiple identical first network structures to obtain a fusion feature: Using the feedforward network module included in the current first network structure to extract features from the multidimensional feature graph to obtain shallow features; Using the attention module included in the current first network structure to fuse the shallow features to obtain current fused features; Determine the current fusion feature as a multidimensional feature map input to the next first network structure; The fusion feature is processed by using the second network structure to obtain the sleep quality level.

6. The detection method according to claim 5, characterized in that: The attention module includes multiple convolutional layers, a global average pooling layer, a first fully connected layer, a linear rectification function layer, a second fully connected layer, a normalization layer and a feature fusion layer in sequence, wherein the feature fusion layer is used to multiply the convolutional features output by the multiple convolutional layers with the channel weights output by the normalization layer.

7. The detection method according to claim 6, characterized in that: The number of the multiple convolutional layers is 2.

8. The detection method according to claim 1 or 2, characterized in that: The first environmental feature sequence includes at least one of the following: a first environmental temperature sequence, a first environmental noise sequence, and a first environmental illumination sequence.

9. The detection method according to claim 3, characterized in that: The correcting the abnormal data according to the normal data to obtain a target sequence corresponding to the first sequence comprises: Interpolate the sequence position of the first sequence where the abnormal data is located according to the normal data to obtain corrected data; The target sequence is obtained according to the normal data and the correction data.

10. A detection device for assisting in determining a sleep quality level, characterized in that: The detection device comprises: A first obtaining module is used to continuously collect coherent light emitted from the multimode optical fiber within a preset time after placing the multimode optical fiber at a predetermined position of the target object, and continuously detect the heart rate, muscle state, and state of the target object's environment to obtain a coherent image sequence, a first heart rate sequence, a first electromyography sequence, and a first environmental feature sequence; A second obtaining module is used to perform singular value decomposition on a plurality of coherent images included in the coherent image sequence to obtain a first posture feature sequence and a first motion feature sequence; A third obtaining module is used to perform abnormal data detection and correction on the first posture feature sequence, the first motion feature sequence, the first heart rate sequence, the first electromyographic sequence and the first environment feature sequence respectively based on an agglomerative hierarchical clustering algorithm to obtain a second posture feature sequence, a second motion feature sequence, a second heart rate sequence, a second electromyographic sequence and a second environment feature sequence; a fourth obtaining module, for calculating, for each of the second posture feature sequence, the second motion feature sequence, the second heart rate sequence, and the second electromyography sequence, a correlation between the data of the second sequence and the data of the second environment feature sequence at the same time, and performing data screening on the second sequence according to the correlation to obtain a third sequence; a fifth obtaining module, configured to perform modal decomposition on the plurality of third sequences respectively to obtain a plurality of connotation modal components corresponding to the plurality of third sequences, wherein the plurality of third sequences correspond one-to-one to the plurality of second sequences; The sixth obtaining module is used to combine and arrange the multiple connotation modal components to obtain a multi-dimensional feature map, and input the multi-dimensional feature map into the target prediction model to obtain the sleep quality level.

Images