Sleep music generation method and device and terminal equipment

Abstract

The invention provides a sleep music generation method and device and terminal equipment, and is suitable for the technical field of data processing, and the method comprises the steps: obtaining a sleep music generation result according to user age information, user gender information, current user physiological state information, current user behavior state information, historical user physiological state information and historical user behavior state information; obtaining current user rhythm information and current user sleep state characterization information; generating current user sleep state confirmation information according to the current user rhythm information and the current user sleep state representation information; and in response to the generation of the current user sleep state confirmation information, generating sleep music information according to the current user physiological state information, the sleep music generation model and a preset sleep music set. According to the method and the device, the individual difference of the user and the sleep state of dynamic change are accurately adapted, the accuracy and adaptability of sleep music generation are improved, and personalized sleep adjustment is effectively realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data processing, and particularly relates to a sleep music generation method and device and a terminal device. BACKGROUND

[0002] In the era of rapid development of digitization and intelligence, the field of sleep assistance technology and music generation technology is booming. With the increasing emphasis on sleep quality, especially the concern for the sleep health of special groups such as infants, the exploration and application of related technologies are increasingly in-depth.

[0003] In the prior art, sleep assistance and music generation are usually achieved by fixed modes or simple algorithms, for example, simple physiological parameters of a user are collected by a wearable device, a sleep state is determined according to a specific threshold, and then suitable music is selected from a pre-prepared music library, or a specific music time sequence information is captured by means of a deep learning algorithm to create music.

[0004] However, relying only on simple physiological parameters and threshold values to determine the sleep state cannot accurately and comprehensively reflect the complex and variable real sleep state of the user, especially infants, resulting in poor matching of the selected or generated sleep music to the actual needs. The sleep music generated by concatenating music materials according to fixed rules has low listening quality and few syllable changes, and it is difficult to meet the sleep assistance needs of the user. SUMMARY

[0005] Therefore, the embodiments of the present application provide a sleep music generation method, device and terminal device, aiming to solve the problems of inaccurate sleep state monitoring, lack of individualization and dynamic optimization of sleep music generation, inability to fully utilize multi-modal data, and difficulty in deeply combining long-term sleep information of the user in the prior art.

[0006] A first aspect of the embodiments of the present application provides a sleep music generation method, comprising:

[0007] obtaining user age information, user gender information, current user physiological state information, current user behavior state information, current user sleep behavior information, historical user physiological state information, historical user behavior state information, historical user sleep behavior information, and a sleep music generation model;

[0008] obtaining current user rhythm information and current user sleep state representation information according to the user age information, the user gender information, the current user physiological state information, the current user behavior state information, the historical user physiological state information, and the historical user behavior state information;

[0009] generating current user sleep state confirmation information according to the current user rhythm information and the current user sleep state representation information;

[0010] In response to generation of the current user sleep state confirmation information, sleep music information is generated according to the current user physiological state information, the sleep music generation model, and a preset sleep music set.

[0011] A second aspect of the embodiment of the present application provides a sleep music generation device, comprising:

[0012] An information acquisition module is configured to acquire user age information, user gender information, current user physiological state information, current user behavior state information, current user sleep behavior information, historical user physiological state information, historical user behavior state information, historical user sleep behavior information, and a sleep music generation model.

[0013] A user rhythm information and user sleep state representation information generation module is configured to obtain current user rhythm information and current user sleep state representation information according to the user age information, the user gender information, the current user physiological state information, the current user behavior state information, the historical user physiological state information, and the historical user behavior state information.

[0014] A current user sleep state confirmation information generation module is configured to generate current user sleep state confirmation information according to the current user rhythm information and the current user sleep state representation information.

[0015] A sleep music information generation module is configured to generate sleep music information in response to generation of the current user sleep state confirmation information according to the current user physiological state information, the sleep music generation model, and a preset sleep music set.

[0016] A third aspect of the embodiment of the present application provides a terminal device, comprising a memory and a processor, the memory stores a computer program that can be run on the processor, and the processor implements the steps of the sleep music generation method in the first aspect described above when executing the computer program.

[0017] A fourth aspect of the embodiment of the present application provides a computer readable storage medium, comprising a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the sleep music generation method in the first aspect described above.

[0018] Compared with the prior art, the embodiment of the present application has the beneficial effects that the embodiment of the present application precisely adapts to individual differences and dynamically changing sleep states of users, realizes personalized sleep adjustment, constructs rhythm information and sleep state representation information based on multi-modal data, makes the generated sleep music more suitable for current sleep needs of users, and thus improves the accuracy, adaptability, and effectiveness of sleep music generation. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0020] Figure 1 is an implementation flow diagram of the sleep music generation method provided by the first embodiment of the present application, and is specifically described as follows:

[0021] Figure 2 is an implementation flow diagram of the sleep music generation method provided by the second embodiment of the present application, and is specifically described as follows:

[0022] Figure 3 is an implementation flow diagram of the sleep music generation method provided by the third embodiment of the present application, and is specifically described as follows:

[0023] Figure 4 is an implementation flow diagram of the sleep music generation method provided by the fourth embodiment of the present application, and is specifically described as follows:

[0024] Figure 5 is an implementation flow diagram of the sleep music generation method provided by the fifth embodiment of the present application, and is specifically described as follows:

[0025] Figure 6 is an implementation flow diagram of the sleep music generation method provided by the sixth embodiment of the present application, and is specifically described as follows:

[0026] Figure 7 is a structural diagram of the sleep music generation device provided by the embodiment of the present application, and is specifically described as follows:

[0027] Figure 8 is a schematic diagram of the terminal device provided by the embodiment of the present application, and is specifically described as follows: DETAILED DESCRIPTION

[0028] In the following description, specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced without these specific details. In other instances, well-known systems, devices, circuits, and methods have not been described in detail so as not to obscure the description of the present application.

[0029] In order to illustrate the technical solutions described in the present application, the following will be described by specific embodiments.

[0030] Figure 1 The implementation flow diagram of the sleep music generation method provided by the first embodiment of the present application is shown, and is specifically described as follows:

[0031] In step S101, the user age information, the user gender information, the current user physiological state information, the current user behavior state information, the current user sleep behavior information, the historical user physiological state information, the historical user behavior state information, the historical user sleep behavior information, and the sleep music generation model are acquired.

[0032] In the embodiment, the user can be an infant. The user age information can refer to the user's week / month / age, which is used to match the standard parameters of the user's sleep rhythm development stage, such as the sleep duration range for different months of age, which can be obtained by manual input by the parent through the interactive interface of the intelligent central control console. The user gender information refers to the gender of the user, which is one of the individual difference characteristics and a reference for the sleep rhythm. It can be understood that different gender users can have different sleep patterns. The user gender information can be obtained by input by the parent through the interactive interface of the intelligent central control console. The current user physiological state information can include real-time physiological data such as heart rate, respiratory rate, and body temperature, which can be used to determine the user's activity state, drowsy state, light sleep state, or deep sleep state, such as heart rate > 100 times / minute corresponding to recognition as an active state. The current user physiological state information can be collected in real time by a millimeter wave radar (heart rate, respiration) and an infrared non-contact body temperature sensor. The current user behavior state information can refer to the user's pre-sleep wake-up activities, such as feeding, playing; can refer to the user's drowsy behavior between wake-up and sleep, such as yawning, rubbing eyes; can refer to the user's transition behavior from sleep to wake-up, such as turning over, getting up, etc., which can be used to assist in determining the current state. Among them, the user's pre-sleep wake-up activities can be obtained by querying the parent through the intelligent central control console, and the user's drowsy behavior between wake-up and sleep and the user's transition behavior from sleep to wake-up can be captured in real time by a camera. The current user sleep behavior information can cover real-time sleep data and periodic sleep data, which can be used to analyze sleep quality and rhythm, and can be obtained by collecting sleep monitoring mattress and millimeter wave radar sensors, and can be analyzed and processed by combining wavelet transform, random forest, etc. Among them, the real-time sleep data can include sleep posture, number of turns, etc.; the periodic sleep data can include sleep time, sleep duration, sleep cycle, 24-hour sleep-wake alternation times, etc. The historical user physiological state information can refer to the user's past physiological data such as heart rate, respiratory rate, and body temperature, which is used to compare and analyze the user's sleep state change rule and assist in personalized adjustment, which can be collected in advance by the system before formal operation and updated daily to form a historical record. The historical user behavior state information can refer to the user's past pre-sleep wake-up activities, such as feeding, playing; can refer to the user's past drowsy behavior between wake-up and sleep, such as yawning, rubbing eyes; can refer to the user's past transition behavior from sleep to wake-up, such as turning over, getting up, etc., which can be used to assist in determining the current state. The historical user sleep behavior information can refer to the user's past sleep duration, sleep time, sleep cycle, etc. data, which can be used to build a personalized circadian rhythm template, such as setting the evening as a "drowsy period" for a 4-month-old baby, which can be obtained by the system continuously recording and accumulating updates.The sleep music generation model can be a conditional control diffusion model based on score estimation, which can be used to dynamically generate music matching the sleep state of the user according to the user physiological data, and can be realized by training the user physiological data-music data set, combining a VAE pre-training model (mapping mel spectrum features) and a UNet+Attention model (estimating a score function), introducing forward and reverse diffusion processes in the training process, thereby generating mel spectrum features and converting them into audio through a HiFi-GAN model.

[0033] In the embodiment, the current user physiological state information includes current user heart rate information, current user breathing information and current user body temperature information; the current user behavior state information includes current user wake-up active behavior information, current user sleep transition behavior information and current user wake-up transition behavior information; the current user sleep behavior information includes current user sleep posture behavior information, current user sleep turning-over information and current user sleep behavior cycle information; the historical user physiological state information includes historical user heart rate information, historical user breathing information and historical user body temperature information; the historical user behavior state information includes historical user wake-up active behavior information, historical user sleep transition behavior information and historical user wake-up transition behavior information; the historical user sleep behavior information includes historical user sleep posture behavior information, historical user sleep turning-over information and historical user sleep behavior cycle information.

[0034] The current user heart rate information can refer to the real-time heartbeat frequency of the user, which is an important indicator for determining the activity state, drowsiness state, light sleep state or deep sleep state, such as an activity state heart rate > 100 times / minute, and a deep sleep state heart rate of 60-70 times / minute. The current user respiratory information can refer to the real-time respiratory rate of the user, and different sleep states correspond to different ranges, such as an activity state respiratory rate > 30 times / minute, and a deep sleep state respiratory rate of 15-20 times / minute. The current user body temperature information can refer to the real-time body temperature data of the user, which changes with the sleep state, such as an activity state body temperature > 36.8°C, and a deep sleep state body temperature of 36.0-36.3°C. The current user wakefulness active behavior information can refer to the behavior of the user when he / she is awake and not sleepy, such as eating milk, playing, pre-sleep activities, frequent turning over in sleep, getting up, etc. The current user sleep transition behavior information can refer to the behavior when transitioning from wakefulness to sleep, such as yawning, rubbing eyes, closing eyes, etc. The current user wake-up transition behavior information can refer to the behavior when transitioning from sleep to wakefulness, such as turning over in sleep, sucking, etc. The current user sleep posture behavior information can refer to the body posture of the user when sleeping, such as supine, lateral recumbency, etc. The current user sleep turning over information can refer to the frequency and number of limb turning over during sleep, which is the basis for determining light sleep or deep sleep, such as a light sleep state pressure fluctuation frequency > 1 time / minute. The current user sleep behavior cycle information can include sleep time, sleep duration, single sleep duration, sleep-wake alternation times within 24 hours, etc. The historical user heart rate information, historical user respiratory information and historical user body temperature information can respectively refer to the past heart rate, respiratory rate and body temperature data of the user, reflecting the change rule of the physiological state, which can be used for comparison with the current state and training of a personalized sleep music generation model. The historical user wakefulness active behavior information, historical user sleep transition behavior information and historical user wake-up transition behavior information can refer to the behavior records of the user in the past wakefulness active, sleep transition and wake-up transition stages, which are used to analyze the association mode of behavior and sleep state, and assist in current state determination and strategy optimization. The historical user sleep posture behavior information, historical user sleep turning over information and historical user sleep behavior cycle information can respectively refer to the past sleep posture, turning over frequency and sleep cycle data of the user, which can be used to construct a personalized sleep rhythm template, such as an ideal sleep cycle for different ages, and calibrate the model through daily data.The heart rate and respiration information can be collected in real time by a millimeter wave radar non-contact sensor, the body temperature information can be obtained by an infrared non-contact body temperature sensor, the body movement information in the wakeful active behavior and the wake-up transition behavior can be collected by a camera, the sleep preparation activity and other details can be obtained by inquiring the parents through an intelligent center console, the sleep position and turning-over information can be obtained in real time by a sleep monitoring mattress and a millimeter wave radar sensor, the sleep cycle information can be obtained by collecting data through a sensor and analyzing the data by using algorithms such as wavelet transform and random forest, the historical user heart rate information, the historical user respiration information and the historical user body temperature information can be collected in advance before the system is formally run, and the physiological data collected daily after the system is run is superimposed and updated with the historical data to form a continuous historical record, the historical user wakeful active behavior information, the historical user sleep transition behavior information and the historical user wake-up transition behavior information can be recorded synchronously with the current behavior information, and the historical data is accumulated by camera collection and parent input and is updated daily, the historical user sleep position behavior information, the historical user sleep turning-over information and the historical user sleep behavior cycle information can be recorded continuously by a sleep monitoring mattress, a millimeter wave radar and other devices, the system automatically accumulates and dynamically updates the information to store as historical data.

[0035] In step S102, current user rhythm information and current user sleep state representation information are obtained according to the user age information, the user gender information, the current user physiological state information, the current user behavior state information, the historical user physiological state information and the historical user behavior state information.

[0036] In this embodiment, the user age information and the user gender information can be one-hot encoded first to generate a basic feature vector as the core identifier of individual differences. Then, the current user physiological state information and the current user behavior state information can be combined to determine the state type of the current user in real time through threshold judgment and random forest algorithm. The state type of the current user can include an active state, a drowsy state, a light sleep state or a deep sleep state. For example, if the current heart rate is 90 beats per minute, the respiratory rate is 22 times per minute, and the camera captures slight body twisting, such as body movement frequency of 2 times per 10 minutes, it is determined to be a light sleep state. Then, based on the historical user physiological state information and the historical user behavior state information, in combination with the standard sleep rhythm parameters corresponding to the user age, a dynamic sleep rhythm feature space can be constructed through an LSTM+Transformer hybrid time sequence architecture, including 24-hour sleep-wake alternation times, single continuous sleep duration and other key rhythm features. The rhythm similarity between the current user and similar age / gender users in the database is calculated using a collaborative filtering algorithm, the rhythm pattern of the high similarity sample is referred to, the rhythm judgment of the current user is optimized, and thus the current user rhythm information (such as sleep period, wake period or sleep period / wake period transition) and the current user sleep state representation information (such as active state, deep sleep state and other specific state descriptions) are obtained.

[0037] In this embodiment, the rhythm information of the current user can be represented as The rhythm of the similar age / gender user in the database can be represented as The similarity between the current user and other users can be calculated through a collaborative filtering algorithm . It can be recorded as:

[0038]

[0039] Step S103, generating current user sleep state confirmation information according to the current user rhythm information and the current user sleep state representation information.

[0040] In this embodiment, it can be understood that in this application, the current user rhythm information is related to the sleep of the user. The current user rhythm information can be matched with the sleep state representation information based on a logically preset matrix. For example, if the current user rhythm information is "sleep period" and the current user sleep state is "active state", it corresponds to "need to lull to sleep"; if the current user rhythm information is "wake period" and the current user sleep state is "light sleep state", it corresponds to "need to wake up"; if the current user rhythm information is "sleep period / wake period" and the current user sleep state is "abnormal state" (such as sudden increase in heart rate + frequent crying), it corresponds to "need to alarm". The mapping rule can be dynamically calibrated in combination with the adjustment effect of the same rhythm-state combination in the historical data, such as the sleep time under the past "sleep period + drowsy state", for example, if the historical data shows that a certain user needs 15 minutes to fall asleep under "sleep period + drowsy state", the expected time is labeled synchronously during the current determination, and then the structured confirmation information containing the current sleep state type, adjustment demand and expected target is output as the core basis for subsequent music generation.

[0041] Step S104, in response to the generation of the current user sleep state confirmation information, generating sleep music information according to the current user physiological state information, the sleep music generation model and the preset sleep music set.

[0042] In this embodiment, the preset sleep music set can be an open source music data set, and the sleep music generation model can be a comparative physiological music model (CPMP). The current user physiological state information (real-time data such as heart rate and respiration) can be input into the comparative physiological music model (CPMP), encoded into text features by the BERT pre-training model, processed by the self-attention mechanism, and the context vector is extracted as the physiological feature vector, which reflects the matching demand of the current physiological state and the music. It can be a score estimation based conditional control diffusion model, taking the physiological feature vector as the control condition, processing the preset sleep music set, adding Gaussian noise to the low-dimensional latent vector of the music feature in the forward diffusion process; in the reverse diffusion process, the score function is estimated by the UNet+Attention model, and the generated mel spectrum feature is gradually denoised, and then the generated mel spectrum feature is input into the HiFi-GAN, and the sleep music information is generated by the HiFi-GAN and played.

[0043] The sleep music generation method provided by the embodiment of the application accurately adapts to the individual differences and dynamically changing sleep state of the user, realizes personalized sleep adjustment, constructs rhythm information and sleep state representation information based on multi-modal data, so that the generated sleep music is more suitable for the current sleep demand of the user, thereby improving the accuracy, adaptability and effectiveness of the sleep music generation.

[0044] Figure 2 An implementation flowchart of the sleep music generation method provided by Embodiment Two of the present application is shown, which is different from Embodiment One described above in that the step S104 specifically comprises:

[0045] In step S201, based on a preset sleep music spectrum feature information extraction rule, sleep music spectrum feature information is generated from a preset sleep music set.

[0046] In this embodiment, the preset sleep music spectrum feature information extraction rule can refer to a standardized process of processing music data by short-time Fourier transform, mel frequency scaling and mel filter bank. A plurality of music data can be selected from the preset sleep music set, and then short-time Fourier transform can be performed on each music data in sequence to obtain time-frequency distribution features, linear frequency is converted into mel frequency conforming to human ear hearing characteristics through mel frequency scaling, and finally mel spectrum features are extracted by using a mel filter bank to generate the sleep music spectrum feature information corresponding to each music, i.e., mel spectrum features, as one of the input features for subsequent model training.

[0047] In step S202, the current user physiological state information is encoded and processed to obtain current user physiological state feature information.

[0048] In this embodiment, the current user physiological state information includes current user heart rate information, current user breathing information and current user body temperature information. The encoding process is as follows: first, convert the real-time collected heart rate, breathing frequency and body temperature data into structured numerical values, such as heart rate 80 times / minute, breathing frequency 20 times / minute and body temperature 36.5℃; then input these numerical values into a large language model to generate corresponding text description data, such as “heart rate normal, breathing stable, body temperature appropriate”; and then the text description data can be input into the BERT pre-training model in the comparative physiological music model (CPMP), and after processing by the self-attention mechanism, the context vector is extracted as the current user physiological state feature information, which can map the association between physiological state and music features, providing physiological dimension input for model training.

[0049] The training process of the comparative physiological music model (CPMP) can use a bidirectional loss function:

[0050] The loss function for retrieving music data using user physiological data can be expressed as:

[0051]

[0052] wherein the numerator of the fraction represents the similarity of the positive sample (i.e., the user physiological data and the Similarity between individual music data), in the denominator Indicates the first The sum of similarities between a user's physiological data and all music data. This indicates the amount of user physiological data;

[0053] The loss function for retrieving user physiological data using music data can be expressed as:

[0054]

[0055] Among them, in the molecule Represents the similarity of positive samples (i.e., the first...) The music data and the first Similarity between individual user physiological data), in the denominator Indicates the first The sum of similarities between music data and all user physiological data. This indicates the amount of music data, which is understandable: .

[0056] Step S203: Based on the sleep music spectrum feature information and the current user physiological state feature information, train the sleep music generation model to obtain the trained sleep music generation model.

[0057] In this embodiment, the sleep music generation model can be a conditionally controlled diffusion model based on score estimation. The training process can begin by using the encoder in the VAE pre-trained model to map the generated sleep music spectral feature information to a low-dimensional latent feature vector. Then, the low-dimensional latent feature vector and the current user's physiological state feature information can be used as input to the score-based conditionally controlled diffusion model. During the forward diffusion process, zero-mean, time-dependent Gaussian noise is progressively added to the low-dimensional latent feature vector to make its distribution approximate a standard Gaussian distribution. Then, during the backward diffusion process, the current user's physiological state feature information is used as the control condition, and the score function at each time step is estimated using the UNet+Attention model. Noise is gradually removed, and the low-dimensional latent feature vector is reconstructed. This is then mapped back to the reconstructed Mel spectral features through the VAE decoder. The error between the reconstructed Mel spectral features and the original sleep music spectral feature information is calculated. The model parameters are updated through backpropagation, and the training process is repeated until the error converges, resulting in the trained sleep music generation model. The score function at each time step can be the log probability density gradient of the data.

[0058] The forward diffusion process can be defined as follows:

[0059]

[0060]

[0061] in, It is a discrete noise diffusion coefficient. It is a continuous noise diffusion coefficient. It is a tiny increment of Wiener motion. .

[0062] The purpose of the reverse diffusion process is to estimate the scores of data at different times, i.e., to calculate the log probability density gradient of the data, thereby gradually removing noise. This is achieved using the feature vector of user physiological data. As a control condition, the UNet+Attention model is used to estimate data at different times. Score Fitting data at different times True score , among which The model obtains feature vectors from user physiological data. The loss function of the diffusion model based on score estimation can be expressed as:

[0063]

[0064] in, , This is the score function estimated by the diffusion model based on score estimation.

[0065] Step S204: Generate sleep music information based on the current user's physiological state characteristics and the trained sleep music generation model.

[0066] In this embodiment, a random Gaussian noise vector can be generated as the initial input. The Gaussian noise vector and the current user's physiological state feature information are then input into the trained sleep music generation model to perform a reverse diffusion process. Guided by the current user's physiological state feature information, the trained sleep music generation model gradually removes noise through an estimated score function, generating a low-dimensional latent feature vector that matches the current physiological state. The low-dimensional latent feature vector is then input into the decoder in the VAE pre-trained model to map the corresponding sleep music spectral feature information. Subsequently, the Mel spectral feature is input into the HiFi-GAN pre-trained model to convert it into a playable audio signal, generating the final sleep music information.

[0067] The final sampling formula for the reverse process can be expressed as:

[0068]

[0069] Wherein, the step size is determined by the parameter control.

[0070] The sleep music generation method provided by the embodiment of the present application enables the sleep music generation model to more accurately learn the association between physiological states and music features, and the generated sleep music information is more in line with the real-time physiological needs of the current user, thereby improving the effect of personalized sleep regulation.

[0071] Figure 3 An implementation flowchart of the sleep music generation method provided by the third embodiment of the present application is shown, which is different from the second embodiment described above in that the step S203 specifically includes:

[0072] In step S301, the current user physiological state feature information is subjected to vector extraction to obtain a current user physiological state context feature vector.

[0073] In the embodiment, the current user physiological state feature information is a feature vector generated by encoding the BERT pre-training model in the comparative physiological music model (CPMP), which can be a context vector separated from the current user physiological state feature information.

[0074] In the embodiment, given N user physiological data (such as electroencephalogram data, respiratory data) corresponding text data, after encoding by the user physiological data encoder in the comparative physiological music model (CPMP) N dimensional vectors can be denoted as:

[0075]

[0076] wherein n represents the n-th user physiological data, and represents the user's physiological data.

[0077] In step S302, the current user physiological state context feature vector and the preset user group physiological state feature vector are subjected to merging processing to obtain a to-be-mapped user physiological state feature matrix.

[0078] In the embodiment, the preset user group physiological state feature vector refers to the physiological state feature vector of other users in the database, and the matrix formed by splicing the current user physiological state context feature vector and the preset user group physiological state feature vector by rows can be taken as the to-be-mapped user physiological state feature matrix.

[0079] In the embodiment, the user group physiological state feature vectors of N user physiological data and the current user physiological state context feature vector can be merged to obtain the to-be-mapped user physiological state feature matrix ​The physiological state feature matrix of the user to be mapped The shape is .

[0080] Step S303: Based on the user physiological state feature matrix to be mapped, the preset feature query matrix, the preset feature key matrix, and the preset feature value matrix, obtain the user physiological state feature correlation information and the user physiological state value feature matrix.

[0081] In this embodiment, the preset feature query matrix, preset feature key matrix, and preset feature value matrix can be learnable parameter matrices in the self-attention mechanism. Alternatively, the user's physiological state feature matrix to be mapped can be multiplied by the preset feature query matrix, preset feature key matrix, and preset feature value matrix respectively to obtain the query matrix, key matrix, and user physiological state value feature matrix. Then, the dot product of the query matrix and key matrix is ​​calculated, and this dot product is subjected to dimensionality reduction to avoid gradient vanishing, resulting in an attention score matrix, which represents the correlation information of user physiological state features. This can be called the attention score, used to measure the correlation of physiological features among different user samples.

[0082] In this embodiment, the Query matrix in the self-attention mechanism is used. The physiological state feature matrix of the user to be mapped Mapping to a matrix It can be written as:

[0083]

[0084] Using the Key matrix in the self-attention mechanism The physiological state feature matrix of the user to be mapped Mapping to a matrix It can be written as:

[0085]

[0086] Using the Value matrix in the self-attention mechanism The physiological state feature matrix of the user to be mapped Mapped to the feature matrix of user physiological state values It can be written as:

[0087]

[0088] For the user physiological state feature matrix to be mapped For each position i in the dataset, calculate the attention score. This attention score is used to measure the correlation between position i and position j, that is, the correlation information of user physiological state characteristics, which can be denoted as:

[0089]

[0090] wherein, is the th Query vector, is the th Key vector, , divided by is to prevent the gradient of the softmax function from vanishing due to the dot product result being too large.

[0091] In step S304, the user physiological state feature correlation degree information is normalized to obtain user physiological state feature weight information.

[0092] In this embodiment, the normalization processing can adopt a softmax function, which can be a softmax operation on each row of the user physiological state feature correlation degree information to obtain an attention weight matrix as the user physiological state feature weight information. The user physiological state feature weight information can be denoted as:

[0093]

[0094] In step S305, the user physiological state value feature processing matrix is generated according to the user physiological state feature weight information and the user physiological state value feature matrix.

[0095] In this embodiment, the user physiological state feature weight information and the user physiological state value feature matrix are weighted and summed, and the result of the weighted summation is taken as the user physiological state value feature processing matrix. The weighted summation can be represented as:

[0096]

[0097] In step S306, the user physiological state value feature representation vector is obtained by performing vector extraction on the user physiological state value feature processing matrix.

[0098] In this embodiment, the vector at the first position in the user physiological state value feature processing matrix is selected as the user physiological state value feature representation vector.

[0099] In step S307, the user physiological state value feature representation normalized vector is obtained by performing normalization processing on the user physiological state value feature representation vector based on a preset user physiological state conversion matrix.

[0100] In the embodiment, the preset user physiological state conversion matrix can be used to map the user physiological state value feature representation vector to a matrix consistent with the sleep music spectrum feature information dimension, can be used to multiply the user physiological state value feature representation vector by the preset user physiological state conversion matrix, obtain a dimension-adapted intermediate vector, and then perform normalization processing on the intermediate vector to generate a user physiological state value feature representation normalized vector.

[0101] In step S308, the sleep music generation model is trained according to the sleep music spectrum feature information and the user physiological state value feature representation normalized vector, and a trained sleep music generation model is obtained.

[0102] In the embodiment, the sleep music generation model can be a conditional control diffusion model based on score estimation. The encoder in the VAE pre-training model can be used to map the sleep music spectrum feature information to a low-dimensional latent music feature vector, and then the low-dimensional latent music feature vector and the user physiological state value feature representation normalized vector are spliced to form a fusion feature vector as the input of the sleep music generation model. In the forward diffusion process, zero-mean and time-dependent Gaussian noise is gradually added to the fusion feature vector, so that its distribution gradually approaches the standard Gaussian distribution. In the reverse diffusion process, the user physiological state value feature representation normalized vector is used as the control condition, the score function at each time step is estimated by the UNet+Attention model, the noise is gradually removed, the low-dimensional latent music feature vector is reconstructed, and the reconstructed mel spectrum feature is mapped back to the VAE decoder. The error between the reconstructed mel spectrum feature and the original sleep music spectrum feature information is calculated, the model parameters are updated through back propagation, and the training process is repeated until the error converges, and the trained sleep music generation model is obtained.

[0103] The sleep music generation method provided by the embodiment of the application introduces a self-attention mechanism to fuse individual and group physiological features, and realizes accurate matching of physiological features and music features through dimension adaptation and normalization processing, improves the sensitivity of the sleep music generation model to the user's physiological state and the pertinence of feature learning, and generates sleep music information that is more in line with the user's personalized sleep needs, thereby improving the sleep regulation effect.

[0104] Figure 4 An implementation flowchart of the sleep music generation method provided by the fourth embodiment of the application is shown, which is different from the third embodiment described above in that the step S303 specifically includes:

[0105] In step S401, the user physiological state feature matrix to be mapped is mapped based on a preset feature query matrix to generate a user physiological state query feature matrix.

[0106] In the embodiment, the preset feature query matrix can be a learnable parameter matrix in the self-attention mechanism. The matrix multiplication operation can be performed on the user physiological state feature matrix to be mapped and the preset feature query matrix to obtain a user physiological state query feature matrix, which can be used for subsequent calculation of the correlation weight between different user physiological features.

[0107] In step S402, the user physiological state feature matrix to be mapped is mapped based on the preset feature key matrix to generate a user physiological state key feature matrix.

[0108] In the embodiment, the preset feature key matrix can be a learnable parameter matrix in the self-attention mechanism. The matrix multiplication operation can be performed on the user physiological state feature matrix to be mapped and the preset feature key matrix to generate a user physiological state key feature matrix, which is used in cooperation with the user physiological state query feature matrix to measure the correlation of different user physiological features.

[0109] In step S403, the user physiological state feature matrix to be mapped is mapped based on the preset feature value matrix to generate a user physiological state value feature matrix.

[0110] In the embodiment, the preset feature value matrix can be a learnable parameter matrix in the self-attention mechanism. The matrix multiplication operation can be performed on the user physiological state feature matrix to be mapped and the preset feature value matrix to generate a user physiological state value feature matrix, which is used to store the user physiological feature information to be fused and subsequently fused by the attention weight.

[0111] In step S404, the user physiological state feature correlation information is obtained according to the user physiological state query feature matrix and the user physiological state key feature matrix.

[0112] In the embodiment, the dot product of the user physiological state query feature matrix and the transpose matrix of the user physiological state key feature matrix can be calculated to obtain an original correlation matrix. To avoid the gradient vanishing problem caused by too high feature dimension, each element in the original correlation matrix can be divided by the key feature dimension for scaling processing, so that the obtained matrix is the user physiological state feature correlation information, which is used to quantify the correlation degree between different user sample physiological features.

[0113] The sleep music generation method provided in the embodiment makes the correlation measurement of different user physiological features more accurate, provides a more reliable basis for subsequent feature weighted fusion, and thus improves the adaptability of the sleep music generation model to the user physiological state and the individualization degree of music generation.

[0114] Figure 5An implementation flowchart of the sleep music generation method provided by Embodiment Five of the present application is shown, which is different from Embodiment One in that after the step S104, it further includes:

[0115] In step S501, feature extraction is performed on the sleep music information to obtain sleep music feature information; the sleep music feature information includes sleep music rhythm feature information, sleep music melody feature information, and sleep music type information.

[0116] In this embodiment, the feature extraction process can use audio signal processing techniques, such as converting the audio signal to the frequency domain using Fourier transform, determining the rhythm by detecting the energy peak interval; such as extracting the pitch sequence and scale distribution of the audio, and using Mel Frequency Cepstral Coefficients (MFCC) to capture the frequency variation characteristics of the melody; such as inputting the extracted rhythm and melody features into a pre-trained music classification model (such as a CNN-based classifier) for determination, and the music type can include lullaby, natural white noise (ocean waves, rain), classical light music, etc.

[0117] In step S502, based on a preset mapping relationship between sleep music feature information and mattress vibration frequency information, mattress vibration frequency adjustment information is generated according to the sleep music rhythm feature information and the sleep music melody feature information, so as to adjust the mattress vibration frequency through the mattress vibration frequency adjustment information.

[0118] In this embodiment, the preset mapping relationship can be a rule base constructed based on historical data. If the sleep music rhythm feature information shows a slow rhythm and the melody feature is smooth, low-frequency adjustment information is generated; if the rhythm is slightly fast and the melody fluctuates moderately, medium-frequency adjustment information is generated. The mattress vibration frequency adjustment information is sent to the piezoelectric ceramic array in the form of electrical signal instructions to control it to vibrate at a specified frequency, realizing coordination with the music rhythm.

[0119] In step S503, based on a preset mapping relationship between sleep music feature information and sleep environment lighting information, sleep environment lighting adjustment information is generated according to the sleep music type information and the sleep music rhythm feature information; the sleep environment lighting adjustment information includes sleep environment lighting brightness adjustment information, sleep environment lighting color tone adjustment information, and sleep environment lighting flicker frequency adjustment information.

[0120] In the embodiment, the preset mapping relationship between the sleep music feature information and the mattress vibration frequency information and the preset mapping relationship between the sleep music feature information and the sleep environment illumination information can be the best mapping relationship obtained through experiments, and specifically can cover four types of corresponding relationships: music rhythm BPM corresponding to mattress vibration frequency and light flicker frequency, melody fluctuation corresponding to mattress amplitude and illumination color tone, music loudness corresponding to illumination brightness, and music type corresponding to light color. The specific construction process of the experiment and the mapping relationship can be: first, extract the core features (such as rhythm BPM value, melody fluctuation degree, and loudness decibel value) from the preset sleep music set, and set the gradient combination for the mattress vibration parameters (such as vibration frequency set to 5-15Hz gradient, amplitude set to 0.5-2mm gradient) and the illumination parameters (such as flicker frequency set to 0-1.5Hz gradient, color tone set to 2000-3500K gradient, brightness set to 1-15lux gradient). Then collect the physiological and behavioral feedback of the user under different music-vibration-illumination combinations, based on the goal of physiological state approaching stable sleep state (such as heart rate fluctuation ≤5 times / minute, sleep latency ≤25 minutes), determine the initial mapping rule through statistical analysis, for example: 60BPM soothing lullaby corresponding to 5-8Hz vibration frequency, 0Hz (constant) flicker frequency, low fluctuation melody corresponding to 0.5-1mm amplitude, 2000-2500K warm light, and 40dB low loudness corresponding to 1-5lux illumination brightness.

[0121] In the embodiment, the preset mapping relationship can be a multi-dimensional rule matrix preset by a person, for example, if the sleep music type is a lullaby and the rhythm is slow, generate low-brightness adjustment information.

[0122] Step S504, adjust the environment illumination brightness through the sleep environment illumination brightness adjustment information.

[0123] In the embodiment, the sleep environment illumination brightness adjustment information is transmitted to the light control device, so that the device adjusts the light intensity according to the duty cycle parameter in the signal.

[0124] Step S505, adjust the environment illumination color tone through the sleep environment illumination color tone adjustment information.

[0125] In the embodiment, the sleep environment illumination color tone adjustment information is transmitted to the light control device, and the light control device adjusts the light emitting proportion of red, green, and blue to realize the specified color tone illumination output.

[0126] Step S506, adjust the environment illumination flicker frequency through the sleep environment illumination flicker frequency adjustment information.

[0127] In the embodiment, the sleep environment light illumination flicker frequency adjustment information can be transmitted to the light control device, and the light control device can realize regular flickering of the illumination by controlling the alternating switch.

[0128] The sleep music generation method provided by the embodiment of the application realizes the coordinated adjustment of music and a multi-modal environment, improves the comprehensiveness and adaptability of sleep induction by constructing a linkage mechanism of music, touch, and light environment, accurately matches the real-time sleep state of a user, and significantly improves the sleep induction effect.

[0129] Figure 6 An implementation flowchart of the sleep music generation method provided by the sixth embodiment of the application is shown, which is different from the fifth embodiment described above in that after the step S506, the method further includes:

[0130] In step S601, the adjusted user physiological state information, the adjusted user behavior state information, and the adjusted user sleep behavior information are acquired.

[0131] In the embodiment, the adjusted user physiological state information can include adjusted user heart rate information, adjusted user respiration information, and adjusted user body temperature information, which can be collected in real time by a millimeter wave radar and an infrared non-contact body temperature sensor during the sleep music information playing and multi-modal environment adjustment. The adjusted user behavior state information includes adjusted user wake-up active behavior information, adjusted user sleep transition behavior information, and adjusted user wake-up transition behavior information, which are obtained by capturing the behavior characteristics such as body movement and expression of the infant after adjustment through a camera and supplementing the information by inquiring the parents through an intelligent center console. The adjusted user sleep behavior information includes adjusted user sleep posture behavior information, adjusted user sleep turning-over information, and adjusted user sleep behavior cycle information, which are collected by a sleep monitoring mattress and a millimeter wave radar sensor, and analyzed by a wavelet transform algorithm and a random forest algorithm to obtain adjusted sleep cycle data.

[0132] In step S602, the adjusted user physiological state information, the adjusted user behavior state information, and the adjusted user sleep behavior information are sent to a user terminal.

[0133] In the embodiment, the user terminal can be an interactive terminal such as a mobile phone APP or a smart central control screen held by the parents of the infant. The collected adjusted user physiological state information, adjusted user behavior state information, and adjusted user sleep behavior information can be formatted and converted into visual charts, and then sent to the user terminal through a wireless communication module, so that the parents can view the influence effect of multi-modal adjustment on the sleep state of the infant in real time.

[0134] In the embodiment, preferably, after the adjusted user physiological state information, the adjusted user behavior state information and the adjusted user sleep behavior information are sent to the user terminal, it is further determined whether the sleep state of the user at this time has changed compared with before the adjustment according to the adjusted user physiological state information, the adjusted user behavior state information and the adjusted user sleep behavior information, the state type of the user after the adjustment is judged based on the adjusted user physiological state information, the adjusted user behavior state information and the adjusted user sleep behavior information, the sleep music information is regenerated, and the mattress vibration frequency, the sleep environment light illumination adjustment information, the sleep environment light color tone adjustment information and the sleep environment light flicker frequency adjustment information are adjusted according to the regenerated sleep music information. The state type of the user after the adjustment can be judged in real time through threshold discrimination and random forest algorithm.

[0135] For example, when the state before adjustment is determined to be the light sleep state and the state after adjustment is still the light sleep state, it indicates that the generated sleep music information and the adjustment effect of the mattress vibration frequency, the sleep environment light brightness adjustment information, the sleep environment light color adjustment information, and the sleep environment light flicker frequency adjustment information are weak, so music adapted to the deep sleep state is selected from the pre-prepared music library, and music adapted to the deep sleep state is regenerated as sleep music information, and then the mattress vibration frequency, the sleep environment light brightness adjustment information, the sleep environment light color adjustment information, and the sleep environment light flicker frequency adjustment information are controlled based on the regenerated sleep music information to induce the user to enter the deep sleep state. For example, when the state before adjustment is determined to be the light sleep state and the state after adjustment is the deep sleep state, it indicates that the generated sleep music information and the adjustment effect of the mattress vibration frequency, the sleep environment light brightness adjustment information, the sleep environment light color adjustment information, and the sleep environment light flicker frequency adjustment information have a certain adjustment effect, so the current sleep music information and the mattress vibration frequency, the sleep environment light brightness adjustment information, the sleep environment light color adjustment information, and the sleep environment light flicker frequency adjustment information can be maintained unchanged, or music adapted to the deep sleep state is selected from the pre-prepared music library, and music adapted to the deep sleep state is regenerated as sleep music information, and then the mattress vibration frequency, the sleep environment light brightness adjustment information, the sleep environment light color adjustment information, and the sleep environment light flicker frequency adjustment information are controlled based on the regenerated sleep music information to induce the user to maintain the deep sleep state. For example, when the state before adjustment is determined to be the deep sleep state and the state after adjustment is the wake state, it indicates that the generated sleep music information and the adjustment effect of the mattress vibration frequency, the sleep environment light brightness adjustment information, the sleep environment light color adjustment information, and the sleep environment light flicker frequency adjustment information have already failed to adapt to the current wake state, so music adapted to the wake state is selected from the pre-prepared music library, and music information adapted to the wake state is regenerated, and then the mattress vibration frequency, the sleep environment light brightness adjustment information, the sleep environment light color adjustment information, and the sleep environment light flicker frequency adjustment information are controlled based on the regenerated music information adapted to the wake state to induce the user to maintain the wake state.

[0136] The sleep music generation method provided by the embodiments of the present application forms a closed-loop optimization mechanism by obtaining and feeding back the adjusted multi-modal data, which not only enables parents to master the sleep state changes of infants in real time, but also provides a basis for continuously and dynamically adjusting the sleep music generation strategy and multi-modal coordination parameters, thereby improving the accuracy and effectiveness of personalized sleep adjustment and realizing continuous optimization of sleep induction effect.

[0137] Corresponding to the method of the above embodiments, Figure 7A structural block diagram of the sleep music generation apparatus provided by the embodiment of the present application is shown. For the convenience of description, only the parts related to the embodiment of the present application are shown. Figure 7 The sleep music generation apparatus can be an execution subject of the sleep music generation method provided by the first embodiment.

[0138] With reference to Figure 7 The sleep music generation apparatus comprises:

[0139] The information acquisition module 710 is configured to acquire user age information, user gender information, current user physiological state information, current user behavior state information, current user sleep behavior information, historical user physiological state information, historical user behavior state information, historical user sleep behavior information, and a sleep music generation model.

[0140] The user rhythm information and user sleep state representation information generation module 720 is configured to obtain current user rhythm information and current user sleep state representation information according to the user age information, user gender information, current user physiological state information, current user behavior state information, historical user physiological state information, and historical user behavior state information.

[0141] The current user sleep state confirmation information generation module 730 is configured to generate current user sleep state confirmation information according to the current user rhythm information and current user sleep state representation information.

[0142] The sleep music information generation module 740 is configured to generate sleep music information according to the current user physiological state information, sleep music generation model, and preset sleep music set in response to the generation of the current user sleep state confirmation information.

[0143] The processes in which the modules in the sleep music generation apparatus provided by the embodiment of the present application realize their respective functions can be specifically referred to the description of the first embodiment provided above, and will not be described here. Figure 1

[0144] It should be understood that the sequence numbers of the steps in the above embodiments do not mean the execution sequence, and the execution sequence of the processes should be determined according to their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0145] It should be understood that when used in the present application and the appended claims, the term "comprise" indicates the existence of the described features, integers, steps, operations, elements and / or components, but does not exclude one or more other features, integers, steps, operations, elements, components and / or sets thereof.

[0146] ​It should also be understood that, in the description of the present application and the appended claims, the terms "and / or" are used to mean one or more of the items in the list joined by "and / or" and that it is not intended to, on its own, exclude any combination of one or more of the items in the list. It should also be understood that, in the description of the present application and the appended claims, the term "comprises the following features, structures, or characteristics, " means "comprising, but not limited to, the following features, structures, or characteristics," unless otherwise specifically specified.

[0147] As used in the description of the application and the appended claims, the term "if' can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "once it is determined" or "in response to determining" or "once [the described condition or event] is detected" or "in response to detecting [the described condition or event]," depending on the context.

[0148] In addition, the description in the specification of the application and the appended claims uses the terms "first," "second," "third," etc. to refer to the features described in the text simply for identification purposes, and not to imply or suggest a relative importance of the elements. It should also be understood that, although the terms "first," "second," etc. are used in the text to describe various elements, these elements should not be limited by these terms. These terms are simply used to distinguish one element from another. For example, a first table can be named a second table, and similarly, a second table can be named a first table without departing from the scope of the various described embodiments. The first table and the second table are both tables, but they are not the same table.

[0149] Reference in the specification to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places in the specification are not necessarily all referring to the same embodiment, although it can. The terms "including," "containing," "having," and variations thereof are meant to encompass the terms "including but not limited to."

[0150] The sleep music generation method provided by the embodiments of the present application can be applied to a terminal device such as a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and the like. The embodiments of the present application do not limit the specific type of the terminal device.

[0151] For example, the terminal device can be a station (STATION, STA) in a WLAN, and can be a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication function, a computing device, or other processing device connected to a wireless modem, a vehicle-mounted device, a vehicle networking terminal, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite radio device, a wireless modem card, a television set top box (STB), a customer premise equipment (CPE), and / or other devices for communicating over a wireless system, and a next-generation communication system, for example, a mobile terminal in a 5G network or a mobile terminal in a future evolved Public Land Mobile Network (PLMN) network, and the like.

[0152] Figure 8 FIG. 1 is a structural schematic diagram of a terminal device according to an embodiment of the present application. As shown in the figure, the terminal device 8 according to the embodiment includes at least one processor 80 (only one processor is shown in the figure), a memory 81, and the memory 81 stores a computer program 82 executable on the processor 80. Figure 8 Figure 8 When the processor 80 executes the computer program 82, the steps in each of the sleep music generation method embodiments described above are implemented, for example, steps S101 to S104 shown in the figure. Figure 1 Alternatively, when the processor 80 executes the computer program 82, the functions of each module / unit in each of the device embodiments described above are implemented, for example, the functions of the modules 710 to 740 shown in the figure. Figure 7

[0153] ​​The terminal device 8 can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The terminal device can include, but is not limited to, a processor 80, a memory 81. Those skilled in the art can understand that Figure 8 The terminal device 8 is only an example and does not constitute a limitation on the terminal device 8, and can include more or fewer components than shown, or combine some components, or include different components, for example, the terminal device can also include an input sending device, a network access device, a bus, and the like.

[0154] The processor 80 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0155] The memory 81 can be an internal storage unit of the terminal device 8 in some embodiments, for example, a hard disk or a memory of the terminal device 8. The memory 81 can also be an external storage device of the terminal device 8, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like. Further, the memory 81 can include both the internal storage unit and the external storage device of the terminal device 8. The memory 81 is used to store an operating system, an application program, a boot loader, data, and other programs, for example, program codes of the computer program, and the like. The memory 81 can also be used to temporarily store data that has been transmitted or will be transmitted.

[0156] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0157] The embodiment of the present application further provides a terminal device, which comprises at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor, wherein the processor executes the computer program to enable the terminal device to implement the steps in any of the above method embodiments.

[0158] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps in any of the above method embodiments.

[0159] The embodiment of the present application provides a computer program product, which enables a terminal device to implement the steps in any of the above method embodiments when the computer program product is executed on the terminal device.

[0160] The integrated modules / units, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the embodiment of the present application can also be implemented by a computer program to instruct related hardware to complete all or part of the above method embodiments, and the computer program can be stored in a computer readable storage medium, and the computer program can implement the steps in the above method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0161] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0162] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0163] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may also be distributed to multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0164] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for generating sleep music, characterized in that, include: Acquire user age information, user gender information, current user physiological state information, current user behavioral state information, current user sleep behavior information, historical user physiological state information, historical user behavioral state information, historical user sleep behavior information, and a sleep music generation model; Based on the user's age information, gender information, current user physiological state information, current user behavioral state information, historical user physiological state information, and historical user behavioral state information, the current user's circadian rhythm information and current user sleep state representation information are obtained. Based on the current user's circadian rhythm information and the current user's sleep state representation information, generate current user sleep state confirmation information; In response to the generation of the current user's sleep status confirmation information, sleep music information is generated based on the current user's physiological status information, the sleep music generation model, and a preset set of sleep music.

2. The sleep music generation method as described in claim 1, characterized in that, The current user's physiological status information includes the current user's heart rate information, current user's respiratory information, and current user's body temperature information; The current user behavior status information includes the current user's waking and active behavior information, the current user's sleep transition behavior information, and the current user's waking and transition behavior information. The current user sleep behavior information includes the current user sleeping posture behavior information, the current user sleeping turning information, and the current user sleep behavior cycle information; The historical user physiological status information includes historical user heart rate information, historical user respiratory information, and historical user body temperature information. The historical user behavior status information includes historical user awake and active behavior information, historical user sleep transition behavior information, and historical user wake-up transition behavior information. The historical user sleep behavior information includes historical user sleeping posture behavior information, historical user sleep turning information, and historical user sleep behavior cycle information.

3. The sleep music generation method as described in claim 1, characterized in that, The step of generating sleep music information in response to the generation of the current user's sleep state confirmation information, based on the current user's physiological state information, the sleep music generation model, and a preset set of sleep music, specifically includes: Based on the preset rules for extracting spectral features of sleep music, spectral features of sleep music are generated according to the preset set of sleep music. The current user's physiological state information is encoded to obtain the current user's physiological state feature information; The sleep music generation model is trained based on the sleep music spectrum feature information and the current user physiological state feature information to obtain the trained sleep music generation model. Based on the current user's physiological state characteristics and the trained sleep music generation model, sleep music information is generated.

4. The sleep music generation method as described in claim 3, characterized in that, The step of training the sleep music generation model based on the sleep music spectral feature information and the current user's physiological state feature information to obtain the trained sleep music generation model specifically includes: Vector extraction is performed on the current user's physiological state feature information to obtain the current user's physiological state context feature vector; The current user physiological state context feature vector and the preset user group physiological state feature vector are merged to obtain the physiological state feature matrix of the user to be mapped. Based on the user physiological state feature matrix to be mapped, the preset feature query matrix, the preset feature key matrix, and the preset feature value matrix, the user physiological state feature correlation information and the user physiological state value feature matrix are obtained. The correlation information of the user's physiological state features is normalized to obtain the weight information of the user's physiological state features. Based on the user physiological state feature weight information and the user physiological state value feature matrix, a user physiological state value feature processing matrix is ​​generated. Vector extraction is performed on the user physiological state value feature processing matrix to obtain the user physiological state value feature representation vector; Based on the preset user physiological state transformation matrix, the user physiological state value feature representation vector is normalized to obtain the user physiological state value feature representation normalized vector. The sleep music generation model is trained based on the sleep music spectrum feature information and the normalized vector of the user's physiological state value feature representation to obtain the trained sleep music generation model.

5. The sleep music generation method as described in claim 4, characterized in that, The step of obtaining user physiological state feature correlation information and user physiological state value feature matrix based on the user physiological state feature matrix to be mapped, the preset feature query matrix, the preset feature key matrix, and the preset feature value matrix specifically includes: Based on a preset feature query matrix, the physiological state feature matrix of the user to be mapped is mapped to generate a user physiological state query feature matrix. Based on a preset feature key matrix, the physiological state feature matrix of the user to be mapped is mapped to generate a user physiological state key feature matrix. Based on a preset feature value matrix, the physiological state feature matrix of the user to be mapped is mapped to generate a physiological state feature matrix of the user. Based on the user physiological state query feature matrix and the user physiological state key feature matrix, the user physiological state feature correlation information is obtained.

6. The sleep music generation method as described in claim 1, characterized in that, After the step of generating sleep music information based on the current user's physiological state information, the sleep music generation model, and a preset set of sleep music in response to the generation of the current user's sleep state confirmation information, the method further includes: Feature extraction is performed on the sleep music information to obtain sleep music feature information; the sleep music feature information includes sleep music rhythm feature information, sleep music melody feature information, and sleep music type information; Based on the preset mapping relationship between sleep music feature information and mattress vibration frequency information, mattress vibration frequency adjustment information is generated according to the sleep music rhythm feature information and sleep music melody feature information, so as to adjust the mattress vibration frequency through the mattress vibration frequency adjustment information. Based on the preset mapping relationship between sleep music feature information and sleep environment lighting information, sleep environment lighting adjustment information is generated according to the sleep music type information and sleep music rhythm feature information; the sleep environment lighting adjustment information includes sleep environment lighting intensity adjustment information, sleep environment lighting hue adjustment information, and sleep environment lighting flicker frequency adjustment information. The ambient light intensity is adjusted using the aforementioned sleep environment light intensity adjustment information; The ambient light tone is adjusted using the aforementioned sleep environment light tone adjustment information; The ambient light flicker frequency is adjusted using the aforementioned sleep environment light flicker frequency adjustment information.

7. The sleep music generation method as described in claim 6, characterized in that, After the step of adjusting the ambient light flicker frequency using the sleep ambient light flicker frequency adjustment information, the method further includes: Acquire user physiological state information, user behavioral state information, and user sleep behavior information after adjustment; The adjusted user physiological state information, adjusted user behavioral state information, and adjusted user sleep behavior information are sent to the user terminal.

8. A sleep music generating device, characterized in that, include: The information acquisition module is used to acquire user age information, user gender information, current user physiological state information, current user behavioral state information, current user sleep behavior information, historical user physiological state information, historical user behavioral state information, historical user sleep behavior information, and sleep music generation model. The user circadian rhythm information and user sleep state representation information generation module is used to obtain the current user circadian rhythm information and the current user sleep state representation information based on the user age information, user gender information, current user physiological state information, current user behavior state information, historical user physiological state information and historical user behavior state information. The current user sleep state confirmation information generation module is used to generate current user sleep state confirmation information based on the current user circadian rhythm information and the current user sleep state representation information. The sleep music information generation module is used to generate sleep music information in response to the generation of the current user's sleep state confirmation information, based on the current user's physiological state information, the sleep music generation model, and a preset set of sleep music.

9. A terminal device, characterized in that, The terminal device includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images