Sleep-aid audio generation method, device, equipment, and storage medium

Abstract

The present application provides a method, device, equipment, and storage medium for generating sleep-aiding audio. The method includes: collecting the brain wave sequence and respiratory signal sequence of the sleep-aiding subject; after time-aligning the brain wave sequence and the respiratory signal sequence, sampling at a preset sampling frequency, and connecting the brain wave and respiratory signal at the sampling time point as a state signal each time; determining the sleep state feature vector and the current sleep state based on the time series signal composed of the state signal; determining the reference audio that matches the current sleep state; and generating sleep-aiding audio that assists the sleep-aiding subject to enter the next sleep state based on the reference audio and the sleep state feature vector through a diffusion model. The method of the present application generates sleep-aiding audio based on the brain wave sequence and respiratory signal sequence of the sleep-aiding subject, and can adaptively generate sleep-aiding audio based on the sleep state of the sleep-aiding subject, thereby achieving a better sleep-aiding effect.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for generating sleep-aid audio. Background Art

[0002] Sleep-aid audio is a specific type of audio played through headphones that is designed to help people relax their body and mind, relieve stress and anxiety, make it easier to fall asleep, or promote deep sleep, making it easier to fall asleep.

[0003] Sleep-aid audio includes many categories, such as nature sounds, soothing music, guided meditations, and white noise. Nature sounds, including the sounds of waves, rain, and birdsong, simulate the natural environment and create a peaceful and comfortable sleep environment. Soothing music includes instrumental music, light music, and piano music to meet the different musical relaxation preferences of different users. Guided meditations provide professional meditation audio to help users relax and fall asleep through meditation. White noise, a sound that encompasses all frequencies, can help mask background noise and make it easier to fall asleep.

[0004] Existing sleep-aid audio is obtained by manually collecting these types of sounds in advance, supplemented by post-editing and production. When used, the audio type and parameters are pre-selected in the headphones to accompany sleep.

[0005] In the existing audio technology of sleep-aid headphones, the audio playback content and parameter settings can only be pre-set by the user before going to bed, and the audio cannot be automatically adjusted. Summary of the Invention

[0006] In order to solve one of the above technical defects, the present application provides a sleep-aid audio generation method, device, equipment, and storage medium.

[0007] In a first aspect, the present application provides a method for generating sleep-aiding audio, the method comprising:

[0008] Collect brain wave sequences and breathing signal sequences of the sleep-aiding subject;

[0009] After the brain wave sequence and the respiratory signal sequence are time-aligned, sampling is performed at a preset sampling frequency, and the brain wave and respiratory signal at the sampling time point are connected as a state signal each time sampling is performed;

[0010] determining a sleep state feature vector and a current sleep state according to a time series signal formed by the state signal;

[0011] Determine reference audio that matches the current sleep state;

[0012] Through the diffusion model, based on the reference audio and the sleep state feature vector, a sleep-aiding audio is generated to help the sleep-aiding subject enter the next sleep state.

[0013] Optionally, collecting a breathing signal sequence of the sleep-aiding subject includes:

[0014] The video stream of the superficial temporal artery and external carotid artery of the sleep aid subject is collected through an infrared long-wave sensor and an image acquisition card;

[0015] The time series signal of the first-order moment of each frame image in the video stream is extracted to form a respiratory signal sequence.

[0016] Optionally, determining the sleep state feature vector and the current sleep state according to the time series signal formed by the state signal includes:

[0017] The time series signal composed of the state signal is divided according to a preset time length to obtain multiple subsequences;

[0018] The temporal features of each subsequence are extracted through two convolutional neural networks; the initial filter size of one convolutional neural network is F s / 2, step size is F s / 16; the starting filter size of another convolutional neural network is F s ×4, step size is F s / 2;F s is the preset sampling frequency;

[0019] The temporal features of each subsequence are encoded through a bidirectional long short-term memory network to obtain the sleep state feature vector;

[0020] Convert the sleep state feature vector into the sleep state probability;

[0021] Predict the current sleep state of the sleep aid subject based on the sleep state probability.

[0022] Optionally, determining a reference audio that matches the current sleep state includes:

[0023] From the audio database, an audio that matches the current sleep state is randomly selected as a reference audio; wherein the audio database is composed of multiple sleep-aiding audio data, and each sleep-aiding audio data is marked with the corresponding sleep state.

[0024] Optionally, generating sleep-aiding audio for assisting the sleep-aided subject to enter the next sleep state based on the reference audio and the sleep state feature vector through a diffusion model includes:

[0025] Convert the reference audio into a spectrum graph through Mel spectrum conversion;

[0026] The spectrum graph is encoded and compressed through the variational autoencoder in the diffusion model to obtain the reference spectrum features;

[0027] Through the denoising network in the diffusion model, multiple denoising processes are performed based on the reference spectrum features and the sleep state feature vector to obtain auxiliary audio features;

[0028] By generating an efficient and high-fidelity speech synthesis model based on a generative adversarial network, the auxiliary audio features are transcoded into the original waveform to obtain sleep-aiding audio that helps the sleep-aided subject enter the next sleep state.

[0029] Optionally, a denoising network in the diffusion model is used to perform multiple denoising processes based on the reference spectrum features and the sleep state feature vector to obtain auxiliary audio features, including:

[0030] For any denoising process, the cross-attention module of the denoising network in the diffusion model is used to connect the reference spectrum features and the sleep state feature vector, map the connected feature vector to the Key and Value feature spaces, and map the noise spectrum features to the Query feature space. Based on the mapping features of the Query feature space, the mapping vector of the Key feature space, and the mapping vector of the Value feature space, auxiliary audio is generated.

[0031] The noise spectrum feature is a spectrum graph composed of initialized Gaussian noise.

[0032] Optionally, generating auxiliary audio based on the mapping features of the Query feature space, the mapping vectors of the Key feature space, and the mapping vectors of the Value feature space includes:

[0033] By formula Generate auxiliary audio features;

[0034] Among them, O is the auxiliary audio feature, Q is the Key feature space, K is the mapping vector of the Key feature space, V is the mapping vector of the Value feature space, T is the transpose, and d is the number of channels in the Key feature space; Softmax() is the Softmax processing, and Conv() is the mapping processing.

[0035] In a second aspect of the present application, a sleep-aid audio generation device is provided, the device comprising:

[0036] An acquisition module, used to acquire brain wave sequences and respiratory signal sequences of the sleep-aiding subject;

[0037] The sampling module is used to time-align the brain wave sequence and the respiratory signal sequence collected by the acquisition module, sample them at a preset sampling frequency, and connect the brain wave and respiratory signal at the sampling time point as a state signal each time;

[0038] The sleep-aiding audio generation module is used to determine the sleep state feature vector and the current sleep state based on the time series signal composed of the state signal obtained by the sampling module; determine the reference audio that matches the current sleep state; and generate sleep-aiding audio to assist the sleep-aiding subject to enter the next sleep state based on the reference audio and the sleep state feature vector through the diffusion model.

[0039] In a third aspect of the present application, an electronic device is provided, comprising:

[0040] Memory;

[0041] processor; and

[0042] computer programs;

[0043] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect above.

[0044] In a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored; the computer program is executed by a processor to implement the method described in the first aspect above.

[0045] The present application provides a method, device, equipment, and storage medium for generating sleep-aiding audio. The method includes: collecting the brain wave sequence and respiratory signal sequence of the sleep-aiding subject; after time-aligning the brain wave sequence and the respiratory signal sequence, sampling at a preset sampling frequency, and connecting the brain wave and respiratory signal at the sampling time point as a state signal each time; determining the sleep state feature vector and the current sleep state based on the time series signal composed of the state signal; determining the reference audio that matches the current sleep state; and generating sleep-aiding audio that assists the sleep-aiding subject to enter the next sleep state based on the reference audio and the sleep state feature vector through a diffusion model. The method of the present application determines the sleep state feature vector and the current sleep state based on the brain wave sequence and respiratory signal sequence of the sleep-aiding subject, and after determining the reference audio that matches the current sleep state, adjusts the reference audio based on the sleep state feature vector to generate sleep-aiding audio that assists the sleep-aiding subject to enter the next sleep state. The sleep-aiding audio can be adaptively generated according to the sleep state of the sleep-aiding subject, thereby achieving a better sleep-aiding effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0047] Figure 1 A flowchart of a sleep-aid audio generation method provided in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of the structure of a sleep-aiding audio generation module provided in an embodiment of the present application;

[0049] Figure 3 A schematic diagram of the training principle of a diffusion model provided in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of the implementation principle of a sleep-aid audio generation method provided in an embodiment of the present application;

[0051] Figure 5 A schematic structural diagram of a sleep-aiding audio generation device provided in an embodiment of the present application;

[0052] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the technical solutions and advantages of the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, and are not an exhaustive list of all the embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other unless they conflict.

[0054] Existing sleep-aid audio is created by manually collecting these types of sounds in advance and then editing them. When using the headphones, the user pre-selects the audio type and configures the parameters before falling asleep. However, due to the audio technology of existing sleep-aid headphones, the audio content and parameters can only be set by the user before bedtime, and the audio cannot be automatically adjusted.

[0055] In order to solve this problem, the present application provides a method, device, equipment, and storage medium for generating sleep-aiding audio. The method includes: collecting the brain wave sequence and respiratory signal sequence of the sleep-aiding subject; after time-aligning the brain wave sequence and the respiratory signal sequence, sampling at a preset sampling frequency, and connecting the brain wave and respiratory signal at the sampling time point as a state signal each time; determining the sleep state feature vector and the current sleep state according to the time series signal composed of the state signal; determining the reference audio that matches the current sleep state; through a diffusion model, generating sleep-aiding audio that assists the sleep-aiding subject to enter the next sleep state according to the reference audio and the sleep state feature vector. The method of the present application determines the sleep state feature vector and the current sleep state according to the brain wave sequence and respiratory signal sequence of the sleep-aiding subject, and after determining the reference audio that matches the current sleep state, adjusts the reference audio according to the sleep state feature vector to generate sleep-aiding audio that assists the sleep-aiding subject to enter the next sleep state. The sleep-aiding audio can be adaptively generated according to the sleep state of the sleep-aiding subject, thereby achieving a better sleep-aiding effect.

[0056] The present embodiment provides a method for generating sleep-aiding audio. In specific implementation, the sleep-aiding audio generation method provided by the present embodiment can be implemented by a system consisting of an acquisition module, an in-ear micro-sensor, an infrared long-wave sensor, an image acquisition card, a sampling module, and a sleep-aiding audio generation module. Data communication can be carried out between the acquisition module, the in-ear micro-sensor, the infrared long-wave sensor, the image acquisition card, the sampling module, and the sleep-aiding audio generation module through wired (such as through a bus, etc.), or through wireless (such as 4G, 5G, etc.). That is to say, all or part of the acquisition module, the in-ear micro-sensor, the infrared long-wave sensor, the image acquisition card, the sampling module, and the sleep-aiding audio generation module can be located in the same device or in different devices. The present embodiment does not limit the communication method between the acquisition module, the in-ear micro-sensor, the infrared long-wave sensor, the image acquisition card, the sampling module, and the sleep-aiding audio generation module.

[0057] Step 101 can be implemented by an acquisition module, an in-ear micro sensor, an infrared long-wave sensor, and an image acquisition card. Step 102 can be implemented by a sampling module, and subsequent steps 103 to 105 can be implemented by a sleep-aiding audio generation module.

[0058] Among them, the sleep-aiding audio generation module structure can be as follows Figure 2 As shown, it includes TSE (Time-Series Encoder, time series signal encoding), Database (sleep state corresponding audio database), and RASD (Retrieval-Augmented Spectrogram Diffusion, spectrum diffusion model based on search enhancement).

[0059] TSE consists of at least two convolutional neural networks, a bidirectional long short-term memory network and a softmax layer.

[0060] See also Figure 1 The implementation process of the method provided in this embodiment is as follows:

[0061] 101. Collect brain wave sequences and breathing signal sequences of the sleep assistance subject.

[0062] In this step, the brainwave information of the sleep-aided subject is collected and a brainwave sequence is formed based on the brainwave information. The breathing information of the sleep-aided subject is also collected and a breathing signal sequence is formed based on the breathing information.

[0063] 1. The process of collecting brain wave sequences

[0064] Electroencephalography (EEG) information is collected through in-ear micro-sensors to obtain brain wave sequences.

[0065] Among them, the ear-type microsensor can be the SpiralIE system, which is a deformable support with a spiral radius slightly larger than the radius of the ear canal. It triggers the shape memory effect by applying Joule heating generated by an external electric field to fit the inner wall of the ear canal of the person being assisted to sleep. The electrodes on the support sense tiny voltage changes in the ear canal to obtain EEG waveforms.

[0066] 2. Acquisition process of respiratory signal sequence

[0067] The researchers used a long-wave infrared sensor and an image acquisition card to capture video streams of the sleep-aided subjects' superficial temporal arteries (STA) and external carotid arteries (ECA). They extracted the time series signal (i.e., the respiration signal) from each frame of the video stream to form a respiration signal sequence.

[0068] For example, the Ostu method (an adaptive image threshold segmentation algorithm) is used to binarize each frame image in the video stream to obtain a sequence image, and the time series signal of the first-order moment of the sequence image is extracted, which is the breathing signal of the sleep-aiding subject.

[0069] 102. After the brain wave sequence and the respiratory signal sequence are time-aligned, sampling is performed at a preset sampling frequency, and the brain wave and respiratory signal at the sampling time point are connected as a state signal each time sampling is performed.

[0070] In specific implementation, the sampling module can align the brain wave sequence and the respiratory signal sequence in time, and then sample the brain wave sequence at a preset sampling frequency (such as F s) are sampled and then aligned in the time domain to form a state signal at each time point, thus forming a time series signal.

[0071] 103. Determine a sleep state feature vector and a current sleep state according to the time series signal formed by the state signal.

[0072] Step 103 can be achieved by Figure 2 The TSE implementation in , TSE includes at least two convolutional neural networks, a bidirectional long short-term memory network and a softmax layer. The implementation process of this step is:

[0073] 103 - 1 , dividing the time series signal composed of the state signal according to a preset time length to obtain a plurality of subsequences.

[0074] The preset time length is, for example, 30 seconds.

[0075] For example, Figure 2 The TSE in the paper divides the continuous time series signal (i.e. Figure 2 The Time Series S in is split into 30-second periods to obtain multiple subsequences.

[0076] 103-2, the temporal features of each subsequence are extracted through two convolutional neural networks.

[0077] Among them, the starting filter size of a convolutional neural network is F s / 2, step size is F s / 16. The starting filter size of another convolutional neural network is F s ×4, step size is F s / 2. F s is the preset sampling frequency.

[0078] For example, Figure 2 The TSE in

[15] includes two convolutional neural networks, which are used to extract time-invariant features for each 30-second period. The first layer of each convolutional neural network uses filters of different sizes to capture features of different scales. The starting filter size of the first convolutional neural network is F s / 2, step size is F s / 16, to better obtain the time domain part of the time series; the starting filter size of the second convolutional neural network is F s ×4, step size is F s / 2, in order to better obtain the frequency domain part of the time series.

[0079] 103-3, the temporal features of each subsequence are encoded through a bidirectional long short-term memory network to obtain the sleep state feature vector.

[0080] For example, Figure 2 The TSE in

[15] includes a bidirectional long short-term memory network, which can encode the time information in the time series period (i.e., encode the time characteristics of each subsequence) and obtain the sleep state feature vector to learn the transition rules of the sleep state.

[0081] 103-4, converting the sleep state feature vector into a sleep state probability, and predicting the current sleep state of the sleep-aided subject based on the sleep state probability.

[0082] Among them, sleep states such as light sleep, deep sleep or rapid eye movement.

[0083] For example, Figure 2 The TSE in

[15] includes a Softmax layer, which converts the sleep state feature vector into a sleep state probability and predicts the current sleep state of the sleep-aided subject based on the sleep state probability.

[0084] In specific implementation, TSE can be optimized through supervised learning. For example, through a large amount of sample data, the probability of sleep state and sleep state are predicted, and TSE is optimized based on the cross entropy loss between the predicted results and the actual sleep state of the sample data.

[0085] 104 , determining a reference audio that matches the current sleep state.

[0086] In this step, the audio database (such as Figure 2 In the Database in , randomly select an audio that matches the current sleep state as the reference audio.

[0087] Among them, the audio database is composed of multiple sleep-aiding audio data, and each sleep-aiding audio data is marked with the corresponding sleep state.

[0088] Step 104 passed Figure 2 The TSE and Database in the . Database can be constructed by collecting existing sleep-aiding audio data. For example, each sleep-aiding audio data is a two-tuple, and the two-tuple will mark the sleep state corresponding to the sleep-aiding audio data. For example, the two-tuple of any sleep-aiding audio data is (i, R ij ), where i is the sleep state identifier, j is the sleep-aiding audio data identifier, and R ij is the jth sleep-aiding audio data, and corresponds to sleep state i. Thus, Database can be defined as a two-tuple set {(i, {R ij})}.

[0089] Among them, {R ij} is the set of all sleep-aiding audio data corresponding to sleep state i.

[0090] In step 104, the TSE compares the current sleep state obtained in step 103 with the sleep states corresponding to all the sleep-aiding audio data in the database, obtains a set of all the sleep-aiding audio data corresponding to the current sleep state, and then randomly obtains a sleep-aiding audio data from the set of all the sleep-aiding audio data corresponding to the current sleep state. The sleep-aiding audio data is the reference audio (such as Figure 2 The reference audio is used in step 105 to guide the diffusion model to generate audio that matches the current sleep state, thereby providing a more personalized and efficient sleep aid solution.

[0091] 105 , generating a sleep-aiding audio for assisting the sleep-aiding subject to enter the next sleep state based on the reference audio and the sleep state feature vector using a diffusion model.

[0092] Before executing step 105, a diffusion model can be trained. In step 105, sleep-aiding music is generated by the trained diffusion model. Figure 2 RASD in.

[0093] The training process is as follows Figure 3 As shown, the diffusion model adopts the linear noise mechanism of the Denoising Diffusion Probabilistic Model (DDPM), and the inherent parameters β1=0.0001,β T = 0.01, using this parameter and the forward process of DDPM to add noise, and the reverse denoising network to perform denoising. Where t is the step number identifier, T is the maximum step number, β t is the intrinsic parameter at step t.

[0094] If the training sample data is audio data, the training sample data is converted into a spectrum graph through Mel spectrum conversion to obtain a variable Z0 of the spectrum graph.

[0095] During training, the forward noise addition process of the denoising diffusion model is a process from the initial variable (i.e. Z0) to the terminal variable (i.e. Gaussian noise variable Z T ) diffusion Markov chain, which can be formalized as:

[0096]

[0097] Among them, Z t is the variable of the tth step, I is a diagonal matrix, q() is the probability distribution function in the noise addition process, and N() is the Gaussian distribution function.

[0098] The reverse process of the denoising diffusion model converts the Gaussian noise variable Z T Converted to Z0, it can be formally defined as:

[0099]

[0100] p θ (Z t-1 |Z t ):=N(Z t-1 ;μ θ (Z t ,t),∑ θ (Z t ,t)).

[0101] Among them, θ is the denoising parameter of the diffusion model, p θ () is the probability distribution function in the denoising process with θ as the denoising parameter, μ θ () is the mean calculation function of Gaussian noise in the denoising process with θ as the denoising parameter, ∑ θ () is the variance calculation function of Gaussian noise in the denoising process of the denoising parameter.

[0102] The reverse process of the denoising diffusion model can be implemented by a denoising neural network, where θ is a parameter of the denoising neural network. Step 105 can be implemented by a denoising neural network of a trained noise diffusion model, and the implementation process is:

[0103] 105-1, converting the reference audio into a spectrum graph through Mel Spectrogram conversion.

[0104] like Figure 3 In the reference audio (i.e. Figure 3 Reference Audio R) in is converted to Spectrogram via Mel Spectrogram.

[0105] 105-2, the spectrum graph is encoded and compressed through the variational autoencoder in the diffusion model to obtain the reference spectrum features.

[0106] like Figure 3 In the Reference Audio R, the Spectrogram converted from Mel Spectrogram is compressed by the variational autoencoder to obtain the feature Z of the VAE Encoder R (i.e. reference spectrum characteristics).

[0107] 105-3, through the denoising network in the diffusion model, multiple denoising processes are performed based on the reference spectrum features and the sleep state feature vector to obtain auxiliary audio features.

[0108] For example, for any denoising process, the reference spectrum features and the sleep state feature vectors are connected through the cross-attention module of the denoising network in the diffusion model, the connected feature vectors are mapped to the Key and Value feature spaces, the noise spectrum features are mapped to the Query feature space, and auxiliary audio is generated based on the mapping features of the Query feature space, the mapping vector of the Key feature space, and the mapping vector of the Value feature space.

[0109] The noise spectrum feature is a spectrum graph composed of initialized Gaussian noise.

[0110] Generate auxiliary audio based on the mapping features of the Query feature space, the mapping vectors of the Key feature space, and the mapping vectors of the Value feature space, including:

[0111] By formula Generate auxiliary audio features.

[0112] Where O is the auxiliary audio feature, Q is the Key feature space, K is the mapping vector of the Key feature space, V is the mapping vector of the Value feature space, T is the transpose, and d is the number of channels in the Key feature space. Softmax() is the Softmax process, and Conv() is the mapping process.

[0113] like Figure 3 As shown, in step 105-3, the cross attention module of the denoising network (i.e. Figure 3 Cross-Attention in the reference spectrum feature Z R , sleep state feature vector (i.e. Figure 3 Sleep Embedding E) in the multi-step denoising process to obtain auxiliary audio features (i.e. Figure 2 Spectrogram U in ).

[0114] In any denoising process (such as the t-th denoising process), the cross attention module combines the sleep state feature vector E and the reference spectrum feature Z R Then, the connected feature vector is mapped to the Key and Value feature space through a trainable linear layer; at the same time, the cross attention module converts the noise spectrum feature Z t The intermediate layer feature variables Mapped to the Query feature space; the cross-attention module uses the cross-attention mechanism between Query, Key, and Value, the forward neural network, and the residual connection to output the generated audio features, so that the generated audio incorporates the reference audio features and is aligned with the current sleep state features.

[0115] The process of mapping the connected feature vectors to the Key and Value feature spaces and mapping the noise spectrum features to the Query feature space can be formally expressed as:

[0116]

[0117] Among them, Q is the Key feature space, K is the mapping vector of the Key feature space, V is the mapping vector of the Value feature space, Concat() is the connection process, Concat([Z R ,E]) is the sleep state feature vector E and the reference spectrum feature Z R Conv() is the mapping process.

[0118] The process of generating auxiliary audio based on the mapping features of the Query feature space, the mapping vectors of the Key feature space, and the mapping vectors of the Value feature space can be formally expressed as follows:

[0119] O is the fused feature. After multiple steps of denoising, the noise spectrum feature Z0 can be finally decoded by the variational autoencoder into a spectrum graph U, and then re-encoded into the original waveform V through the standard HiFi-GAN algorithm to obtain playable audio (i.e. Figure 3 The VAE Decoder in the decoder gets the Spectrogram, and then the Audio is obtained through HiFi-GAN).

[0120] 105-4, by generating an efficient and high-fidelity speech synthesis model of a confrontation network, the auxiliary audio features are transcoded into the original waveform to obtain the sleep-aiding audio that helps the sleep-aided subject enter the next sleep state (i.e. Figure 2 Audio V in ).

[0121] After the sleep-aiding audio is generated in step 5, the sleep-aiding audio can be played to the sleep-aiding target, and then the sleep-aiding audio can be adaptively adjusted to achieve a better sleep-aiding effect.

[0122] The content of the sleep-aiding audio obtained by the sleep-aiding audio generation method provided in this embodiment can automatically switch as the sleep state of the sleep-aiding subject changes, ensuring that the audio at different stages better meets the actual needs of the sleep-aiding subject.

[0123] In addition, in a specific implementation, after the current sleep state is obtained in step 103, the volume can be adaptively adjusted, such as adjusting the volume according to the sleep depth, so as to avoid stimulating the user and affecting the sleep quality.

[0124] In addition, after obtaining the current sleep state through step 103, the audio can be automatically turned on and off, and it can be determined whether the sleep-aiding audio needs to be played according to the sleep state of the sleep-aiding object, thereby achieving high efficiency and energy saving, while providing sleep aid that is more in line with actual needs.

[0125] The sleep-aid audio generation method provided in this embodiment can effectively improve the intelligence level of sleep-aid audio, provide users with a more efficient, stable and comfortable sleep assistance experience, and has broad application prospects.

[0126] The sleep-inducing audio generation method provided in this embodiment is a sleep-state-adaptive sleep-inducing audio generation method suitable for use with sleep-inducing headphones. Unlike existing audio technologies, this method can sense and detect the real-time sleep state of the person being assisted, automatically adjusting audio playback based on the sleep state. It can also adaptively adjust the music content, volume, and audio duration, achieving a better sleep-inducing effect.

[0127] like Figure 4 As shown, the sleep-aiding audio generation method provided in this embodiment, in step 101, is recorded by an in-ear electroencephalogram (ie Figure 4 In-ear EEGs recording) to collect brain waves and form a single-channel EEG (i.e. Figure 4 The single-channel EEG in the channel is the brain wave sequence. Figure 4 STA / ECA monitoring in the video stream acquisition of the superficial temporal artery and the external carotid artery, extract the time series signal of the first-order moment of each frame image in the video stream, and form a respiratory signal sequence (i.e. Figure 4 In step 102, a state signal is obtained based on the brain wave sequence and the respiratory signal sequence. Figure 4 The Sleep-Audio Generator in the code executes steps 103 to 105 to adaptively generate a sleep-aiding audio (i.e. Figure 4 After the sleep-aiding audio is played to the sleep-aiding subject, the sleep-aiding audio can adapt to the current sleep state of the sleep-aiding subject and assist the user to enter the next sleep state.

[0128] In this process, the collected brain wave sequence and respiratory signal sequence are continuous time waveforms, which are sampled at a preset frequency (such as F s ) and a preset time length to generate multiple subsequences, which are input into the sleep-aiding audio generation module through the network. The sleep-aiding audio generation module synthesizes playable audio and then transmits it back to the headphone player for playback.

[0129] The sleep-aid audio generation method provided in this embodiment optimizes the diffusion model through supervised learning to accurately predict the sleep state. In addition, the brain wave sequence and the breathing signal sequence are processed to obtain a state signal, and the state signal is used to obtain the current sleep state, and the reference audio matching the current sleep state is retrieved from the audio database as a key guide for generating audio. Finally, the diffusion model converts the reference audio into spectral features, and combines the sleep state features through a cross-attention mechanism to achieve alignment of the audio features with the sleep state, and generate sleep-aid audio that is highly matched with the current state. The sleep-aid audio generation method provided in this embodiment can provide a more accurate and efficient sleep-aid solution by deeply combining sleep state features, expert reference audio and personalized audio generation.

[0130] The sleep-aiding audio generation method provided in this embodiment can predict the specific sleep state (such as light sleep, deep sleep, or rapid eye movement) of the sleep-aiding subject based on real-time collected sleep signals (such as EEG and breathing signals). It dynamically generates sleep-aiding audio that matches the current sleep state, ensuring the pertinence and adaptability of the audio content.

[0131] The present embodiment provides a method for generating sleep-aiding audio, which collects the brain wave sequence and breathing signal sequence of the sleep-aiding subject; after time-aligning the brain wave sequence and the breathing signal sequence, samples are performed at a preset sampling frequency, and the brain wave and breathing signal at the sampling time point are connected as a state signal each time; based on the time series signal composed of the state signal, the sleep state feature vector and the current sleep state are determined; the reference audio that matches the current sleep state is determined; through a diffusion model, based on the reference audio and the sleep state feature vector, a sleep-aiding audio that assists the sleep-aiding subject to enter the next sleep state is generated. The method of the present embodiment determines the sleep state feature vector and the current sleep state based on the brain wave sequence and breathing signal sequence of the sleep-aiding subject, and after determining the reference audio that matches the current sleep state, adjusts the reference audio according to the sleep state feature vector to generate sleep-aiding audio that assists the sleep-aiding subject to enter the next sleep state. The sleep-aiding audio can be adaptively generated according to the sleep state of the sleep-aiding subject, thereby achieving a better sleep-aiding effect.

[0132] Based on the same inventive concept of the sleep-aiding audio generation method, this embodiment provides a sleep-aiding audio generation device, see Figure 5 , the device comprises:

[0133] The acquisition module 501 is used to acquire the brain wave sequence and breathing signal sequence of the sleep-aiding subject.

[0134] The sampling module 502 is used to time-align the brain wave sequence and the respiratory signal sequence collected by the collection module 501, sample them at a preset sampling frequency, and connect the brain wave and respiratory signal at the sampling time point into a state signal each time.

[0135] The sleep-aiding audio generation module 503 is configured to determine a sleep state feature vector and the current sleep state based on the time series signal formed by the state signal obtained by the sampling module 502. Reference audio matching the current sleep state is determined. Using a diffusion model, based on the reference audio and the sleep state feature vector, sleep-aiding audio is generated to assist the sleep-aiding subject in entering the next sleep state.

[0136] The acquisition module 501 is used to acquire video streams of the superficial temporal artery and external carotid artery of the sleep aided subject using an infrared long-wave sensor and an image acquisition card, and extract the time series signal of the first-order moment of each frame in the video stream to form a respiratory signal sequence.

[0137] The sleep-aiding audio generation module 503 is used to segment the time series signal composed of the state signal according to the preset time length to obtain multiple subsequences. The time features of each subsequence are extracted through two convolutional neural networks. The starting filter size of a convolutional neural network is F s / 2, step size is F s / 16. The starting filter size of another convolutional neural network is F s ×4, step size is F s / 2. F s is the preset sampling frequency. The temporal features of each subsequence are encoded using a bidirectional long short-term memory network to obtain a sleep state feature vector. The sleep state feature vector is converted into a sleep state probability. The current sleep state of the sleep aid recipient is predicted based on the sleep state probability.

[0138] The sleep-aiding audio generation module 503 is configured to randomly select an audio that matches the current sleep state from the audio database as a reference audio. The audio database is composed of a plurality of sleep-aiding audio data, each of which is labeled with a corresponding sleep state.

[0139] Among them, the sleep-aiding audio generation module 503 is used to convert the reference audio into a spectrum graph through Mel-spectrogram conversion. The spectrum graph is encoded and compressed through the variational autoencoder in the diffusion model to obtain the reference spectrum features. The denoising network in the diffusion model performs multiple denoising processes based on the reference spectrum features and the sleep state feature vector to obtain the auxiliary audio features. The auxiliary audio features are transcoded into the original waveform through the efficient and high-fidelity speech synthesis model of the generative adversarial network to obtain the sleep-aiding audio that helps the sleep-aiding subject enter the next sleep state.

[0140] Among them, the sleep-aiding audio generation module 503 is used to connect the reference spectrum features and the sleep state feature vectors through the cross-attention module of the denoising network in the diffusion model for any denoising process, map the connected feature vectors to the Key and Value feature spaces, map the noise spectrum features to the Query feature space, and generate auxiliary audio based on the mapping features of the Query feature space, the mapping vector of the Key feature space, and the mapping vector of the Value feature space.

[0141] The noise spectrum feature is a spectrum graph composed of initialized Gaussian noise.

[0142] Among them, the sleep-aiding audio generation module 503 is used to generate the sleep-aiding audio through the formula Generate auxiliary audio features.

[0143] Where O is the auxiliary audio feature, Q is the Key feature space, K is the mapping vector of the Key feature space, V is the mapping vector of the Value feature space, T is the transpose, and d is the number of channels in the Key feature space. Softmax() is the Softmax process, and Conv() is the mapping process.

[0144] The device provided in this embodiment determines the sleep state feature vector and the current sleep state based on the brain wave sequence and breathing signal sequence of the sleep-aiding subject. After determining the reference audio that matches the current sleep state, the reference audio is adjusted according to the sleep state feature vector to generate sleep-aiding audio that assists the sleep-aiding subject to enter the next sleep state. The sleep-aiding audio can be adaptively generated according to the sleep state of the sleep-aiding subject, thereby achieving a better sleep-aiding effect.

[0145] Based on the same inventive concept of the sleep-aiding audio generation method, this embodiment provides an electronic device, such as Figure 6 As shown, it includes: a memory 601, a processor 602, and a computer program.

[0146] The computer program is stored in the memory 601 and is configured to be executed by the processor 602 to implement the above-mentioned sleep-aid audio generation method.

[0147] Specifically,

[0148] Collect brain wave sequences and breathing signal sequences of the sleep-aided subjects.

[0149] After the brain wave sequence and the respiratory signal sequence are time-aligned, sampling is performed at a preset sampling frequency, and the brain wave and respiratory signal at the sampling time point are connected as a state signal each time sampling is performed.

[0150] The sleep state feature vector and the current sleep state are determined according to the time series signal formed by the state signal.

[0151] Determines reference audio that matches the current sleep state.

[0152] Through the diffusion model, based on the reference audio and the sleep state feature vector, a sleep-aiding audio is generated to help the sleep-aiding subject enter the next sleep state.

[0153] Optionally, collecting a breathing signal sequence of the sleep-aiding subject includes:

[0154] The video streams of the superficial temporal artery and external carotid artery of the sleep-aiding subjects were collected using an infrared long-wave sensor and an image acquisition card.

[0155] The time series signal of the first-order moment of each frame image in the video stream is extracted to form a respiratory signal sequence.

[0156] Optionally, determining the sleep state feature vector and the current sleep state according to the time series signal formed by the state signal includes:

[0157] The time series signal composed of the state signal is divided according to a preset time length to obtain multiple subsequences.

[0158] The temporal features of each subsequence are extracted through two convolutional neural networks. The initial filter size of one convolutional neural network is F s / 2, step size is F s / 16. The starting filter size of another convolutional neural network is F s ×4, step size is F s / 2. F s is the preset sampling frequency.

[0159] The temporal features of each subsequence are encoded through a bidirectional long short-term memory network to obtain the sleep state feature vector.

[0160] Convert the sleep state feature vector into sleep state probability.

[0161] Predict the current sleep state of the sleep aid subject based on the sleep state probability.

[0162] Optionally, determining a reference audio that matches the current sleep state includes:

[0163] Randomly select an audio that matches the current sleep state from the audio database as a reference audio. The audio database consists of multiple sleep-aiding audio data, each of which is labeled with the corresponding sleep state.

[0164] Optionally, generating sleep-aiding audio for assisting the sleep-aided subject to enter the next sleep state based on the reference audio and the sleep state feature vector through a diffusion model includes:

[0165] Convert the reference audio into a spectrum graph through Mel spectrum conversion.

[0166] The spectrum graph is encoded and compressed through the variational autoencoder in the diffusion model to obtain the reference spectrum features.

[0167] Through the denoising network in the diffusion model, multiple denoising processes are performed based on the reference spectrum features and sleep state feature vectors to obtain auxiliary audio features.

[0168] By generating an efficient and high-fidelity speech synthesis model based on a generative adversarial network, the auxiliary audio features are transcoded into the original waveform to obtain sleep-aiding audio that helps the sleep-aided subject enter the next sleep state.

[0169] Optionally, a denoising network in the diffusion model is used to perform multiple denoising processes based on the reference spectrum features and the sleep state feature vector to obtain auxiliary audio features, including:

[0170] For any denoising process, the reference spectrum features and the sleep state feature vector are connected through the cross-attention module of the denoising network in the diffusion model, the connected feature vector is mapped to the Key and Value feature spaces, and the noise spectrum features are mapped to the Query feature space. Auxiliary audio is generated based on the mapping features of the Query feature space, the mapping vector of the Key feature space, and the mapping vector of the Value feature space.

[0171] The noise spectrum feature is a spectrum graph composed of initialized Gaussian noise.

[0172] Optionally, generating auxiliary audio based on the mapping features of the Query feature space, the mapping vectors of the Key feature space, and the mapping vectors of the Value feature space includes:

[0173] By formula Generate auxiliary audio features.

[0174] Where O is the auxiliary audio feature, Q is the Key feature space, K is the mapping vector of the Key feature space, V is the mapping vector of the Value feature space, T is the transpose, and d is the number of channels in the Key feature space. Softmax() is the Softmax process, and Conv() is the mapping process.

[0175] The electronic device provided by this embodiment has a computer program executed by a processor to determine a sleep state feature vector and a current sleep state based on a brain wave sequence and a breathing signal sequence of a sleep-aiding subject. After determining a reference audio that matches the current sleep state, the reference audio is adjusted according to the sleep state feature vector to generate sleep-aiding audio that assists the sleep-aiding subject to enter the next sleep state. The sleep-aiding audio can be adaptively generated according to the sleep state of the sleep-aiding subject, thereby achieving a better sleep-aiding effect.

[0176] Based on the same inventive concept of the sleep-aiding audio generation method, this embodiment provides a computer-readable storage medium having a computer program stored thereon. The computer program is executed by a processor to implement the sleep-aiding audio generation method.

[0177] Specifically,

[0178] Collect brain wave sequences and breathing signal sequences of the sleep-aided subjects.

[0179] After the brain wave sequence and the respiratory signal sequence are time-aligned, sampling is performed at a preset sampling frequency, and the brain wave and respiratory signal at the sampling time point are connected as a state signal each time sampling is performed.

[0180] The sleep state feature vector and the current sleep state are determined according to the time series signal formed by the state signal.

[0181] Determines reference audio that matches the current sleep state.

[0182] Through the diffusion model, based on the reference audio and the sleep state feature vector, a sleep-aiding audio is generated to help the sleep-aiding subject enter the next sleep state.

[0183] Optionally, collecting a breathing signal sequence of the sleep-aiding subject includes:

[0184] The video streams of the superficial temporal artery and external carotid artery of the sleep-aiding subjects were collected using an infrared long-wave sensor and an image acquisition card.

[0185] The time series signal of the first-order moment of each frame image in the video stream is extracted to form a respiratory signal sequence.

[0186] Optionally, determining the sleep state feature vector and the current sleep state according to the time series signal formed by the state signal includes:

[0187] The time series signal composed of the state signal is divided according to a preset time length to obtain multiple subsequences.

[0188] The temporal features of each subsequence are extracted through two convolutional neural networks. The initial filter size of one convolutional neural network is F s / 2, step size is F s / 16. The starting filter size of another convolutional neural network is F s ×4, step size is F s / 2. F s is the preset sampling frequency.

[0189] The temporal features of each subsequence are encoded through a bidirectional long short-term memory network to obtain the sleep state feature vector.

[0190] Convert the sleep state feature vector into sleep state probability.

[0191] Predict the current sleep state of the sleep aid subject based on the sleep state probability.

[0192] Optionally, determining a reference audio that matches the current sleep state includes:

[0193] Randomly select an audio that matches the current sleep state from the audio database as a reference audio. The audio database consists of multiple sleep-aiding audio data, each of which is labeled with the corresponding sleep state.

[0194] Optionally, generating sleep-aiding audio for assisting the sleep-aided subject to enter the next sleep state based on the reference audio and the sleep state feature vector through a diffusion model includes:

[0195] Convert the reference audio into a spectrum graph through Mel spectrum conversion.

[0196] The spectrum graph is encoded and compressed through the variational autoencoder in the diffusion model to obtain the reference spectrum features.

[0197] Through the denoising network in the diffusion model, multiple denoising processes are performed based on the reference spectrum features and sleep state feature vectors to obtain auxiliary audio features.

[0198] By generating an efficient and high-fidelity speech synthesis model based on a generative adversarial network, the auxiliary audio features are transcoded into the original waveform to obtain sleep-aiding audio that helps the sleep-aided subject enter the next sleep state.

[0199] Optionally, a denoising network in the diffusion model is used to perform multiple denoising processes based on the reference spectrum features and the sleep state feature vector to obtain auxiliary audio features, including:

[0200] For any denoising process, the reference spectrum features and the sleep state feature vector are connected through the cross-attention module of the denoising network in the diffusion model, the connected feature vector is mapped to the Key and Value feature spaces, and the noise spectrum features are mapped to the Query feature space. Auxiliary audio is generated based on the mapping features of the Query feature space, the mapping vector of the Key feature space, and the mapping vector of the Value feature space.

[0201] The noise spectrum feature is a spectrum graph composed of initialized Gaussian noise.

[0202] Optionally, generating auxiliary audio based on the mapping features of the Query feature space, the mapping vectors of the Key feature space, and the mapping vectors of the Value feature space includes:

[0203] By formula Generate auxiliary audio features.

[0204] Where O is the auxiliary audio feature, Q is the Key feature space, K is the mapping vector of the Key feature space, V is the mapping vector of the Value feature space, T is the transpose, and d is the number of channels in the Key feature space. Softmax() is the Softmax process, and Conv() is the mapping process.

[0205] The computer-readable storage medium provided in this embodiment has a computer program thereon executed by a processor to determine a sleep state feature vector and a current sleep state based on a brain wave sequence and a breathing signal sequence of a sleep-aiding subject. After determining a reference audio that matches the current sleep state, the reference audio is adjusted based on the sleep state feature vector to generate sleep-aiding audio that assists the sleep-aiding subject in entering the next sleep state. The sleep-aiding audio can be adaptively generated based on the sleep state of the sleep-aiding subject, thereby achieving a better sleep-aiding effect.

[0206] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiment of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal translation scripting language JavaScript, etc.

[0207] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0208] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0209] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 individual streams

[0210] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0211] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for generating sleep-aiding audio, characterized in that: The method comprises: Collect brain wave sequences and breathing signal sequences of the sleep-aiding subject; After the brain wave sequence and the respiratory signal sequence are time-aligned, sampling is performed at a preset sampling frequency, and the brain wave and respiratory signal at the sampling time point are connected as a state signal each time sampling is performed; determining a sleep state feature vector and a current sleep state according to a time series signal formed by the state signal; determining a reference audio that matches the current sleep state; Generate, by using a diffusion model, sleep-aiding audio for assisting the sleep-aiding subject to enter a next sleep state based on the reference audio and the sleep state feature vector; comprising: Converting the reference audio into a spectrum graph through Mel spectrum conversion; The spectrum graph is encoded and compressed by a variational autoencoder in a diffusion model to obtain a reference spectrum feature; Through the denoising network in the diffusion model, multiple denoising processes are performed based on the reference spectrum features and the sleep state feature vector to obtain auxiliary audio features; including: For any denoising process, the reference spectrum feature and the sleep state feature vector are concatenated through the cross-attention module of the denoising network in the diffusion model, the concatenated feature vector is mapped to the Key and Value feature spaces, the noise spectrum feature is mapped to the Query feature space, and auxiliary audio is generated based on the mapping features of the Query feature space, the mapping vector of the Key feature space, and the mapping vector of the Value feature space; The noise spectrum feature is a spectrum graph composed of initialized Gaussian noise; and the auxiliary audio is generated based on the mapping features of the Query feature space, the mapping vector of the Key feature space, and the mapping vector of the Value feature space, including: By formula , generate auxiliary audio features; in, is the auxiliary audio feature, is the Key feature space, is the mapping vector of the Key feature space, is the mapping vector of the Value feature space, is the transpose, is the number of channels in the Key feature space; for deal with, For mapping processing; By generating an efficient and high-fidelity speech synthesis model of a generative adversarial network, the auxiliary audio features are transcoded into the original waveform to obtain sleep-aiding audio that assists the sleep-aiding subject to enter the next sleep state.

2. The method according to claim 1, characterized in that Collect the breathing signal sequence of the sleep aid subject, including: The video stream of the superficial temporal artery and external carotid artery of the sleep aid subject is collected through an infrared long-wave sensor and an image acquisition card; A time series signal of the first-order moment of each frame image in the video stream is extracted to form a respiratory signal sequence.

3. The method according to claim 1, characterized in that The step of determining the sleep state feature vector and the current sleep state based on the time series signal formed by the state signal includes: The time series signal composed of the state signal is divided according to a preset time length to obtain multiple subsequences; The temporal features of each subsequence are extracted through two convolutional neural networks; the starting filter size of one convolutional neural network is , the step size is ; The starting filter size of the other convolutional neural network is , the step size is ; is the preset sampling frequency; The temporal features of each subsequence are encoded through a bidirectional long short-term memory network to obtain the sleep state feature vector; Converting the sleep state feature vector into a sleep state probability; The current sleep state of the sleep-aided subject is predicted according to the sleep state probability.

4. The method according to claim 1, wherein The determining of the reference audio matching the current sleep state includes: An audio that matches the current sleep state is randomly selected from an audio database as a reference audio; wherein the audio database is composed of a plurality of sleep-aiding audio data, and each sleep-aiding audio data is marked with a corresponding sleep state.

5. A sleep-aiding audio generation device, characterized in that: The device comprises: An acquisition module, used to acquire brain wave sequences and respiratory signal sequences of the sleep-aiding subject; A sampling module is used to time-align the brain wave sequence and the respiratory signal sequence collected by the acquisition module, sample them at a preset sampling frequency, and connect the brain wave and respiratory signal at the sampling time point into a state signal each time; A sleep-aiding audio generation module is configured to determine a sleep state feature vector and a current sleep state based on a time series signal formed by the state signal obtained by the sampling module; determine a reference audio that matches the current sleep state; and generate sleep-aiding audio that assists the sleep-aiding subject in entering a next sleep state based on the reference audio and the sleep state feature vector using a diffusion model. The module comprises: Converting the reference audio into a spectrum graph through Mel spectrum conversion; The spectrum graph is encoded and compressed by a variational autoencoder in a diffusion model to obtain a reference spectrum feature; Through the denoising network in the diffusion model, multiple denoising processes are performed based on the reference spectrum features and the sleep state feature vector to obtain auxiliary audio features; including: For any denoising process, the reference spectrum feature and the sleep state feature vector are concatenated through the cross-attention module of the denoising network in the diffusion model, the concatenated feature vector is mapped to the Key and Value feature spaces, the noise spectrum feature is mapped to the Query feature space, and auxiliary audio is generated based on the mapping features of the Query feature space, the mapping vector of the Key feature space, and the mapping vector of the Value feature space; The noise spectrum feature is a spectrum graph composed of initialized Gaussian noise; and the auxiliary audio is generated based on the mapping features of the Query feature space, the mapping vector of the Key feature space, and the mapping vector of the Value feature space, including: By formula , generate auxiliary audio features; in, is the auxiliary audio feature, is the Key feature space, is the mapping vector of the Key feature space, is the mapping vector of the Value feature space, is the transpose, is the number of channels in the Key feature space; for deal with, For mapping processing; By generating an efficient and high-fidelity speech synthesis model of a generative adversarial network, the auxiliary audio features are transcoded into the original waveform to obtain sleep-aiding audio that assists the sleep-aiding subject to enter the next sleep state.

6. An electronic device, characterized in that: include: Memory; processor; as well as computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that A computer program is stored thereon; the computer program is executed by a processor to implement the method according to any one of claims 1 to 4.

Images