Immersive sleep-aiding music synthesis method and immersive sleep-aiding music synthesis system fusing environment sound

Abstract

The invention discloses an immersive sleep-aiding music synthesis method and system fusing environment sound. The method comprises the following steps: acquiring an environment audio signal and a multi-mode environment parameter; based on the environment audio signals and the multi-modal environment parameters, a fusion feature is generated through a multi-modal information fusion module; performing dynamic attention time-frequency analysis on the fusion features to obtain key time-frequency features; performing personalized feature learning based on the key time-frequency features to obtain personalized features; performing self-correction cycle processing on the personalized features to obtain corrected features; extracting beneficial environment elements from the corrected features; synthesizing and mixing music based on the beneficial environment elements; optimizing the mixed music according to user feedback; through multi-modal information fusion and a dynamic attention mechanism, the system can sense and adapt to a complex sound environment in real time and effectively process various interference noises; the method can quickly learn and adapt to the personal preference of the user, and provides customized sleep-aiding experience.

Description

Technical Field

[0001] The present invention relates to the technical field of immersive sleep-aiding music synthesis methods, and more specifically, to an immersive sleep-aiding music synthesis method and a system thereof that integrates ambient sounds. Background Art

[0002] Sleep quality is essential to human health. In recent years, with the accelerated pace of life and increased environmental noise, sleep disorders have become a common health problem. In order to improve sleep quality, various sleep-aiding technologies and devices have emerged.

[0003] The current status of the existing technology mainly includes the following aspects:

[0004] 1. White noise sleep aid: This is one of the earliest and simplest sleep aid devices. It masks the disturbing sounds in the environment by playing white noise of a constant frequency. Although it is simple and easy to use, it lacks personalization and environmental adaptability.

[0005] 2. Natural sound sleep aids: These devices play pre-recorded natural sounds, such as rain, waves, etc. Compared with white noise, these sounds are more pleasant to the ear, but still lack the ability to interact with the actual environment.

[0006] 3. Music sleep aid apps: These apps provide specially designed sleep aid music, and some even adjust the music tempo according to the user's heart rate. However, they usually ignore the impact of environmental sounds.

[0007] 4. Ambient sound fusion system: This is a relatively new type of technology that attempts to integrate ambient sound into sleep-aiding audio. However, existing systems often use simple superposition or mixing methods and lack intelligent and personalized processing.

[0008] The main problems of the existing technology:

[0009] 1. Lack of environmental adaptability: Most existing technologies are unable to perceive and adapt to the user’s sound environment in real time, resulting in poor results in complex noisy environments.

[0010] 2. Insufficient personalization: Existing technologies usually adopt a one-size-fits-all approach, ignoring the fact that different users have significant differences in their preferences and sensitivities to sound.

[0011] 3. Low level of intelligence: Lack of real-time monitoring and feedback adjustment of the user's sleep status, and inability to dynamically adjust the audio output according to the user's sleep stage.

[0012] 4. Decreased effectiveness with long-term use: Many users have reported that the effects of sleep aids will gradually weaken after long-term use of the same sleep aid device or application, which may be due to the body's adaptability to fixed-pattern sounds.

[0013] 5. Insufficient technology integration: Existing technologies often focus on a single aspect (such as music generation or environmental sound processing) and lack the fusion and comprehensive utilization of multimodal information.

[0014] Taking the environmental sound fusion system as an example, the specific problems existing in the existing technology include:

[0015] 1. Simple superposition leads to audio confusion: Many systems simply superimpose ambient sounds and preset sleep-aiding audio, which can easily cause sound confusion and affect sleep.

[0016] 2. Fixed algorithms are difficult to adapt to complex environments: Most systems use fixed sound processing algorithms that cannot effectively cope with changing environmental noise, especially the complex soundscapes in urban environments.

[0017] 3. Lack of user feedback mechanism: The system usually does not take into account the user's real-time physiological feedback and cannot determine whether the current audio really helps the user fall asleep.

[0018] 4. Limited sound conversion capabilities: Existing systems can often only process preset types of environmental sounds and are not effective for unknown or complex sound types.

[0019] 5. Privacy issues: Some systems may involve recording when processing environmental sounds, causing users to worry about privacy.

[0020] In view of the above problems, it is particularly necessary to develop a sleep aid system that can intelligently perceive the environment, adjust strategies in real time, and is highly personalized and privacy-protecting. This is exactly the core problem that the present invention aims to solve. Summary of the invention

[0021] The present invention is aimed at the above problems and proposes an immersive sleep-aiding music synthesis method and system that integrates environmental sounds, including environmental information collection, multimodal information fusion, dynamic attention time-frequency analysis, personalized feature learning, self-correction loop processing, beneficial environmental element extraction, music synthesis and mixing, and optimization based on user feedback. This method comprehensively uses advanced technologies such as multimodal information processing, deep learning, and nonlinear dynamics to innovatively solve many problems faced by existing technologies.

[0022] The present invention provides an immersive sleep-aiding music synthesis method that integrates environmental sounds, comprising the following steps: acquiring environmental audio signals and multimodal environmental parameters; generating fusion features through a multimodal information fusion module based on the environmental audio signals and the multimodal environmental parameters; performing dynamic attention time-frequency analysis on the fusion features to obtain key time-frequency features; performing personalized feature learning based on the key time-frequency features to obtain personalized features; performing self-correction loop processing on the personalized features to obtain corrected features; extracting beneficial environmental elements from the corrected features; synthesizing and mixing music based on the beneficial environmental elements; and optimizing the mixed music according to user feedback.

[0023] Specifically, the multimodal environmental parameters include light intensity, temperature, humidity, air pressure and air quality.

[0024] Specifically, the multimodal information fusion module adopts a quantum-inspired attention mechanism, where:

[0025] X fused =QAM([X A , X L , X T , X H , X P , X Q ])

[0026]

[0027] |ψ i >=(cos(θ i / 2), exp(iφ i )sin(θ i / 2))

[0028] θ i =π*sigmoid(W θ *X i +b θ )

[0029] φ i =2π*tanh(W φ *X i +b φ )

[0030] Among them, X fused represents the fused feature vector QAM represents the quantum inspired attention module; X A , X L , X T , X H , X P , X QThe feature vectors representing audio, light, temperature, humidity, air pressure and air quality respectively; |ψ i > represents the quantum state; θ i and φ i Respectively represent the polar angle and azimuth angle of the quantum state; W θ , W φ is the weight matrix, b θ , b φ are bias vectors, all of which are learnable parameters; i represents the index of different modes; ∑ represents the sum; sigmoid and tanh represent Sigmoid functions respectively.

[0031] Specifically, the dynamic attention time-frequency analysis includes: performing nonlinear time-frequency transformation: S = CWT (X fused ) Calculate dynamic attention:

[0032] A t =softmax(D α (V t *tanh(W t *S+b t )))

[0033] A f =softmax(D ， (V f *tanh(W f *S+b f )))

[0034] Perform attention weighting: S a ttended=S⊙A t ⊙A f ,

[0035] Where CWT stands for continuous wavelet transform; S stands for time-frequency spectrum; A t and A f Represent the attention weights in the time domain and frequency domain respectively; D α and D β represents the fractional differential operator, α and β are learnable fractional parameters in the range of (0, 1); V t , V f , W t , W f is the weight matrix, b t , b f is a bias vector, which are all learnable parameters; softmax represents the Softmax function; ⊙ represents the Hadamard product; S attended Represents the weighted time-frequency spectrum.

[0036] Specifically, the personalized feature learning adopts a fractal meta-learning algorithm, including:

[0037] Meta-learning optimization:

[0038] Personalized fine-tuning:

[0039] Among them, θ represents the model parameters; α and β are learning rates; L represents the loss function based on Tsallis entropy: fθ represents the neural network function with parameter θ, represents the gradient with respect to θ; y and y personal denote the general label and the personalized label respectively; R(θ) denotes the regularization term based on the fractal dimension: R(θ) = λ*(FD(θ)-FD target ) 2 ; FD represents the fractal dimension calculation function, FD target is the target fractal dimension; λ is the regularization strength; q is the Tsallis entropy parameter, which is a learnable parameter.

[0040] Specifically, the self-correction cyclic processing adopts a nonlinear oscillating neural network and a self-correction mechanism based on Lyapunov exponents, including:

[0041]

[0042] error t =||h t -h t-1 ||

[0043]

[0044] h t，corrected =h t +γ·λ t ·(h t-1 -h t )

[0045] Where h represents the hidden state of the neural network, t represents the time step; μ, ω and ε are learnable parameters; error t represents the error between adjacent time steps; λ t is the local Lyapunov exponent; δh0 represents the initial disturbance; γ is the self-correction coefficient; ||·|| represents the Euclidean norm; h t,corrected Represents the hidden state after self-correction.

[0046] Specifically, the extraction of the beneficial environmental elements includes: using an improved isometric mapping algorithm to perform nonlinear dimensionality reduction:

[0047] D = geodesic_distance (ht，corrected )

[0048] Y=MDS(D)

[0049] Use the improved fuzzy C-means clustering algorithm for clustering and identification:

[0050]

[0051] Where D represents the geodesic distance matrix, geodesic_distance represents the geodesic distance calculation function; Y represents the data after dimensionality reduction, MDS represents the multidimensional scaling function; U ik is the membership matrix, indicating the degree to which the i-th data point belongs to the k-th cluster; c k is the center of the kth cluster; y i represents the i-th data point; m is the fuzziness parameter, which is usually greater than 1; ||·|| represents the Euclidean norm; ∑ represents the sum.

[0052] Specifically, the music synthesis and mixing includes: generating basic music using a fractal interpolation algorithm: M(t) = ∑ k a k *sin(2πf k *t D )

[0053] Nonlinearly mix the extracted ambient sound elements with the base music:

[0054]

[0055] Apply spherical harmonics-based stereo sound field reconstruction:

[0056]

[0057] Where M(t) represents the generated basic music; t represents time; a k represents the amplitude coefficient; f k represents the fundamental frequency; D represents the fractal dimension; S mixed (t) represents the mixed audio signal, w i Represents the weight of the environmental sound element E i (t) represents the extracted environmental sound element, f m represents the modulation frequency; P(r, θ, φ, t) represents the sound pressure distribution of the three-dimensional sound field; r, θ, φ represent the radial distance, polar angle and azimuth angle in the spherical coordinate system respectively; Y l，m represents spherical harmonics; H l represents the spherical Hankel function; k represents the wave number; S lm (t) represents the acoustic field coefficient; l and m represent the order and degree of the spherical harmonic function, respectively.

[0058] Specifically, the optimization includes collecting physiological signals of the user, including heart rate variability R(t) and brain wave energy B(t); and using an improved fuzzy cognitive map to evaluate the sleep state:

[0059]

[0060] Optimizing the mixing parameters using the Q-learning algorithm based on fractional differentiation

[0061]

[0062] Among them, R(t) represents the heart rate variability at time t, B(t) represents the brain wave energy at time t, and C i represents the i-th concept node in the fuzzy cognitive graph, w ji represents the weight from node j to node i; f represents the activation function; Q(s, a) represents the Q value of taking action a in state s, s represents the current state, including physiological signals and audio features, a represents the action, that is, the adjustment of the mixed parameters; α represents the learning rate; γ represents the discount factor; r represents the reward, which is calculated based on the fuzzy cognitive map state evaluation: r = ∑ i v i *C i ;v i Represents the importance weight of the i-th concept node.

[0063] An immersive sleep-aiding music synthesis system that integrates ambient sounds, including:

[0064] Environmental information acquisition module, used to obtain environmental audio signals and multimodal environmental parameters;

[0065] A multimodal information fusion module, used to generate fusion features based on the environmental audio signal and the multimodal environmental parameters;

[0066] A dynamic attention time-frequency analysis module is used to analyze the fusion features to obtain key time-frequency features;

[0067] A personalized feature learning module, used for learning based on the key time-frequency features to obtain personalized features;

[0068] A self-correction loop processing module, used for processing the personalized features to obtain corrected features;

[0069] A beneficial environmental element extraction module, used to extract beneficial environmental elements from the corrected features;

[0070] a music synthesis and mixing module, for synthesizing and mixing music based on the beneficial environmental elements; and

[0071] An optimization module is used to optimize the mixed music according to user feedback.

[0072] The beneficial effects of the present invention are mainly reflected in the following aspects:

[0073] 1. Excellent environmental adaptability: Through multimodal information fusion and dynamic attention mechanism, the system can perceive and adapt to complex sound environments in real time and effectively handle various interfering noises.

[0074] 2. Highly personalized: Using fractal meta-learning algorithms and continuous optimization mechanisms, the system can quickly learn and adapt to the user's personal preferences to provide a tailored sleep-aiding experience.

[0075] 3. Intelligent adjustment: Based on the user's physiological feedback and sleep status, the system can adjust the audio output in real time to optimize sleep quality throughout the process.

[0076] 4. Continuous effectiveness: Through self-correcting loop processing and dynamic music synthesis, the system can maintain its effectiveness over long periods of time and avoid user adaptation.

[0077] 5. Immersive experience: Utilize 3D audio technology and intelligent conversion of ambient sound to create a natural and comfortable auditory environment and enhance the user's sense of immersion.

[0078] 6. Privacy protection: The system processes ambient sound in real time without storing original audio data, effectively protecting user privacy.

[0079] 7. Health Improvement: By significantly shortening the time to fall asleep and improving sleep quality, the present invention is expected to make an important contribution to improving the user's overall health.

[0080] In general, the present invention provides an innovative and comprehensive solution that not only overcomes the limitations of the existing technology, but also creates a new paradigm of personalized and intelligent sleep aid technology. It has the potential to revolutionize the field of sleep aid and bring good news to the majority of patients with sleep disorders and people who pursue high-quality sleep. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 It is a logic block diagram of the system module of the present invention.

[0082] Figure 2 This is the core algorithm logic block diagram of the present invention.

[0083] Figure 3 This is a logic block diagram of the multimodal information fusion module of the present invention.

[0084] Figure 4 This is the logic block diagram of the beneficial environmental element extraction module of the present invention.

[0085] Figure 5It is a logic block diagram of the music synthesis and mixing module of the present invention.

[0086] Figure 6 It is the logic block diagram of the optimization module of the present invention. DETAILED DESCRIPTION

[0087] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects thereof are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0088] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0089] The present invention relates to an immersive sleep-aiding music synthesis method and a system thereof that integrates environmental sounds, and in particular to a method and a system that can perceive the environment in real time, perform personalized adjustments, and generate immersive sleep-aiding music.

[0090] In order to better understand the present invention, we will take a user who uses the system in an urban apartment as an example to describe the working process of the present invention in detail. Figure 1-6 As shown, the method of the present invention comprises the following steps:

[0091] 1. Environmental information collection steps

[0092] In this step, the system first collects the ambient audio signal A(t) through a high-sensitivity microphone array (such as the Earthworks M50 measurement-grade microphone). In our scenario, this includes traffic noise outside the window, the conversations of neighbors, and the operation of the indoor air conditioner. The system uses a sampling rate of 96kHz and a bit depth of 24 bits to ensure that high-quality audio signals are captured.

[0093] At the same time, the system also collects a variety of environmental parameters:

[0094] The TSL2591 light sensor measures visible and infrared light intensity L(t), capturing street light and possible moonlight shining through the curtains.

[0095] Use the BME680 temperature sensor to measure the ambient temperature T(t) and record the room temperature after air conditioning, which is usually between 20-25℃.

[0096] The relative humidity H(t) is measured using the BME680 humidity sensor, with typical values ​​between 40%-60%.

[0097] Use the BME680 air pressure sensor to measure the atmospheric pressure P(t), which is usually between 980-1020hPa.

[0098] Use the PMS5003 particle sensor to measure PM2.5 and PM10 concentrations Q(t) and assess indoor air quality.

[0099] 2. Multimodal information fusion step. In this step, the system uses the multimodal information fusion module (MIFM) to process the data collected from various sensors. First, preliminary feature extraction is performed on the information of each modality:

[0100] Audio characteristics: X A =f A (A(t))

[0101] Among them, f A It is a feature extraction function based on fractal dimension and wavelet packet transform:

[0102] f A (A) = WPT (FD (A)) FD represents fractal dimension calculation, WPT represents wavelet packet transform

[0103] In our scenario, this step can effectively extract the features of traffic noise, neighbor conversations, and air conditioning sounds while preserving their time-frequency characteristics. Fractal dimension calculation can capture the complexity and self-similarity of these sounds, while wavelet packet transform can analyze the time-frequency characteristics of sounds at different scales. For example, traffic noise may present a high fractal dimension (such as 1.6-1.8), indicating its complexity, while air conditioning sounds may have a low fractal dimension (such as 1.2-1.4), indicating its relatively stable characteristics.

[0104] Temperature characteristics: X T =f T (T(t))

[0105]

[0106] This function uses hyperbolic sine and logarithmic functions to emphasize temperature deviations. T0 is a reference temperature (such as 20°C), T mean is the average temperature over a period of time. In our scenario, this can highlight temperature fluctuations caused by air conditioning. For example, if the air conditioning is set to 24℃, but the actual temperature fluctuates between 23.5℃ and 24.5℃, this function will amplify these small changes.

[0107] Humidity characteristics: X H =f H (H(t))

[0108]

[0109] This function uses the hyperbolic tangent function to normalize the humidity value, and then uses the square root function to emphasize changes in humidity. mean is the average humidity over time. In our urban apartment scenario, if the humidity fluctuates between 45% and 55%, this function will highlight this variation.

[0110] Air pressure characteristics: X P =f P (P(t))

[0111]

[0112] This function uses the error function (erf) to normalize the pressure value. standard is standard atmospheric pressure (1013.25hPa), P std is the standard deviation of air pressure (usually about 10hPa). This function can effectively capture the fluctuations in air pressure caused by weather changes.

[0113] Air quality characteristics: X Q =f Q (Q(t))

[0114]

[0115] This function uses logarithmic and exponential functions to highlight changes in air quality. max is the predefined maximum particle concentration value (for example, for PM2.5, it can be set to 500μg / m 3 ). In our urban apartment scenario, if the PM2.5 concentration is between 10-50 μg / m 3 This function will clearly reflect the change.

[0116] Next, the system fuses this multimodal information using an improved quantum-inspired attention mechanism:

[0117] X fused =QAM([X A , X L , X T , X H , X P , X Q ])

[0118] Among them, QAM is the quantum-inspired attention module, which is defined as follows:

[0119]

[0120] |ψ i >=(cos(θ i / 2), exp(iφ i )sin(θ i / 2))

[0121] θ i =π*sigmoid(W θ *X i +b θ )

[0122] φ i =2π*tanh(W φ *X i +b φ )

[0123] Among them, X fused represents the fused feature vector QAM represents the quantum inspired attention module; X A , X L , X T , X H , X P , X Q The feature vectors representing audio, light, temperature, humidity, air pressure and air quality respectively; |ψ i > represents the quantum state; θ i and φ i Respectively represent the polar angle and azimuth angle of the quantum state; W θ , W φ is the weight matrix, b θ , b φ are bias vectors, all of which are learnable parameters; i represents the index of different modes; ∑ represents the sum; sigmoid and tanh represent Sigmoid functions respectively.

[0124] This quantum-inspired attention mechanism allows the system to dynamically adjust the importance of each modality when fusing information from different modalities. For example, if significant traffic noise is detected, the system may increase the weight of audio features; if light changes are detected, the system may increase the weight of light features. In our scenario, assuming it is late at night, the system may pay more attention to audio and light features, and relatively less attention to temperature and humidity features.

[0125] 3. Dynamic attention time-frequency analysis step. In this step, the system performs dynamic attention time-frequency analysis on the fused features to extract key time-frequency features.

[0126] First, the system performs a nonlinear time-frequency transform:

[0127] S=CWT(X fused )

[0128] The improved continuous wavelet transform (CWT) is used here:

[0129]

[0130] Among them, ψ a , b(t)=|a| -1 / 2 *ψ((tb) / a)

[0131] We use the improved Morlet wavelet as the mother wavelet:

[0132]

[0133] f b and f c is a learnable parameter. This improved Morlet wavelet allows the system to better adapt to different types of environmental sounds, such as continuous air conditioning sound and intermittent traffic noise.

[0134] Next, the system uses the fractional differential attention mechanism to calculate the time domain and frequency domain attention:

[0135] A t =softmax(D α (V t *tanh(W t *S+b t )))

[0136] A f =softmax(D β (V f *tanh(W f *S+b f )))

[0137] Perform attention weighting: S a ttended=S⊙A t ⊙A f ,

[0138] Where CWT stands for continuous wavelet transform; S stands for time-frequency spectrum; A t and A f Represent the attention weights in the time domain and frequency domain respectively; D α and D β represents the fractional differential operator, α and β are learnable fractional parameters in the range of (0, 1); V t , V f , W t , W f is the weight matrix, b t , b f is a bias vector, which are all learnable parameters; softmax represents the Softmax function; ⊙ represents the Hadamard product; S attendedRepresents the weighted time-frequency spectrum.

[0139] Among them, D α and D β represents the fractional differential operator and is defined as follows:

[0140]

[0141] α and β are learnable fractional-order parameters in the range of (0, 1). This fractional-order differential attention mechanism allows the system to capture the characteristics of signals at different time scales and frequency scales, which is particularly suitable for processing long-term dependencies and sudden events in environmental sounds.

[0142] Finally, the system performs attention weighting:

[0143] S attended =S⊙A t ⊙A f

[0144] Wherein, ⊙ represents the Hadamard product.

[0145] The output of this step is S attended Represents key time-frequency features of ambient sound and other environmental parameters. For example, in our urban apartment scene, it might highlight the frequency range (e.g. 50-500Hz) and temporal patterns (e.g. traffic rush every 5-10 minutes), and the persistent nature of the air conditioning sound (e.g. a steady 60Hz hum).

[0146] 4. Personalized feature learning step. In this step, the system uses the improved fractal meta-learning algorithm to perform personalized feature learning.

[0147] The first is meta-learning optimization:

[0148]

[0149] Among them, L is the loss function based on Tsallis entropy

[0150]

[0151] q is the Tsallis entropy parameter, which is a learnable parameter, usually between 1.1 and 2.0. This Tsallis entropy-based loss function allows the system to have better performance when dealing with non-stationary and long-tailed environmental sounds.

[0152] Then personalize it:

[0153]

[0154] Among them, R(θ) is a regularization term based on the fractal dimension:

[0155] R(θ)=λ*(FD(θ)-FD target ) 2

[0156] FD represents fractal dimension, FD target is the target fractal dimension (usually between 1.2 and 1.8), and L is the regularization strength (usually between 0.01 and 0.1). This step allows the system to quickly adapt to the user's personal preferences. For example, in our scenario, if the user is particularly sensitive to high-frequency traffic noise, the system can learn to pay more attention to and process this type of sound. The system may map the characteristics of this type of sound to lower-frequency, rhythmic sounds, such as simulating the sound of distant waves.

[0157] 5. Self-correction loop processing step, in this step, the system uses a nonlinear oscillating neural network and a self-correction mechanism based on the Lyapunov exponent to process the timing information.

[0158] First, the timing information is processed using a modified VanderPol oscillator neural network:

[0159]

[0160] Among them, μ, ω, and ∈ are learnable parameters. μ is usually between 0.1 and 2, controlling the nonlinear strength; ω is usually between 0.5 and 5, controlling the oscillation frequency; ∈ is usually between 0.01 and 0.5, controlling the influence of external input. This nonlinear oscillating neural network can capture the periodicity and nonlinear dynamic characteristics of environmental sounds.

[0161] The system then applies self-correction based on the Lyapunov exponent:

[0162] error t =||h t -h t-1 ||

[0163]

[0164] h t，corrected =h t +γ·λ t ·(h t-1 -h t )

[0165] Where h represents the hidden state of the neural network, t represents the time step; μ, ω and ε are learnable parameters; error t represents the error between adjacent time steps; λ tis the local Lyapunov exponent; δh0 represents the initial perturbation; ||·|| represents the Euclidean norm; h t，corrected represents the hidden state after self-correction, and γ is the self-correction coefficient (usually between 0.01 and 0.1). This self-correction mechanism can handle sudden changes and long-term trends in environmental sounds, ensuring that the system maintains stable performance when running for a long time (such as all night).

[0166] In our urban apartment scenario, this step helps the system adapt to changes such as the sudden appearance of car horns late at night, or the gradual increase in bird chirping in the early morning. The system dynamically adjusts its internal state based on these changes to keep the audio output coherent and adaptive.

[0167] 6. Beneficial environmental elements extraction step: in this step, the system extracts beneficial environmental elements from the corrected features.

[0168]

[0169] Y=MDS(D)

[0170] Among them, geodesic_distance calculates the geodesic distance, and MDS is multidimensional scaling. This step can reveal the global structure of the data while retaining the local structure, and is particularly suitable for processing nonlinear environmental sound data.

[0171] Then, the improved fuzzy C-means clustering algorithm is used for clustering and identification:

[0172]

[0173] Among them, U ik is the membership matrix, c k is the cluster center, y i represents the i-th data point, and m is the fuzziness parameter (usually between 1.5 and 3). This fuzzy clustering method allows each sound element to partially belong to multiple categories, which is particularly useful for processing complex environmental sounds.

[0174] In our urban apartment scenario, this step might classify the environmental sound elements into several main categories, such as continuous background noise (including air conditioning), intermittent external noise (including traffic noise), and natural environmental sounds (if any, which might include wind or distant birdsong). The system will determine which environmental elements are beneficial based on the characteristics of these categories and the user's preferences.

[0175] For example, the system might identify the low-frequency hum of an air conditioner as a steady, beneficial element that helps mask other distracting noises, while the system might selectively extract the low-frequency components of intermittent traffic noise as the basis for simulating distant thunder or the sound of ocean waves.

[0176] 7. Music synthesis and mixing step, in this step, the system synthesizes and mixes music based on the extracted beneficial environmental elements.

[0177] First, generate basic music using a fractal interpolation algorithm:

[0178]

[0179] Among them, f k is the fundamental frequency (usually between 20Hz and 500Hz), a k is the amplitude (usually between 0 and 1), and D is the fractal dimension (usually between 1.1 and 1.9). This fractal-based music generation method can create music with a natural and harmonious feel, which is particularly suitable as the basis for sleep-inducing music.

[0180] The extracted ambient sound elements are then mixed non-linearly with the base music:

[0181]

[0182] Among them, E i (t) is the extracted environmental sound element, w i is the weight (usually between 0 and 1), f m is the modulation frequency (usually between 0.1Hz and 1Hz). This nonlinear mixing method allows ambient sound elements to be naturally integrated into the music rather than simply added.

[0183] Finally, apply the spherical harmonics based stereo sound field reconstruction:

[0184]

[0185] Among them, Y l,m is a spherical harmonic function, H l is the spherical Hankel function, S lm is the sound field coefficient, l and m represent the order and degree of the spherical harmonic function respectively (usually l ranges from 0 to 4, and m ranges from -I to 1). This 3D audio technology can create a more immersive auditory experience, making users feel surrounded by sound and helping them fall asleep faster.

[0186] In our urban apartment scenario, the resulting synthesized music might include some soft background music, mixed with slight air conditioning white noise (as a steady background sound), and processed, slow-paced traffic sounds (perhaps transformed into a rhythmic sound similar to ocean waves). These sounds are presented in the form of 3D audio to give the user the feeling of being in a peaceful environment. For example, the hum of the air conditioner might be processed to sound like a distant waterfall, coming from the left rear of the user; while the processed traffic noise might become a sound similar to ocean waves, coming from the right front, creating the feeling of being at the seaside.

[0187] 8. Optimization step, in this step, the system optimizes the mixed music based on user feedback.

[0188] First, the system collects the user's physiological signals, including:

[0189] Heart rate variability R(t): collected using a PPG sensor, usually once every minute.

[0190] EEG energy B(t): A dry electrode EEG cap is used to collect energy in the δ, θ, α, and β bands, usually once per second.

[0191] Then, the sleep state was assessed using a modified fuzzy cognitive map (FCM):

[0192]

[0193] Among them, C i is a concept node (such as relaxation level, sleep depth, etc.), ji is the weight (usually between -1 and 1), and f is the activation function (usually the sigmoid function). This fuzzy cognitive map can take into account multiple factors and give an overall sleep quality assessment.

[0194] Finally, the Q-learning algorithm based on fractional differentiation is used to optimize the mixing parameters:

[0195]

[0196] Where state s includes physiological signals and audio features, and action a is the adjustment of the mixing parameters. α is the learning rate (usually between 0.01 and 0.1), and γ is the discount factor (usually between 0.9 and 0.99). r is the reward function based on FCM state evaluation: r = ∑ i v i ·C i .

[0197] v iis the importance weight of each concept node (summing to 1). This optimization process is ongoing. For example, in our urban apartment scenario, if the system detects an increase in the user's heart rate variability (indicating relaxation) and an increase in alpha waves (indicating a calm wakeful state), it may slightly increase the proportion of air conditioning white noise; if it detects an increase in delta waves (indicating deep sleep), it may gradually reduce the overall volume.

[0198] In this way, the system can continuously adjust and optimize the music to suit the user's sleep state and personal preferences, providing the best sleep-aiding effect. For example, if the user responds well to certain sound elements (such as simulated waves) in the early stages of falling asleep, the system will increase the proportion of such elements in subsequent nights. If the user often wakes up slightly during a certain period of time (such as around 3 a.m.), the system may pay special attention to maintaining the stability of the sound before and after this period of time to avoid sudden changes.

[0199] Please refer to Figure 1 In order to implement the above method, the present invention also provides an immersive sleep-aiding music synthesis system integrating environmental sounds, the system comprising:

[0200] Environmental information acquisition module 1 is used to obtain environmental audio signals and multimodal environmental parameters. This module includes a high-sensitivity microphone array (such as EarthworksM50) and various environmental sensors (such as TSL2591 light sensor, BME680 temperature, humidity and pressure sensor, and PMS5003 particle sensor).

[0201] Multimodal information fusion module 2 is used to generate fusion features based on environmental audio signals and multimodal environmental parameters. This module implements a quantum-inspired attention mechanism that can dynamically adjust the importance of different modal information. The quantum-inspired attention mechanism implemented using PyTorch runs on the NPU to improve efficiency.

[0202] Dynamic attention time-frequency analysis module 3 is used to analyze the fusion features and obtain key time-frequency features. This module uses the improved continuous wavelet transform and fractional differential attention mechanism to capture signal features at different time and frequency scales. Based on the improved continuous wavelet transform library and the customized fractional differential operator, it is implemented in C++ to improve performance.

[0203] The personalized feature learning module 4 is used to learn based on key time-frequency features to obtain personalized features. This module implements a fractal meta-learning algorithm based on Tsallis entropy, which can quickly adapt to the user's personal preferences. The fractal meta-learning algorithm implemented using TensorFlow supports online learning and model updates.

[0204] The self-correction loop processing module 5 is used to process the personalized features to obtain the corrected features. This module uses a nonlinear oscillating neural network and a self-correction mechanism based on the Lyapunov exponent to ensure that the system maintains stable performance during long-term operation. It is implemented based on a custom nonlinear dynamics library, including VanderPol oscillators and Lyapunov exponent calculations.

[0205] The beneficial environmental element extraction module 6 is used to extract the beneficial environmental elements from the corrected features. This module uses an improved isometric mapping algorithm and a fuzzy C-means clustering algorithm to effectively identify and extract beneficial environmental sound elements.

[0206] Music Synthesis and Mixing Module 7: Used to synthesize and mix music based on beneficial environmental elements. This module uses a fractal interpolation algorithm to generate basic music and creates an immersive auditory experience through nonlinear mixing and 3D audio technology. It uses the SuperCollider audio synthesis engine combined with a custom fractal music generation algorithm.

[0207] Optimization module 8 is used to optimize the mixed music based on user feedback. This module collects the user's physiological signals (such as heart rate variability and brain waves), uses fuzzy cognitive maps to evaluate sleep status, and continuously optimizes music parameters through a Q-learning algorithm based on fractional differentials. A reinforcement learning environment implemented based on the OpenAI Gym framework uses a custom fractional Q-learning algorithm.

[0208] The various modules of this system work closely together to form a closed-loop workflow. From the collection of environmental information to the final music optimization, each module provides the necessary input for the next module, while being able to adjust based on the feedback of subsequent modules. This design ensures that the system can continue to learn and adapt to provide users with the best sleep-aiding experience.

[0209] The hardware part of the present invention includes a central processing unit, which uses a high-performance ARM Cortex-A76 processor with a main frequency of 3.0GHz for fast calculation of complex algorithms; a neural network processor, which uses a dedicated NPU (Neural Processing Unit) with an AI computing power of 8TOPS (trillion operations per second) to accelerate the operation of deep learning models; an audio acquisition system, which uses an EarthworksM50 measurement-grade microphone array with a sampling rate of 96kHz and a bit depth of 24 bits to ensure high-quality environmental sound acquisition. Physiological signal acquisition equipment, including a heart rate PPG (photoplethysmography) sensor and a dry electrode EEG cap, collects δ, θ, α, and β band signals; and an audio output, which uses a high-fidelity stereo speaker system and supports 3D audio technology.

[0210] The software part includes the operating system, an embedded operating system customized based on the real-time Linux kernel, which optimizes the real-time performance of audio processing and sensor data acquisition.

[0211] In order to verify the superiority of the present invention, we conducted a series of comparative experiments. The following is a detailed comparison of Example 1 of the present invention and three comparative examples:

[0212] Example 1: Complete implementation of the present invention

[0213] Comparative Example 1: Traditional white noise sleep aid

[0214] Comparative Example 2: Fixed-mode natural sound sleep aid

[0215] Comparative Example 3: Simple ambient sound fusion sleep aid (without personalization and adaptive functions)

[0216] Test indicators, standards and methods:

[0217] 1. Time to fall asleep (minutes): The time from lying down to entering the light sleep stage was monitored using electroencephalogram (EEG).

[0218] 2. Sleep quality score (0-100 points): Based on the EEG data of the whole night, combined with sleep stage and cycle integrity assessment.

[0219] 3. Subjective comfort (1-10 points): The user’s subjective rating after waking up.

[0220] 4. Environmental adaptability (1-10 points): The system's ability to adapt to different noise environments is scored by professional evaluators based on the audio output of the system.

[0221] 5. Personalization level (1-10 points): After one week of testing, the system’s learning of user preferences is scored by professional evaluators.

[0222] The test results are shown in the following table:

[0223] Test indicators Time to fall asleep (minutes) Sleep quality score Subjective comfort Environmental adaptability Degree of personalization Example 1 12.3 87.5 8.7 9.2 9.5 Comparative Example 1 22.7 72.1 6.2 5.1 1 Comparative Example 2 18.5 76.8 7.1 6.3 3.2 Comparative Example 3 16.8 79.3 7.5 7.8 4.7

[0224] Example 1 demonstrates the superiority of the present invention. Compared with other comparative examples, the present invention exhibits obvious advantages in all test indicators:

[0225] 1. Time to fall asleep: The present invention helps users fall asleep in an average of 12.3 minutes, which is 4.5 minutes faster than the closest comparative example 3. This shows that the personalized and adaptive features of the present invention can guide users to fall asleep faster.

[0226] 2. Sleep quality score: The high score of 87.5 points shows that the present invention can not only help users fall asleep quickly, but also maintain high-quality sleep throughout the night. This may be due to the system's ability to adjust the audio in real time to suit the user's sleep stage.

[0227] 3. Subjective comfort: The high score of 8.7 reflects the high recognition of users for the audio generated by the present invention, which shows that the present invention successfully transforms the ambient sound into a pleasant auditory experience.

[0228] 4. Environmental adaptability: The score of 9.2 points shows the excellent performance of the invention in different noise environments. This proves the effectiveness of multimodal information fusion and dynamic attention mechanism, which enables the system to intelligently process various environmental sounds.

[0229] 5. Personalization: The high score of 9.5 points highlights the outstanding advantages of the present invention in personalization, which shows that the fractal meta-learning algorithm and continuous optimization mechanism can effectively capture and adapt to the user's personal preferences.

[0230] The in-depth analysis is as follows:

[0231] 1. Advantages of multimodal fusion: The high score (9.2) of Example 1 in environmental adaptability far exceeds that of other comparative examples, proving that the multimodal information fusion module of the present invention can fully perceive and adapt to complex sleeping environments.

[0232] 2. Dynamic adaptability: The significant improvement in sleep onset time and sleep quality indicates that the dynamic attention time-frequency analysis module of the present invention is able to adjust the audio output in real time to maximize sleep promotion.

[0233] 3. Effect of personalized learning: The high score (9.5) of the degree of personalization reflects the excellent performance of the personalized feature learning module and optimization module of the present invention, which can quickly learn and adapt to the unique needs of users.

[0234] 4. Long-term stability: The advantage of the sleep quality score (87.5) indicates that the self-correcting cycle processing module of the present invention ensures that the system remains stable and efficient throughout the night.

[0235] 5. Improvement of user experience: The high score of subjective comfort (8.7) shows that the present invention not only performs well in objective indicators, but also brings good subjective experience to users, which is crucial to improving user compliance and long-term use effect.

[0236] The test results clearly show that the present invention is significantly superior to traditional methods and simple environmental sound fusion methods in all aspects. It not only helps users fall asleep faster, but also provides a higher quality sleep experience. The core innovations of the present invention, including multimodal information fusion, dynamic attention mechanism, personalized learning and adaptive optimization, work together to create an intelligent, efficient and personalized sleep aid system. These results verify the significant advantages and potential of the present invention in improving sleep quality and providing personalized sleep solutions.

[0237] Through the above detailed description, those skilled in the art should be able to understand and implement the method and system of the present invention. The present invention provides users with a highly personalized and adaptive sleep aid solution through innovative algorithm design and multimodal information processing, which is expected to significantly improve the sleep quality of users.

[0238] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An immersive sleep-aiding music synthesis method integrating ambient sound, characterized in that: The following steps are involved: Acquire an ambient audio signal and a multimodal environmental parameter; based on the ambient audio signal and the multimodal environmental parameter, generate a fusion feature through a multimodal information fusion module; Performing dynamic attention time-frequency analysis on the fusion features to obtain key time-frequency features; performing personalized feature learning based on the key time-frequency features to obtain personalized features; Performing a self-correction cycle on the personalized features to obtain corrected features; Extracting beneficial environmental elements from the modified features; synthesizing and mixing music based on the beneficial environmental elements; and optimizing the mixed music according to user feedback.

2. The method according to claim 1, characterized in that The multimodal environmental parameters include light intensity, temperature, humidity, air pressure and air quality.

3. The method according to claim 1, characterized in that The multimodal information fusion module adopts a quantum-inspired attention mechanism, where: X fused =QAM([X A ,X L ,X T ,X H ,X P ,X Q ]) |ψ i >=(cos(θ) i / 2),exp(iφ i )sin(θ i / 2)) i i =π*sigmoid(W θ *X i +b θ ) φ i =2π*tanh(W φ *X i +b φ ) Among them, X fused represents the fused feature vector QAM represents the quantum inspired attention module; X A ,X L ,X T ,X H ,X P ,X Q The feature vectors representing audio, light, temperature, humidity, air pressure and air quality respectively; |ψ i > represents the quantum state; θ i and φ i Respectively represent the polar angle and azimuth angle of the quantum state; W θ ,W φ is the weight matrix, b θ ,b φ are bias vectors, all of which are learnable parameters; i represents the index of different modes; ∑ represents the sum; sigmoid and tanh represent Sigmoid functions respectively.

4. The method according to claim 1, characterized in that: The dynamic attention time-frequency analysis includes: performing nonlinear time-frequency transformation: S = CWT (X fused ) Calculate dynamic attention: A t =softmax(D α (V t *tanh(W t *S+b t ))) A f =softmax(D β (V f *tanh(W f *S+b f ))) Perform attention weighting: S a ttended=S⊙A t ⊙A f , Where CWT stands for continuous wavelet transform; S stands for time-frequency spectrum; A t and A f Represent the attention weights in the time domain and frequency domain respectively; D α and D β represents the fractional differential operator, α and β are learnable fractional parameters in the range of (0,1); V t ,V f ,W t ,W f is the weight matrix, b t ,b f is a bias vector, which are all learnable parameters; softmax represents the Softmax function; ⊙ represents the Hadamard product; S attended Represents the weighted time-frequency spectrum.

5. The method according to claim 1, characterized in that The personalized feature learning adopts a fractal meta-learning algorithm, including: Meta-learning optimization: Personalized fine-tuning: Among them, θ represents the model parameters; α and β are learning rates; L represents the loss function based on Tsallis entropy: f θ represents a neural network function with parameter θ, represents the gradient with respect to θ; y and y personal denote the general label and the personalized label respectively; R(θ) denotes the regularization term based on the fractal dimension: R(θ) = λ*(FD(θ)-FD target ) 2 ; FD represents the fractal dimension calculation function, FD target is the target fractal dimension; λ is the regularization strength; q is the Tsallis entropy parameter, which is a learnable parameter.

6. The method according to claim 1, characterized in that The self-correction loop process uses a nonlinear oscillating neural network and a self-correction mechanism based on Lyapunov exponents, including: error t =||h t -h t-1 || h t,corrected =h t +γ·λ t ·(h t-1 -h t ) Where h represents the hidden state of the neural network, t represents the time step; μ, ω and ε are learnable parameters; error t represents the error between adjacent time steps; λ t is the local Lyapunov index; δh0 represents the initial disturbance; γ is the self-correction coefficient; ∥·∥ represents the Euclidean norm; h t,corrected Represents the hidden state after self-correction.

7. The method according to claim 1, characterized in that The extraction of the beneficial environmental elements includes: using an improved isometric mapping algorithm to perform nonlinear dimensionality reduction: D=geodesic_distance(h t,corrected ) Y=MDS(D) Use the improved fuzzy C-means clustering algorithm for clustering and identification: Where D represents the geodesic distance matrix, geodesic_distance represents the geodesic distance calculation function; Y represents the data after dimensionality reduction, MDS represents the multidimensional scaling function; U ik is the membership matrix, indicating the degree to which the i-th data point belongs to the k-th cluster; c k is the center of the kth cluster; y i represents the i-th data point; m is the fuzziness parameter, which is usually greater than 1; ∥·∥ represents the Euclidean norm; ∑ represents the sum.

8. The method according to claim 1, characterized in that The music synthesis and mixing includes: generating basic music using a fractal interpolation algorithm: M(t)=∑ k a k *sin(2πf k *t D ) Nonlinearly mix the extracted ambient sound elements with the base music: Apply spherical harmonics-based stereo sound field reconstruction: Where M(t) represents the generated basic music; t represents time; a k represents the amplitude coefficient; f k represents the fundamental frequency; D represents the fractal dimension; S mixed (t) represents the mixed audio signal, w i Represents the weight of the environmental sound element E i (t) represents the extracted environmental sound element, f m represents the modulation frequency; P(r,θ,φ,t) represents the sound pressure distribution of the three-dimensional sound field; r,θ,φ represent the radial distance, polar angle and azimuth angle in the spherical coordinate system respectively; Y l,m represents spherical harmonics; H l represents the spherical Hankel function; k represents the wave number; S lm (t) represents the acoustic field coefficient; l and m represent the order and degree of the spherical harmonic function, respectively.

9. The method according to claim 1, characterized in that: The optimization includes collecting physiological signals of the user, including heart rate variability R(t) and brain wave energy B(t); and using an improved fuzzy cognitive map to evaluate the sleep state: Optimizing the mixing parameters using the Q-learning algorithm based on fractional differentiation Among them, R(t) represents the heart rate variability at time t, B(t) represents the brain wave energy at time t, and C i represents the i-th concept node in the fuzzy cognitive graph, w ji represents the weight from node j to node i; f represents the activation function; Q(s,a) represents the Q value of taking action a in state s, s represents the current state, including physiological signals and audio features, a represents the action, that is, the adjustment of the mixed parameters; α represents the learning rate; γ represents the discount factor; r represents the reward, which is calculated based on the fuzzy cognitive map state evaluation: r = ∑ i v i *C i ;v i Represents the importance weight of the i-th concept node.

10. An immersive sleep-aiding music synthesis system integrating ambient sound, characterized in that: include: Environmental information acquisition module, used to obtain environmental audio signals and multimodal environmental parameters; A multimodal information fusion module, used to generate fusion features based on the environmental audio signal and the multimodal environmental parameters; A dynamic attention time-frequency analysis module is used to analyze the fusion features to obtain key time-frequency features; A personalized feature learning module, used for learning based on the key time-frequency features to obtain personalized features; A self-correction loop processing module, used for processing the personalized features to obtain corrected features; A beneficial environmental element extraction module, used to extract beneficial environmental elements from the corrected features; A music synthesis and mixing module, used for synthesizing and mixing music based on the beneficial environmental elements; as well as An optimization module is used to optimize the mixed music according to user feedback.

Images