Smart home RGBW dimming environment and background music cooperative control method and system

Abstract

The invention relates to the technical field of smart home, and discloses a smart home RGBW dimming environment and background music cooperative control method and system, and the method comprises the steps: obtaining a next music segment of a music player in a smart home, respectively extracting the pitch features and energy features of the music segment, and carrying out the feature decision processing of the pitch features and energy features; constructing a tonality-color mapping library between a preset music tonality sample and a preset light color parameter, and performing crossover mutation operation on tonality-color units in the tonality-color mapping library; based on current color parameters of a light controller in the smart home, analyzing user preference, tonality-color matching degree and coherence between the crossover and variation unit and music tonality, and generating comprehensive fitness between the crossover and variation unit and the music tonality; and performing color evolution on the current color parameter according to the crossover variation unit and the comprehensive fitness. According to the invention, the consideration of personalized dimming-music cooperative control can be improved.

Description

Technical Field

[0001] The present invention relates to a method and system for collaborative control of a smart home RGBW dimming environment and background music, and belongs to the technical field of smart homes. Background Art

[0002] The RGBW dimming technology refers to achieving various color lighting effects through the mixing of three primary colors of light, namely red, green, and blue (RGB), and then adjusting the brightness of the white light (W) to provide basic lighting or form a soft lighting effect by mixing with colored light. The main dimming parameters include color, brightness, color temperature, etc. Secondly, the process of collaborative control of a smart home RGBW dimming environment and background music refers to adjusting the color and brightness of the lights in real time according to the rhythm, frequency, or emotional characteristics of the music, etc.

[0003] Currently, existing patents (such as the dimming technology patents of Hangzhou Tuya) mostly rely on preset rules or static learning models. For example, they automatically adjust the light brightness and color temperature according to user habits, but lack the ability of real-time dynamic adjustment. Similarly, although the smart home control solution of Skyworth - RGB supports scene switching, it requires manual setting of the mode by the user and cannot be automatically optimized according to the music content or environmental changes. Existing music - light linkage technologies (such as Philips Hue Sync) are mostly based on fixed frequency or volume control. For example, mapping low frequencies to red and high frequencies to blue, lacking in-depth analysis of the tonality and emotion of music (such as the brightness of C major and the melancholy of D minor). Traditional methods (such as lighting control based on the DMX protocol) are prone to failure in multi-tonality music or ambiguous tonality scenarios. For example, the lighting effects become chaotic due to frequent key changes in jazz improvisation segments. Therefore, the existing patented technologies lack sufficient consideration for personalized dimming - music collaborative control. Summary of the Invention

[0004] The present invention provides a method and system for collaborative control of a smart home RGBW dimming environment and background music, and its main purpose is to enhance the consideration for personalized dimming - music collaborative control.

[0005] To achieve the above object, a method for collaborative control of a smart home RGBW dimming environment and background music provided by the present invention includes:

[0006] Obtaining the next music segment of a music player in a smart home, respectively extracting the pitch feature and energy feature of the music segment, performing feature decision processing on the pitch feature and the energy feature to obtain a feature decision score, so as to identify the music tonality of the music segment through the feature decision score;

[0007] Construct a tonality-color mapping library between preset music tonality samples and preset light color parameters, and perform a crossover mutation operation on the tonality-color units in the tonality-color mapping library to obtain crossover mutation units, where the tonality-color mapping library is composed of preset tonality-color units;

[0008] Based on the current color parameters of the light controller in the smart home, analyze the user preferences, tonality-color matching degree, and coherence between the crossover mutation units and the music tonality, and generate a comprehensive fitness between the crossover mutation units and the music tonality by using the user preferences, the tonality-color matching degree, and the coherence;

[0009] According to the crossover mutation units and the comprehensive fitness, perform color evolution on the current color parameters to obtain evolved color parameters, and detect whether the current music segment of the music player has played to the next music segment;

[0010] When it is detected that the current music segment of the music player has played to the next music segment, based on the evolved color parameters, use the light controller to complete the collaborative control with the next music segment to obtain a collaborative control result.

[0011] Optionally, obtaining the next music segment of the music player in the smart home includes:

[0012] Query the start time of playback of the music player;

[0013] Analyze the average duration from sampling the music to lighting control;

[0014] Extract the current music segment corresponding to the start time of playback lasting the average duration;

[0015] Obtain the entire piece of music corresponding to the current music segment;

[0016] Use the remaining music segments in the entire piece of music except the current music segment as the next music segment of the music player.

[0017] Optionally, separately extracting the pitch feature and energy feature of the music segment includes:

[0018] Use a preset CREPE model to extract the frame pitch sequence of the music segment;

[0019] Perform sequence alignment on the frame pitch sequence through a preset dynamic time warping algorithm to obtain a frame smoothed pitch;

[0020] Analyze the pitch mean, pitch variance, pitch range, and semitone distribution histogram of the frame smoothed pitch;

[0021] Take the pitch mean value, the pitch variance, the pitch range, and the semitone distribution histogram as the pitch features of the music segment;

[0022] Perform a Fourier transform on the music segment to obtain a music spectrum;

[0023] Perform spectrum compression on the music spectrum to obtain a compressed spectrum;

[0024] Locate the fundamental frequency points and harmonic points in the compressed spectrum;

[0025] Extract the point amplitudes of the fundamental frequency points and the harmonic points;

[0026] According to the point amplitudes, use the following formula to calculate the harmonic energy of the music segment:

[0027]

[0028] where E represents the harmonic energy, f0 represents the frequency corresponding to the fundamental frequency point and the harmonic point, k represents the harmonic order of the fundamental frequency point and the harmonic point, and A k (kf0) represents the point amplitude when the harmonic order is k and the frequency is f0;

[0029] Calculate the energy proportion of the harmonic energy in the total spectral energy corresponding to the compressed spectrum;

[0030] Calculate the Bark band energy entropy value of the music spectrum;

[0031] Take the energy proportion and the Bark band energy entropy value as the energy features of the music segment.

[0032] Optionally, the feature decision processing of the pitch features and the energy features to obtain a feature decision score includes:

[0033] Perform feature standardization processing on the pitch features and the energy features to obtain a standard pitch and a standard energy;

[0034] Calculate the interaction weight between the standard pitch and the standard energy through a preset bilinear attention model;

[0035] Fuse the standard pitch and the standard energy using the interaction weight to obtain a fused feature;

[0036] Perform mean pooling on the fused feature to obtain a smoothed feature;

[0037] Output the context enhanced feature of the smoothed feature through a preset GRU network;

[0038] Calculate the cosine similarity between the context-enhanced feature and a preset scale template;

[0039] Based on the cosine similarity, use the following formula to calculate the feature decision score corresponding to the pitch feature and the energy feature:

[0040] H i = c i (1 - q|E harmonic - E i |)

[0041] where H i represents the feature decision score corresponding to the i-th musical key, c i represents the cosine similarity between the context-enhanced feature and the scale template of the i-th musical key, q represents the attenuation coefficient, E i represents the preset harmonic energy ratio of the i-th musical key, and E harmonic represents the energy ratio of the harmonic energy to the total spectral energy corresponding to the compressed spectrum;

[0042] where the bilinear attention model includes:

[0043] a = σ(W p h p + W e h e + b)

[0044] where a represents the interaction weight, h p represents the standard pitch, W o represents the weight value of the standard pitch, h e represents the standard energy, W e represents the weight value of the standard energy, and b represents the bias of the bilinear attention model.

[0045] Optionally, constructing the key-color mapping library between the preset musical key samples and the preset light color parameters includes:

[0046] Query the basic hue, basic saturation, basic brightness, and basic white light intensity corresponding to the basic musical key in the musical key sample from the light color parameters;

[0047] Construct the basic association unit between the basic musical key and the basic hue, the basic saturation, the basic brightness, and the basic white light intensity;

[0048] Determine the hue range of the basic hue;

[0049] Use the Gaussian distribution of the basic hue to determine the hue perturbation amplitude of the basic hue;

[0050] Randomly perturb the base hue based on the hue range and the hue perturbation amplitude to obtain a perturbed hue;

[0051] Randomly perturb the base saturation using the following formula to obtain a perturbed saturation:

[0052]

[0053] where S new represents the perturbed saturation, S base represents the base saturation, ΔS represents the saturation error value, and V base represents the base brightness;

[0054] Randomly perturb the base brightness using the following formula to obtain a perturbed brightness:

[0055]

[0056] where V new represents the perturbed brightness, ΔV represents the brightness error value, and S base represents the base saturation;

[0057] Set the white light intensity range and the environmental perturbation amplitude of the base white light intensity;

[0058] Randomly perturb the base white light intensity using the white light intensity range and the environmental perturbation amplitude to obtain a perturbed white light intensity;

[0059] Construct a perturbation correlation unit between the base music key and the perturbed hue, the perturbed saturation, the perturbed brightness, and the perturbed white light intensity;

[0060] Randomly generate a random hue, a random saturation, a random brightness, and a random white light intensity corresponding to the base music key;

[0061] Construct a random correlation unit between the base music key and the random hue, the random saturation, the random brightness, and the random white light intensity;

[0062] Based on a preset RGBW safety threshold and a low-saturation - high-brightness range, perform unit screening on the base correlation unit, the perturbation correlation unit, and the random correlation unit to obtain a screened base unit, a screened perturbation unit, and a screened random unit;

[0063] Based on a preset ratio among the screened base unit, the screened perturbation unit, and the screened random unit, perform structural adjustment on the screened base unit, the screened perturbation unit, and the screened random unit to obtain an adjusted base unit, an adjusted perturbation unit, and an adjusted random unit;

[0064] Taking the adjustment basic unit, the adjustment perturbation unit, and the adjustment random unit as a tonality-color unit to determine a tonality-color mapping library through the tonality-color unit.

[0065] Optionally, performing a crossover mutation operation on the tonality-color units in the tonality-color mapping library to obtain crossover mutation units, including:

[0066] Calculating the mutation probability of the tonality-color unit using the following formula:

[0067]

[0068] p new = p base + min(w1e″, p max - p base )

[0069] where p new represents the mutation probability, p base represents the original mutation probability, m represents the number of times the light controller in the smart home has been manually adjusted in the historical period, t now represents the current moment when calculating the mutation probability, t j represents the jth moment of being manually adjusted, d represents the attenuation coefficient affecting w1, w1 represents the weight value affected by manual adjustment, e″ represents the gain coefficient, p max represents the maximum mutation probability within the historical period, p base represents the basic mutation probability within the historical period;

[0070] Calculating the crossover probability of the tonality-color unit using the following formula:

[0071]

[0072] where p′ new represents the crossover probability, p′ base represents the basic crossover probability within the historical period, e′ represents the gain coefficient when calculating the crossover probability, m represents the number of times the light controller in the smart home has been manually adjusted in the historical period, n total represents the total number of iterations;

[0073] Performing a parameter crossover operation on the same-modal parameters of different tonality-color units, different-modal parameters of the same tonality-color unit, and different-modal parameters of different tonality-color units according to the crossover probability to obtain crossover units;

[0074] Performing a mutation operation on the crossover units according to the mutation probability to obtain crossover mutation units.

[0075] Optionally, analyzing the user preference, tonality-color matching degree, and coherence between the crossover and mutation unit and the music tonality based on the current color parameters of the light controller in the smart home includes:

[0076] Calculating the user preference of the crossover and mutation unit using the following formula:

[0077]

[0078] where UserScore represents the user preference, u gene represents the color parameter of any crossover and mutation unit, and P user adjust represents the average value of the color parameters of the light controller that have been manually adjusted in the historical period, and P range represents the value range of the color parameter of any crossover and mutation unit, I represents the category index of the color parameter of any crossover and mutation unit, and n represents the number of categories of the color parameter of any crossover and mutation unit;

[0079] Calculating the tonality-color matching degree of the crossover and mutation unit using the following formula:

[0080]

[0081] where MatchScore represents the tonality-color matching degree, u' gene represents the vector of the color parameter of any crossover and mutation unit, and u' template represents the template vector of the color parameter of any crossover and mutation unit;

[0082] Based on the current color parameters, calculating the coherence of the crossover and mutation unit using the following formula:

[0083]

[0084] where SmoothScore represents the coherence, T represents the time window length, and ΔH t = H t - H t-1 , ΔH t represents the hue change amount, H t represents the hue in the color parameter of any crossover and mutation unit, H t-1 represents the current hue in the current color parameters, ΔS t represents the saturation change amount, ΔV t represents the brightness change amount, and t represents the moment index within the time window.

[0085] Optionally, generating the comprehensive fitness between the crossover and mutation unit and the music tonality using the user preference, the tonality-color matching degree, and the coherence includes:

[0086] Allocate preference weights, matching weights, and coherence weights for the user preferences, the tonality-color matching degree, and the coherence respectively;

[0087] Use the preference weights, the matching weights, and the coherence weights to perform weighted summation on the user preferences, the tonality-color matching degree, and the coherence to obtain a comprehensive fitness.

[0088] Optionally, evolving the current color parameters according to the crossover mutation unit and the comprehensive fitness to obtain evolved color parameters includes:

[0089] Query the parent unit in the crossover mutation unit;

[0090] Based on the comprehensive fitness, select a target unit that meets a preset quantile in the parent unit;

[0091] Use the target unit to determine the final unit of the current color parameters;

[0092] Obtain the evolved color parameters from the final unit.

[0093] To solve the above problems, the present invention also provides a smart home RGBW dimming environment and background music collaborative control system, and the system includes:

[0094] A tonality recognition module, configured to obtain the next music segment of a music player in the smart home, extract the pitch feature and the energy feature of the music segment respectively, perform feature decision processing on the pitch feature and the energy feature to obtain a feature decision score, so as to identify the music tonality of the music segment through the feature decision score;

[0095] A unit mutation module, configured to construct a tonality-color mapping library between a preset music tonality sample and a preset lighting color parameter, perform a crossover mutation operation on the tonality-color units in the tonality-color mapping library to obtain a crossover mutation unit, wherein the tonality-color mapping library is composed of preset tonality-color units;

[0096] A fitness generation module, configured to analyze the user preference, the tonality-color matching degree, and the coherence between the crossover mutation unit and the music tonality based on the current color parameters of the lighting controller in the smart home, and generate a comprehensive fitness between the crossover mutation unit and the music tonality by using the user preference, the tonality-color matching degree, and the coherence;

[0097] A music detection module, configured to perform color evolution on the current color parameters according to the crossover and mutation unit and the comprehensive fitness, obtain evolved color parameters, and detect whether the current music segment of the music player has played to the next music segment;

[0098] A collaborative control module, configured to, when detecting that the current music segment of the music player has played to the next music segment, perform collaborative control with the next music segment by using the lighting controller based on the evolved color parameters, and obtain a collaborative control result.

[0099] Compared with the problems described in the background art, the embodiment of the present invention obtains the music segment of the next segment of the music player in the smart home to obtain the subsequent music segment of the whole song in the currently played segment, and generates an RGBW lighting parameter based on the subsequent music segment. Compared with other patents that generate lighting parameters at each moment based on music data at each moment, the embodiment of the present invention will not cause the lighting effect to be chaotic due to frequent key changes, thus considering the personalization in unique scenarios such as jazz improvisation scenarios. Further, the embodiment of the present invention extracts the pitch feature and energy feature of the music segment respectively to analyze the music key of the entire music segment based on the pitch feature and energy feature. Further, the embodiment of the present invention performs feature decision processing on the pitch feature and the energy feature to fuse the pitch feature and the energy feature, and outputs the score of the fused feature for each music key. Subsequently, the music key corresponding to the highest score is selected, and the key change trend can be identified through the temporal context modeling of bidirectional GRU to dynamically adjust the music key score. The embodiment of the present invention constructs a key-color mapping library between the preset music key samples and the preset lighting color parameters to remove the intermediate role of emotional color, changing from the previous music key-emotional color mapping relationship and emotional color-lighting color parameter mapping relationship to a music key-lighting color parameter mapping relationship, and adding a perturbation parameter to the mapping relationship, thereby increasing the population diversity in the key-color mapping library, preparing for subsequent mining of personalized key-lighting color pairings suitable for users from the diverse key-color mapping library. Further, the embodiment of the present invention performs crossover and mutation operations on the key-color units in the key-color mapping library to dynamically adjust the color parameters through the crossover and mutation operations, and combines the manual adjustment records feedback by the user to optimize the parameters of the key-color mapping in real time to adapt to the personalized scenario of the user. The embodiment of the present invention analyzes the user preference, key-color matching degree and coherence between the crossover and mutation unit and the music key based on the current color parameter of the lighting controller in the smart home, so as to optimize the color parameter based on the user preference, thereby adapting to the personalization of the user scenario. The embodiment of the present invention performs color evolution on the current color parameter according to the crossover and mutation unit and the comprehensive fitness, so as to perform an optimization operation on the color parameter by using the genetic algorithm to find the best-performing color parameter. Therefore, the smart home RGBW dimming environment and background music collaborative control method and system provided by the embodiment of the present invention can improve the consideration of personalized dimming-music collaborative control. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Figure 1 It is a schematic flowchart of a smart home RGBW dimming environment and background music collaborative control method provided by an embodiment of the present invention;

[0101] Figure 2 This is a schematic diagram of the module for implementing the collaborative control system of the smart home RGBW dimming environment and background music provided by an embodiment of the present invention.

[0102] The implementation, functional features, and advantages of the present invention will be further described in conjunction with the embodiments with reference to the accompanying drawings. Specific Embodiments

[0103] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0104] The embodiments of the present application provide a method for collaborative control of a smart home RGBW dimming environment and background music. The execution subject of the method for collaborative control of the smart home RGBW dimming environment and background music includes, but is not limited to, at least one of electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiments of the present application. In other words, the method for collaborative control of the smart home RGBW dimming environment and background music can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to: a single server, a server cluster, a cloud server, or a cloud server cluster, etc.

[0105] Embodiment 1:

[0106] Referring to Figure 1 As shown, it is a flowchart of the method for collaborative control of a smart home RGBW dimming environment and background music provided by an embodiment of the present invention. In this embodiment, the method for collaborative control of the smart home RGBW dimming environment and background music includes:

[0107] S1. Obtain the next music segment of the music player in the smart home, respectively extract the pitch feature and energy feature of the music segment, perform feature decision processing on the pitch feature and the energy feature to obtain a feature decision score, so as to identify the music key of the music segment through the feature decision score.

[0108] In the embodiments of the present invention, the smart home mainly includes a music player and a lighting controller. The music player mainly controls the music playback in the room, and the lighting controller mainly controls the RGBW transformation of the indoor lights.

[0109] Furthermore, in the embodiments of the present invention, by obtaining the next music segment of the music player in the smart home to obtain the subsequent music segment of the whole song in the current playing segment, and generating an RGBW lighting parameter based on the subsequent music segment. Compared with other patents that generate lighting parameters at each moment based on music data at each moment, the embodiments of the present invention will not cause the lighting effect to be chaotic due to frequent key changes, thus considering the personalization in unique scenarios such as jazz improvisation scenarios.

[0110] Wherein, the next music segment refers to the subsequent music segment of the entire song in the currently playing segment.

[0111] In an embodiment of the present invention, obtaining the next music segment of the music player in the smart home includes: querying the start time of playback of the music player; analyzing the average duration from sampling the music to controlling the lighting; extracting the current music segment corresponding to the start time of playback lasting for the average duration; obtaining the entire music corresponding to the current music segment; and using the remaining music segment of the entire music except the current music segment as the next music segment of the music player.

[0112] Wherein, the start time of playback refers to the start time of each song. For example, when switching to song A for playback currently, it is the start time of song A. The average duration refers to the duration from the data processing unit in the smart home collecting the music data of the music player to the lighting parameters of the lighting controller changing. It should be noted that since this solution is based on multiple experiments, therefore, the average duration within the historical period of multiple experiments can be statistically calculated. Again, the reason for selecting the average duration as the duration to extend backward from the start time of playback is that within the duration from the data processing unit collecting the music data of the music player to the lighting parameters of the lighting controller changing, song A also plays music for the average duration. When the lighting parameters of the lighting controller change, the music segment of song A just reaches the start time of the remaining music segment. At this time, the lighting parameters of the lighting controller are adapted to the music tonality of the remaining music segment. Further, the current music segment refers to the music segment in the entire music starting from the start time of playback and lasting for the average duration.

[0113] Further, in the embodiment of the present invention, the pitch feature and energy feature of the music segment are respectively extracted to analyze the music tonality of the entire music segment based on the pitch feature and energy feature.

[0114] Wherein, the music tonality refers to the 24 major and minor keys that the music possesses. Each of the 24 major and minor keys has a corresponding emotional color. For example, the pure white C major among the 24 major and minor keys corresponds to the emotional colors of purity and initial.

[0115] In an embodiment of the present invention, the separately extracting the pitch feature and the energy feature of the music segment includes: using a preset CREPE model to extract the frame pitch sequence of the music segment; performing sequence alignment on the frame pitch sequence through a preset dynamic time warping algorithm to obtain a frame smoothed pitch; analyzing the pitch mean, pitch variance, pitch range, and semitone distribution histogram of the frame smoothed pitch; taking the pitch mean, the pitch variance, the pitch range, and the semitone distribution histogram as the pitch features of the music segment; performing Fourier transform on the music segment to obtain a music spectrum; performing spectrum compression on the music spectrum to obtain a compressed spectrum; locating the fundamental frequency points and harmonic points in the compressed spectrum; extracting the point amplitudes of the fundamental frequency points and the harmonic points; calculating the harmonic energy of the music segment according to the point amplitudes by using the following formula:

[0116]

[0117] where E represents the harmonic energy, f0 represents the frequencies corresponding to the fundamental frequency points and the harmonic points, k represents the harmonic order of the fundamental frequency points and the harmonic points, and A k (kf0) represents the point amplitude when the harmonic order is k and the frequency is f0;

[0118] calculating the energy proportion of the harmonic energy in the total spectral energy corresponding to the compressed spectrum; calculating the Bark band energy entropy value of the music spectrum; taking the energy proportion and the Bark band energy entropy value as the energy features of the music segment.

[0119] Among them, the core of the CREPE model is a pre-trained model based on the Transformer architecture, which is a sequence modeling technology widely respected in the field of NLP. By combining the Transformer with two training strategies, autoregressive and masked language modeling (MLM), Crepe can better capture the context information of the text and make predictions, thus achieving high-quality text generation. The frame pitch sequence refers to a sequence of consecutive time points composed of the fundamental frequency values of each frame segment in the music segment. Each frame refers to each time interval in the music segment. For example, if the music segment is 3 minutes, each frame refers to every 20 ms. The dynamic time warping algorithm is a time series analysis technique based on dynamic programming, used to measure the similarity between two time series. It allows the sequences to be matched by stretching and compressing the non-linear time axis, thereby measuring the similarity between them. Furthermore, an optimal path is found through the dynamic programming algorithm to align the two sequences on the time axis. Therefore, it is necessary to prepare a template of the pitch sequence in advance, calculate the similarity between the template and the current frame pitch sequence, and finally find an optimal path through the dynamic programming algorithm to align the two sequences on the time axis. The pitch range refers to the range between the maximum pitch and the minimum pitch. The chroma distribution histogram refers to an image obtained through a visualization tool used to analyze the chroma distribution law in music. The visualization tool shows the usage frequency or distribution of chromas in the music melody in the form of a histogram, helping researchers or creators understand the melody structure and interval change law of the music. The fundamental frequency point is often represented by the symbol f0, and the harmonic points refer to the positions where 2f0, 3f0, 4f0, 5f0, 6f0, 7f0, 8f0 are located. The point amplitude is the amplitude of the fundamental frequency point and the harmonic points. The calculation formula of the total spectral energy corresponding to the compressed spectrum is similar to the principle of calculating the Bark band energy in the following. The difference is that the calculation of the Bark band energy is based on 12 frequency bands, while the total spectral energy corresponding to the compressed spectrum is calculated based on all frequencies. Regarding the Bark bands in the Bark band energy entropy value, the 20 Hz - 20 kHz audible range is divided into 24 standard Bark bands. In the music analysis scenario, the bands are often merged or key frequency bands are selected, and the 24 bands are compressed into 12 Bark bands.

[0120] Optionally, the process of performing spectral compression on the music spectrum to obtain a compressed spectrum refers to downsampling the music spectrum by an integer multiple of the fundamental frequency (such as 1×, 2×, 3× the fundamental frequency) and multiplying point by point. For example, if the music spectrum is from 0 to 8 kHz and the fundamental frequency is 200 Hz, then the first harmonic (200 Hz) retains the full spectrum, the second harmonic (400 Hz) is compressed to 0 to 4 kHz and multiplied by the original spectrum, the third harmonic (600 Hz) is compressed to 0 to 2.66 kHz and continues to be multiplied. Further, the process of calculating the Bark band energy entropy value of the music spectrum is as follows: Calculate the Bark band energy: E(b) = ∑ f∈频带b |X(f)| 2 , where X(f) is the spectral amplitude at frequency f, E(b) is the energy of the b-th Bark band. The ratio between the energy of the b-th Bark band and the total energy of 12 Bark bands is used as the Bark band energy proportion. The Bark band energy proportion is used as the input data for the information entropy formula to calculate the Bark band energy entropy value. The information entropy formula is a commonly used formula in the industry. For example p b is the proportion of the energy of the b-th Bark band, and the formula for the total spectral energy corresponding to the compressed spectrum is similar to E(b) = ∑ f∈频带b |X(f)| 2 , and only need to replace f ∈ band b with all frequencies in the compressed spectrum.

[0121] Further, in the embodiment of the present invention, the pitch feature and the energy feature are subjected to feature decision processing to fuse the pitch feature and the energy feature, and the scores of the fusion feature for each music key are output. In the subsequent step, the music key corresponding to the highest score is selected, and the key transition trend can be identified through the temporal context modeling of the bidirectional GRU, and the music key score is dynamically adjusted.

[0122] In one embodiment of the present invention, the feature decision processing of the pitch feature and the energy feature to obtain a feature decision score includes: performing feature normalization processing on the pitch feature and the energy feature to obtain a standard pitch and a standard energy; calculating the interaction weight between the standard pitch and the standard energy through a preset bilinear attention model; fusing the standard pitch and the standard energy using the interaction weight to obtain a fusion feature; performing mean pooling on the fusion feature to obtain a smoothed feature; outputting the context-enhanced feature of the smoothed feature through a preset GRU network; calculating the cosine similarity between the context-enhanced feature and a preset scale template; based on the cosine similarity, calculating the feature decision score corresponding to the pitch feature and the energy feature using the following formula:

[0123] H i = c i (1 - q|Eharmonic -E i |)

[0124] Among them, H i represents the characteristic decision score corresponding to the i-th musical key, c i represents the cosine similarity between the context-enhanced feature and the scale template of the i-th musical key, q represents the attenuation coefficient, and E i represents the preset harmonic energy proportion of the i-th musical key, and E harmonic represents the energy proportion of the harmonic energy in the total spectral energy corresponding to the compressed spectrum;

[0125] Among them, the bilinear attention model includes:

[0126] a = σ(W p h p + W e h e + b)

[0127] Among them, a represents the interaction weight, h p represents the standard pitch, W p represents the weight value of the standard pitch, h e represents the standard energy, W e represents the weight value of the standard energy, and b represents the bias of the bilinear attention model.

[0128] Among them, the GRU network refers to the gated recurrent unit, which is a variant of the recurrent neural network (RNN) designed to solve the problems of long-term memory and gradient vanishing and gradient explosion in backpropagation in RNN. The scale template refers to 24 major and minor scale templates. For example, the scale template of C major is composed of feature vectors similar to the context-enhanced features (feature vectors constructed manually in advance). The scale templates of other keys are the same as that of C major. The value range of the attenuation coefficient is from 0.05 to 0.2. The smaller the value of the attenuation coefficient, the larger (1 - q|E harmonic - E i |). (1 - q|E harmonic - E i |) is used as the weight value of c i affected by the harmonic energy, and can dynamically adjust the size of c i . E i is a reference value and template value determined manually in advance. W p , W e , and b are parameters trained in advance through neural network algorithms such as training samples and gradient descent algorithms.

[0129] Optionally, the process of performing feature standardization on the pitch feature and the energy feature is as follows: Perform Z-score standardization on the pitch mean, pitch variance, and pitch range to eliminate the dimensional difference. Convert the 12-dimensional semitone distribution histogram (mapping MIDI pitch 0-127 to 12 semitone classes) into a probability density vector, e.g., [C: 0.35, C#: 0.02, D: 0.1,..., B: 0.08], where C represents the pure white C major, C# represents the off-white C# major, and the same applies to others. Apply logarithmic transformation (log(1 + x)) to the harmonic energy ratio (0-1) to enhance the sensitivity of the low ratio interval. Divide the continuous Bark band energy entropy value into 4 levels (low / mid-low / mid-high / high), and convert the levels into a One-hot vector. Further, use the interaction weight to fuse the standard pitch and the standard energy. The commonly used fusion formula in the industry for obtaining the fusion feature is ah p +(1 - a)h e , where the mean pooling is also called the average pooling process.

[0130] Optionally, the process of identifying the musical key of the music segment through the feature decision score refers to selecting the musical key corresponding to the highest score as the musical key of the music segment.

[0131] S2. Construct a key-color mapping library between the preset musical key samples and the preset lighting color parameters, and perform cross-mutation operations on the key-color units in the key-color mapping library to obtain cross-mutated units, where the key-color mapping library is composed of preset key-color units.

[0132] In the embodiment of the present invention, by constructing a key-color mapping library between the preset musical key samples and the preset lighting color parameters, the intermediate role of emotional color is removed, and the previous musical key-emotional color mapping relationship and emotional color-lighting color parameter mapping relationship are transformed into a musical key-lighting color parameter mapping relationship. And a perturbation parameter is added to the mapping relationship, thereby increasing the population diversity in the key-color mapping library, preparing for subsequent mining of personalized key-lighting color pairings suitable for users from the diverse key-color mapping library.

[0133] Among them, the key-color mapping library is composed of key-color units with different pairings.

[0134] In an embodiment of the present invention, constructing a tonality-color mapping library between a preset music tonality sample and preset light color parameters includes: querying a basic hue, a basic saturation, a basic brightness, and a basic white light intensity corresponding to the basic music tonality in the music tonality sample from the light color parameters; constructing a basic association unit between the basic music tonality and the basic hue, the basic saturation, the basic brightness, and the basic white light intensity; determining a hue range of the basic hue; determining a hue perturbation amplitude of the basic hue by using a Gaussian distribution of the basic hue; randomly perturbing the basic hue based on the hue range and the hue perturbation amplitude to obtain a perturbed hue; randomly perturbing the basic saturation by using the following formula to obtain a perturbed saturation:

[0135]

[0136] where S new represents the perturbed saturation, S base represents the basic saturation, ΔS represents a saturation error value, and V base represents the basic brightness;

[0137] Randomly perturbing the basic brightness by using the following formula to obtain a perturbed brightness:

[0138]

[0139] where V new represents the perturbed brightness, ΔV represents a brightness error value, and S base represents the basic saturation;

[0140] Set the white light intensity range and the environmental disturbance amplitude of the basic white light intensity; randomly disturb the basic white light intensity by using the white light intensity range and the environmental disturbance amplitude to obtain a disturbed white light intensity; construct a disturbance correlation unit between the basic music key and the disturbed hue, the disturbed saturation, the disturbed brightness and the disturbed white light intensity; randomly generate a random hue, a random saturation, a random brightness and a random white light intensity corresponding to the basic music key; construct a random correlation unit between the basic music key and the random hue, the random saturation, the random brightness and the random white light intensity; based on a preset RGBW safety threshold and a low saturation-high brightness range, perform unit screening on the basic correlation unit, the disturbance correlation unit and the random correlation unit to obtain a screened basic unit, a screened disturbance unit and a screened random unit; based on a preset ratio among the screened basic unit, the screened disturbance unit and the screened random unit, perform structural adjustment on the screened basic unit, the screened disturbance unit and the screened random unit to obtain an adjusted basic unit, an adjusted disturbance unit and an adjusted random unit; use the adjusted basic unit, the adjusted disturbance unit and the adjusted random unit as key-color units to determine a key-color mapping library through the key-color units.

[0141] Among them, the basic hue, basic saturation, basic brightness, and basic white light intensity corresponding to the basic music tonality refer to the conventional music tonality-HSVW pairings in the past and historical periods. The basic music tonality refers to any pitch, and the basic hue, basic saturation, basic brightness, and basic white light intensity refer to the HSVW mapped to this basic music tonality. The basic association unit refers to the data formed by the association relationship between the basic music tonality and the basic hue, basic saturation, basic brightness, and basic white light intensity. The hue range refers to the conventional range of the basic hue. For example, the basic hue of a bright tonality (such as the C major basic music tonality) is in the warm color area (0° to 90°,In the hue range of 270° to 360°), regarding ΔS and ΔV, ΔS and ΔV are parameters that the user requests to increase or decrease. For example, if the user requests to increase the saturation by ΔS = +20%, regarding the white light intensity range and the environmental disturbance amplitude, the environmental disturbance amplitude refers to the environmental disturbance amplitude preset according to the simulated environmental light. For example, the example: the base W = 50%, the disturbance range during the day mode is ±30% (i.e., the environmental disturbance amplitude), and the night mode is ±10% (i.e., the environmental disturbance amplitude). The white light intensity range refers to the white light intensity range of a certain music tonality, which is similar to the meaning of the aforementioned hue range. The meanings of the disturbance correlation unit, the random correlation unit, and the aforementioned base correlation unit are similar and will not be elaborated here. The RGBW safety threshold refers to the RGBW sum threshold. For example, R + G + B + W ≤ 300%, and 300% is the RGBW safety threshold. Since the RGBW safety threshold is related to RGB, when comparing the RGBW safety threshold subsequently, it is necessary to convert RGBW to the HSVW space, and the conversion of RGBW to the HSVW space is a commonly used method in the industry. The low-saturation - high-brightness range refers to the combination of low saturation (S < 20%) and high brightness (V > 90%). Since the combination of low saturation (S < 20%) and high brightness (V > 90%) of the light makes the human eye uncomfortable, it is necessary to exclude the combination of low saturation (S < 20%) and high brightness (V > 90%). The refers to the screening base unit, the screening disturbance unit, and the screening random unit refer to the units that do not exceed the RGBW safety threshold and remove the combined units in the low-saturation - high-brightness range. The preset ratio refers to the number of screening base units: the number of screening disturbance units: the number of screening random units. For example, 20%:50%:30%. The adjusted base unit, the adjusted disturbance unit, and the adjusted random unit refer to the units whose numbers are adjusted to conform to 20%:50%:30%. When adjusting the numbers here, there is no need to add new units, just randomly remove the extra units. Regarding the Gaussian distribution of the base hue, hue is an attribute of color, representing the type of color (such as red, green, blue, etc.), usually represented by an angular value (0 to 360 degrees). Since hue has periodicity (for example, 0 degrees and 360 degrees are the same), the ordinary Gaussian distribution (linear Gaussian distribution) is not directly applicable, and the circular Gaussian distribution (also known as the von Mises distribution) is usually used to describe the Gaussian distribution of hue.,

[0142] Optionally, the process of determining the hue perturbation amplitude of the base hue using the Gaussian distribution of the base hue refers to calculating the mean and variance of the Gaussian distribution so that the hue perturbation amplitude of the base hue is controlled within the mean and variance of the Gaussian distribution. For example, if the hue perturbation amplitude of the base hue is a, and a new hue is obtained by adding a to the base hue, calculate the mean and variance between the new hue and all hues in the Gaussian distribution. If the newly calculated mean and variance are similar to the old mean and variance, then the a hue perturbation amplitude can be used; otherwise, it cannot. Further, the process of randomly perturbing the base hue based on the hue range and the hue perturbation amplitude to obtain the perturbed hue is to perform a numerical transformation on the base hue within the hue range using the hue perturbation amplitude (this amplitude value is not unique and is all amplitude values that conform to the mean and variance of the aforementioned Gaussian distribution). The transformed hue value needs to be within the hue range.

[0143] Further, in an embodiment of the present invention, cross-mutation operations are performed on the key-tone - color mapping units in the key-tone - color mapping library to dynamically adjust color parameters through cross-mutation operations, and the parameters of the key-tone - color mapping are optimized in real time by combining the manual adjustment records fed back by the user to adapt to the personalized scenarios of the user.

[0144] Among them, the cross-mutation unit refers to a data unit including parents, offspring, grandchildren, and subsequent descendants, and the parent unit among them is all the key-tone - color units in the key-tone - color mapping library.

[0145] In an embodiment of the present invention, performing cross-mutation operations on the key-tone - color units in the key-tone - color mapping library to obtain cross-mutation units includes: calculating the mutation probability of the key-tone - color unit using the following formula:

[0146]

[0147] p new =p base +min(w1e″,p max -p base )

[0148] Among them, p new represents the mutation probability, p base represents the original mutation probability, m represents the number of times the lighting controller in the smart home has been manually adjusted in the historical period, t now represents the current moment of calculating the mutation probability, t j represents the jth moment of being manually adjusted, d represents the attenuation coefficient affecting w1, w1 represents the weight value affected by manual adjustment, e″ represents the gain coefficient, p max represents the maximum mutation probability within the historical period, p baseRepresents the basic mutation probability within the historical period;

[0149] Calculate the crossover probability of the tonality-color unit using the following formula:

[0150]

[0151] where p′ new represents the crossover probability, p′ base represents the basic crossover probability within the historical period, e′ represents the gain coefficient when calculating the crossover probability, m represents the number of times the lighting controller in the smart home has been manually adjusted during the historical period, n total represents the total number of iterations;

[0152] According to the crossover probability, perform parameter crossover operations on the same-modal parameters of different tonality-color units, different-modal parameters of the same tonality-color unit, and different-modal parameters of different tonality-color units to obtain crossover units; according to the mutation probability, perform mutation operations on the crossover units to obtain crossover mutation units.

[0153] It should be noted that d represents the attenuation coefficient that affects w1, and the value range of this attenuation coefficient is from 0.1 to 0.3, which is mainly used to dynamically adjust the magnitude of the weight w1. Similar to the meaning of the aforementioned q attenuation coefficient, the gain coefficients (e″, e′) are used to control the adjustment range of the algorithm parameters by user feedback. The basic mutation probability and the basic crossover probability are both taken as the average value. For p and p′, as the number of times the user manually adjusts the lights increases, the values of p and p′ will also change dynamically. Therefore, different values of p and p′ will be generated within the historical period. For n total , represents the round of crossover. For example, the crossover is between two parents of the initial population. After obtaining the offspring, the offspring also need to be crossed with each other, and then the grandchildren are crossed. For the parents, offspring, and grandchildren, n total represents the number of times of these three generations, that is, the value 3. Here, it should be noted that regarding the mutation probability and the crossover probability, the mutation probability and the crossover probability need to be recalculated each time an iteration is performed. For example, when performing crossover mutation on the parents, the mutation probability and the crossover probability need to be calculated once. When performing crossover mutation on the offspring, the mutation probability and the crossover probability need to be calculated once. As for the iteration threshold, it needs to be set in advance according to manual experience. For example, it is set that iterating three times (that is, three generations of parents, offspring, and grandchildren) is the iteration threshold, and n total is less than or equal to the iteration threshold.

[0154] Optionally, the process of performing parameter crossover operations on the same-modal parameters of different tonality-color units, different-modal parameters of the same tonality-color unit, and different-modal parameters of different tonality-color units according to the crossover probability to obtain crossover units is as follows: For example, the same-modal parameters of different tonality-color units: The saturation of "C major" and "c minor" are exchanged; The different-modal parameters of the same tonality-color unit: The saturation and brightness of "C major" exchange parameters with each other; The different-modal parameters of different tonality-color units: The saturation of "C major" and the brightness of "c minor" are exchanged with each other. Finally, it is only necessary to make the proportion of crossover consistent with the crossover probability. Further, the process of performing mutation operations on the crossover units according to the mutation probability to obtain crossover mutation units is a random mutation process, and it is only necessary to make the proportion of mutation consistent with the mutation probability.

[0155] S3. Based on the current color parameters of the lighting controller in the smart home, analyze the user preferences, tonality-color matching degree, and coherence between the crossover mutation unit and the music tonality, and generate the comprehensive fitness between the crossover mutation unit and the music tonality by using the user preferences, the tonality-color matching degree, and the coherence.

[0156] In an embodiment of the present invention, by analyzing the user preferences, tonality-color matching degree, and coherence between the crossover mutation unit and the music tonality based on the current color parameters of the lighting controller in the smart home, the color parameters are optimized based on user preferences, so as to adapt to the personalization of the user scenario.

[0157] Among them, the current color parameters refer to the HSVW parameters corresponding to the current music segment.

[0158] In an embodiment of the present invention, the analyzing the user preferences, tonality-color matching degree, and coherence between the crossover mutation unit and the music tonality based on the current color parameters of the lighting controller in the smart home includes: calculating the user preference of the crossover mutation unit by using the following formula:

[0159]

[0160] Among them, UserScore represents the user preference, u gene represents the color parameter of any crossover mutation unit, P user adjust represents the average value of the color parameters of the lighting controller after being manually adjusted in the historical period, P range represents the value range of the color parameter of any crossover mutation unit, I represents the category index of the color parameter of any crossover mutation unit, and n represents the number of categories of the color parameter of any crossover mutation unit;

[0161] Calculate the tonality-color matching degree of the crossover mutation unit using the following formula:

[0162]

[0163] Where MatchScore represents the tonality-color matching degree, and u′ gene represents the vector of the color parameters of any crossover mutation unit, and u′ template represents the template vector of the color parameters of any crossover mutation unit;

[0164] Based on the current color parameters, calculate the coherence of the crossover mutation unit using the following formula:

[0165]

[0166] Where SmoothScore represents coherence, T represents the time window length, and ΔH t =H t -H t-1 ΔH t represents the hue change amount, H t represents the hue in the color parameters of any crossover mutation unit, H t-1 represents the current hue in the current color parameters, ΔS t represents the saturation change amount, ΔV t represents the brightness change amount, and t represents the moment index within the time window.

[0167] It should be noted that n represents the number of categories of the color parameters of any crossover mutation unit, that is, the number of the four of H, S, V, and W, and u′ gene represents the vector of the color parameters of any crossover mutation unit, for example, a 1-row 4-column matrix vector composed of H, S, V, and W, such as C major = [60°, 80%, 70%, 50%].

[0168] In an embodiment of the present invention, generating the comprehensive fitness between the crossover mutation unit and the music tonality using the user preference, the tonality-color matching degree, and the coherence includes: respectively assigning preference weights, matching degree weights, and coherence weights to the user preference, the tonality-color matching degree, and the coherence; performing weighted summation on the user preference, the tonality-color matching degree, and the coherence using the preference weights, the matching degree weights, and the coherence weights to obtain the comprehensive fitness.

[0169] It should be noted that the preference weight, the matching degree weight, and the coherence weight are pre-determined weight coefficients. After obtaining the historical user preference samples, the tone-color matching degree samples, and the coherence samples from the historical samples, a weighted summation model is constructed between the preference weight, the matching degree weight, and the coherence weight and the historical user preference samples, the tone-color matching degree samples, and the coherence samples. Then, the fixed parameters of the preference weight, the matching degree weight, and the coherence weight are obtained by fitting using the least squares method. Further, each gene unit (parent, child, grandchild, etc.) has a corresponding comprehensive fitness.

[0170] Optionally, the process of using the preference weight, the matching degree weight, and the coherence weight to perform weighted summation on the user preference, the tone-color matching degree, and the coherence to obtain the comprehensive fitness means that the preference weight multiplied by the user preference + the matching degree weight multiplied by the tone-color matching degree + the coherence weight multiplied by the coherence = the comprehensive fitness. The higher the value of the comprehensive fitness, the better the color parameters of any crossover and mutation unit.

[0171] S4. According to the crossover and mutation unit and the comprehensive fitness, perform color evolution on the current color parameters to obtain evolved color parameters, and detect whether the current music segment of the music player has played to the next music segment.

[0172] In the embodiment of the present invention, by performing color evolution on the current color parameters according to the crossover and mutation unit and the comprehensive fitness, the genetic algorithm is used to optimize the color parameters to find the best-performing color parameters.

[0173] In an embodiment of the present invention, the performing color evolution on the current color parameters according to the crossover and mutation unit and the comprehensive fitness to obtain evolved color parameters includes: querying the parent unit in the crossover and mutation unit; based on the comprehensive fitness, selecting the target unit that meets the preset quantile in the parent unit; using the target unit to determine the final unit of the current color parameters; and obtaining the evolved color parameters from the final unit.

[0174] Optionally, the process of selecting the target unit that meets the preset quantile in the parent unit based on the comprehensive fitness is, for example, directly retaining the top 10% of the high-fitness parent gene units (i.e., the target units). The process of using the target unit to determine the final unit of the current color parameters is, for example: retaining the top 10% of the high-fitness offspring from the offspring of the target unit, and then retaining the top 10% of the high-fitness grandchildren from the grandchildren, until only one gene unit is available for retention, or when reaching the last generation, selecting any one gene unit from the last generation as the final unit.

[0175] Optionally, the process of detecting whether the current music segment of the music player has played to the next music segment means whether the current music segment has played to the last moment and the starting moment of the next music segment.

[0176] S5. When detecting that the current music segment of the music player has played to the next music segment, based on the evolutionary color parameter, use the lighting controller to complete the collaborative control with the next music segment to obtain a collaborative control result.

[0177] Optionally, when performing the process of completing the collaborative control with the next music segment by using the lighting controller based on the evolutionary color parameter, if the current color parameter of the lighting controller is consistent with the evolutionary color parameter, there is no need to change the lighting and it can remain unchanged. If not, the current color parameter of the lighting controller can be adjusted to the evolutionary color parameter.

[0178] Compared with the problems described in the background art, the embodiment of the present invention obtains the next music segment of the music player in the smart home to obtain the subsequent music segment of the whole song in the currently played segment, and generates an RGBW lighting parameter based on the subsequent music segment. Compared with other patents that generate lighting parameters for each moment based on the music data at each moment, the embodiment of the present invention will not cause the lighting effect to be chaotic due to frequent key changes, thus considering the personalization in unique scenarios such as jazz improvisation scenarios. Further, the embodiment of the present invention extracts the pitch feature and energy feature of the music segment respectively to analyze the music key of the entire music segment based on the pitch feature and energy feature. Further, the embodiment of the present invention performs feature decision processing on the pitch feature and the energy feature to fuse the pitch feature and the energy feature, and outputs the score of the fused feature for each music key. Subsequently, the music key corresponding to the highest score is selected, and the key change trend can be identified through the temporal context modeling of the bidirectional GRU to dynamically adjust the music key score. The embodiment of the present invention constructs a key-color mapping library between the preset music key samples and the preset lighting color parameters to remove the intermediate role of emotional color, changing from the previous music key-emotional color mapping relationship and emotional color-lighting color parameter mapping relationship to a music key-lighting color parameter mapping relationship, and adding a perturbation parameter to the mapping relationship, thereby increasing the population diversity in the key-color mapping library and preparing for mining the personalized key-lighting color pairing suitable for users from the diverse key-color mapping library in the future. Further, the embodiment of the present invention performs crossover and mutation operations on the key-color units in the key-color mapping library to dynamically adjust the color parameters through the crossover and mutation operations, and combines the manual adjustment records fed back by the user to optimize the parameters of the key-color mapping in real time to adapt to the personalized scenario of the user. The embodiment of the present invention analyzes the user preference, key-color matching degree and coherence between the crossover and mutation unit and the music key based on the current color parameter of the lighting controller in the smart home, and optimizes the color parameter based on the user preference, so as to adapt to the personalization of the user scenario. The embodiment of the present invention performs color evolution on the current color parameter according to the crossover and mutation unit and the comprehensive fitness, and uses the genetic algorithm to perform an optimization operation on the color parameter to find the best-performing color parameter. Therefore, the smart home RGBW dimming environment and background music collaborative control method and system provided by the embodiment of the present invention can improve the consideration of personalized dimming-music collaborative control.

[0179] Embodiment 2:

[0180] As Figure 2 shown, it is a functional module diagram of a smart home RGBW dimming environment and background music collaborative control system of the present invention.

[0181] The intelligent home RGBW dimming environment and background music collaborative control system 200 described in the present invention can be installed in an electronic device. According to the functions achieved, the intelligent home RGBW dimming environment and background music collaborative control system may include a tonality recognition module 201, a unit mutation module 202, a fitness generation module 203, a music detection module 204, and a collaborative control module 205. The modules described in the present invention may also be referred to as units, which refer to a series of computer program segments that can be executed by a processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.

[0182] In the embodiments of the present invention, the functions of each module / unit are as follows:

[0183] The tonality recognition module 201 is used to obtain the next music segment of the music player in the intelligent home, respectively extract the pitch feature and energy feature of the music segment, perform feature decision processing on the pitch feature and the energy feature to obtain a feature decision score, and identify the music tonality of the music segment through the feature decision score;

[0184] The unit mutation module 202 is used to construct a tonality-color mapping library between a preset music tonality sample and a preset light color parameter, perform a crossover mutation operation on the tonality-color units in the tonality-color mapping library to obtain crossover mutation units, wherein the tonality-color mapping library is composed of preset tonality-color units;

[0185] The fitness generation module 203 is used to analyze the user preference, tonality-color matching degree, and coherence between the crossover mutation unit and the music tonality based on the current color parameter of the light controller in the intelligent home, and generate a comprehensive fitness between the crossover mutation unit and the music tonality by using the user preference, the tonality-color matching degree, and the coherence;

[0186] The music detection module 204 is used to perform color evolution on the current color parameter according to the crossover mutation unit and the comprehensive fitness to obtain an evolved color parameter, and detect whether the current music segment of the music player has played to the next music segment;

[0187] The collaborative control module 205 is used to, when detecting that the current music segment of the music player has played to the next music segment, complete collaborative control with the next music segment based on the evolved color parameter by using the light controller to obtain a collaborative control result.

[0188] Specifically, when the modules in the smart home RGBW dimming environment and background music collaborative control system 200 in the embodiments of the present invention are used, they adopt the same technical means as those in the Figure 1 smart home RGBW dimming environment and background music collaborative control method described above, and can produce the same technical effects, which will not be elaborated here.

[0189] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for collaborative control of a smart home RGBW dimming environment and background music, characterized in that, The method includes: Obtaining a music segment of the next segment of a music player in a smart home, respectively extracting the pitch feature and the energy feature of the music segment, performing feature decision processing on the pitch feature and the energy feature to obtain a feature decision score, and identifying the music key of the music segment through the feature decision score; Constructing a key-color mapping library between a preset music key sample and preset lighting color parameters, performing crossover mutation operations on the key-color units in the key-color mapping library to obtain crossover mutation units, where the key-color mapping library is composed of preset key-color units; Based on the current color parameters of the lighting controller in the smart home, analyzing the user preference, key-color matching degree, and coherence between the crossover mutation unit and the music key, and generating a comprehensive fitness between the crossover mutation unit and the music key by using the user preference, the key-color matching degree, and the coherence; According to the crossover mutation unit and the comprehensive fitness, performing color evolution on the current color parameters to obtain evolved color parameters, and detecting whether the current music segment of the music player has been played to the music segment of the next segment; When it is detected that the current music segment of the music player has been played to the music segment of the next segment, based on the evolved color parameters, using the lighting controller to complete the collaborative control with the music segment of the next segment to obtain a collaborative control result.

2. The collaborative control method for a smart home RGBW dimming environment and background music according to claim 1, wherein The obtaining of the music segment of the next segment of the music player in the smart home includes: Querying the playback start time of the music player; Analyzing the average duration from sampling the music to lighting control; Extracting the current music segment corresponding to the playback start time lasting for the average duration; Obtaining the whole music corresponding to the current music segment; Taking the remaining music segments in the whole music except the current music segment as the music segment of the next segment of the music player.

3. The collaborative control method for smart home RGBW dimming environment and background music according to claim 1, characterized in that The respectively extracting of the pitch feature and the energy feature of the music segment includes: Using a preset CREPE model to extract the frame pitch sequence of the music segment; Performing sequence alignment on the frame pitch sequence through a preset dynamic time warping algorithm to obtain a frame smoothed pitch; Analyzing the pitch mean value, pitch variance, pitch range, and semitone distribution histogram of the frame smoothed pitch; Taking the pitch mean value, the pitch variance, the pitch range, and the semitone distribution histogram as the pitch features of the music segment; Performing Fourier transform on the music segment to obtain a music spectrum; Performing spectrum compression on the music spectrum to obtain a compressed spectrum; Locating the fundamental frequency points and harmonic points in the compressed spectrum; Extracting the point amplitudes of the fundamental frequency points and the harmonic points; According to the point amplitudes, calculating the harmonic energy of the music segment by using the following formula: Among them, E represents the harmonic energy, f0 represents the frequency corresponding to the fundamental frequency point and the harmonic point, k represents the harmonic order between the fundamental frequency point and the harmonic point, and A k (kf0) represents the point amplitude at the harmonic order of k and the frequency of f0; Calculating the energy proportion of the harmonic energy in the total spectrum energy corresponding to the compressed spectrum; Calculating the Bark band energy entropy value of the music spectrum; Taking the energy proportion and the Bark band energy entropy value as the energy features of the music segment.

4. The intelligent home RGBW dimming environment and background music collaborative control method according to claim 1, characterized in that Performing feature decision processing on the pitch feature and the energy feature to obtain a feature decision score, including: Performing feature standardization processing on the pitch feature and the energy feature to obtain a standard pitch and a standard energy; Calculating the interaction weight between the standard pitch and the standard energy through a preset bilinear attention model; Fusing the standard pitch and the standard energy by using the interaction weight to obtain a fused feature; Performing mean pooling on the fused feature to obtain a smoothed feature; Outputting a context-enhanced feature of the smoothed feature through a preset GRU network; Calculating the cosine similarity between the context-enhanced feature and a preset scale template; Based on the cosine similarity, calculating the feature decision score corresponding to the pitch feature and the energy feature by using the following formula: H i = c i (1 - q|E harmonic - E i |) Among them, H i represents the characteristic decision score corresponding to the i-th musical key, c i represents the cosine similarity between the context-enhanced feature and the scale template of the i-th musical key, q represents the attenuation coefficient, E i represents the preset harmonic energy proportion of the i-th musical key, E harmonic represents the energy proportion of the harmonic energy in the total spectral energy corresponding to the compressed spectrum; Wherein, the bilinear attention model includes: a = σ(W p h p + W e h e + b) Among them, a represents the interaction weight, h p represents the standard pitch, W p represents the weight value of the standard pitch, h e represents the standard energy, W e represents the weight value of the standard energy, and b represents the bias of the bilinear attention model.

5. The collaborative control method for a smart home RGBW dimming environment and background music according to claim 1, characterized in that, Constructing a tonality-color mapping library between a preset music tonality sample and preset lighting color parameters, including: Querying the basic hue, basic saturation, basic brightness and basic white light intensity corresponding to the basic music tonality in the music tonality sample from the lighting color parameters; Constructing a basic association unit between the basic music tonality, the basic hue, the basic saturation, the basic brightness and the basic white light intensity; Determining the hue range of the basic hue; Determining the hue perturbation amplitude of the basic hue by using the Gaussian distribution of the basic hue; Performing random perturbation on the basic hue based on the hue range and the hue perturbation amplitude to obtain a perturbed hue; Performing random perturbation on the basic saturation by using the following formula to obtain a perturbed saturation: Among them, S new represents the perturbation saturation, and S base represents the basic saturation. ΔS represents the saturation error value, and V base represents the basic brightness; Performing random perturbation on the basic brightness by using the following formula to obtain a perturbed brightness: Among them, V new represents the disturbance brightness, ΔV represents the brightness error value, and S base represents the basic saturation; Setting the white light intensity range and the environmental perturbation amplitude of the basic white light intensity; Performing random perturbation on the basic white light intensity by using the white light intensity range and the environmental perturbation amplitude to obtain a perturbed white light intensity; Constructing a perturbed association unit between the basic music tonality, the perturbed hue, the perturbed saturation, the perturbed brightness and the perturbed white light intensity; Randomly generating a random hue, a random saturation, a random brightness and a random white light intensity corresponding to the basic music tonality; Constructing a random association unit between the basic music tonality, the random hue, the random saturation, the random brightness and the random white light intensity; Based on a preset RGBW safety threshold and a low saturation-high brightness range, performing unit screening on the basic association unit, the perturbed association unit and the random association unit to obtain a screened basic unit, a screened perturbed unit and a screened random unit; Based on a preset ratio among the screened basic unit, the screened perturbed unit and the screened random unit, performing structural adjustment on the screened basic unit, the screened perturbed unit and the screened random unit to obtain an adjusted basic unit, an adjusted perturbed unit and an adjusted random unit; Taking the adjusted basic unit, the adjusted perturbed unit and the adjusted random unit as tonality-color units to determine the tonality-color mapping library through the tonality-color units.

6. The collaborative control method of the smart home RGBW dimming environment and background music according to claim 1, characterized in that, Performing a cross-mutation operation on the key-color units in the key-color mapping library to obtain cross-mutated units, including: Calculating the mutation probability of the key-color unit using the following formula: p new = p base + min(w1e″, p max - p base ) Among them, p new represents the mutation probability, p base represents the original mutation probability, m represents the number of times the lighting controller in the smart home has been manually adjusted during the historical period, t now represents the current moment of calculating the mutation probability, t j represents the moment of the j-th manual adjustment, d represents the attenuation coefficient affecting w1, w1 represents the weight affected by the manual adjustment, e″ represents the gain coefficient, p max represents the maximum mutation probability within the historical period, p base represents the basic mutation probability within the historical period; Calculating the crossover probability of the key-color unit using the following formula: Among them, p' new represents the crossover probability, p' base represents the basic crossover probability within the historical period, e' represents the gain coefficient when calculating the crossover probability, m represents the number of times the lighting controller in the smart home has been manually adjusted during the historical period, n total represents the total number of iterations; According to the crossover probability, performing parameter crossover operations on the same-modal parameters of different key-color units, different-modal parameters of the same key-color unit, and different-modal parameters of different key-color units to obtain crossover units; Performing a mutation operation on the crossover units according to the mutation probability to obtain cross-mutated units.

7. The collaborative control method for a smart home RGBW dimming environment and background music according to claim 1, characterized in that, Analyzing the user preference, key-color matching degree, and coherence between the cross-mutated units and the music key based on the current color parameters of the light controller in the smart home, including: Calculating the user preference of the cross-mutated unit using the following formula: Among them, UserScore represents user preference, u gene represents the color parameter of any crossover mutation unit, P user adjust represents the average value of the color parameters after being manually adjusted by the lighting controller in the historical period, P range represents the value range of the color parameter of any crossover mutation unit, I represents the category index of the color parameter of any crossover mutation unit, and n represents the number of categories of the color parameter of any crossover mutation unit; Calculating the key-color matching degree of the cross-mutated unit using the following formula: Among them, MatchScore represents the tonality-color matching degree, and u′ gene represents the vector of the color parameters of any crossover and mutation unit, and u′ template represents the template vector of the color parameters of any crossover and mutation unit; Based on the current color parameters, calculating the coherence of the cross-mutated unit using the following formula: Among them, SmoothScore represents coherence, T represents the time window length, and ΔH t = H t - H t-1 , ΔH t represents the hue change amount, H t represents the hue in the color parameters of any crossover mutation unit, H t-1 represents the current hue in the current color parameters, ΔS t represents the saturation change amount, ΔV t represents the brightness change amount, and t represents the moment index within the time window.

8. The collaborative control method for a smart home RGBW dimming environment and background music according to claim 1, characterized in that, Generating the comprehensive fitness between the cross-mutated unit and the music key using the user preference, the key-color matching degree, and the coherence, including: Respectively assigning preference weights, matching degree weights, and coherence weights to the user preference, the key-color matching degree, and the coherence; Performing weighted summation on the user preference, the key-color matching degree, and the coherence using the preference weights, the matching degree weights, and the coherence weights to obtain the comprehensive fitness.

9. The intelligent home RGBW dimming environment and background music collaborative control method according to claim 1, characterized in that, Performing color evolution on the current color parameters according to the cross-mutated unit and the comprehensive fitness to obtain evolved color parameters, including: Querying the parent unit in the cross-mutated unit; Based on the comprehensive fitness, selecting target units that meet the preset quantile in the parent units; Determining the final unit of the current color parameters using the target units; Obtaining the evolved color parameters from the final unit.

10. A smart home RGBW dimming environment and background music collaborative control system, characterized in that, The system includes: A key recognition module, configured to obtain the next music segment of the music player in the smart home, respectively extract the pitch feature and the energy feature of the music segment, perform feature decision processing on the pitch feature and the energy feature to obtain a feature decision score, and identify the music key of the music segment through the feature decision score; A unit mutation module, configured to construct a key-color mapping library between a preset music key sample and a preset light color parameter, perform a cross-mutation operation on the key-color units in the key-color mapping library to obtain cross-mutated units, where the key-color mapping library is composed of preset key-color units; A fitness generation module, configured to analyze the user preference, the key-color matching degree, and the coherence between the cross-mutated units and the music key based on the current color parameters of the light controller in the smart home, and generate the comprehensive fitness between the cross-mutated units and the music key using the user preference, the key-color matching degree, and the coherence; A music detection module, which is used to perform color evolution on the current color parameters according to the crossover and mutation unit and the comprehensive fitness to obtain evolved color parameters, and detect whether the current music segment of the music player has played to the next music segment; A collaborative control module, which is used to, when detecting that the current music segment of the music player has played to the next music segment, complete collaborative control with the next music segment based on the evolved color parameters by using the lighting controller to obtain a collaborative control result.

Images