A music fountain intelligent control method and system

Abstract

The present application belongs to the field of program control, and particularly relates to a music fountain intelligent control method and system. The method comprises: obtaining real-time state data of a music fountain system and audio data of music to be played, framing the music audio data and extracting the composite acoustic characteristics of each frame to obtain future music characteristics in a target prediction time domain; at each control sampling time, calculating a target terminal state according to the beat strength and the harmonic feature of the future music characteristics, obtaining a terminal invariant set centered on the state as a terminal constraint; constructing a performance target function, combining the transient change rate of the future music characteristics and the harmonic entropy adjustment function to adjust the weights of the punishment water type change rate and the state deviation; online solving an optimal control problem of minimizing the target function under the terminal constraint to obtain a control sequence for driving the water pump and the valve. The present application can balance the smoothness and fluency of the waterscape and the dynamic expressiveness of the music, and improve the overall artistic effect of the music fountain.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of program control, and particularly relates to a music fountain intelligent control method and system. BACKGROUND

[0002] Music fountain is a comprehensive art form integrating waterscape, light and music, realizing the deep synchronization of water type change and music melody and rhythm. The advancement of music fountain control technology directly determines the artistic presentation effect. At present, the mainstream music fountain control methods in the industry mainly include pre-programmed control and real-time control.

[0003] Pre-programmed control relies on manual design of water type and light for specific music pieces in advance, and the control logic is solidified into a program. Manual programming consumes a large amount of time and labor cost, especially for complex pieces, the production cycle often lasts for several weeks or even months; on the other hand, the solidified program is only suitable for preset pieces, and when a new piece is needed, the whole process of programming needs to be re-performed, which has poor flexibility.

[0004] Real-time control technology improves flexibility by dynamically generating control instructions to drive fountain equipment through real-time acquisition and analysis of acoustic characteristics of music. However, in the real-time control process, the acquisition of music signals, the analysis of acoustic characteristics, the control decision and the response of the physical system (pump and valve) all consume time, and the total delay formed by the superposition of the three usually exceeds the perception threshold of human eyes and ears for music picture synchronization. This delay difference makes the water type change always lag behind the music rhythm, and it is difficult to achieve true synchronization. In addition, the existing real-time control technology focuses on the surface characteristics of music such as rhythm, and it is difficult to deeply analyze the harmonic structure and emotional ups and downs of music, resulting in that the water type change can only follow the simple switching of the music rhythm, and the complexity and expressiveness of the water type cannot be adjusted according to the emotional expression of the music, for example, when the music enters the lyric section with rich harmony, the water type still maintains a single form, which is out of touch with the delicate emotions conveyed by the music. Furthermore, the control rules of the existing real-time control technology for water type are mostly fixed settings, and cannot be flexibly adjusted according to the dynamic characteristics of music. When the music rhythm is fast and the transient change is intense, the fixed water type change rate constraint will limit the rapid response of the water type, resulting in insufficient visual impact; and when the music harmony is complex and the emotion is full, the fixed target water type tracking accuracy cannot support the water type to show multi-level combined forms, and it is difficult to match the rich connotation of the music.

[0005] From the above analysis, it can be seen that although the flexibility of the existing real-time control technology is improved, it is still difficult to achieve the best balance between the smoothness of waterscape and the dynamic expressiveness of music, which limits the improvement of the overall artistic effect of music fountain. SUMMARY

[0006] In view of this, the purpose of the present application is to provide a music fountain intelligent control method and system to solve the technical problem that the existing music fountain real-time control technology is difficult to balance the smoothness of water scene and the dynamic performance of music, thereby limiting the overall artistic effect of music fountain.

[0007] To solve the above problems, the technical scheme of a music fountain intelligent control method provided by the present application is as follows:

[0008] A music fountain intelligent control method, comprising the following steps:

[0009] Obtaining real-time state data of a music fountain system and music audio data to be played; frame the music audio data, extract the composite acoustic features of each music frame, and obtain the future music features in the target prediction time domain;

[0010] At each control sampling time, based on the beat intensity and the harmonic feature of the future music feature, the target terminal state is calculated, and the terminal invariant set centered on the target terminal state is obtained as the terminal constraint;

[0011] Construct a performance target function, and adjust the weights of the performance target function for punishing the water type change rate and the water type state deviation according to the transient change rate and the harmonic entropy of the future music feature, wherein the transient change rate and the weight of the punishment water type change rate are inversely proportional, and the harmonic entropy and the weight of the punishment water type state deviation are inversely proportional;

[0012] Solving the optimal control problem of minimizing the performance target function under the terminal constraint online to obtain the control sequence of driving the water pump and valve in the music fountain.

[0013] Further, the frame of the music audio data, the extraction of the composite acoustic features of each music frame, comprises:

[0014] The music audio data is divided into music frames with a length of 25 milliseconds, and the overlapping length between adjacent music frames is 10 milliseconds;

[0015] Extracting the mel-frequency cepstral coefficient, chroma feature and spectral contrast feature of each music frame, and splicing the obtained mel-frequency cepstral coefficient, chroma feature and spectral contrast feature into a high-dimensional feature vector as the composite acoustic features of the music frame.

[0016] Further, the obtaining method of the future music features in the target prediction time domain comprises:

[0017] The mel-frequency cepstral coefficient, chroma feature and spectral contrast feature extracted from each music frame are arranged in the order of their appearance time in the music to form an acoustic feature sequence;

[0018] The acoustic feature sequence is input into an LSTM network pre-trained with massive amounts of music data. The LSTM network uses a three-layer stacked Long Short-Term Memory network as a deep temporal model. The first layer of the memory network learns to recognize the features of a single note or chord. The second layer of the memory network learns to recognize short musical phrases or rhythmic patterns based on the output of the first layer of the memory network. The third layer of the memory network understands the musical structure and emotional direction of longer passages based on this.

[0019] The LSTM network predicts a composite acoustic feature sequence within the next 2 seconds based on the historical input acoustic feature sequence. This composite acoustic feature sequence contains future music features that include beat, harmony, and timbre information.

[0020] Furthermore, the calculation of the target terminal state based on the beat intensity and harmonic features of the future music features includes:

[0021] Determine whether the normalized value of the beat intensity is greater than the preset value w1. If so, set the target terminal state to the preset climax water pattern combination.

[0022] Otherwise, determine whether the harmonic feature is a major triad. If so, set the target terminal state to a preset stable and wide water-type combination.

[0023] Preferably, the preset value w1 is in the range of 0.6. <w1<0.9。

[0024] Furthermore, obtaining the terminal invariant set centered on the target terminal state as the terminal constraint includes:

[0025] A mathematical model of the musical fountain system is established, which adopts a linear state-space model: ,in, It is the state vector of the musical fountain system at the k-th time in the future. Is the musical fountain system the next... The state vector at time t, Satisfying state constraints , and These are the minimum and maximum constraints of the state vector, respectively. It is the control input vector that satisfies the input constraints. , and These represent the minimum and maximum constraints of the input vector, respectively; A and B represent the system dynamics matrices.

[0026] Define a controller, the controller form is as follows: Used to stabilize the musical fountain system at the target terminal state. wherein, is the steady state control input required to maintain the state, is the controller gain matrix;

[0027] iteratively computing the maximal positively invariant set : through iterative computation, find the set of points that satisfy all the following conditions, this set is the maximal positively invariant set , the points in the set satisfy the state constraints; the control vectors generated by applying the points in the set to the controller satisfy the input constraints, and the state of the musical fountain system at the next moment is still located in the set.

[0028] Further, the performance objective function is:

[0029] ;

[0030] wherein, N is the length of the prediction horizon, is the desired target state vector at the future kth moment; is the weight of the water type state deviation; is the increment of the control input at the future kth moment ( - ); is the weight of the water type change rate.

[0031] Further, the adjustment of the weights for punishing the water type change rate and the water type state deviation in the performance objective function according to the transient change rate and the harmonic entropy of the future music features comprises:

[0032] When the transient change rate of the future music features is greater than a preset value w2, the weight coefficient of the water type change rate is set to q1% of the base value;

[0033] When the harmonic entropy of the future music features is less than a preset value w3, the weight coefficient of the water type state deviation is set to q2% of the base value;

[0034] wherein, 0.5<w2<1, 10<q1<50, 0<w3<0.5, 150<q2<300.

[0035] Further, the optimal control problem is solved by using the interior point method, and the optimal control sequence is obtained by solving the optimal control problem in each control sampling period.

[0036] The technical scheme of the musical fountain intelligent control system provided by the application is as follows:

[0037] The application discloses an intelligent control system for a music fountain, which comprises a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to realize an intelligent control method for the music fountain.

[0038] The application has the following advantages:

[0039] The application can obtain future music features in a target prediction time domain by frame dividing the music audio data and extracting compound acoustic features, so that the control system can predict music rhythm and harmonic change, plan control actions in advance, and reserve response time for physical devices, thereby solving the problem that water types lag behind music rhythm due to system delay, and truly realizing synchronization of water scenery and music rhythm. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 A step flowchart of the intelligent control method for the music fountain is provided. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the application will be clearly and completely described with reference to the drawings in the embodiments of the application.

[0042] A specific embodiment of the intelligent control method for the music fountain is as follows:

[0043] As shown in the drawings, Figure 1 The intelligent control method for the music fountain comprises the following steps:

[0044] S1, obtaining real-time state data of a music fountain system and music audio data to be played; frame dividing the music audio data, extracting compound acoustic features of each music frame, and obtaining future music features in a target prediction time domain.

[0045] Specifically, the music fountain system mainly comprises a water pump and a valve, the water pressure and water flow direction can be controlled through the water pump and the valve, and the actual rotating speed of the water pump, the actual opening degree of the valve and pipe pressure and other state quantities can be collected through encoders and pressure sensors installed on the water pump and the valve.

[0046] In an alternative embodiment, the music audio data is divided into frames, and a composite acoustic feature is extracted for each music frame, including:

[0047] The music audio data is divided into music frames with a length of 25 milliseconds, and an overlap length of 10 milliseconds between adjacent music frames; a mel-frequency cepstral coefficient, a chroma feature, and a spectral contrast feature are extracted for each music frame, and the three features are concatenated into a high-dimensional feature vector as the composite acoustic feature of the music frame.

[0048] In order to process continuous music signals, the music is divided into a series of short-time audio segments, i.e. music frames. The frame length is set to 25 milliseconds because in such a short time, the characteristics of the music signal can be assumed to be basically stable, which helps to perform accurate spectral analysis. At the same time, a frame overlap of 10 milliseconds is set to ensure smooth transition between frames and avoid losing important information due to window function edge effects. For example, a piece of music with a duration of 3 seconds will be divided into a sequence of about 200 such overlapping music frames.

[0049] The mel-frequency cepstral coefficient can simulate the nonlinear perception characteristics of the human ear to the frequency of the sound, and mainly reflects the tone color of the music, for example, it can distinguish the difference in tone quality when a violin and a piano play the same note. The chroma feature focuses on the pitch and harmony information of the music, which maps the energy of the entire spectrum to twelve semitones, for example, an audio frame of a C major chord will have higher energy values in the C, E, and G chroma components. The spectral contrast feature represents the tonal quality and clarity of the music by calculating the energy difference between the peaks and valleys in the spectrum, for example, the spectral contrast value of a clear flute solo will be significantly higher than that of a noisy background noise. The fusion of the above three features can obtain a more representative composite acoustic feature. For example, for a certain music frame, 13-dimensional mel-frequency cepstral coefficients, 12-dimensional chroma features, and 6-dimensional spectral contrast features are extracted. The above three vectors are concatenated in order to form a 31-dimensional high-dimensional feature vector, which not only contains tone, pitch, and harmony, but also contains information about tone clarity.

[0050] In an optional embodiment, the manner of obtaining the future music features in the target prediction time domain comprises: arranging the mel-frequency cepstral coefficients, the chroma features and the spectral contrast features extracted from each music frame in the order of their occurrence in the music to form an acoustic feature sequence; inputting the acoustic feature sequence into an LSTM network trained in advance using a large amount of music data, and the LSTM network preferably adopts a three-layer stacked long short-term memory network as a deep time sequence model, a single-layer long short-term memory network can capture short-term and long-term dependencies in the acoustic feature sequence, and a three-layer stacked network model can learn deeper time patterns. For example, the first layer of memory network learns to identify the features of a single note or chord, the second layer of memory network learns to identify short phrases or rhythm patterns based on the output of the first layer of memory network, and the third layer of memory network understands the music structure and emotional trend of a longer passage on this basis. The LSTM network predicts a composite acoustic feature sequence in the future 2 seconds based on the historical input acoustic feature sequence, and the composite acoustic feature sequence is the future music feature containing beat, harmony and timbre information.

[0051] Of course, in other embodiments, the manner of obtaining the future music features in the target prediction time domain can also be to load the audio data of a small piece of future music (for example, an audio file of 2 seconds in the future) into the data buffer in advance by the music fountain system, and the music fountain system performs the same analysis on this piece of future audio as on the real-time audio, that is, frames it and extracts its composite acoustic features, thereby directly obtaining the future music features.

[0052] S2, at each control sampling time, based on the beat intensity and the harmony features of the future music features, calculating a target terminal state, and obtaining a terminal invariant set centered on the target terminal state as a terminal constraint.

[0053] In this step, the feature vector at the end of the 2-second future predicted music feature sequence is analyzed, and if the feature vector represents high beat intensity and simple harmony, the target terminal state is set to a single main water column reaching the highest, that is, the main water pump is at 100% speed and the remaining water pumps are turned off; if the harmony is complex, it is set to a combination of multiple water columns in a staggered form according to the harmony structure; a small range of fluctuations around the target terminal state is defined as a terminal invariant set, for example, the water pump speed must be stable within plus or minus 5% of the target speed at the end of the prediction time domain.

[0054] In a preferred embodiment, the terminal invariant set centered on the target terminal state is obtained as a terminal constraint, specifically:

[0055] A mathematical model of the music fountain system is established, and the mathematical model adopts a linear state space model: wherein, is the state vector of the musical fountain system at future time k, is the state vector of the musical fountain system at future time k+1, is the state vector of the musical fountain system at future time k+1, satisfies state constraints , and are the minimum and maximum constraints of the state vector, respectively; is the control input vector, satisfying input constraints , and are the minimum and maximum constraints of the input vector, respectively; A and B represent the matrix of system dynamics;

[0056] A controller is set, which is in the form of , used to stabilize the musical fountain system at the target terminal state , where is the steady-state control input required to maintain state, is the controller gain matrix;

[0057] Iterative calculation of the maximum positive invariant set : Through iterative calculation, find the set of points that satisfy all the following conditions, and this set is the maximum positive invariant set , the points in the set satisfy the state constraints; the control vector generated by applying the points in the set to the controller satisfies the input constraints, and the state of the musical fountain system at the next time is still located in the set.

[0058] Starting from an initial constraint set, through iteration, points that do not satisfy the above conditions are continuously deleted, and the maximum positive invariant set is converged to be represented as , where H and h are the matrix and vector of the calculated safety region boundary, is a d-dimensional Euclidean space, represents the number of system state variables, for example, in the musical fountain system, if the water column height, water pump speed, and water flow rate are selected as three state variables, then 3 is taken.

[0059] For each preset target terminal state, the corresponding H and h are calculated offline and stored. When online control, after the target terminal state is determined, the corresponding H and h are directly read and used as constraint conditions added to the online optimization problem.

[0060] In yet another embodiment, a set of preset constraints are adopted to obtain a terminal invariant set centered on the target terminal state as the terminal constraint, and each target terminal state corresponds to at least one terminal constraint. For example, the terminal constraint corresponding to the climax water type includes the main water column height constraint, the total flow constraint of the pool, the main water pump power / frequency constraint, etc.

[0061] In an optional embodiment, the target terminal state is calculated based on the beat strength and the harmony feature of the future music feature, including:

[0062] It is judged whether the normalized value of the beat strength is greater than a preset value w1. If yes, the target terminal state is set to a preset climax water type combination; wherein 0.6 < w1 < 0.9.

[0063] Otherwise, it is judged whether the harmony feature is a major chord. If yes, the target terminal state is set to a preset stable wide water type combination.

[0064] When the music in the future one second will appear a strong beat point, such as the bass drum heavy hit in symphony or the drum point in rock music, the calculated beat strength normalized value is 0.9. The preset value w1 is 0.8. Since 0.9 is greater than the preset value w1, the target terminal state is set to the climax water type combination. The climax water type combination corresponds to all the nozzles being sprayed to the highest point at the same time, for example, the center main fountain reaches 30 meters, and the surrounding auxiliary fountains form a starry sky pattern to match the climax moment of the music. If the predicted beat strength value is 0.5, it does not meet the condition of being greater than the preset value w1, and the harmony feature will be analyzed. Assuming that the harmony feature of the future music is mainly composed of G, B, and D three notes, which is a G major chord, which is a bright and stable chord. According to the rules, the target terminal state is set to the stable wide water type combination. The stable wide water type combination is represented by dozens of nozzles at a medium height to form a wide and stable water curtain or fan-shaped water surface, which echoes the stable harmony of the music. In an embodiment, in addition to the above two combinations, there can be light and quick decoration type, soft and comfortable stretching type, mysterious and introverted type, etc.

[0065] S3, constructing a performance target function, and adjusting the weights of the performance target function for punishing the water type change rate and the water type state deviation according to the transient change rate and the harmonic entropy of the future music feature, wherein the weight of the transient change rate is inversely proportional to the water type change rate, and the weight of the harmonic entropy is inversely proportional to the water type state deviation.

[0066] The performance objective function is composed of two parts: one is the deviation between the water type state and the target posture mapped from the music features, and the other is the variation rate of the control vector, i.e. the rotation speed of the water pump and the opening degree of the valve. The controller obtains the transient variation rate by calculating the frame difference of the predicted music feature sequence, and obtains the harmonic entropy by analyzing the chroma distribution in the feature sequence. When the music contains strong drum beats, which cause the transient variation rate to increase, the penalty weight of the water type variation rate is reduced to allow the water column to burst instantaneously. When the music contains rich harmonies in the symphony part, which cause the harmonic entropy to increase, the penalty weight of the water type state deviation is reduced to match the music with more complex water type combinations.

[0067] In one embodiment, the performance objective function is:

[0068] ;

[0069] wherein N is the length of the prediction horizon, is the state vector of the musical fountain system at the future k-th time, which preferably contains the water column height, flow rate, water pump rotation speed, etc. of all the nozzles; is the desired target state vector at the future k-th time, which is calculated according to the beat intensity, harmony, etc. of the music. is the state weight matrix, which is an adjustable parameter, and the larger the value, the heavier the penalty for the state deviation, and is the weight of the water type state deviation. is the increment of the control input at the future k-th time - , is the control vector, such as the voltage applied to all water pumps and the opening degree command of the valve. is the control input weight matrix, which is an adjustable parameter, and the larger the value, the heavier the penalty for the control variation, which will tend to produce more gentle and smaller control commands, thereby making the water type dynamic more gentle, and is the weight of the penalty for the water type variation rate.

[0070] In an optional embodiment, adjusting the weights for penalizing the water type variation rate and the water type state deviation in the performance objective function according to the transient variation rate and the harmonic entropy of the future music features comprises:

[0071] when the transient variation rate of the future music features is greater than a preset value w2, setting the weight coefficient of the water type variation rate to q1% of the base value;

[0072] when the harmonic entropy of the future music features is less than a preset value w3, setting the weight coefficient of the water type state deviation to q2% of the base value;

[0073] wherein 0.5 < w2 < 1, 10 < q1 < 50, 0 < w3 < 0.5, and 150 < q2 < 300.

[0074] The water type change rate penalty term in the performance objective function serves to limit the rate of change of the fountain's water column height, shape, etc. The higher the weight, the more gradual the water type change. The transient change rate is a measure of the suddenness of sound energy changes in the music signal, such as the strike of a percussion instrument or the rapid switching of notes. In a calm musical passage, the weight can remain at its base value of 1.0, allowing the water type change to be smooth and fluid.

[0075] When the deep temporal model predicts that the upcoming music segment will contain dense drumbeats or cymbal strikes, the computed transient change rate value is 0.95, and the preset value w2 is 0.9. Since the transient change rate exceeds the preset value w2, the weight coefficient of the water type change rate is automatically reduced from the base value of 1.0 to 20% thereof, i.e., 0.2. The reduction in the weight means that the penalty for rapid water type change is reduced, allowing the optimization controller to generate control instructions that can drive the nozzles to perform rapid and forceful spraying, achieving the effect of the water column jumping and flickering in response to the impulsive transient events in the music. If the transient change rate is less than or equal to w2, the music is relatively calm, and the weight coefficient of the water type change rate is restored to the base value, i.e., 100%.

[0076] The water type state deviation penalty term serves to ensure that the actual state of the fountain can accurately follow the preset target state. The higher the weight, the more stringent the tracking accuracy requirement. The harmonic entropy is an indicator of sound purity, and the lower the harmonic entropy, the clearer the harmonic structure of the sound and the more pure and stable the tone, such as the long notes played by a flute or a violin. In a musical passage with complex structure, the weight can remain at its base value of 1.0. When it is predicted that the future music will enter a long and pure melody played by a clarinet, the computed harmonic entropy value is 0.2, which is lower than the preset value w3 = 0.3. This indicates that the music has entered a highly stable and harmonious stage, triggering the adjustment mechanism to increase the weight coefficient of the water type state deviation to 200% of the base value, i.e., from 1.0 to 2.0. The increase in the weight will cause the optimization controller to more strictly control the water pump and the valve, and to minimize any slight fluctuations in the water column height, thereby forming a stable waterscape. If the harmonic entropy is greater than or equal to w3, the music tone is complex or close to noise, and the weight coefficient of the water type state deviation is restored to the base value, i.e., 100%.

[0077] S4, solving the optimal control problem that minimizes the performance objective function under the terminal constraints to obtain a control sequence for driving the water pump and the valve in the music fountain.

[0078] In each control cycle, for example, every 100 milliseconds, the controller constructs a quadratic programming or nonlinear programming problem with the physical model of the musical fountain, the performance objective function, and various constraints, including terminal constraints and device physical limit constraints; then uses a numerical optimization algorithm to solve the problem to obtain the optimal control quantity sequence for each step in the next 2 seconds, for example, the pump target speed sequence for the next 20 steps; the controller only executes the first control instruction in the sequence, that is, sets the pump speed for the next 100 milliseconds, and then repeats the entire prediction and optimization process according to the latest system state and music data in the next control cycle to achieve rolling optimization control.

[0079] In an optional embodiment, the optimal control problem is numerically solved by using an interior point method, and the solution of the optimal control problem is obtained in each control sampling cycle to obtain an optimal control sequence.

[0080] For example, assuming that the sampling period of the control system is set to 100 milliseconds, every 100 milliseconds, the controller must predict the current state of the fountain according to the latest music characteristics and recalculate an optimal control scheme for a period of time in the future. The interior point method is particularly suitable for solving the constraint optimization problem commonly encountered in model predictive control. At the beginning of each 100-millisecond control cycle, the constructed performance objective function, fountain dynamics model, and terminal state constraints are input into a solver based on the interior point method. The solver finds a control input sequence that minimizes the future performance objective function value, for example, the pump frequency and valve opening sequence for the next 20 steps, that is, 2 seconds, through iterative calculation within milliseconds. The controller only executes the first control instruction in the sequence, and then repeats the entire process when the next 100-millisecond cycle arrives.

[0081] The present application plans control actions by using a model to analyze future music characteristics in advance, compensates for the response delay of the physical system, and ultimately realizes true synchronization of water type changes and music rhythm. In addition, the present application can deeply mine the harmony and emotional connotation of music by extracting and analyzing the composite acoustic characteristics of music, so that the changes of water type are more in line with the expression of music emotions. Furthermore, the system can also adaptively adjust the control target according to the characteristics such as the transient change rate and harmonic complexity of music, and ultimately significantly improves the overall artistic effect and ornamental value of the musical fountain.

[0082] The present application provides a specific embodiment of a musical fountain intelligent control system:

[0083] The musical fountain intelligent control system comprises a processor and a memory, and the memory stores a computer program. The processor realizes the musical fountain intelligent control method in the above embodiment by executing the computer program.

[0084] The music fountain intelligent control system further comprises other components well known to those skilled in the art such as a communication bus and a communication interface, the settings and functions of which are known in the art and thus will not be described here.

[0085] In this application, the aforementioned memory can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. For example, the computer-readable storage medium can be any suitable magnetic storage medium or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high bandwidth memory (HBM), hybrid memory cube (HMC), and the like, or any other medium that can be used to store the desired information and that can be accessed by an application, module, or both. Any such computer storage media can be part of a device or accessible or connectable thereto.

[0086] Although the present specification has shown and described a number of embodiments of the application, it will be apparent to those skilled in the art that many modifications, changes and substitutions can be made thereto without departing from the spirit and scope of the present application.

Claims

1. A method for intelligent control of a musical fountain, characterized in that, Includes the following steps: Acquire real-time status data of the musical fountain system and audio data of the music to be played; divide the audio data into frames, extract the composite acoustic features of each music frame, and obtain the future music features in the target prediction time domain; At each control sampling time, based on the beat intensity and harmonic features of future music characteristics, the target terminal state is calculated, and the terminal invariant set centered on the target terminal state is obtained as the terminal constraint. Construct a performance objective function, and adjust the weights of water pattern change rate and water pattern state deviation in the performance objective function according to the transient change rate and harmonic entropy of future music features. The transient change rate is inversely proportional to the weight of the water pattern change rate, and the harmonic entropy is inversely proportional to the weight of the water pattern state deviation. The optimal control problem that minimizes the performance objective function under terminal constraints is solved online, resulting in the control sequence for driving the water pumps and valves in the musical fountain. Based on the beat intensity and harmonic features of future music characteristics, calculate the target terminal state, including: determining whether the normalized value of the beat intensity is greater than the preset value w1. If so, set the target terminal state to the preset climax water pattern combination; otherwise, determine whether the harmonic feature is a major triad. If so, set the target terminal state to the preset stable and broad water pattern combination. Based on the transient rate of change and harmonic entropy of future music features, adjust the weights in the performance objective function used to penalize the rate of change of water patterns and the deviation of water pattern state, including: when the transient rate of change of future music features is greater than a preset value w2, set the weight coefficient of the rate of change of water patterns to the base value q1; when the harmonic entropy of future music features is less than a preset value w3, set the weight coefficient of the deviation of water pattern state to the base value q2; where 0.5 <w2<1,10<q1<50,0<w3<0.5,150<q2<300; If the transient rate of change is less than or equal to w2, the weighting coefficient of the water pattern rate of change is restored to the base value; if the harmonic entropy is greater than or equal to w3, the weighting coefficient of the water pattern state deviation is restored to the base value.

2. The intelligent control method for a musical fountain according to claim 1, characterized in that, The music audio data is segmented into frames, and the composite acoustic features of each music frame are extracted, including: The music audio data is divided into music frames with a length of 25 milliseconds, and the overlap length between adjacent music frames is 10 milliseconds. Mel frequency cepstral coefficients, chromaticity features, and spectral contrast features of each music frame are extracted, and the obtained Mel frequency cepstral coefficients, chromaticity features, and spectral contrast features are concatenated into a high-dimensional feature vector, which serves as the composite acoustic feature of the music frame.

3. The intelligent control method for a musical fountain according to claim 2, characterized in that, Methods for obtaining future music features within the target prediction time domain include: The Mel frequency cepstral coefficients, chromaticity features, and spectral contrast features extracted from each music frame are arranged in chronological order of their appearance in the music to form an acoustic feature sequence. The acoustic feature sequence is input into an LSTM network pre-trained with massive amounts of music data. The LSTM network uses a three-layer stacked Long Short-Term Memory network as a deep temporal model. The first memory network learns to recognize the features of a single note or chord, the second memory network learns to recognize short musical phrases or rhythmic patterns based on the output of the first memory network, and the third memory network understands the musical structure and emotional direction of longer passages based on this. The LSTM network predicts a composite acoustic feature sequence within the next 2 seconds based on the historical input acoustic feature sequence. This composite acoustic feature sequence contains future music features that include beat, harmony, and timbre information.

4. The intelligent control method for a musical fountain according to claim 3, characterized in that, The default value w1 has a range of 0.

6. <w1<0.9。 5. A method for intelligent control of a musical fountain according to claim 3 or 4, characterized in that, Obtain the terminal invariant set centered on the target terminal state as the terminal constraint, including: A mathematical model of the musical fountain system is established, which adopts a linear state-space model: ,in, It is the state vector of the musical fountain system at the k-th time in the future. Is the musical fountain system the next... The state vector at time t, Satisfying state constraints , and These are the minimum and maximum constraints of the state vector, respectively. It is the control input vector that satisfies the input constraints. , and These represent the minimum and maximum constraints of the input vector, respectively; A and B represent the system dynamics matrices. Define a controller, the controller form is as follows: Used to stabilize the musical fountain system at the target terminal state. ,in, Is it maintained at The steady-state control input required for the state It is the controller gain matrix; Iterative computation of the maximum positive invariant set The set of points that satisfy all of the following conditions is found through iterative calculation; this set is the maximal positive invariant set. The points within the set satisfy the state constraints; when the points within the set are applied to the controller, the resulting control vector satisfies the input constraints, and the state of the musical fountain system at the next moment still lies within the set.

6. The intelligent control method for a musical fountain according to claim 5, characterized in that, The performance objective function is: ； Where N is the length of the prediction time domain, Let be the desired target state vector at the k-th future time. The weight of the water type state deviation; To determine the increment of the control input at time k in the future ( ); The weight of the rate of change of water type.

7. The intelligent control method for a musical fountain according to claim 6, characterized in that, The optimal control problem is solved numerically using the interior point method. The optimal control problem is solved within each control sampling period to obtain the optimal control sequence.

8. A smart control system for a musical fountain, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the intelligent control method for a musical fountain as described in any one of claims 1 to 7.

Images