A dynamic lighting effect control method for piano lamp

Abstract

The present application provides a method for controlling dynamic lighting effects of a piano lamp, comprising: using an audio acquisition device to obtain a real-time audio signal of a piano performance, performing spectrum analysis on the audio signal through a Fourier transform algorithm, and obtaining frequency distribution characteristics of the piano sound; according to the spectrum analysis result, using a peak detection algorithm to identify notes in the audio signal, calculating the start time, duration and pitch of each note, and constructing a note sequence data structure; predicting the frequency change of the note sequence through a Kalman filter algorithm, calculating the inertial characteristics of the note frequency, and obtaining the frequency change trend and speed; using a graphics rendering engine to generate a dynamic effect of light projection, controlling the rhythm of light changes according to the time information of the note sequence, and realizing the synchronization of sound and light.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a method for controlling dynamic lighting effects of a piano lamp. Background Art

[0002] The dynamic lighting effect of the piano lamp can bring a good experience of light and shadow combination to the performer and the audience during piano performance. In the process of realizing the dynamic effect control of the piano lamp, the piano sound frequency inertia analysis and light projection dynamic continuation system face a key technical problem in the implementation process: how to accurately capture and analyze the slight changes and continuity characteristics of the piano sound frequency in a real-time performance environment. Due to the rich timbre of the piano, the large frequency span between different notes, and the various variations in playing speed and intensity, the traditional spectrum analysis method is difficult to accurately identify the subtle transitions between notes. At the same time, there is a time delay between the sound frequency analysis results and the light projection control, which affects the synchronization of the audio-visual experience. In addition, the light projection system needs to present a continuous dynamic effect that matches the sound frequency changes within a limited projection area, which puts higher requirements on the real-time and smoothness of the projection algorithm. How to extract effective frequency features in a complex acoustic environment and quickly convert them into smooth light dynamic effects while ensuring the low latency and high precision of the system is the core challenge facing this technology. Solving this problem requires in-depth research and innovation in multiple fields such as sound signal processing, machine learning, and real-time control. Summary of the invention

[0003] The present invention provides a method for controlling dynamic lighting effects of a piano lamp, which mainly includes:

[0004] The audio acquisition device is used to obtain the real-time audio signal of the piano performance, and the frequency distribution characteristics of the piano sound are obtained by performing spectrum analysis on the audio signal through the Fourier transform algorithm;

[0005] According to the spectrum analysis results, the peak detection algorithm is used to identify the notes in the audio signal, calculate the start time, duration and pitch of each note, and construct the note sequence data structure;

[0006] For note sequence data, the sliding window technology is applied to calculate the time interval and pitch difference between adjacent notes, judge the continuity and transition characteristics of the notes, and generate the note coherence index;

[0007] The frequency change of the note sequence is predicted by the Kalman filter algorithm, the inertial characteristics of the note frequency are calculated, and the frequency change trend and speed are obtained;

[0008] Based on the note continuity index and frequency inertia characteristics, the dynamic change rules of light projection are designed to map the continuity and frequency inertia of notes to the color, brightness and position parameters of the light.

[0009] A graphics rendering engine is used to generate dynamic effects of light projection, and the rhythm of light changes is controlled according to the time information of the note sequence to achieve synchronization between sound and light;

[0010] The lighting control instructions are sent to the projection device on the piano lamp through the network communication module, driving the projection device to adjust the projection image in real time to present dynamic lighting effects that match the piano performance.

[0011] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0012] The present invention discloses a method for processing real-time audio signals of piano performance and generating dynamic changes in light projection. In this business scenario, the present invention obtains real-time audio signals of piano performance through an acquisition device, performs spectrum analysis on the audio signals using a Fourier transform algorithm, and identifies notes in the audio signals through a peak detection algorithm. In order to ensure accurate note data, the present invention calculates the start time, duration, and pitch of each note, and constructs a note sequence data structure. Subsequently, the present invention uses a sliding window technique to calculate the time interval and pitch difference between adjacent notes, thereby determining the continuity and transition characteristics of the notes, and generating a note coherence index. Based on these data, the present invention predicts the frequency changes of the note sequence through a Kalman filter algorithm, thereby calculating the inertial characteristics of the note frequency, and obtaining the trend and speed of the frequency change.

[0013] The unique problem for this business scenario is how to transform the real-time analyzed audio data into visual effects, namely dynamic lighting effects. The present invention maps the continuity and frequency inertia of the notes to the color, brightness and position parameters of the lights by designing the dynamic change rules of the light projection, thereby achieving the synchronization of sound and light. In this process, the present invention uses a graphics rendering engine to generate the dynamic effects of light projection, and controls the rhythm of light changes according to the time information of the note sequence. Finally, the light control instructions are sent to the projection device on the piano lamp through the network communication module, driving the projection device to adjust the projection picture in real time, presenting a dynamic lighting effect that matches the piano performance.

[0014] In general, the present invention realizes the effective combination of high-precision processing of audio signals and real-time dynamic visual performance. This technical effect not only improves the audience's viewing experience and enhances the expressiveness and attractiveness of music performances, but also provides an innovative interactive method for live performances, making the combination of music and visual arts closer and more diversified. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The present invention is a flow chart of a method for controlling dynamic lighting effects of a piano lamp.

[0016] Figure 2It is a schematic diagram of a method for controlling dynamic lighting effects of a piano lamp according to the present invention.

[0017] Figure 3 It is another schematic diagram of a method for controlling dynamic lighting effects of a piano lamp according to the present invention. DETAILED DESCRIPTION

[0018] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and embodiments. The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are only used to explain the relevant inventions, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0019] like Figure 1-3 In this embodiment, a method for controlling dynamic lighting effects of a piano lamp specifically includes:

[0020] Step S101, using an audio acquisition device to obtain a real-time audio signal of a piano performance, performing spectrum analysis on the audio signal using a Fourier transform algorithm, and obtaining frequency distribution characteristics of the piano sound.

[0021] The real-time audio signal of the piano performance is obtained by audio acquisition equipment, and the analog audio signal is converted into a digital signal through an analog-to-digital converter. According to the Nyquist sampling theorem, the sampling frequency needs to be greater than twice the highest frequency of the audio signal to avoid aliasing distortion. After obtaining the digital audio signal, the fast Fourier transform algorithm is used to perform spectrum analysis on the audio signal to convert the time domain signal into a frequency domain signal. Through spectrum analysis, the frequency distribution characteristics of the piano sound are obtained, including fundamental frequency, overtone and other frequency components. According to the standard frequency of the piano notes, the notes corresponding to the main frequency components in the spectrum are determined. The pitch detection algorithm based on frequency recognition is used to determine the dominant frequency in each time frame and map it to the corresponding MIDI note number. Through continuous analysis of the time frame, a complete MIDI note sequence is obtained to represent the melody and chord information of the piano performance. According to the MIDI note sequence, the score generation algorithm is used to generate the score representation of the piano performance, including pitch, duration, dynamics and other information. Through the score alignment algorithm, the generated score is compared with the standard score to calculate the accuracy and expressiveness index of the performance. According to the evaluation indicators, the performer's skills and level are evaluated and fed back.

[0022] Exemplarily, in the process of real-time audio signal acquisition of piano performance, a high-fidelity condenser microphone can be used, and the microphone can be placed 20 cm above the piano soundboard to obtain a clear audio signal. The sampling frequency is set to 41kHz, which is twice higher than the fundamental frequency of the highest piano note, 4186Hz, to avoid the generation of aliasing distortion. The collected analog audio signal is converted into a digital signal by a 24-bit analog-to-digital converter to ensure the dynamic range and signal-to-noise ratio of the signal. The digital audio signal is subjected to a fast Fourier transform, and the Hanning window function is used for windowing. The window length is set to 2048 sampling points, and the corresponding frequency resolution is about 25Hz. Through spectrum analysis, the energy distribution characteristics of the piano sound on the fundamental frequency and overtones are obtained. According to the standard frequency of the 88 keys of the piano, a mapping table of frequency and MIDI note number is established. The frequency-based pitch detection algorithm YIN is used to calculate the fundamental frequency of each time frame through the autocorrelation function, and quantize it to the closest MIDI note number. By continuously analyzing the time frame of the audio signal, a complete MIDI note sequence is obtained, which reflects the pitch, duration and dynamic changes of the piano performance. Based on the MIDI note sequence, the score generation algorithm is used to consider the note duration, dynamics, speed and other parameters to generate a five-line score representation that conforms to the rules of music theory. Through the dynamic time warping algorithm, the generated score is aligned with the standard score, and the performance deviation and dynamic error of each note are calculated to obtain quantitative indicators of performance accuracy and expressiveness. Finally, based on the evaluation indicators, the performer's skill level is comprehensively evaluated, and targeted practice suggestions and feedback are provided to help performers improve the quality of their performance.

[0023] Step S102, based on the spectrum analysis result, a peak detection algorithm is used to identify the notes in the audio signal, calculate the start time, duration and pitch of each note, and construct a note sequence data structure.

[0024] According to the spectrum analysis results of the audio signal, the peak detection algorithm is used to identify the notes in the audio signal. The peak detection algorithm sets a threshold and determines the peak position in the spectrum that exceeds the threshold as the frequency of the note. According to the note frequency obtained by the spectrum analysis, the fundamental frequency estimation algorithm is used to calculate the fundamental frequency of each note, and then the pitch of the note is determined. By analyzing the change of the note frequency over time, the start time and duration of the note are determined. The start time, duration and pitch of the identified note are constructed into a note sequence data structure. The note sequence data structure contains the start time, duration, pitch and other attributes of each note, as well as the order relationship between the notes. By analyzing the note sequence data structure, high-level music information such as melody and rhythm in the audio signal is obtained.

[0025] Exemplarily, the audio signal is first subjected to spectrum analysis, and the time domain signal is converted into a frequency domain signal using the Fast Fourier Transform (FFT) algorithm to obtain a spectrum diagram. Then, the peak detection threshold is set to 20% of the spectrum amplitude, and the frequency exceeding the threshold is determined as the frequency of the note. For example, if a peak is detected near the frequency of the C4 note of 266Hz, it is considered that the audio contains the C4 note. Next, the fundamental frequency of the spectrum of each note is estimated using the autocorrelation method, and the fundamental frequency of the note is determined by finding the first peak position of the autocorrelation function. If the fundamental frequency of the C4 note is estimated to be 260Hz, the pitch of the note is C4. By analyzing the change of the note frequency over time, the start time and duration of the note are determined with 50ms as the time window. For example, if the C4 note is detected in the time period from 2s to 8s, the start time of the C4 note is 2s and the duration is 6s. Finally, the identified note information is constructed as a note sequence data structure. The start time, duration, pitch and other attributes of each note are stored in chronological order in the structure. By analyzing the note sequence, high-level information such as melody and rhythm in the audio can be obtained. For example, by counting the duration distribution of each note, it can be determined that the rhythm of the audio is 4 / 4. For another example, by analyzing the change trend of the pitch in the note sequence, it can be determined that the melody is rising.

[0026] Step S103, for the note sequence data, the sliding window technology is applied to calculate the time interval and pitch difference between adjacent notes, determine the continuity and transition characteristics of the notes, and generate a note continuity index.

[0027] According to the input note sequence data, the sliding window technology is used to obtain the time interval and pitch difference between adjacent notes. By setting the time interval threshold and pitch difference threshold, it is judged whether the adjacent notes are continuous. If the time interval between adjacent notes is less than or equal to the time interval threshold, and the pitch difference is less than or equal to the pitch difference threshold, the two notes are considered to be continuous, otherwise they are considered to be discontinuous. For continuous note sequences, the Markov chain model is used to calculate the transition probability matrix between notes. The transition characteristics of the note sequence are obtained through the transition probability matrix. According to the transition characteristics, the coherence index of the note sequence is determined. The coherence index can be used to determine whether the note sequence is coherent by setting a threshold. If the coherence index is greater than or equal to the threshold, the note sequence is considered to be coherent, otherwise it is considered to be incoherent. For discontinuous note sequences, the dynamic time warping algorithm is used to align and regularize the discontinuous note sequences. The time interval and pitch difference between adjacent notes are recalculated through the aligned and regularized note sequences, and the above steps are repeated until all note sequences are judged to be continuous or discontinuous. Finally, the coherence index of the entire music clip is calculated based on the coherence index of all note sequences. By setting the overall coherence index threshold, it is determined whether the entire music clip is coherent. If the overall coherence index is greater than or equal to the threshold, the music clip is considered coherent, otherwise it is considered incoherent.

[0028] Exemplarily, in the note sequence data, the time interval threshold can be set to 5 seconds and the pitch difference threshold can be set to 2 semitones. Through the sliding window technology, the time interval and pitch difference between adjacent notes are calculated in sequence with a step size of 1 second. For example, for a certain note sequence, the time interval between adjacent notes is 3 seconds and the pitch difference is 1 semitone, then the two notes are considered to be continuous. For a continuous note sequence, a Markov chain model is used to calculate the transition probability matrix between notes. Assuming that in the transition probability matrix, the transition probability from note A to note B is 8, and the transition probability from note B to note C is 6, the transition characteristics of the note sequence can be obtained. According to the transition characteristics, the coherence index threshold is set to 7. If the coherence index is greater than or equal to 7, the note sequence is considered to be coherent. For discontinuous note sequences, a dynamic time warping algorithm is used to calculate the optimal alignment between note sequences through dynamic programming, and the discontinuous note sequences are aligned and regularized. The time interval and pitch difference between adjacent notes are recalculated, and the above steps are repeated until all note sequences are judged to be continuous or discontinuous. Finally, according to the coherence index of all note sequences, the overall coherence index threshold is set to 8, and the coherence index of the entire music clip is calculated. If the overall coherence index is greater than or equal to 8, the entire music clip is considered coherent, otherwise it is considered incoherent. This method can effectively judge the coherence of the music clip and provide an important reference for music analysis and processing.

[0029] Step S104, predicting the frequency change of the note sequence through the Kalman filter algorithm, calculating the inertial characteristics of the note frequency, and obtaining the frequency change trend and speed.

[0030] According to the frequency change of the note sequence, the Kalman filter algorithm is used to predict the frequency change of the note sequence. First, the frequency data of the note sequence is obtained and used as the input of the Kalman filter algorithm. Then, the frequency change of the note sequence is modeled by the Kalman filter algorithm to obtain the state equation and observation equation of the note frequency change. Then, according to the state equation and observation equation, the Kalman filter algorithm is used to predict the frequency change of the note sequence to obtain the predicted value of the note frequency at the future moment. On the basis of predicting the note frequency change, the inertial characteristics of the note frequency are calculated. The speed of the note frequency change is obtained by the first-order difference of the note frequency prediction value. Then, the acceleration of the note frequency change is obtained by the second-order difference of the note frequency prediction value. The speed and acceleration of the note frequency change are used as the inertial characteristics of the note frequency. According to the predicted value and inertial characteristics of the note frequency, the change trend and speed of the note frequency are judged. If the note frequency prediction value shows an upward trend and the change speed is fast, it is judged that the note frequency change trend is rising and the change speed is fast. If the note frequency prediction value shows a downward trend and the change speed is slow, it is judged that the note frequency change trend is falling and the change speed is slow. By analyzing the predicted value of note frequency and inertial characteristics, the changing trend and speed of note frequency are obtained.

[0031] Exemplarily, first obtain a sequence of notes played on a piano, which contains 100 notes, and the frequency range of each note is between 266Hz and 1045Hz. Input these frequency data into the Kalman filter algorithm to establish the state equation and observation equation of note frequency change. The state equation describes the dynamic process of note frequency change, assuming that the change of note frequency obeys a first-order Markov process, that is, the frequency at the current moment is only related to the frequency at the previous moment. The observation equation describes the relationship between the observed frequency and the true frequency, assuming that the observation noise obeys a Gaussian distribution. Through the Kalman filter algorithm, the optimal estimate of the note frequency at each moment and the covariance matrix of the estimation error are estimated. The first-order difference and second-order difference of the note frequency are calculated based on the estimated value to obtain the speed and acceleration of the note frequency change. For example, at the 50th note, the predicted frequency is 523Hz. Compared with the predicted frequency at the previous moment, the first-order difference is 12Hz, indicating that the note frequency rises faster; compared with the predicted frequency at the previous two moments, the second-order difference is 5Hz, indicating that the speed of note frequency rise has slowed down. According to these inertial characteristics, the change trend and speed of the note frequency are judged. Between the 50th note and the 80th note, the note frequency prediction value shows a continuous upward trend, the first-order difference is greater than 5Hz, and the second-order difference is greater than 1Hz, so it is judged that the note frequency change trend during this period is a rapid increase. After the 80th note, the note frequency prediction value shows a downward trend, the first-order difference is less than -3Hz, and the second-order difference is less than -5Hz, so it is judged that the note frequency change trend during this period is a slow decrease. By analyzing the note frequency prediction value and inertial characteristics, the change of note frequency can be judged in real time, providing a basis for the analysis and generation of musical expressions.

[0032] Step S105, based on the note continuity index and frequency inertia characteristics, design the dynamic change rules of light projection, and map the continuity and frequency inertia of the notes to the color, brightness and position parameters of the light.

[0033] According to the coherence index and frequency inertia characteristics of musical notes, the convolutional neural network algorithm is used to extract the features of the note sequence to obtain the note coherence index and frequency inertia feature vector. The extracted feature vectors are classified by the support vector machine algorithm to determine the coherence and frequency inertia degree of the note sequence. If the coherence index of the note sequence is higher than the preset threshold, the note coherence is mapped to the continuous gradient of the light color; if the coherence index of the note sequence is lower than the preset threshold, the note coherence is mapped to the random change of the light color. According to the frequency inertia feature vector of the note sequence, the K-means clustering algorithm is used to cluster the frequency inertia features to obtain the cluster centers under different frequency inertia levels. The frequency inertia level of the current note is determined by the Euclidean distance between the cluster center and the frequency inertia feature of the current note. If the frequency inertia level of the current note is high, the frequency inertia is mapped to the increase of the light brightness; if the frequency inertia level of the current note is low, the frequency inertia is mapped to the decrease of the light brightness. According to the time position of the notes in the music clip, the linear interpolation algorithm is used to calculate the corresponding light projection position coordinates to obtain the mapping relationship between the note time position and the light projection position. By integrating the light color change rules mapped by the note continuity index, the light brightness change rules mapped by the frequency inertia feature, and the light projection position coordinates mapped by the note time position, the weighted average algorithm is used to determine the final light projection dynamic change parameters, including color, brightness, and position coordinates. According to the determined light projection dynamic change parameters, the light projection equipment is controlled to adjust the color, brightness, and position of the projection light in real time to achieve the synchronous mapping of the music note characteristics and the dynamic changes of the light. During the music playback process, the note sequence is continuously obtained, and the above steps are repeated to dynamically update the light projection effect to form a light art expression that echoes the rhythm and emotion of the music.

[0034] Exemplarily, based on the coherence index and frequency inertia characteristics of musical notes, a convolutional neural network algorithm is used to extract features from the note sequence. First, the note sequence is converted into a frequency-time matrix with a matrix size of 128×128, representing 128 time steps and 128 frequency intervals. Then, three convolution layers and two pooling layers are used to extract features from the matrix. The convolution kernel sizes are 3×3, 5×5, and 7×7, respectively. Each convolution layer is followed by a ReLU activation function. The pooling layer uses maximum pooling, with a pooling kernel size of 2×2 and a step size of 2. After convolution and pooling operations, a 256-dimensional feature vector is obtained, which represents the coherence index and frequency inertia characteristics of the note sequence. Next, the extracted feature vector is input into a support vector machine for classification, using a Gaussian kernel function with a penalty coefficient C of 10 to determine the coherence and frequency inertia of the note sequence. If the coherence index is higher than 8, the note coherence is mapped to a continuous gradient of light color. The speed of color change is proportional to the coherence index. The higher the index, the smoother the color change. If the coherence index is lower than 8, the note coherence is mapped to a random change of light color. The change range is the RGB color domain. For the frequency inertia feature, the K-means clustering algorithm is used for clustering. The number of clusters K is set to 3 to obtain the cluster centers of the three frequency inertia levels of low, medium and high. The Euclidean distance between the frequency inertia feature of the current note and each cluster center is calculated. The cluster with the smallest distance is the frequency inertia level to which the current note belongs. If it belongs to the high frequency inertia level, the light brightness is increased by 20%; if it belongs to the low frequency inertia level, the light brightness is reduced by 20%. According to the time position of the note in the 4-minute music clip, the time is normalized to the range of 0-1, and then multiplied by the length of the light projection area to obtain the light projection position coordinates corresponding to the note time position. Finally, the final light projection parameters are determined by weighted average algorithm based on the light color change mapped by the note coherence index, the light brightness change mapped by the frequency inertia feature, and the light projection position coordinates mapped by the note time position. The weight of color change is 4, the weight of brightness change is 35, and the weight of position change is 25. By adjusting the color, brightness and position of the light in real time, the synchronous mapping of the musical note characteristics and the dynamic changes of the light is achieved, creating an immersive light art atmosphere that echoes the rhythm and emotion of the music.

[0035] Step S106, using a graphics rendering engine to generate a dynamic effect of light projection, controlling the rhythm of light changes according to the time information of the note sequence, and achieving synchronization of sound and light.

[0036] According to the time information of the note sequence, the Fourier transform algorithm is used to perform frequency domain analysis on the audio signal to obtain the frequency and amplitude characteristics of the audio. The audio frequency is mapped to the corresponding light color through the frequency mapping algorithm, and the light brightness is determined according to the amplitude. A three-dimensional graphics rendering engine is used to build a virtual scene, and the light projection effect is dynamically generated according to the audio frequency and amplitude characteristics. The audio timeline is aligned with the light animation timeline through the time synchronization algorithm to achieve the synchronous change of sound and light. During the rendering process, the color and brightness of the light projection are dynamically adjusted according to the change of audio frequency, and the smooth transition of the light effect is achieved through the interpolation algorithm. At the same time, the space mapping algorithm is used to map the light position in the virtual scene to the actual physical space, and the actual light equipment is controlled by the light control system to present the virtual light effect in the physical environment. Finally, the light effect is dynamically adjusted through the real-time feedback algorithm, and the light projection is updated in real time according to the change of the audio signal to maintain the synchronization and consistency of sound and light.

[0037] Exemplarily, first, the audio signal is sampled, the sampling frequency is 41kHz, and the sampling bit number is 16 bits. Then, the sampled audio data is divided into frames with a length of 1024 sampling points, and each frame of data is subjected to a fast Fourier transform (FFT) to obtain the frequency spectrum of the frame. Next, the frequency spectrum is divided into 20 frequency bands, each frequency band corresponds to a color, for example, the low frequency band corresponds to red, the medium frequency band corresponds to green, and the high frequency band corresponds to blue. According to the energy size of each frequency band, it is mapped to the brightness value of the corresponding color. The greater the energy, the higher the brightness. A virtual scene is created in the Unity3D engine, and the scene contains a sphere model to represent the light projection. According to the frequency and amplitude characteristics of the audio, the color and brightness of the sphere are dynamically adjusted. For example, when the low-frequency energy is dominant, the sphere is red, and when the high-frequency energy is dominant, the sphere is blue. Through the linear interpolation algorithm, a smooth transition is made between different colors to avoid the color change being too abrupt. In order to achieve the synchronization of sound and light, the audio timeline needs to be aligned with the light animation timeline. Assuming that the duration of the audio is 120 seconds and the frame rate of the light animation is 30 frames per second, the total number of frames of the light animation is 3600. Through the time mapping function, the playback progress of the audio is mapped to the frame index of the light animation to achieve synchronous changes of the two. In the actual physical environment, the lighting equipment is controlled by the DMX512 protocol. According to the position of the sphere in the virtual scene, the corresponding DMX address is calculated, and the color and brightness value of the sphere are sent to the corresponding lighting equipment, so that the virtual lighting effect is presented in the real environment. At the same time, through the real-time feedback mechanism, the lighting effect is dynamically adjusted according to the changes in the audio signal to maintain the synchronization of sound and light. For example, when the audio energy suddenly increases, the brightness of the light also increases, and when the audio energy decreases, the brightness of the light also decreases. Through this real-time feedback, dynamic lighting effects synchronized with the rhythm of the music can be created.

[0038] Step S107, sending the lighting control instruction to the projection device on the piano lamp through the network communication module, driving the projection device to adjust the projection picture in real time to present a dynamic lighting effect that matches the piano performance.

[0039] According to the notes and rhythm information of piano playing, the convolutional neural network algorithm is used to extract and analyze the features of audio data to obtain the time, force, pitch and other parameters of the notes. The extracted audio features are classified and identified by the support vector machine algorithm to determine the type of notes and the playing style of the current performance. According to the identified note type and playing style, the decision tree algorithm is used to generate the matching lighting control instructions, including parameters such as light color, brightness, and flashing frequency. The generated lighting control instructions are sent to the projection device on the piano lamp in real time through the network communication module. After the projection device receives the lighting control instructions, the optical flow algorithm is used to parse the instructions to obtain the specific parameters of the lighting effect. According to the parsed lighting parameters, the lighting effect is rendered in real time to the specified position on the piano surface through the projection mapping algorithm to form a dynamic lighting projection that matches the piano performance. In the process of light projection, the Kalman filter algorithm is continuously used to filter the audio data to remove noise interference and improve the accuracy of note recognition. At the same time, the adaptive gain control algorithm is used to adjust the light brightness in real time to ensure that the best lighting effect can be presented under different ambient lighting conditions. Through the cyclic iteration of the above steps, the real-time synchronization of piano performance and light projection is achieved, creating an immersive music performance atmosphere. During the entire process, a long short-term memory neural network algorithm is used to learn and memorize historical note data and lighting control instructions, and continuously optimize the lighting effect generation strategy, so that the light projection can be dynamically adapted to the piano playing style and enhance the artistic expression of the performance.

[0040] Exemplarily, during the piano playing process, the piano audio signal is collected in real time by an audio acquisition device with a sampling frequency of 41kHz and a bit depth of 16bit. The collected audio data is preprocessed by a pre-emphasis filter, and then the Mel frequency cepstral coefficient (MFCC) is used to extract the audio data to extract the 13-dimensional MFCC feature vector. The extracted MFCC feature vector is input into a convolutional neural network consisting of 3 convolutional layers and 2 fully connected layers for feature analysis. The convolution kernel sizes are 3x3, 4x4 and 5x5 respectively. After each convolution layer, the ReLU activation function and the maximum pooling operation are used. The time, intensity and pitch parameters of the notes are obtained through network training. The note feature parameters output by the convolutional neural network are input into the support vector machine (SVM) classifier for the identification of note types and playing styles. The SVM uses the RBF kernel function and finds the optimal penalty factor C and kernel function parameter g through the grid search optimization algorithm, so that the recognition accuracy of the classifier reaches more than 95%. According to the note type and playing style identified by SVM, the decision tree algorithm is used to generate the matching lighting control instructions. The feature selection of the decision tree adopts the Gini index. The decision tree is pruned and optimized by 10-fold cross validation, so that the matching degree between the generated lighting control instructions and the note type reaches more than 90%. The lighting control instructions generated by the decision tree are sent to the piano light projection device in real time through the network communication module. The projection device performs image rendering. After receiving the control instructions, the optical flow algorithm is used to parse the instructions. According to the parameters such as light color, brightness, and flashing frequency contained in the instructions, the projection mapping algorithm is used to render the lighting effect to the piano surface in real time. The rendering frame rate reaches 60fps, and the lighting projection and piano performance are precisely synchronized. During the lighting rendering process, the Kalman filter algorithm is continuously used to filter the audio data, and the order and bandwidth of the filter are adaptively adjusted to effectively remove environmental noise interference, so that the accuracy of note recognition is increased by more than 5%. At the same time, the adaptive gain control algorithm is used to adjust the light brightness in real time. According to the ambient light intensity and the reflective characteristics of the piano surface material, the brightness value of the light projection is dynamically adjusted. The brightness adjustment range is 0-255, ensuring that the best lighting effect can be presented under different lighting conditions. Finally, a long short-term memory network composed of a three-layer LSTM neural network is used to learn and memorize historical note data and lighting control instructions. The number of hidden layer nodes of LSTM is 128, 256 and 128 respectively. Through continuous parameter tuning and model iteration, the lighting effect generation strategy can be dynamically adapted to the piano playing style. During the performance, the synchronization rate of light projection and music rhythm reaches more than 98%, which greatly enhances the artistic expression of piano performance.

[0041] What is disclosed above is only a preferred embodiment of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. A person skilled in the art can understand that all or part of the processes of the above embodiments and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A method for controlling dynamic lighting effects of a piano lamp, characterized in that: The method comprises: S1, using an audio acquisition device to obtain a real-time audio signal of a piano performance, performing spectrum analysis on the audio signal using a Fourier transform algorithm, and obtaining frequency distribution characteristics of the piano sound; S2. According to the spectrum analysis results, the peak detection algorithm is used to identify the notes in the audio signal, calculate the start time, duration and pitch of each note, and construct a note sequence data structure; S3. For the note sequence data, the sliding window technology is applied to calculate the time interval and pitch difference between adjacent notes, judge the continuity and transition characteristics of the notes, and generate the note coherence index; S4. Predict the frequency change of the note sequence through the Kalman filter algorithm, calculate the inertial characteristics of the note frequency, and obtain the frequency change trend and speed; S5. Based on the note continuity index and frequency inertia characteristics, design the dynamic change rules of light projection, and map the continuity and frequency inertia of the notes to the color, brightness and position parameters of the light; S6. Use a graphics rendering engine to generate dynamic effects of light projection, control the rhythm of light changes according to the time information of the note sequence, and achieve synchronization of sound and light; S7, sending the lighting control command to the projection device on the piano lamp through the network communication module, driving the projection device to adjust the projection image in real time, and presenting a dynamic lighting effect that matches the piano performance; In step S3, the following steps are also included: A1. Based on the input note sequence data, the sliding window technology is used to obtain the time interval and pitch difference between adjacent notes; A2. By setting a time interval threshold and a pitch difference threshold, determine whether adjacent notes are continuous; if the time interval between adjacent notes is less than or equal to the time interval threshold, and the pitch difference is less than or equal to the pitch difference threshold, the two notes are considered to be continuous, otherwise they are considered to be discontinuous; A3. For a continuous sequence of notes, a Markov chain model is used to calculate the transition probability matrix between notes. The transition characteristics of the note sequence are obtained through the transition probability matrix. A4. Determine the continuity index of the note sequence based on the transition characteristics; The coherence index can determine whether the note sequence is coherent by setting a threshold; If the coherence index is greater than or equal to the threshold, the note sequence is considered coherent, otherwise it is considered incoherent; A5. For discontinuous note sequences, a dynamic time warping algorithm is used to align and regularize the discontinuous note sequences; A6, recalculating the time interval and pitch difference between adjacent notes through the aligned and regularized note sequences, and repeating steps A1-A5 until all note sequences are judged to be continuous or discontinuous; A7. Finally, the coherence index of the entire music clip is calculated based on the coherence indexes of all note sequences; By setting the overall coherence index threshold, we can determine whether the entire music clip is coherent; If the overall coherence index is greater than or equal to the threshold, the music clip is considered coherent, otherwise it is considered incoherent.

2. The method according to claim 1, wherein: The method adopts an audio acquisition device to obtain a real-time audio signal of a piano performance, performs spectrum analysis on the audio signal through a Fourier transform algorithm, and obtains the frequency distribution characteristics of the piano sound, including: An audio acquisition device is used to obtain the real-time audio signal of the piano performance, and the analog audio signal is converted into a digital signal through an analog-to-digital converter; According to the Nyquist sampling theorem, the sampling frequency needs to be greater than twice the highest frequency of the audio signal to avoid aliasing distortion; After acquiring the digital audio signal, the fast Fourier transform algorithm is used to perform spectrum analysis on the audio signal to convert the time domain signal into a frequency domain signal; Through spectrum analysis, the frequency distribution characteristics of piano sound are obtained, including fundamental frequency and overtone frequency components; According to the standard frequency of piano notes, determine the notes corresponding to the main frequency components in the spectrum; A frequency recognition-based pitch detection algorithm is used to determine the dominant frequency in each time frame and map it to the corresponding MIDI note number; Through continuous analysis of time frames, a complete MIDI note sequence is obtained, which represents the melody and chord information of the piano performance; According to the MIDI note sequence, a score generation algorithm is used to generate a score representation of the piano performance, including pitch, duration and velocity information; Through the score alignment algorithm, the generated score is compared with the standard score to calculate the accuracy and expressiveness index of the performance; Based on the evaluation indicators, the performers' skills and levels are evaluated and given feedback.

3. The method according to claim 1, wherein: The method comprises: identifying the notes in the audio signal by using a peak detection algorithm according to the spectrum analysis result, calculating the start time, duration and pitch of each note, and constructing a note sequence data structure, including: According to the spectrum analysis results of the audio signal, a peak detection algorithm is used to identify the notes in the audio signal; The peak detection algorithm sets a threshold and determines the peak position in the spectrum that exceeds the threshold as the frequency of the note; Based on the note frequencies obtained by spectrum analysis, the fundamental frequency estimation algorithm is used to calculate the fundamental frequency of each note, thereby determining the pitch of the note; By analyzing the change of note frequency over time, the starting time and duration of the note can be determined; Constructing the start time, duration and pitch information of the identified notes into a note sequence data structure; The note sequence data structure contains the start time, duration and pitch attributes of each note, as well as the order of the notes. By analyzing the note sequence data structure, the melody and rhythm information in the audio signal can be obtained.

4. The method according to claim 1, wherein: The method predicts the frequency change of the note sequence by using the Kalman filter algorithm, calculates the inertial characteristics of the note frequency, and obtains the frequency change trend and speed, including: According to the frequency change of the note sequence, the Kalman filter algorithm is used to predict the frequency change of the note sequence; First, obtain the frequency data of the note sequence and use it as the input of the Kalman filter algorithm; Then, the frequency variation of the note sequence is modeled through the Kalman filter algorithm to obtain the state equation and observation equation of the note frequency variation; Then, according to the state equation and observation equation, the Kalman filter algorithm is used to predict the frequency change of the note sequence to obtain the predicted value of the note frequency at the future moment; Based on the predicted note frequency changes, the inertial characteristics of the note frequency are calculated; The speed of note frequency change is obtained by taking the first-order difference of the predicted value of note frequency; Then, the acceleration of the note frequency change is obtained by taking the second-order difference of the note frequency prediction value; The speed and acceleration of the note frequency change are taken as the inertial characteristics of the note frequency; According to the predicted value and inertial characteristics of the note frequency, determine the changing trend and speed of the note frequency; If the predicted value of the note frequency shows an upward trend and changes rapidly, it is judged that the note frequency change trend is upward and the change speed is rapid; If the predicted value of the note frequency shows a downward trend and the changing speed is slow, it is judged that the note frequency changing trend is downward and the changing speed is slow; By analyzing the predicted value of note frequency and inertial characteristics, the changing trend and speed of note frequency are obtained.

5. The method according to claim 1, wherein: Based on the note continuity index and frequency inertia characteristics, the dynamic change rules of light projection are designed to map the continuity and frequency inertia of the notes to the color, brightness and position parameters of the light, including: According to the continuity index and frequency inertia characteristics of musical notes, the convolutional neural network algorithm is used to extract the features of the note sequence to obtain the note continuity index and frequency inertia feature vector; The extracted feature vectors are classified by the support vector machine algorithm to determine the coherence and frequency inertia of the note sequence; If the continuity index of the note sequence is higher than a preset threshold, the note continuity is mapped to a continuous gradient of light color; If the continuity index of the note sequence is lower than a preset threshold, the note continuity is mapped to a random change in light color; According to the frequency inertia feature vector of the note sequence, the K-means clustering algorithm is used to cluster the frequency inertia features and obtain the cluster centers at different frequency inertia levels. The frequency inertia level of the current note is determined by the Euclidean distance between the cluster center and the frequency inertia feature of the current note; If the frequency inertia level of the current note is high, the frequency inertia is mapped to an increase in the light brightness; If the frequency inertia level of the current note is low, the frequency inertia is mapped to a decrease in light brightness; According to the time position of the note in the music clip, the corresponding light projection position coordinates are calculated using a linear interpolation algorithm to obtain the mapping relationship between the note time position and the light projection position; By integrating the light color change rules mapped by the note continuity index, the light brightness change rules mapped by the frequency inertia feature, and the light projection position coordinates mapped by the note time position, a weighted average algorithm is used to determine the final light projection dynamic change parameters, including color, brightness, and position coordinates; According to the determined dynamic change parameters of light projection, the light projection equipment is controlled to adjust the color, brightness and position of the projection light in real time, so as to realize the synchronous mapping of the characteristics of music notes and the dynamic changes of light; During the music playing process, the note sequence is continuously acquired, and the above steps are repeated to dynamically update the light projection effect to form a light art expression that echoes the rhythm and emotion of the music.

6. The method according to claim 1, wherein: The method uses a graphics rendering engine to generate a dynamic effect of light projection, controls the rhythm of light changes according to the time information of the note sequence, and realizes the synchronization of sound and light, including: According to the time information of the note sequence, the Fourier transform algorithm is used to perform frequency domain analysis on the audio signal to obtain the frequency and amplitude characteristics of the audio; The audio frequency is mapped to the corresponding light color through the frequency mapping algorithm, and the light brightness is determined according to the amplitude; A 3D graphics rendering engine is used to build virtual scenes, and light projection effects are dynamically generated based on audio frequency and amplitude characteristics; The audio timeline is aligned with the lighting animation timeline through the time synchronization algorithm to achieve synchronous changes of sound and lighting; During the rendering process, the color and brightness of the light projection are dynamically adjusted according to the changes in the audio frequency, and the smooth transition of the light effect is achieved through the interpolation algorithm; At the same time, a spatial mapping algorithm is used to map the lighting position in the virtual scene to the actual physical space, and the actual lighting equipment is controlled by the lighting control system to present the virtual lighting effect in the physical environment; Finally, the lighting effect is dynamically adjusted through a real-time feedback algorithm, and the light projection is updated in real time according to the changes in the audio signal to maintain the synchronization and consistency of sound and light.

7. The method according to claim 1, wherein: The light control instruction is sent to the projection device on the piano lamp through the network communication module, and the projection device is driven to adjust the projection picture in real time to present a dynamic light effect matching the piano performance, including: According to the notes and rhythm information of piano playing, the convolutional neural network algorithm is used to extract and analyze the features of the audio data to obtain the time, intensity and pitch parameters of the notes; The extracted audio features are classified and identified through the support vector machine algorithm to determine the type of note currently being played and the playing style; According to the identified note type and playing style, a decision tree algorithm is used to generate matching lighting control instructions, including light color, brightness and flashing frequency parameters; The generated lighting control instructions are sent to the projection device on the piano lamp in real time through the network communication module; After receiving the lighting control command, the projection device uses the optical flow algorithm to analyze the command and obtain the specific parameters of the lighting effect; According to the analyzed lighting parameters, the lighting effect is rendered in real time to the specified position on the piano surface through the projection mapping algorithm, forming a dynamic light projection that matches the piano performance; During the light projection process, the Kalman filter algorithm is continuously used to filter the audio data to remove noise interference and improve the accuracy of note recognition; At the same time, an adaptive gain control algorithm is used to adjust the light brightness in real time to ensure the best lighting effect under different ambient lighting conditions; Through the iteration of the above steps, the real-time synchronization of piano performance and light projection is achieved, creating an immersive music performance atmosphere; During the entire process, a long short-term memory neural network algorithm is used to learn and memorize historical note data and lighting control instructions, and continuously optimize the lighting effect generation strategy, so that the light projection can be dynamically adapted to the piano playing style and enhance the artistic expression of the performance.

Images