Music signal analysis and light dynamic mapping method based on multi-feature fusion

Abstract

This invention discloses a method for music signal analysis and dynamic lighting mapping based on multi-feature fusion, relating to the field of intelligent lighting control technology. The method reads the music signal, segments it by time granularity, and performs amplitude normalization, silent segment removal, and burst noise suppression to obtain an audio analysis segment sequence. Each audio analysis segment is then windowed and subjected to Fast Fourier Transform, dividing it into low-frequency, mid-frequency, and high-frequency bands. Amplitude information for each band is statistically analyzed, and a spectral state set is generated. A reference segment is determined based on the spectral state set, and brightness and hue adjustment indicators are identified, generating a lighting fluctuation primitive sequence. The lighting fluctuation primitives are then checked, merged, filled with gaps, and time-corrected in chronological order, expanding into a dynamic lighting recording sequence. Commands are encapsulated and sequentially issued using a lighting control protocol, and subsequent control commands are synchronously corrected based on the current playback time and dominant frequency band.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent lighting control, and specifically to a method for music signal analysis and lighting dynamic mapping based on multi-feature fusion. Background Technique

[0002] Existing lighting control mainly adopts methods such as manual switches, fixed timing triggers, and preset program playback. The control content mainly includes on-off switching, color rotation, and basic rhythm coordination, which are suitable for stage lighting and scene display with relatively simple structures. With the gradual development of audio acquisition, digital signal processing, and lighting communication protocols, lighting systems have begun to shift from static program control to linkage control based on music signals, and the control basis has gradually expanded from single manual triggers to processing methods such as volume detection, beat recognition, and spectrum reading. During this development process, the linkage relationship between music and lighting has gradually evolved from single-point triggering to segment-level analysis and continuous timing control, and a processing idea of jointly participating in lighting drive with multiple audio features has also been formed.

[0003] In existing music-linked lighting solutions, the more common method is to directly extract audio intensity, beat points, or single-spectrum peaks as the control basis, and then map the corresponding results to on-off, color-changing, or flashing instructions. Most of these processing methods focus on single acoustic features, lack the recognition of the relationship between consecutive audio segments, and rarely jointly determine the dominant frequency band, sub-dominant frequency band, and spectral segment switching state, resulting in lighting control often only reflecting local instantaneous changes and being difficult to express the structural changes of music within a continuous time range. At the same time, existing solutions lack sufficient consideration of sorting and verification of control records, merging of adjacent elements, correction of time gaps, and subsequent instruction correction during playback, which easily causes a disconnection between lighting instructions and the music playback process and is difficult to meet the usage requirements of timing consistency and state coherence in continuous mapping scenarios. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a method for music signal analysis and lighting dynamic mapping based on multi-feature fusion, which solves the problems in the above background technique.

[0005] To achieve the above objectives, the present invention is realized through the following technical solutions: A method for music signal analysis and lighting dynamic mapping based on multi-feature fusion, including the following steps: S1. Read the music signal, segment it according to the time granularity and perform audio processing to obtain a sequence of audio analysis segments; S2. Perform windowing processing and fast Fourier transform on each audio analysis segment, statistically analyze the band amplitudes and determine the dominant frequency band and spectral segment state, and generate a set of spectral segment states; S3. Determine the reference segment for each audio analysis segment based on the spectral state set, and determine the brightness adjustment identifier and hue adjustment identifier respectively to generate the light fluctuation primitive sequence; S4. Perform verification, merging and correction on the light fluctuation primitive sequence in chronological order, and expand it into the corresponding light dynamic record to generate a light dynamic sequence; S5. Encapsulate the dynamic sequence of lights into control commands and issue them according to the target execution time. Correct subsequent control commands based on the current playback time and the dominant frequency band.

[0006] Preferably, S1 includes S11 and S12; S11. Read the music signal and divide it according to the time granularity. Divide the original music signal into several continuous audio segments. Each audio segment has the same duration and is arranged in chronological order. Configure the corresponding segment number and time identifier for each audio segment according to the start position, end position and arrangement order of each audio segment in the music signal. S12. Perform audio processing on each audio segment to obtain an audio analysis segment sequence; The audio processing includes amplitude normalization, silent segment removal, and burst noise suppression.

[0007] Preferably, S2 includes S21; S21. Windowing is performed on each audio analysis segment using a window function. After windowing, the spectral amplitude data corresponding to the frequency points in each audio analysis segment is extracted by Fast Fourier Transform to obtain the spectral data sequence corresponding to each audio analysis segment. According to the frequency band division table, the spectral data sequence is divided into low-frequency band, mid-frequency band, and high-frequency band according to the frequency range. The cumulative value of the spectral amplitude, the maximum amplitude, and the frequency position corresponding to the maximum amplitude in each frequency band are statistically analyzed to obtain the frequency division statistics results of each audio analysis segment.

[0008] Preferably, S2 further includes S22 and S23; S22. Sort the cumulative amplitude values ​​of the low-frequency band, mid-frequency band, and high-frequency band corresponding to each audio analysis segment, and determine the frequency band ranked first as the current dominant frequency band and the frequency band ranked second as the current secondary dominant frequency band. Read the dominant frequency bands corresponding to the previous and next audio analysis segments of the current audio analysis segment, and perform state flag determination. The specific state flag determination is as follows; When the dominant frequency band of the current audio analysis segment and the previous audio analysis segment are both low frequency bands, it is marked as a low frequency dominant state; When the dominant frequency band of the current audio analysis segment is a high-frequency band, and the dominant frequency band of the previous audio analysis segment is not a high-frequency band, it is marked as a high-frequency prominence state; When the dominant and secondary dominant frequency bands of the current audio analysis segment still correspond to the same pair of frequency bands in the previous audio analysis segment, and their order has been interchanged, it is marked as a bispectral concurrent state. When the dominant frequency band changes twice in three consecutive audio analysis segments, it is marked as a spectral band switching state; When the same dominant frequency band remains unchanged in three consecutive audio analysis segments, it is marked as a spectral band stable state; S23. Read the time markers corresponding to each audio analysis segment, write the state marker judgment results of each audio analysis segment to the corresponding time position, and generate a set of spectral states arranged in chronological order.

[0009] Preferably, S3 includes S31 and S32; S31. Read the spectrum segment status set and the time identifier corresponding to each audio analysis segment, and establish a light control record for each audio analysis segment; The lighting control record includes start time, end time, brightness adjustment indicator, and hue adjustment indicator; The brightness adjustment indicator is used to indicate whether the brightness of the light corresponding to the current audio analysis segment is increased, decreased, or maintained, and the hue adjustment indicator is used to indicate whether the hue of the light corresponding to the current audio analysis segment is switched in, rolled back, or maintained. S32. Read the state flag determination results of adjacent audio analysis segments before and after the current audio analysis segment, and determine the audio analysis segment marked as having a stable spectral state as the reference segment. When there are multiple audio analysis segments with stable spectral states, determine the audio analysis segment with the closest time distance to the current audio analysis segment as the reference segment, and read the low-frequency amplitude accumulation value corresponding to the current audio analysis segment and the reference segment to determine the brightness adjustment flag. The specific brightness adjustment flag determination is as follows. When the cumulative amplitude value of the low-frequency band corresponding to the current audio analysis segment is greater than the cumulative amplitude value of the low-frequency band corresponding to the reference segment, the brightness adjustment flag is set to "increase" until it is consistent with the cumulative amplitude value of the low-frequency band corresponding to the reference segment. When the cumulative amplitude value of the low-frequency band corresponding to the current audio analysis segment is less than the cumulative amplitude value of the low-frequency band corresponding to the reference segment, the brightness adjustment flag is set to "lower" until it is consistent with the cumulative amplitude value of the low-frequency band corresponding to the reference segment. When the cumulative amplitude value of the low-frequency band corresponding to the current audio analysis segment is the same as the cumulative amplitude value of the low-frequency band corresponding to the reference segment, the brightness adjustment flag is set to hold.

[0010] Preferably, S3 further includes S33 and S34; S33. Read the dominant frequency bands corresponding to the current audio analysis segment and the reference segment to determine the tone adjustment identifier, as follows; When the dominant frequency band of the current audio analysis segment is a high-frequency band and the dominant frequency band of the reference segment is not a high-frequency band, the tone adjustment flag is set to "cut in". When the dominant frequency band of the current audio analysis segment is not a high-frequency band and the dominant frequency band of the reference segment is a high-frequency band, the tone adjustment flag is set to back. When the dominant frequency band of the current audio analysis segment is the same as the dominant frequency band of the reference segment, the tone adjustment flag is set to "hold". S34. Arrange the brightness adjustment flag determination results and hue adjustment flag determination results of each audio analysis segment after the writing process is completed in chronological order to generate a light fluctuation primitive sequence.

[0011] Preferably, S4 includes S41 and S42; S41. Based on the start and end times of each light wave element, sort the light wave elements from front to back according to their start times, and check the order of the end time with the start time of the next light wave element to obtain an ordered control structure sequence. S42. Based on the comparison of the start time, end time, brightness adjustment flag, and hue adjustment flag of two adjacent light wave primitives in the ordered control structure sequence, perform structural adjustment on the adjacent light wave primitives in the ordered control structure sequence, as follows; When the end time and start time of two adjacent light wave elements are consecutive, and the brightness adjustment indicator and the hue adjustment indicator are the same, the two adjacent light wave elements are merged into one continuous light wave element. When there is a gap between the end time of the previous light wave element and the start time of the next light wave element, the gap time is written into the end time of the previous light wave element. When the end time of the next light wave element is earlier than the start time of the previous light wave element, the start time of the next light wave element is corrected to the end time of the previous light wave element.

[0012] Preferably, S4 further includes S43; S43. Read the start time, end time, brightness adjustment flag, hue adjustment flag, continuation flag, and segmentation flag of each adjusted light wave element, and execute control record expansion for each light wave element based on the structural adjustment results, as follows; When the brightness adjustment indicator is set to "increase", a brightness increment execution flag is written into the corresponding light dynamic record. When the brightness adjustment indicator is set to "lower", a brightness decrease execution flag is written into the corresponding light dynamic record. When the brightness adjustment flag is set to hold, write the brightness hold execution flag into the corresponding light dynamic record; When the hue adjustment flag is set to "cut in", write the hue cut in execution flag into the corresponding lighting dynamics record; When the hue adjustment flag is set to rollback, write a hue rollback execution flag into the corresponding lighting dynamics record; When the hue adjustment flag is set to hold, write the hue hold execution flag into the corresponding lighting dynamics record; Each light fluctuation primitive is expanded into a corresponding light dynamic record, generating a light dynamic record sequence.

[0013] Preferably, S5 includes S51; S51. Read the dynamic recording sequence of lights and the channel configuration data corresponding to the existing lighting control protocol, encapsulate and process the instructions for each dynamic recording of lights, and generate a control instruction sequence. The channel configuration data includes brightness control channel, hue control channel, and command sending order; The start time, end time, brightness execution flag, hue execution flag, duration, and time synchronization flag of each light dynamic record are read separately. Based on the channel format corresponding to the existing light control protocol, the brightness control channel, hue control channel, and target execution time are written to generate control instructions corresponding to each light dynamic record. The control instructions are then arranged in order of target execution time to generate a control instruction sequence.

[0014] Preferably, S5 further includes S52; S52. Based on the dominant frequency band corresponding to the current music playback time, the control command sequence is modified according to the control record, as follows; When the audio analysis segment corresponding to the current music playback time is in a stable spectral state and the dominant frequency band is the low frequency band, the brightness execution flag corresponding to the next consecutive time period in the control instruction sequence will be kept as the current brightness execution flag. When the audio analysis segment corresponding to the current music playback time is in a stable spectral state and the dominant frequency band is the high frequency band, the hue execution flag corresponding to the next consecutive time period in the control instruction sequence will be kept as the current hue execution flag. When the audio analysis segment corresponding to the current music playback time is in a state of rapid spectral switching, the control commands corresponding to the next consecutive time period will be maintained in a state of being sent one by one.

[0015] This invention provides a method for music signal analysis and dynamic lighting mapping based on multi-feature fusion. It has the following beneficial effects: (1) This method performs unified segmentation, normalization, silence filtering, and burst noise suppression on the music signal. Combined with windowing and fast Fourier transform, it performs segmented statistics and state determination on the low-frequency, mid-frequency, and high-frequency segments within each time slice. This processing path no longer stops at simply triggering lights based on volume or beat points, but decomposes the music content into a set of spectral states that can be read in chronological order. This gives the time position, frequency band structure, and state change relationship of the music segments a clear corresponding basis, which is used to solve the problems of coarse music feature extraction, unclear frequency band distinction, and single mapping basis in the existing light following scheme.

[0016] (2) This method selects reference segments based on the spectral state set, writes brightness adjustment identifiers and hue adjustment identifiers for each audio analysis segment, and organizes the segment-level control results into a sequence of light fluctuation primitives; then, the light fluctuation primitives are sequentially checked, adjacent ones are merged, gaps are corrected, and control records are expanded. Through this processing link, the changing relationships in the music spectrum are rewritten into a dynamic sequence of lights with start time, end time, and control semantics, so that light changes are no longer a direct stacking of isolated instructions, but a continuous control structure with sequential connections. Compared with the currently more common fixed threshold mapping method, this method organizes the continuity, switching, and overlapping relationships between adjacent time slices, and improves the problems of abrupt jumps, segment breaks, and conflicts between adjacent instructions during the light change process.

[0017] (3) This method encapsulates the dynamic sequence of lights into control commands that conform to existing control protocols. It then combines the current playback time and the dominant frequency band to synchronously modify subsequent control commands, ensuring that the light output is not only delivered according to the predetermined execution time but also adjusted based on the actual state of the music playback. Compared to pre-writing hard-coded light scripts or simply outputting commands in a fixed loop, this method employs different command holding or sequential sending strategies based on the low-frequency stable segment, the high-frequency dominant segment, and the rapid switching segment of the spectrum during music playback. This maintains a closer correspondence between light control and music evolution. Overall, this method establishes a complete processing flow around "music feature analysis—state marking—control generation—structure organization—protocol delivery," achieving a continuous mapping between music signals and dynamic light control. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the steps of the music signal analysis and dynamic light mapping method based on multi-feature fusion of the present invention; Figure 2 This is a block diagram illustrating the logical principle of the music signal analysis and dynamic lighting mapping method based on multi-feature fusion of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1 Please see Figure 1 This invention provides a method for music signal analysis and dynamic lighting mapping based on multi-feature fusion. To achieve the above objectives, this invention is implemented through the following technical solution, including the following steps: S1. Read the music signal, segment it according to time granularity and perform audio processing to obtain an audio analysis segment sequence; S2. Windowing and fast Fourier transform are applied to each audio analysis segment to statistically analyze the frequency band amplitude and determine the dominant frequency band and spectral state, generating a spectral state set. S3. Determine the reference segment for each audio analysis segment based on the spectral state set, and determine the brightness adjustment identifier and hue adjustment identifier respectively to generate the light fluctuation primitive sequence; S4. Perform verification, merging and correction on the light fluctuation primitive sequence in chronological order, and expand it into the corresponding light dynamic record to generate a light dynamic sequence; S5. Encapsulate the dynamic sequence of lights into control commands and issue them according to the target execution time. Correct subsequent control commands based on the current playback time and the dominant frequency band.

[0021] In this embodiment, the music signal is first segmented at the time granularity, processed at the segment level, and decomposed at the spectral density through steps S1 and S2. Then, the frequency amplitude, dominant frequency band, and spectral state of each segment are sequentially determined, transforming the original music data from a continuous sound wave into a structured result that can be located by time, distinguished by frequency band, and read by state. Compared to existing processing methods that trigger lights based solely on overall volume, fixed beat points, or single frequency band thresholds, this method decomposes the music content more finely. It can distinguish the changes in the spectral center of gravity in different time slices and identify the spectral stability, prominence, and switching relationships between consecutive segments. This allows subsequent light control to no longer rely on coarse triggering conditions but to be based on segment-level spectral states. Steps S3 and S4 determine the reference segment based on the spectral state set and determine the brightness adjustment marker and hue adjustment marker. Then, the light fluctuation primitives are checked, merged, corrected, and expanded in chronological order, further rewriting the music analysis results into a dynamic light sequence with start time, end time, and control semantics. Compared to existing technologies that directly map audio features to scattered lighting commands, this method incorporates reference segment comparison, adjacent structure organization, and temporal continuity correction before control generation. This makes the sequential relationships in the lighting process clearer, addresses issues such as jumps, gaps, overlaps, and repetitions between segments, and results in more coherent lighting output in terms of timing. The structure of the control record is also more suitable for subsequent protocol encapsulation and device execution. By encapsulating the dynamic lighting sequence into control commands using S5 and issuing them according to the target execution time, and simultaneously adjusting subsequent control commands in conjunction with the current playback time and dominant frequency band, the entire method not only possesses front-end music analysis capabilities but also provides a practical control path for actual lighting equipment. Compared to current preset script-based, fixed-rule-based, or single-generation lighting control methods that do not adjust after generation, this method can continuously adjust subsequent commands based on the current segment state during music playback, ensuring greater consistency between brightness changes during low-frequency dominance, hue changes during high-frequency dominance, and control rhythm during rapid transitions with the music's evolution. Overall, these five steps form a complete processing chain around "music fragment extraction - spectrum status determination - control identifier generation - dynamic sequence arrangement - control command issuance". The purpose is to enable a finer correspondence and clearer timing logic between music signals and light actions.

[0022] Example 2 Please refer to Figure 2 Specifically: S1 includes S11 and S12; S11. Read the music signal and segment it according to the time granularity. Divide the original music signal into several continuous audio segments. Each audio segment has the same duration and is arranged in chronological order. Assign a corresponding segment number and time identifier to each audio segment according to the start position, end position and arrangement order of each audio segment in the music signal. By assigning a time identifier to each segment, the subsequent spectrum analysis and light mapping process can ensure seamless connection between each segment and avoid mapping distortion caused by time misalignment. The time granularity is a time segment ranging from 1 second to 10 seconds; S12. Perform audio processing on each audio segment to obtain an audio analysis segment sequence; The audio processing includes amplitude normalization, silent segment removal, and burst noise suppression; The amplitude normalization process is used to read the amplitude distribution range of each audio segment and perform proportional compression or proportional expansion on each sample value in the segment according to a unified amplitude benchmark, so that different audio segments are under the same amplitude scale. The silent segment filtering is used to identify low-energy silent segments based on the energy occupancy status and duration of continuous sample values ​​in an audio segment, and to remove or mark the corresponding segments as non-mapping segments. The burst noise suppression is used to identify abnormal pulse signals in audio segments that are short in duration, have abrupt amplitude jumps, and are discontinuous with the changes in previous and subsequent samples, and to suppress the current abnormal part by using a neighboring sample smoothing replacement method.

[0023] In this embodiment, the music signal to be processed is read and divided into multiple audio segments with consecutive beginnings and endings and consistent durations according to a preset time granularity. Then, based on the start position, end position, and arrangement order of each audio segment in the original music signal, segment numbers and time identifiers are written respectively, forming a set of segments with time sequence attributes. Subsequently, each audio segment is processed sequentially. Among them, amplitude normalization is used to unify the sampling amplitude scale of each segment, so that different time segments can be directly compared under the same amplitude benchmark. Silent segment screening is used to identify silent segments with continuous low energy and duration that meet the conditions, and these segments are removed or marked as non-mapped segments. Burst noise suppression is used to identify abnormal pulses with short duration, sudden amplitude jumps, and no connection with the sampling trend before and after, and is adjusted by the nearest sampling smoothing replacement method. Finally, the audio analysis segment sequence is output. Through the aforementioned processing path, this step goes beyond simply slicing the music signal. It first completes four fundamental tasks: time positioning, amplitude unification, silence detection, and anomaly correction. This ensures that the data entering subsequent spectrum analysis has a continuous, regular, and comparable segment structure. The aim is to establish a consistent data entry point for determining the dominant frequency band, identifying spectral states, and mapping lighting control. Compared to current methods that directly extract the spectrum from the original music stream or trigger lights based on overall amplitude, this step addresses issues such as time misalignment, silence interference, abnormal pulse insertion, and inconsistent amplitude scales between segments beforehand. This makes subsequent spectral statistics of each segment easier to compare horizontally, the lighting mapping sequence clearer, the basis for brightness and hue control more focused, and the segment continuity, mapping accuracy, and output stability in the overall control chain are in a state more suitable for engineering applications.

[0024] Example 3 Please refer to Figure 2 Specifically: S2 includes S21; S21. Windowing is performed on each audio analysis segment using a window function. After windowing, the spectral amplitude data corresponding to the frequency points in each audio analysis segment is extracted by Fast Fourier Transform to obtain the spectral data sequence corresponding to each audio analysis segment. According to the frequency band division table, the spectral data sequence is divided into low-frequency band, mid-frequency band, and high-frequency band according to the frequency range. The cumulative value of the spectral amplitude, the maximum amplitude, and the frequency position corresponding to the maximum amplitude in each frequency band are statistically analyzed to obtain the frequency division statistics results of each audio analysis segment.

[0025] S2 also includes S22 and S23; S22. Sort the cumulative amplitude values ​​of the low-frequency band, mid-frequency band, and high-frequency band corresponding to each audio analysis segment, and determine the frequency band ranked first as the current dominant frequency band and the frequency band ranked second as the current secondary dominant frequency band. Read the dominant frequency bands corresponding to the previous and next audio analysis segments of the current audio analysis segment, and perform state flag determination. The specific state flag determination is as follows; When the dominant frequency band of the current audio analysis segment and the previous audio analysis segment are both low frequency bands, it is marked as a low frequency dominant state; When the dominant frequency band of the current audio analysis segment is a high-frequency band, and the dominant frequency band of the previous audio analysis segment is not a high-frequency band, it is marked as a high-frequency prominence state; When the dominant and secondary dominant frequency bands of the current audio analysis segment still correspond to the same pair of frequency bands in the previous audio analysis segment, and their order has been interchanged, it is marked as a bispectral concurrent state. When the dominant frequency band changes twice in three consecutive audio analysis segments, it is marked as a spectral band switching state; When the same dominant frequency band remains unchanged in three consecutive audio analysis segments, it is marked as a spectral band stable state; S23. Read the time markers corresponding to each audio analysis segment, write the state marker judgment results of each audio analysis segment to the corresponding time position, and generate a set of spectral states arranged in chronological order.

[0026] In this embodiment, a window function is applied to each audio analysis segment to smooth the transition between the beginning and end of the segment. Then, a Fast Fourier Transform is performed on the windowed audio analysis segment to convert the time-domain sampled data into spectral amplitude data at the corresponding frequency points. Subsequently, the spectral data is divided into low-frequency, mid-frequency, and high-frequency bands according to the frequency range, and the cumulative amplitude value, maximum amplitude value, and frequency position corresponding to the maximum amplitude value of each frequency band are statistically analyzed to form the frequency division statistics of each audio analysis segment. Based on this, the cumulative amplitude values ​​of the low-frequency band, mid-frequency band, and high-frequency band are sorted to determine the dominant and secondary dominant frequency bands of the current audio analysis segment. Combining the changes in the dominant frequency bands corresponding to the previous and subsequent audio analysis segments, the current segment is judged as having a low-frequency dominant state, a high-frequency prominent state, a dual-spectral-band concurrent state, a spectral-band switching state, and a spectral-band stable state. Finally, the state markers corresponding to each audio analysis segment are written to the corresponding time positions to generate a set of spectral-band states arranged in chronological order. Through this implementation method, the spectral distribution relationship, primary and secondary frequency band relationship, and spectral band change trajectory between adjacent segments in the music signal are organized into a state result that can be continuously read in time, which is used to establish a unified basis for subsequent reference segment determination, brightness adjustment mark determination, and hue adjustment mark determination. Compared with the processing method that only makes a rough identification based on the overall volume or a single frequency band, this step can distinguish the spectral center of gravity within the current segment, and can also identify the continuation, prominence, interchange, and switching relationship between adjacent segments.

[0027] Example 4 Please refer to Figure 2 Specifically: S3 includes S31 and S32; S31. Read the spectrum segment status set and the time identifier corresponding to each audio analysis segment, and establish a light control record for each audio analysis segment; The lighting control record includes start time, end time, brightness adjustment indicator, and hue adjustment indicator; The brightness adjustment indicator is used to indicate whether the brightness of the light corresponding to the current audio analysis segment is increased, decreased, or maintained, and the hue adjustment indicator is used to indicate whether the hue of the light corresponding to the current audio analysis segment is switched in, rolled back, or maintained. S32. Read the state flag determination results of adjacent audio analysis segments before and after the current audio analysis segment, and determine the audio analysis segment marked as having a stable spectral state as the reference segment. When there are multiple audio analysis segments with stable spectral states, determine the audio analysis segment with the closest time distance to the current audio analysis segment as the reference segment, and read the low-frequency amplitude accumulation value corresponding to the current audio analysis segment and the reference segment to determine the brightness adjustment flag. The specific brightness adjustment flag determination is as follows. When the cumulative amplitude value of the low-frequency band corresponding to the current audio analysis segment is greater than the cumulative amplitude value of the low-frequency band corresponding to the reference segment, the brightness adjustment flag is set to "increase" until it is consistent with the cumulative amplitude value of the low-frequency band corresponding to the reference segment. When the cumulative amplitude value of the low-frequency band corresponding to the current audio analysis segment is less than the cumulative amplitude value of the low-frequency band corresponding to the reference segment, the brightness adjustment flag is set to "lower" until it is consistent with the cumulative amplitude value of the low-frequency band corresponding to the reference segment. When the cumulative amplitude value of the low-frequency band corresponding to the current audio analysis segment is the same as the cumulative amplitude value of the low-frequency band corresponding to the reference segment, the brightness adjustment flag is set to hold.

[0028] S3 also includes S33 and S34; S33. Read the dominant frequency bands corresponding to the current audio analysis segment and the reference segment to determine the tone adjustment identifier, as follows; When the dominant frequency band of the current audio analysis segment is a high-frequency band and the dominant frequency band of the reference segment is not a high-frequency band, the tone adjustment flag is set to "cut in". When the dominant frequency band of the current audio analysis segment is not a high-frequency band and the dominant frequency band of the reference segment is a high-frequency band, the tone adjustment flag is set to back. When the dominant frequency band of the current audio analysis segment is the same as the dominant frequency band of the reference segment, the tone adjustment flag is set to "hold". S34. Arrange the brightness adjustment flag determination results and hue adjustment flag determination results of each audio analysis segment after writing and processing in chronological order to generate a light fluctuation primitive sequence. The light fluctuation primitive is a segment-level light control structure corresponding to an audio analysis segment.

[0029] In this embodiment, the spectral state set and the time identifier of each audio analysis segment are read to establish a corresponding light control record for each audio analysis segment. The start time, end time, brightness adjustment identifier, and hue adjustment identifier are written into the light control record. Subsequently, the state marker determination results of the adjacent segments before and after the current audio analysis segment are read, and the audio analysis segment in the stable state of the spectral segment is selected as the reference segment. When there are multiple selectable reference segments, the current reference object is determined according to the principle of closest time distance. On this basis, the low-frequency amplitude accumulation value of the current audio analysis segment and the reference segment is compared to write the brightness adjustment identifier for adjustment (adjusting up, lowering down, or maintaining). At the same time, the dominant frequency band of the current audio analysis segment and the reference segment are matched to write the hue adjustment identifier for adjustment (cutting in, reverting, or maintaining). Finally, the brightness adjustment identifier determination results and hue adjustment identifier determination results corresponding to each audio analysis segment are arranged in chronological order to generate a light fluctuation primitive sequence. Through the above processing path, music segments no longer directly correspond to single, isolated lighting actions. Instead, they are first compared with reference segments to form a segment-level lighting control structure. This provides a low-frequency reference for brightness changes and a dominant frequency band for hue changes. The aim is to establish a clear connection between the lighting control process and the spectral state within the music segment, the temporal relationship between adjacent segments, and local stable sections. Compared to processing methods that directly trigger brightness or color changes based solely on the instantaneous characteristics of the current segment, this approach incorporates a "stable reference—difference determination—sequential arrangement" processing chain in the control generation stage.

[0030] Example 5 Please refer to Figure 2 Specifically: S4 includes S41 and S42; S41. Based on the start and end times of each light wave element, sort the light wave elements from front to back according to their start times, and check the order of the end time with the start time of the next light wave element to obtain an ordered control structure sequence. S42. Based on the comparison of the start time, end time, brightness adjustment flag, and hue adjustment flag of two adjacent light wave primitives in the ordered control structure sequence, perform structural adjustment on the adjacent light wave primitives in the ordered control structure sequence, as follows; When the end time and start time of two adjacent light wave elements are consecutive, and the brightness adjustment indicator and the hue adjustment indicator are the same, the two adjacent light wave elements are merged into one continuous light wave element. When there is a gap between the end time of the previous light wave element and the start time of the next light wave element, the gap time is written into the end time of the previous light wave element. When the end time of the next light wave element is earlier than the start time of the previous light wave element, the start time of the next light wave element is corrected to the end time of the previous light wave element.

[0031] S4 also includes S43; S43. Read the start time, end time, brightness adjustment flag, hue adjustment flag, continuation flag, and segmentation flag of each adjusted light wave element, and execute control record expansion for each light wave element based on the structural adjustment results, as follows; When the brightness adjustment indicator is set to "increase", a brightness increment execution flag is written into the corresponding light dynamic record. When the brightness adjustment indicator is set to "lower", a brightness decrease execution flag is written into the corresponding light dynamic record. When the brightness adjustment flag is set to hold, write the brightness hold execution flag into the corresponding light dynamic record; When the hue adjustment flag is set to "cut in", write the hue cut in execution flag into the corresponding lighting dynamics record; When the hue adjustment flag is set to rollback, write a hue rollback execution flag into the corresponding lighting dynamics record; When the hue adjustment flag is set to hold, write the hue hold execution flag into the corresponding lighting dynamics record; Each light fluctuation primitive is expanded into a corresponding light dynamic record, generating a light dynamic record sequence.

[0032] In this embodiment, the start time, end time, brightness adjustment flag, and hue adjustment flag of each light wave primitive are read, rearranged from front to back according to the start time, and the time connection relationship between adjacent primitives is checked one by one to obtain an ordered control structure sequence. Then, S42 performs structural adjustment on two adjacent light wave primitives in the ordered control structure sequence. When the two are connected end to end and the brightness adjustment flag and hue adjustment flag are the same, the two are merged into the same continuous light wave primitive. When there is a gap in time between the two, the gap is merged into the end of the previous light wave primitive. When the time position of the subsequent light fluctuation primitive is earlier than the time range of the previous light fluctuation primitive, the start time of the subsequent light fluctuation primitive is adjusted. After the time adjustment and structure merging are completed, S43 reads each adjusted light fluctuation primitive and writes the brightness increase execution flag, brightness decrease execution flag, brightness hold execution flag, hue cut-in execution flag, hue fallback execution flag and hue hold execution flag according to the brightness adjustment flag and hue adjustment flag respectively. The fragment-level light fluctuation primitives are expanded one by one into a light dynamic record sequence that can be directly used for subsequent protocol encapsulation. Through the above processing path, the spectrum state determination results obtained in the previous steps and the comparison results with the reference segment are further transcribed into a dynamic light record that is time-continuous, has clear control semantics, and a clear relationship between preceding and following segments. This is used to handle common issues in existing technologies where audio segments are directly mapped to light commands, such as time breakpoints, adjacent repeated controls, overlapping segments, and scattered control semantics. This allows the light control link to transition from "segment determination" to "continuous recording output," ensuring that subsequent control commands have a unified time reference and unified execution semantics. At the same time, adjacent control structures of the same type can be merged, missing segments can be filled, and time-abnormal segments can be corrected. The dynamic light record is suitable for engineering implementation in terms of structural integrity, continuity, and device-side executability.

[0033] Example 6 Please refer to Figure 2 Specifically: S5 includes S51; S51. Read the dynamic recording sequence of lights and the channel configuration data corresponding to the existing lighting control protocol, encapsulate and process the instructions for each dynamic recording of lights, and generate a control instruction sequence. The channel configuration data includes brightness control channel, hue control channel, and command sending order; The start time, end time, brightness execution flag, hue execution flag, duration, and time synchronization flag of each light dynamic record are read separately. Based on the channel format corresponding to the existing light control protocol, the brightness control channel, hue control channel, and target execution time are written to generate control instructions corresponding to each light dynamic record. The control instructions are then arranged in order of target execution time to generate a control instruction sequence.

[0034] S5 also includes S52; S52. Based on the dominant frequency band corresponding to the current music playback time, the control command sequence is modified according to the control record, as follows; When the audio analysis segment corresponding to the current music playback time is in a stable spectral state and the dominant frequency band is the low frequency band, the brightness execution flag corresponding to the next consecutive time period in the control instruction sequence will be kept as the current brightness execution flag. When the audio analysis segment corresponding to the current music playback time is in a stable spectral state and the dominant frequency band is the high frequency band, the hue execution flag corresponding to the next consecutive time period in the control instruction sequence will be kept as the current hue execution flag. When the audio analysis segment corresponding to the current music playback time is in a state of rapid spectral switching, the control commands corresponding to the next consecutive time period will be maintained in a state of being sent one by one.

[0035] In this embodiment, the dynamic recording sequence of lights is read, and the control protocol and its channel configuration data matching the target lighting device are read synchronously. The start time, end time, brightness execution flag, hue execution flag, duration, and time synchronization flag in each dynamic recording of lights are mapped to the protocol fields one by one, and written to the corresponding brightness control channel, hue control channel, and target execution time to form a control instruction sequence arranged in the order of the target execution time. Subsequently, during music playback, the dominant frequency band and spectral status of the audio analysis segment where the current playback time is located are continuously read. When the current segment is in a stable spectral state and the dominant frequency band is a low frequency band, the brightness execution flag of the subsequent continuous time period is carried over to the current control state. When the current segment is in a stable spectral state and the dominant frequency band is a high frequency band, the hue execution flag of the subsequent continuous time period is carried over to the current control state. When the current segment is in a rapid spectral switching state, the subsequent continuous time period continues to be sent according to the control instructions one by one. According to this implementation, the music segment-level spectrum state, reference segment judgment result, and light fluctuation element processing result formed in the preceding steps are subsequently written into the device-side control link. This establishes a directly executable temporal correspondence between the music analysis results and the light protocol commands. The purpose is to implement the frequency band dominant changes, stability relationships, and switching relationships in the music content into the brightness and hue control process of the lighting equipment. Compared to simply outputting light commands once according to a preset script, this step, after command encapsulation, continues to link and correct subsequent commands based on the current playback state. This makes the brightness change path corresponding to continuous low-frequency bands smoother, the hue change path corresponding to continuous high-frequency bands more concentrated, the command response in the spectrum switching section more refined, and the control connection relationship, temporal correspondence, and the degree of fit between the light actions and the music content during the device execution process clearer.

[0036] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended technical solutions and their equivalents.

Claims

1. A method for music signal analysis and dynamic light mapping based on multi-feature fusion, characterized in that: Includes the following steps: S1. Read the music signal, segment it according to time granularity and perform audio processing to obtain an audio analysis segment sequence; S2. Windowing and fast Fourier transform are applied to each audio analysis segment to statistically analyze the frequency band amplitude and determine the dominant frequency band and spectral state, generating a spectral state set. S3. Determine the reference segment for each audio analysis segment based on the spectral state set, and determine the brightness adjustment identifier and hue adjustment identifier respectively to generate the light fluctuation primitive sequence; S4. Perform verification, merging and correction on the light fluctuation primitive sequence in chronological order, and expand it into the corresponding light dynamic record to generate a light dynamic sequence; S5. Encapsulate the dynamic sequence of lights into control commands and issue them according to the target execution time. Correct subsequent control commands based on the current playback time and the dominant frequency band.

2. The method for music signal analysis and dynamic lighting mapping based on multi-feature fusion according to claim 1, characterized in that: S1 includes S11 and S12; S11. Read the music signal and divide it according to the time granularity. Divide the original music signal into several continuous audio segments. Each audio segment has the same duration and is arranged in chronological order. Configure the corresponding segment number and time identifier for each audio segment according to the start position, end position and arrangement order of each audio segment in the music signal. S12. Perform audio processing on each audio segment to obtain an audio analysis segment sequence; The audio processing includes amplitude normalization, silent segment removal, and burst noise suppression. 3.The music signal analysis and light dynamic mapping method based on multi-feature fusion according to claim 2, characterized in that: S2 includes S21; S21. Windowing is performed on each audio analysis segment using a window function. After windowing, the spectral amplitude data corresponding to the frequency points in each audio analysis segment is extracted by Fast Fourier Transform to obtain the spectral data sequence corresponding to each audio analysis segment. According to the frequency band division table, the spectral data sequence is divided into low-frequency band, mid-frequency band, and high-frequency band according to the frequency range. The cumulative value of the spectral amplitude, the maximum amplitude, and the frequency position corresponding to the maximum amplitude in each frequency band are statistically analyzed to obtain the frequency division statistics results of each audio analysis segment.

4. The music signal analysis and light dynamic mapping method based on multi-feature fusion according to claim 3, characterized in that: S2 also includes S22 and S23; S22. Sort the cumulative amplitude values ​​of the low-frequency band, mid-frequency band, and high-frequency band corresponding to each audio analysis segment, and determine the frequency band ranked first as the current dominant frequency band and the frequency band ranked second as the current secondary dominant frequency band. Read the dominant frequency bands corresponding to the previous and next audio analysis segments of the current audio analysis segment, and perform state flag determination. The specific state flag determination is as follows; When the dominant frequency band of the current audio analysis segment and the previous audio analysis segment are both low frequency bands, it is marked as a low frequency dominant state; When the dominant frequency band of the current audio analysis segment is a high-frequency band, and the dominant frequency band of the previous audio analysis segment is not a high-frequency band, it is marked as a high-frequency prominence state; When the dominant and secondary dominant frequency bands of the current audio analysis segment still correspond to the same pair of frequency bands in the previous audio analysis segment, and their order has been interchanged, it is marked as a bispectral concurrent state. When the dominant frequency band changes twice in three consecutive audio analysis segments, it is marked as a spectral band switching state; When the same dominant frequency band remains unchanged in three consecutive audio analysis segments, it is marked as a spectral band stable state; S23. Read the time markers corresponding to each audio analysis segment, write the state marker judgment results of each audio analysis segment to the corresponding time position, and generate a set of spectral states arranged in chronological order.

5. The music signal analysis and light dynamic mapping method based on multi-feature fusion according to claim 4, characterized in that: S3 includes S31 and S32; S31. Read the spectrum segment status set and the time identifier corresponding to each audio analysis segment, and establish a light control record for each audio analysis segment; The lighting control record includes start time, end time, brightness adjustment indicator, and hue adjustment indicator; The brightness adjustment indicator is used to indicate whether the brightness of the light corresponding to the current audio analysis segment is increased, decreased, or maintained, and the hue adjustment indicator is used to indicate whether the hue of the light corresponding to the current audio analysis segment is switched in, rolled back, or maintained. S32. Read the state flag determination results of adjacent audio analysis segments before and after the current audio analysis segment, and determine the audio analysis segment marked as having a stable spectral state as the reference segment. When there are multiple audio analysis segments with stable spectral states, determine the audio analysis segment with the closest time distance to the current audio analysis segment as the reference segment, and read the low-frequency amplitude accumulation value corresponding to the current audio analysis segment and the reference segment to determine the brightness adjustment flag. The specific brightness adjustment flag determination is as follows. When the cumulative amplitude value of the low-frequency band corresponding to the current audio analysis segment is greater than the cumulative amplitude value of the low-frequency band corresponding to the reference segment, the brightness adjustment flag is set to "increase" until it is consistent with the cumulative amplitude value of the low-frequency band corresponding to the reference segment. When the cumulative amplitude value of the low-frequency band corresponding to the current audio analysis segment is less than the cumulative amplitude value of the low-frequency band corresponding to the reference segment, the brightness adjustment flag is set to "lower" until it is consistent with the cumulative amplitude value of the low-frequency band corresponding to the reference segment. When the cumulative amplitude value of the low-frequency band corresponding to the current audio analysis segment is the same as the cumulative amplitude value of the low-frequency band corresponding to the reference segment, the brightness adjustment flag is set to hold.

6. The method for music signal analysis and dynamic lighting mapping based on multi-feature fusion according to claim 5, characterized in that: S3 also includes S33 and S34; S33. Read the dominant frequency bands corresponding to the current audio analysis segment and the reference segment to determine the tone adjustment identifier, as follows; When the dominant frequency band of the current audio analysis segment is a high-frequency band and the dominant frequency band of the reference segment is not a high-frequency band, the tone adjustment flag is set to "cut in". When the dominant frequency band of the current audio analysis segment is not a high-frequency band and the dominant frequency band of the reference segment is a high-frequency band, the tone adjustment flag is set to back. When the dominant frequency band of the current audio analysis segment is the same as the dominant frequency band of the reference segment, the tone adjustment flag is set to "hold". S34. Arrange the brightness adjustment flag determination results and hue adjustment flag determination results of each audio analysis segment after the writing process is completed in chronological order to generate a light fluctuation primitive sequence.

7. The method for music signal analysis and dynamic lighting mapping based on multi-feature fusion according to claim 6, characterized in that: S4 includes S41 and S42; S41. Based on the start and end times of each light wave element, sort the light wave elements from front to back according to their start times, and check the order of the end time with the start time of the next light wave element to obtain an ordered control structure sequence. S42. Based on the comparison of the start time, end time, brightness adjustment flag, and hue adjustment flag of two adjacent light wave primitives in the ordered control structure sequence, perform structural adjustment on the adjacent light wave primitives in the ordered control structure sequence, as follows; When the end time and start time of two adjacent light wave elements are consecutive, and the brightness adjustment indicator and the hue adjustment indicator are the same, the two adjacent light wave elements are merged into one continuous light wave element. When there is a gap between the end time of the previous light wave element and the start time of the next light wave element, the gap time is written into the end time of the previous light wave element. When the end time of the next light wave element is earlier than the start time of the previous light wave element, the start time of the next light wave element is corrected to the end time of the previous light wave element.

8. The method for music signal analysis and dynamic lighting mapping based on multi-feature fusion according to claim 7, characterized in that: S4 also includes S43; S43. Read the start time, end time, brightness adjustment flag, hue adjustment flag, continuation flag, and segmentation flag of each adjusted light wave element, and execute control record expansion for each light wave element based on the structural adjustment results, as follows; When the brightness adjustment indicator is set to "increase", a brightness increment execution flag is written into the corresponding light dynamic record. When the brightness adjustment indicator is set to "lower", a brightness decrease execution flag is written into the corresponding light dynamic record. When the brightness adjustment flag is set to hold, write the brightness hold execution flag into the corresponding light dynamic record; When the hue adjustment flag is set to "cut in", write the hue cut in execution flag into the corresponding lighting dynamics record; When the hue adjustment flag is set to rollback, write a hue rollback execution flag into the corresponding lighting dynamics record; When the hue adjustment flag is set to hold, write the hue hold execution flag into the corresponding lighting dynamics record; Each light fluctuation primitive is expanded into a corresponding light dynamic record, generating a light dynamic record sequence.

9. The method for music signal analysis and dynamic lighting mapping based on multi-feature fusion according to claim 8, characterized in that: S5 includes S51; S51. Read the dynamic recording sequence of lights and the channel configuration data corresponding to the existing lighting control protocol, encapsulate and process the instructions for each dynamic recording of lights, and generate a control instruction sequence. The channel configuration data includes brightness control channel, hue control channel, and command sending order; The start time, end time, brightness execution flag, hue execution flag, duration, and time synchronization flag of each light dynamic record are read separately. Based on the channel format corresponding to the existing light control protocol, the brightness control channel, hue control channel, and target execution time are written to generate control instructions corresponding to each light dynamic record. The control instructions are then arranged in order of target execution time to generate a control instruction sequence.

10. The method for music signal analysis and dynamic lighting mapping based on multi-feature fusion according to claim 9, characterized in that: S5 also includes S52; S52. Based on the dominant frequency band corresponding to the current music playback time, the control command sequence is modified according to the control record, as follows; When the audio analysis segment corresponding to the current music playback time is in a stable spectral state and the dominant frequency band is the low frequency band, the brightness execution flag corresponding to the next consecutive time period in the control instruction sequence will be kept as the current brightness execution flag. When the audio analysis segment corresponding to the current music playback time is in a stable spectral state and the dominant frequency band is the high frequency band, the hue execution flag corresponding to the next consecutive time period in the control instruction sequence will be kept as the current hue execution flag. When the audio analysis segment corresponding to the current music playback time is in a state of rapid spectral switching, the control commands corresponding to the next consecutive time period will be maintained in a state of being sent one by one.

Images