Audio-based lighting device synchronization control method and system, terminal, and medium

Abstract

This application provides an audio-based method, system, terminal, and medium for synchronous control of lighting equipment. The method includes: acquiring an audio stream and slicing the audio stream according to an adaptive sliding time window to obtain at least one audio frame to be processed; extracting audio features of the audio frame; predicting the occurrence time of a future beat peak at the half-window position of the sliding time window based on the audio features; constructing a beat time grid aligned with the anchor point time based on the occurrence time of the future beat peak; generating a corresponding lighting control segment based on the beat time grid; and sending the lighting control segment to the lighting equipment, which then synchronously executes the lighting actions defined by the lighting transition curve. This application enables high-precision, low-latency, and robust multi-device lighting synchronization control in heterogeneous communication network environments.

Description

Technical Field

[0001] This application belongs to the field of intelligent lighting technology, and relates to an audio-based synchronous control method and system for lighting equipment, as well as a terminal and medium. Background Technology

[0002] With the deep integration of smart lighting and multimedia entertainment systems, dynamic synchronous control of audio-driven lighting equipment has become a key technological direction for enhancing immersive experiences. In applications such as stage lighting, home ambient lighting, and commercial displays, users expect lighting equipment to accurately respond to audio rhythms, achieving highly synchronized visual feedback in dimensions such as beat, intensity, and spectral changes. This not only requires the system to have real-time audio analysis capabilities but also to meet stringent requirements for synchronization accuracy and stability in areas such as multi-device collaboration, network transmission, and timing alignment. Especially in environments with multiple luminaires, heterogeneous communication protocols, and complex networks, achieving low-latency, high-consistency, and interference-resistant synchronous control has become a core technical problem that current smart lighting systems urgently need to solve.

[0003] Although existing technologies have initially achieved the basic function of audio-driven lighting, several key defects still exist in practical applications, severely restricting synchronization effects and user experience:

[0004] First, traditional peak detection suffers from significant lag. Existing solutions typically rely on a complete audio analysis window to identify beat peaks, requiring the window data to be fully acquired before a judgment can be made. This results in a non-negligible time delay between beat detection and light response, making it difficult to achieve real-time synchronization of "what you hear is what you see."

[0005] Secondly, there is clock inconsistency among multiple devices. In distributed lighting systems, each luminaire may use different communication protocols or receive control commands via different routing paths, resulting in differences in propagation delay. Due to the lack of a unified time reference, even if the same clock information is received, the actual execution time of each device will still deviate, leading to visual "mistimed" or "flickering" and disrupting overall coordination.

[0006] Finally, network packet loss disrupts the continuity of rhythm. In a wireless network environment, control commands corresponding to key beats may be lost due to channel congestion, interference, or retransmission failures. Once the beat trigger signal is missing, the lighting fixtures cannot perform the corresponding actions, causing the lighting effects to "break the beat" or abruptly interrupt, severely affecting the sense of rhythm and the smoothness of the visual experience. Summary of the Invention

[0007] This application provides an audio-based lighting equipment synchronization control method, system, terminal, and medium for achieving high-precision, low-latency, and robust multi-device lighting synchronization control.

[0008] In a first aspect, this application provides an audio-based synchronous control method for lighting equipment, applied to a bridge-end device, wherein the bridge-end device is communicatively connected to at least one lighting device; the method includes:

[0009] Acquire the audio stream and slice it according to an adaptive sliding time window to obtain at least one audio frame to be processed;

[0010] Extract the audio features of the audio frame; the audio features include beat time points, beat intensity, beat peaks, spectral variations, and phase information;

[0011] Based on the audio features, at the half-window position of the sliding time window, the occurrence time of future beat peaks is predicted;

[0012] Based on the occurrence time of the future beat peak, a beat time grid aligned with the anchor point time is constructed;

[0013] Based on the beat time grid, a corresponding lighting control segment is generated; the lighting control segment includes anchor point time, segment number and lighting transition curve;

[0014] The lighting control segment is sent to the lighting device, and the lighting device synchronously executes the lighting actions defined by the lighting transition curve.

[0015] In one implementation of the first aspect, predicting the occurrence time of future beat peaks at the half-window position of the sliding time window, based on the audio features, includes:

[0016] Based on the beat time points, calculate the beat period of the audio frame;

[0017] The short-time energy of the audio frame is calculated based on the beat intensity, the beat peak value, and the spectral variation.

[0018] Based on the phase information, the phase change rate of the audio frame is calculated;

[0019] Using the center moment of the sliding time window as a reference time point, and combining the beat period, the short-time energy, and the phase change rate, the occurrence time of the future beat peak is predicted by extrapolation.

[0020] In one implementation of the first aspect, calculating the beat period of the audio frame based on the beat time point includes:

[0021] Calculate the difference between all adjacent beat time points to obtain the original beat time interval;

[0022] Based on the median filtering algorithm, the original beat time interval is smoothed and corrected to obtain the corrected beat time interval.

[0023] The average value of the corrected beat time interval is calculated based on the truncated average algorithm;

[0024] The average value is used as the beat period of the audio frame.

[0025] In one implementation of the first aspect, when the lighting device synchronously performs the lighting action defined by the lighting transition curve, the method further includes synchronously detecting the network state; the network state detection process includes determining whether any of the following conditions are met:

[0026] The packet loss rate of the lighting control segment is greater than a preset threshold;

[0027] The anchor point time deviation exceeds a preset visual safety threshold; the anchor point time deviation is equal to the difference between the expected arrival time of the anchor point broadcast by the bridge end device and the actual arrival time of the anchor point locally sensed by the lighting device.

[0028] The lighting equipment is unable to receive a continuous and valid light control segment;

[0029] The segment numbers received by the lighting device are inconsistent with the expected arrangement order;

[0030] If so, the network status is determined to be abnormal;

[0031] Otherwise, the network status is considered normal.

[0032] In one implementation of the first aspect, the method further includes: activating a packet loss compensation state machine when the network state is abnormal; the packet loss compensation state machine includes a normal alignment state, a packet loss detection state, a short-term extrapolation state, a degraded replay state, and a recovery alignment state, and the transition logic between each state includes:

[0033] In the normal alignment state, the lighting device aligns the execution progress of the local lighting action according to the received anchor point time and the light transition curve; after the alignment is completed, it migrates to the packet loss detection state.

[0034] In the packet loss detection state, the reception integrity of the lighting control segment is continuously monitored; if the lighting device cannot receive a continuous and valid lighting control segment, it transitions to the short-term extrapolation state.

[0035] In the short-term extrapolation state, the occurrence time of the future beat peak is continuously extrapolated; if the number of continuous extrapolations exceeds a preset upper limit, and / or based on the extrapolation results, it is found that the beat stability index does not meet the preset conditions, then the process is moved to the degraded playback state.

[0036] In the downgraded playback state, downgraded playback is performed according to the pre-stored simplified rhythm template or static light sequence; when the continuous and valid light control segment is received again and the beat stability index meets the preset conditions, the process moves to the restored alignment state.

[0037] In the restored alignment state, the lighting actions that were in the degraded playback state are restored; once the restoration is complete, the system moves to the normal alignment state.

[0038] One implementation of the first aspect also includes:

[0039] Obtain the execution progress error periodically reported by the lighting equipment; the execution progress error is equal to the difference between the theoretical execution progress and the actual execution progress.

[0040] Based on the execution progress error, the timing parameters and extrapolation upper limit parameters of the light transition curve are dynamically adjusted; the timing parameters include the execution mode, start time, and time progress calculation rules of the light transition curve on the time axis; the extrapolation upper limit parameter represents the maximum time offset that allows automatic adjustment of the beat time point when network anomalies or synchronization deviations occur.

[0041] In one implementation of the first aspect, the method further includes: introducing a resilient buffering mechanism in the event of network congestion; the network congestion situation is classified into mild congestion, moderate congestion, and severe congestion based on the reception status of the light control segment; the resilient buffering mechanism includes:

[0042] In the case of mild congestion, the light transition curve is dynamically resampled on the time axis;

[0043] In the case of moderate congestion, the missing lighting control segments are reconstructed by interpolation and then combined with the valid lighting control segments for frame synthesis.

[0044] In the event of severe congestion, switch to the pre-stored stable lighting output mode.

[0045] Secondly, this application provides an audio-based lighting device synchronization control system, comprising:

[0046] The bridge-end device is configured to perform the method described in any of the above descriptions;

[0047] Multiple lighting devices, each of which is communicatively connected to the bridge-end device, are configured to receive lighting control segments sent by the bridge-end device and synchronously execute lighting actions defined by lighting transition curves in the lighting control segments.

[0048] Thirdly, this application provides a terminal, including:

[0049] The memory is used to store computer programs;

[0050] A processor, the processor being configured to execute a computer program stored in the memory, so as to cause the terminal to perform any of the methods described above.

[0051] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the preceding claims.

[0052] As described above, the audio-based lighting equipment synchronization control method, system, terminal, and medium of this application can achieve high-precision, low-latency, and robust multi-device lighting synchronization control in heterogeneous communication network environments. Attached Figure Description

[0053] Figure 1 The diagram shown is a structural schematic of a distributed lighting system according to an embodiment of this application.

[0054] Figure 2 The flowchart shown is an embodiment of an audio-based lighting device synchronization control method according to this application.

[0055] Figure 3 The diagram shown is a half-window prediction flowchart according to an embodiment of this application.

[0056] Figure 4 The diagram shown is a schematic representation of a half-window prediction result according to an embodiment of this application.

[0057] Figure 5 The diagram shown is a schematic representation of a lighting control segment according to an embodiment of this application.

[0058] Figure 6 The diagram shown illustrates the release and execution timing of a lighting control segment according to an embodiment of this application.

[0059] Figure 7 The diagram shown is a state transition diagram of a packet loss compensation state machine according to an embodiment of this application.

[0060] Figure 8 The flowchart shown is a method for synchronous control of lighting devices based on audio, according to another embodiment of this application.

[0061] Figure 9 The diagram shown is a structural schematic of an audio-based lighting equipment synchronization control system according to an embodiment of this application.

[0062] Figure 10 The diagram shown is a structural schematic of a terminal according to an embodiment of this application. Detailed Implementation

[0063] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0064] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0065] The following embodiments of this application provide an audio-based lighting equipment synchronization control method, system, terminal, and medium, which can achieve high-precision, low-latency, and robust multi-device lighting synchronization control in heterogeneous communication network environments. This technical solution is suitable for highly dynamic and fast-paced music scenes, and can maintain stable visual synchronization and a smooth user experience even under conditions of dense beats and frequent rhythm changes.

[0066] The following will describe in detail the principle and implementation of an audio-based lighting equipment synchronization control method and system, terminal and medium of this embodiment, so that those skilled in the art can understand the audio-based lighting equipment synchronization control method and system, terminal and medium of this embodiment without creative effort.

[0067] Please see Figure 1 The diagram shown is a schematic diagram of the structure of a distributed lighting system according to an embodiment of this application.

[0068] like Figure 1 As shown, the distributed lighting system includes bridge-end equipment and lighting equipment. The lighting equipment includes luminaire A, luminaire B, and luminaire C.

[0069] In this embodiment, multiple lighting devices can communicate with the same bridge-end device through a heterogeneous communication network. The heterogeneous communication network employs different communication protocols such as Zigbee, Thread, and Wi-Fi, and can also cover communication methods using different network architectures (such as star, mesh, or hybrid topologies) or different physical transmission paths (such as a hybrid wired and wireless approach). By supporting multi-protocol access, the bridge-end device can be compatible with lighting devices from different manufacturers and using different communication standards, thereby improving the system's versatility and deployment flexibility.

[0070] The bridge-end equipment refers to electronic devices with audio processing and network communication capabilities, specifically including smart gateways, multimedia control hosts, smart speakers, or dedicated audio playback terminals.

[0071] Please see Figure 2 The above is a flowchart of an audio-based lighting device synchronization control method according to an embodiment of this application.

[0072] like Figure 2 As shown, this embodiment provides an audio-based synchronous control method for lighting equipment, applied to a bridge-end device, which is communicatively connected to at least one lighting device. The method includes the following steps S100 to S600.

[0073] In step S100, an audio stream is acquired and sliced ​​according to an adaptive sliding time window to obtain at least one audio frame to be processed.

[0074] Specifically, the sliding time window serves as the basic processing unit for audio signal analysis, used to perform finite-length slicing operations on an infinitely long continuous audio stream in the time domain.

[0075] It should be noted that the length and overlap rate of the sliding time window can be adaptively adjusted according to the current audio content. In this implementation, the adaptive frame division mechanism effectively solves the problem of time-frequency resolution imbalance in traditional fixed-window strategies when dealing with diverse music content. Simultaneously, by dividing an infinitely long audio signal into audio frames of finite length, system memory usage and real-time processing latency can be significantly reduced, which is beneficial for achieving efficient and stable synchronization control functions on resource-constrained embedded bridge devices.

[0076] In step S200, the audio features of the audio frame are extracted.

[0077] In one embodiment of this application, extracting the audio features of the audio frame includes: using an audio analysis algorithm to perform multi-dimensional acoustic feature analysis on the audio frame to obtain the audio features.

[0078] Specifically, the audio features include beat timing, beat intensity, beat peak, spectral variation, and phase information.

[0079] The beat time point refers to the time position in an audio frame that has significant rhythmic meaning, usually corresponding to the strong or secondary strong beat in an audio measure. It can be detected using beat tracking algorithms based on autocorrelation functions, dynamic time warping, or deep learning models.

[0080] The beat intensity is used to quantify the salience of rhythmic events at each beat point, reflecting the relative importance of that beat in the overall rhythmic structure. The beat intensity value can be directly mapped to the brightness or saturation adjustment range of lighting equipment, thereby achieving dynamic visual feedback of "strong beats are bright, weak beats are dark".

[0081] The beat peak refers to the maximum value of the audio signal energy or characteristic change that occurs near the beat time point.

[0082] The spectral variation refers to the degree of difference in frequency domain energy distribution between adjacent audio frames. Spectral variation reflects non-rhythmic dynamic information such as musical timbre, instrument switching, or harmonic evolution, and can be used to drive smooth transitions in light color, achieving a semantic mapping of "timbre-light color".

[0083] The phase information includes the phase angles of each frequency component after the audio frame is decomposed in the frequency domain. The continuity and local changes of the phase can help determine the start time and harmonic structure of audio events, which helps improve the robustness of beat detection and supports the generation of spatialized lighting effects in multi-channel audio scenarios.

[0084] In step S300, based on the audio features, the timing of future beat peaks is predicted at the half-window position of the sliding time window.

[0085] Please see Figure 3 The diagram shown is a half-window prediction flowchart according to an embodiment of this application.

[0086] like Figure 3 As shown, based on the audio features, predicting the occurrence time of future beat peaks at the half-window position of the sliding time window includes the following steps S301 to SS304.

[0087] In step S301, the beat period of the audio frame is calculated based on the beat time point.

[0088] Specifically, calculating the beat period of the audio frame based on the beat time points includes: calculating the difference between all adjacent beat time points to obtain the original beat time interval; performing smoothing correction on the original beat time interval based on the median filtering algorithm to obtain the corrected beat time interval; calculating the average value of the corrected beat time interval based on the truncated averaging algorithm; and using the average value as the beat period of the audio frame.

[0089] In this implementation, by combining the median filtering algorithm and the truncated averaging algorithm, the beat time interval is smoothly corrected, which can significantly reduce the fluctuation of the period estimation caused by rhythm transients, detection jitter or background noise, thereby preventing problems such as uneven light response, flickering jumps or synchronization lag caused by abrupt changes in the beat period.

[0090] In step S302, the short-time energy of the audio frame is calculated based on the beat intensity, the beat peak value, and the spectral change.

[0091] Specifically, calculating the short-time energy of the audio frame based on the beat intensity, the beat peak, and the spectral variation includes: calculating the original short-time energy of the audio frame according to a preset energy formula; applying rhythm-aware weights to the original short-time energy based on the beat intensity and the beat peak to obtain a weighted short-time energy; dynamically correcting the weighted short-time energy in conjunction with the spectral variation, and using the corrected short-time energy as the final short-time energy of the audio frame.

[0092] In step S303, the phase change rate of the audio frame is calculated based on the phase information.

[0093] The phase change rate can reflect minute perturbations in the local oscillation frequency of an audio frame. The phase change rate has higher time sensitivity, enabling it to detect the trend of rhythmic events milliseconds in advance.

[0094] In step S304, taking the center moment of the sliding time window as a reference time point, and combining the beat period, the short-time energy, and the phase change rate, the occurrence time of the future beat peak is predicted by extrapolation.

[0095] Specifically, the prediction process for the occurrence time of the future beat peak includes: based on the steady-state rhythm assumption, using the beat period corresponding to the audio frame, initially extrapolating the landing point of the next beat peak; if a significant upward trend in short-term energy is detected near the reference time point, indicating that a rhythmic event is brewing, the prediction time is appropriately advanced; if a peak appears in the phase change rate of the low-frequency band, it is regarded as a strong signal that the beat is about to occur, and the prediction time is further fine-tuned; the dynamic correction amount derived from the short-term energy and phase change rate is superimposed on the initial extrapolation value to obtain the final beat peak prediction time.

[0096] This implementation employs a half-window prediction mechanism, which predicts the next beat peak location based on the currently acquired audio features before the sliding time window ends. Compared to traditional full-window analysis methods that rely on complete window data, this mechanism effectively avoids inherent processing delays and significantly improves the real-time performance of beat detection and light response.

[0097] Please see Figure 4 The image shown is a schematic diagram of a half-window prediction result according to an embodiment of this application.

[0098] In other embodiments, the reference time point can also be flexibly selected as other specific positions within the sliding time window, such as any moment in the first or second half of the window. By dynamically configuring or optimizing the position of the reference time point, an optimal balance can be achieved between real-time performance, robustness, and synchronization accuracy.

[0099] In step S400, a beat time grid aligned with the anchor point time is constructed based on the occurrence time of the future beat peak.

[0100] The anchor point time refers to the global synchronization time marker broadcast by the bridge-end device at a fixed period, which is used to provide a common time reference for all lighting devices. This ensures that even if each lamp is connected through different communication protocols and has different communication or processing delays, it can still execute the preset lighting actions at the same logical moment, thereby avoiding problems such as visual flickering, rhythm lag, or action drift, and achieving a highly consistent and immersive sound and light synergy experience.

[0101] The beat time grid is a discretized, fixed-step time division system that includes multiple predicted beat peak occurrence times. It is used to divide continuous audio time into equal-length "beat grids". Each beat grid corresponds to a logically schedulable light action time unit, and its boundary is aligned with the global time reference defined by the anchor time.

[0102] This application constructs a beat time grid, which constrains all light control actions to be executed within the beat grid. Even with millisecond-level deviations in beat detection or prediction, the light response remains stably locked at the theoretical beat position. Consequently, the light changes exhibit a rhythmic and structural quality highly synchronized with the music, avoiding chaotic flickering caused by transient noise or algorithmic jitter, and significantly improving the rhythmic consistency, visual smoothness, and artistic expressiveness of the synchronization effect.

[0103] In step S500, a corresponding lighting control segment is generated based on the beat time grid.

[0104] Specifically, the bridge-end device generates one or more lighting control segments within each grid cycle to drive the lighting equipment to perform synchronized lighting actions within the corresponding beat grid. The grid cycle is defined as the time interval between two adjacent "beat grids" in the beat time grid, i.e., the fixed step size of the beat time grid on the time axis.

[0105] The lighting control segment guides the lighting equipment to synchronously execute lighting actions defined by the transition curve at a target time. The lighting control segment includes an anchor time, segment number, and lighting transition curve. The meanings of each field are as follows:

[0106] The anchor point time represents the global synchronization reference time to which the lighting control segment belongs. This field has been described in detail in step S400 and will not be repeated here.

[0107] The segment number is a unique identifier assigned to the lighting control segment. In multi-device collaborative scenarios, the segment number can help the lighting fixture determine whether the instruction has been processed, avoiding duplicate execution or out-of-order responses.

[0108] The light transition curve is used to accurately describe the dynamic trajectory of light state changes over time. The types of light transition curves include linear curves, quadratic Bézier curves, and S-shaped curves.

[0109] Please see Figure 5 The image shown is a schematic diagram of a lighting control segment according to an embodiment of this application.

[0110] like Figure 5 As shown, the lighting control segment also includes a beat grid number, beat speed, phase parameter, duration, and degradation strategy. The meanings of each field are as follows:

[0111] The beat grid number indicates the logical position of the light control segment within the beat time grid, i.e., which beat grid it corresponds to. This number corresponds one-to-one with the grid points in the beat time grid, establishing a semantic connection between the light action and the musical structure, and supporting more expressive arrangement strategies.

[0112] The beat speed represents the tempo of the current audio. This parameter can be used to dynamically adjust the time scale of the lighting action, allowing the lighting effects to naturally follow the acceleration, deceleration, or tempo changes of the music, maintaining consistency in the sound and light rhythm.

[0113] The phase parameter is the relative starting offset of the light control segment within its beat cycle; by fine-tuning the phase, the light action can be triggered earlier or later than the beat peak, thereby optimizing the subjective synchronization experience.

[0114] The duration is used to define the effective duration of the lighting control segment from start to finish; this parameter directly determines the rhythm density and visual dwell time of the lighting action.

[0115] The degradation strategy addresses potential anomalies during actual deployment. Each lighting control segment has one or more pre-set fault-tolerant execution strategies to ensure the system maintains basic synchronization even under non-ideal conditions. Typical degradation strategies include: time degradation (if execution cannot be performed at the target beat time, it is delayed to the next available beat cell, or the duration is proportionally stretched to adapt to the actual arrival time); rhythm degradation (automatically switching to a simplified rhythm mode when multiple beats are lost consecutively); effect degradation (reverting to a linear or fixed state when the lighting fixture does not support complex transition curves); and silent discard (abandoning execution in cases of severe synchronization loss to avoid disrupting the overall rhythm).

[0116] In step S600, the lighting control segment is sent to the lighting device, and the lighting device synchronously executes the lighting action defined by the lighting transition curve.

[0117] Please see Figure 6 The diagram shows the timing of the release and execution of a lighting control segment according to an embodiment of this application.

[0118] Specifically, each lighting control segment is associated with a logical target time. This time is neither the physical moment when the luminaire actually begins executing the command, nor the broadcast time of the anchor signal, but rather an ideal synchronization moment calculated based on the musical rhythm structure (such as beat grid, predicted beat peaks, and phase information) at which a change in lighting should occur in terms of musical semantics. After receiving the lighting control segment, each lighting device synchronously executes the lighting action defined by the aforementioned lighting transition curve at the logical target time.

[0119] In this implementation, the changes in lighting can be presented as a controllable, smooth, and dynamic behavior that conforms to the characteristics of musical rhythm and human perceptual psychology on the timeline.

[0120] In one embodiment of this application, when the lighting device synchronously performs the lighting action defined by the lighting transition curve, it also includes synchronously detecting the network status.

[0121] Specifically, the network status detection process includes determining whether any of the following conditions are met:

[0122] (a1) The packet loss rate of the lighting control segment is greater than a preset threshold;

[0123] (a2) The anchor point time deviation exceeds the preset visual safety threshold; the anchor point time deviation is equal to the difference between the expected arrival time of the anchor point broadcast by the bridge end device and the actual arrival time of the anchor point locally sensed by the lighting device.

[0124] (a3) The lighting equipment is unable to receive a continuous and valid lighting control segment;

[0125] (a4) The segment numbers received by the lighting device are inconsistent with the expected arrangement order;

[0126] If so, the network status is determined to be abnormal; otherwise, the network status is determined to be normal.

[0127] In one embodiment of this application, the method further includes: enabling a packet loss compensation state machine when the network status is abnormal.

[0128] Specifically, the packet loss compensation state machine includes a normal alignment state, a packet loss detection state, a short-term extrapolation state, a degraded playback state, and a restored alignment state.

[0129] Please see Figure 7 The diagram shows the state transitions of a packet loss compensation state machine according to an embodiment of this application.

[0130] like Figure 7 As shown, the transition logic between states includes:

[0131] In the normal alignment state, the lighting device aligns the execution progress of the local lighting action according to the received anchor point time and the light transition curve; after the alignment is completed, it migrates to the packet loss detection state.

[0132] In the packet loss detection state, the reception integrity of the lighting control segment is continuously monitored; if the lighting device cannot receive a continuous and valid lighting control segment, it transitions to the short-term extrapolation state.

[0133] In the short-term extrapolation state, the occurrence time of the future beat peak is continuously extrapolated; if the number of continuous extrapolations exceeds a preset upper limit, and / or based on the extrapolation results, it is found that the beat stability index does not meet the preset conditions, then the process is moved to the degraded playback state.

[0134] In the downgraded playback state, downgraded playback is performed according to the pre-stored simplified rhythm template or static light sequence; when the continuous and valid light control segment is received again and the beat stability index meets the preset conditions, the process moves to the restored alignment state.

[0135] In the restored alignment state, the lighting actions that were in the degraded playback state are restored; once the restoration is complete, the system moves to the normal alignment state.

[0136] In this implementation, by enabling the packet loss compensation state machine, visual interruptions can be avoided in complex network environments, ensuring rhythm continuity and user experience stability.

[0137] Please see Figure 8 The above is a flowchart illustrating an audio-based lighting device synchronization control method according to another embodiment of this application.

[0138] like Figure 8 As shown, the audio-based lighting device synchronization control method described in this application further includes the following steps S700 to S800.

[0139] In step S700, the execution progress error periodically reported by the lighting device is obtained.

[0140] The execution progress is defined as the normalized proportional position of the currently executed transition curve in the time or parameter domain, and its value range is usually [0,1]. For example, when the segment just starts executing, the progress is 0; when the segment reaches the midpoint, the progress is 0.5; and when the segment completely ends, the progress is 1.

[0141] The execution progress error is equal to the difference between the theoretical execution progress and the actual execution progress.

[0142] In step S800, based on the execution progress error, the timing parameters and extrapolation upper limit parameters of the light transition curve are dynamically adjusted.

[0143] Specifically, the timing parameters include the execution method, start time, and time progress calculation rules of the light transition curve on the time axis. The extrapolation upper limit parameter represents the maximum time offset allowed for automatic adjustment of the beat time point when network anomalies or synchronization deviations occur.

[0144] In one embodiment of this application, the audio-based lighting device synchronization control method further includes step S900.

[0145] In step S900, a resilient buffering mechanism is introduced in the event of network congestion.

[0146] Specifically, the network congestion situation is divided into mild congestion, moderate congestion, and severe congestion based on the reception status of the light control segment.

[0147] The triggering condition for the elastic buffer includes any one of the following:

[0148] (b1) The lighting control segment is delayed but not lost;

[0149] (b2) The anchor point time deviation is still within the error tolerance range;

[0150] (b3) A brief jitter occurs during the execution of the transition curve.

[0151] Specifically, the elastic buffering mechanism includes the following steps S901 to S903.

[0152] In step S901, under the mild congestion situation, the light transition curve is dynamically resampled on the time axis to ensure the continuity of the transition curve.

[0153] In step S902, under the moderate congestion situation, the missing light control segments are interpolated and reconstructed, and frame synthesis is performed in combination with the valid light control segments to maintain the integrity of the control segments.

[0154] In step S903, under the severe congestion situation, the system switches to a pre-stored stable light output mode to maintain stable light output and avoid flickering that could disrupt the visual experience.

[0155] This application was experimentally validated on a test set containing various music styles. The experimental results show that this application demonstrates significant advantages in several key performance dimensions:

[0156] (1) The 95th percentile value (P95) of the beat alignment error is significantly lower than that of the traditional fixed window or energy threshold method, which effectively reduces the visual “misbeat” phenomenon caused by timing deviation, thereby significantly improving the stability and rhythm fit of the overall visual experience.

[0157] (2) By strictly executing the lighting actions synchronously according to the logical target time at each lighting device, millisecond-level collaboration across devices is achieved, ensuring that the lighting effect is highly uniform and without phase drift under large-scale deployment;

[0158] (3) In the event of network anomalies, by combining the beat extrapolation state machine and the intelligent degradation strategy, the lighting rhythm can be kept running without interruption even in the event of continuous packet loss, effectively avoiding abrupt stagnation or flickering of the screen, and ensuring the continuity of the performance and the stability of the user experience.

[0159] It should be noted that the scope of protection of the audio-based lighting equipment synchronization control method described in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0160] Please see Figure 9 The image shown is a schematic diagram of the structure of an audio-based lighting equipment synchronization control system according to an embodiment of this application.

[0161] like Figure 9 As shown, this embodiment provides an audio-based lighting equipment synchronization control system, including:

[0162] The bridge-end device is configured to perform the method described in any of the above descriptions;

[0163] Multiple lighting devices, each of which is communicatively connected to the bridge-end device, are configured to receive lighting control segments sent by the bridge-end device and synchronously execute lighting actions defined by lighting transition curves in the lighting control segments.

[0164] It should be noted that the execution process of the bridge-end device corresponds one-to-one with the steps in the above-mentioned audio-based lighting equipment synchronization control method, so it will not be repeated here.

[0165] The audio-based lighting equipment synchronization control system provided in this application can implement the audio-based lighting equipment synchronization control method described in this application. However, the implementation device of the audio-based lighting equipment synchronization control method described in this application includes, but is not limited to, the structure of the audio-based lighting equipment synchronization control system listed in this embodiment. All structural modifications and substitutions of the prior art made in accordance with the principles of this application are included within the protection scope of this application.

[0166] Please see Figure 10 The image shown is a schematic diagram of the structure of a terminal according to an embodiment of this application.

[0167] like Figure 10 As shown, this embodiment provides a terminal, including:

[0168] The memory is used to store computer programs;

[0169] A processor, the processor being configured to execute a computer program stored in the memory, so as to cause the terminal to perform any of the methods described above.

[0170] In this application embodiment, the memory may include a computer system readable medium in the form of volatile memory, such as RAM and / or cache memory. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0171] The processor is connected to the memory and is used to execute computer programs stored in the memory, so that the electronic device performs a sewing stitch anomaly detection method based on an unsupervised algorithm.

[0172] For example, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. In other embodiments, the processor can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0173] In some implementations, the electronic device provided in this application embodiment may further include a display. The display is communicatively connected to a memory and a processor, and is used to display a graphical user interface (GUI) related to the sewing stitch anomaly detection method based on an unsupervised algorithm.

[0174] In this embodiment, the display may include a display screen (display panel). In some implementations, the display panel may be configured using a liquid crystal display (LCD), an organic light-emitting diode (OLED), or similar methods. Alternatively, the display may be a touch panel (touchscreen, touch screen), which may include a display screen and a touch-sensitive surface. When the touch-sensitive surface detects a touch operation on or near it, it transmits the information to a processor to determine the type of touch event. The processor then provides corresponding visual output on the display device based on the type of touch event.

[0175] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0176] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0177] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0178] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the methods described in any of the above embodiments. Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0179] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0180] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for synchronous control of lighting equipment based on audio, characterized in that, The method is applied to bridge-end equipment, which is communicatively connected to at least one lighting device; the method includes: Acquire the audio stream and slice it according to an adaptive sliding time window to obtain at least one audio frame to be processed; Extract the audio features of the audio frame; the audio features include beat time points, beat intensity, beat peaks, spectral variations, and phase information; Based on the audio features, at the half-window position of the sliding time window, the occurrence time of future beat peaks is predicted; Based on the occurrence time of the future beat peak, a beat time grid aligned with the anchor point time is constructed; Based on the beat time grid, a corresponding lighting control segment is generated; the lighting control segment includes the anchor point time, segment number, and lighting transition curve; The lighting control segment is sent to the lighting device, and the lighting device synchronously executes the lighting actions defined by the lighting transition curve; Based on the audio features, predicting the occurrence time of future beat peaks at the half-window position of the sliding time window includes: Based on the beat time points, calculate the beat period of the audio frame; The short-time energy of the audio frame is calculated based on the beat intensity, the beat peak value, and the spectral variation. Based on the phase information, the phase change rate of the audio frame is calculated; Using the center moment of the sliding time window as a reference time point, and combining the beat period, the short-time energy, and the phase change rate, the occurrence time of the future beat peak is predicted by extrapolation.

2. The method according to claim 1, characterized in that, Calculating the beat period of the audio frame based on the beat time point includes: Calculate the difference between all adjacent beat time points to obtain the original beat time interval; Based on the median filtering algorithm, the original beat time interval is smoothed and corrected to obtain the corrected beat time interval. The average value of the corrected beat time interval is calculated based on the truncated average algorithm; The average value is used as the beat period of the audio frame.

3. The method according to claim 1, characterized in that, When the lighting equipment synchronously executes the lighting action defined by the lighting transition curve, the process also includes synchronously detecting the network status; the network status detection process includes determining whether any of the following conditions are met: The packet loss rate of the lighting control segment is greater than a preset threshold; The anchor point time deviation exceeds a preset visual safety threshold; the anchor point time deviation is equal to the difference between the expected arrival time of the anchor point broadcast by the bridge end device and the actual arrival time of the anchor point locally sensed by the lighting device. The lighting equipment is unable to receive a continuous and valid light control segment; The segment numbers received by the lighting device are inconsistent with the expected arrangement order; If so, the network status is determined to be abnormal; Otherwise, the network status is considered normal.

4. The method according to claim 1, characterized in that, Also includes: When the network condition is abnormal, the packet loss compensation state machine is activated; the packet loss compensation state machine includes a normal alignment state, a packet loss detection state, a short-term extrapolation state, a degraded replay state, and a recovery alignment state, and the transition logic between each state includes: In the normal alignment state, the lighting device aligns the execution progress of the local lighting action according to the received anchor point time and the light transition curve; after the alignment is completed, it migrates to the packet loss detection state. In the packet loss detection state, the reception integrity of the lighting control segment is continuously monitored; if the lighting device cannot receive a continuous and valid lighting control segment, it transitions to the short-term extrapolation state. In the short-term extrapolation state, the occurrence time of the future beat peak is continuously extrapolated; if the number of continuous extrapolations exceeds a preset upper limit, and / or based on the extrapolation results, it is found that the beat stability index does not meet the preset conditions, then the process is moved to the degraded playback state. In the downgraded playback state, downgraded playback is performed according to the pre-stored simplified rhythm template or static light sequence; when the continuous and valid light control segment is received again and the beat stability index meets the preset conditions, the process moves to the restored alignment state. In the restored alignment state, the lighting actions that were in the degraded playback state are restored; once the restoration is complete, the system moves to the normal alignment state.

5. The method according to claim 1, characterized in that, Also includes: Obtain the execution progress error periodically reported by the lighting equipment; the execution progress error is equal to the difference between the theoretical execution progress and the actual execution progress. Based on the execution progress error, the timing parameters and extrapolation upper limit parameters of the light transition curve are dynamically adjusted; the timing parameters include the execution mode, start time, and time progress calculation rules of the light transition curve on the time axis; the extrapolation upper limit parameter represents the maximum time offset that allows automatic adjustment of the beat time point when network anomalies or synchronization deviations occur.

6. The method according to claim 1, characterized in that, Also includes: In the event of network congestion, an elastic buffering mechanism is introduced; The network congestion situation is classified into mild congestion, moderate congestion, and severe congestion based on the reception status of the light control segment. The elastic buffering mechanism includes: In the case of mild congestion, the light transition curve is dynamically resampled on the time axis; In the case of moderate congestion, the missing lighting control segments are reconstructed by interpolation and then combined with the valid lighting control segments for frame synthesis. In the event of severe congestion, switch to the pre-stored stable lighting output mode.

7. An audio-based synchronous control system for lighting equipment, characterized in that, include: The bridge-end device is configured to perform the method as described in any one of claims 1 to 6; Multiple lighting devices, each of which is communicatively connected to the bridge-end device, are configured to receive lighting control segments sent by the bridge-end device and synchronously execute lighting actions defined by lighting transition curves in the lighting control segments.

8. A terminal, characterized in that, include: The memory is used to store computer programs; A processor for executing a computer program stored in the memory to cause the terminal to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images