A low-latency multi-protocol intelligent lighting synchronization system based on a unified abstraction layer

Abstract

A low-delay multi-protocol intelligent lighting synchronization system based on a unified abstraction layer includes: receiving instructions from a multi-protocol bridge module, obtaining network round-trip time data of terminal equipment through an active probing mechanism; based on the network round-trip time data, a Kalman filtering algorithm is used to construct a delay prediction model to predict the future network delay of the terminal equipment; the instructions are input to a three-level buffer for processing, and the trigger frequency of the active probing mechanism is dynamically adjusted according to the load state of the three-level ring buffer; the buffer queuing delay is calculated according to the load state of the receiving buffer, and the final sending time of the instructions is obtained in combination with the future network delay of the terminal equipment; at the final sending time, the instructions are sent from the sending buffer to the corresponding terminal equipment. The system realizes low-delay and high-stability synchronization of cross-protocol devices in the field of intelligent lighting.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent lighting control and Internet of Things, and particularly relates to a low-delay multi-protocol intelligent lighting synchronization system based on a unified abstraction layer. BACKGROUND

[0002] With the integration of intelligent lighting and Internet of Things technology, the technical solution of analyzing video content to drive real-time changes in lamps to create an immersive visual experience is attracting widespread attention. However, existing such systems face serious challenges in achieving high-precision, low-delay synchronization control. First, the system often needs to control lamps based on different communication protocols at the same time, and the protocol conversion and data distribution process often introduces additional delay and is difficult to maintain synchronization. Second, the inherent jitter of the network environment and the uncertainty of device response make it difficult for the system to predict the arrival time of instructions, which can easily lead to perceptible disconnection between light and picture. In addition, when the video scene is complex or the system processing load is too high, it often causes instruction blocking, which destroys the user experience. Therefore, there is an urgent need in the art for an intelligent lighting synchronization control solution that can achieve cross-protocol precise synchronization, intelligently adapt to network fluctuations, and stably operate under high load. SUMMARY

[0003] To solve the above technical problems, the present application provides a low-delay multi-protocol intelligent lighting synchronization system based on a unified abstraction layer.

[0004] The system aims to solve the problems of high delay, poor protocol compatibility, and weak system load adaptability in real-time video synchronization control of existing intelligent lighting systems. By building a modular and decoupled architecture, the system achieves end-to-end low-delay, high-precision synchronization control.

[0005] The primary innovation of the system lies in the use of intelligent video analysis and efficient data packaging technology. The video analysis engine module processes the original video frames through intelligent downsampling and performs two core algorithms, global uniform color and ROI region mapping, in parallel. In particular, the ROI region mapping integrates a bilinear interpolation scaling algorithm to quickly scale down the image block to a very small size, reducing the time-consuming of calculation while ensuring that the color accuracy is difficult for the human eye to perceive. The extracted color parameters and timestamps are packaged into standardized JSON synchronization frames, providing a unified and efficient data source for downstream processing. This design improves processing efficiency while ensuring the accuracy of visual effects.

[0006] The system establishes a unified data abstraction and distribution layer. The intermediate layer abstraction module implements protocol-independent data distribution using a publish-subscribe mechanism, ensures backward compatibility through versioning analysis, and realizes precise routing of data by relying on a device-protocol registry. This design enables the system to flexibly support multiple communication protocols such as UDP, Zigbee, and Matter, and to achieve hot-plug expansion of protocols through the plug-in architecture of the multi-protocol bridge module. This architecture not only solves the problem of collaborative control in a multi-protocol environment, but also significantly reduces the maintenance and expansion costs of the system.

[0007] The most distinctive innovation of the system is its intelligent synchronization control mechanism. The synchronization control module obtains network delay data through active detection and constructs a prediction model using Kalman filtering algorithm to accurately estimate device delays. When the instruction stream passes through the three-level ring buffer, the sequencing engine performs multi-dimensional intelligent sequencing by combining target presentation timestamps, instruction priorities, and device delay prediction values. The scheduler calculates the final sending time by combining the predicted network delay and buffer queuing delay, and implements forward-looking scheduling. This design enables the system to effectively compensate for network jitter and processing delay, ensuring that all devices respond synchronously within the target time window, achieving extremely high synchronization accuracy.

[0008] In addition, the system also has excellent load adaptation capability. Through the three-level buffer architecture, the system can smooth the instruction stream and combine with the adaptive frame discarding algorithm to intelligently discard non-critical instructions according to real-time load values under high load, prioritizing the synchronization of core lighting effects. This design ensures smooth transition from normal operation to overload degradation, demonstrating excellent system robustness and stability.

[0009] Through the above technical innovations, the intelligent lighting system successfully realizes low latency, high precision, multi-protocol compatibility, and strong robustness in real-time video synchronization control, providing reliable technical support for immersive visual experience. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 is a low-latency multi-protocol intelligent lighting synchronization system module diagram based on a unified abstraction layer provided by an embodiment of the present application;

[0011] Figure 2 is a low-latency multi-protocol intelligent lighting synchronization system flowchart based on a unified abstraction layer provided by an embodiment of the present application;

[0012] Figure 3 is a synchronization control module flowchart provided by an embodiment of the present application.

[0013] Figure 4 is an intermediate layer abstraction module flowchart provided by an embodiment of the present application. DETAILED DESCRIPTION

[0014] The terminology used in the following embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in the description of the application and the appended claims, the singular forms "a," "an" and "the" are intended to include both singular and plural forms, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or," as used in the description of the present application, refers to any or all possible combinations of one or more of the associated listed items.

[0015] Hereinafter, the terms "first", "second" are used only for the purpose of description, and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise stated.

[0016] The present application provides a low-latency multi-protocol intelligent lighting synchronization system based on a unified abstraction layer, see Figure 1 and Figure 2 The system comprises:

[0017] The video analysis engine module: as the signal input end of the system, the module obtains the original video frame through the direct connection of the hardware interface, and extracts the picture color information by using the two core algorithms of global uniform color and ROI region mapping. Through intelligent downsampling and bilinear interpolation technology, the processing efficiency is significantly improved under the premise of ensuring color accuracy. Finally, the color parameters and timestamps are packaged into standardized JSON synchronization frames, providing a low-latency, high-consistency data source for downstream modules. The intermediate layer abstraction module: this module is the data hub of the system, and realizes protocol-independent data distribution based on the publish-subscribe mechanism. Through versioned analysis and field mapping mechanism, it is compatible with multiple versions of data format and extended parameters, and at the same time relies on the device-protocol registry to realize accurate sending of instructions. It realizes the decoupling of data processing and protocol conversion, supports flexible expansion of new protocols and new devices in the system, and ensures data consistency and control accuracy in cross-protocol environment.

[0018] The multi-protocol bridge module: this module supports multi-protocol parallel conversion in a plug-in architecture, converts unified JSON instructions into protocol-specific formats. For UDP protocol, it encapsulates secure binary data packets, adapts to Zigbee / MQTT bridge service topic specifications, and integrates Matter controllers to realize color space conversion and cluster command generation. The hot plug mechanism enables the system to dynamically access new protocols, with high scalability and adaptability.

[0019] Synchronization control module: As the core guarantee of system synchronization performance, this module integrates Kalman filtering algorithm to predict device delay, and calculates instruction sending time through forward-looking scheduling. Combined with three-level ring buffer and adaptive frame dropping mechanism, it smooths the output instruction stream to cope with system load fluctuations. Real-time monitoring and hierarchical discard strategy ensure low-latency synchronization of core functions under overload conditions, achieving stable operation and degradation of the system under high load.

[0020] Video analysis engine module: Calculate each video frame RGB image in the original video. First, to improve the efficiency of subsequent calculations, the high-resolution frame is down-sampled to an appropriate resolution for processing, such as 4K down to 1080p, which can greatly improve performance without losing much color information. Then perform color extraction to obtain color parameters, the core color extraction algorithm includes two types: global uniform color and ROI region mapping.

[0021] Global uniform color: The global uniform color algorithm is responsible for generating an ambient light that represents the overall tone of the entire picture. The execution process of this algorithm is as follows: the system loads the captured complete video frame into the frame buffer in memory. Then, the algorithm initializes three accumulators, corresponding to the red (R), green (G), and blue (B) channels respectively. From front to back, each pixel in the frame buffer is traversed, and the R, G, B values of the pixel are added to the corresponding accumulators. After traversing all pixels, the algorithm divides the accumulated sum of each color channel by the total number of pixels in the frame, calculating the arithmetic mean of all pixels in each color channel. The final output of this RGB triplet is the global representative color of this frame. For example, when processing a frame dominated by a sunset scene, this algorithm will calculate an average color biased towards warm yellow tone, with RGB triplet value (220, 150, 80).

[0022] ROI region mapping: For scenarios that require more fine-grained control, the system enables a ROI region mapping algorithm. This algorithm runs independently for each region of interest that the user has pre-defined on the interactive interface. The system extracts the corresponding image block from the original frame according to the pre-defined bounding box of each region in the screen coordinate system. To cope with the real-time processing pressure that high-resolution inputs such as 4K videos might bring, this algorithm integrates an intelligent down-sampling optimization technique: it uses a bilinear interpolation scaling algorithm to quickly scale down each image block to a fixed, tiny size. For example: 10 pixels by 10 pixels, forming a micro image containing only 100 representative pixels. Subsequently, the algorithm performs a mean value calculation consistent with the global color uniformity on these 100 pixels to obtain the representative color of the region. This optimization measure can reduce the color calculation time of a single region by more than 95%, while the average color deviation introduced is controlled within 0.5. This error value is far below the color difference threshold that the human eye can clearly perceive, i.e. ΔΕ less than 3, thus greatly releasing the system's computing resources while ensuring visual accuracy, ensuring smooth operation even when processing multiple regions of high-frame-rate videos.

[0023] Bilinear interpolation scaling algorithm: First, establish the coordinate mapping relationship between the original image and the target micro image, mapping the coordinates of each pixel point in the target image back to the corresponding position in the original image. Then locate the four nearest pixel points around the coordinates, i.e. the left upper, right upper, left lower and right lower four positions of the original image. Then perform two linear interpolation calculations: first, interpolate the pixel pairs in the horizontal direction, and calculate two intermediate results according to the horizontal coordinate components; then perform the final interpolation on the two intermediate results in the vertical direction, and obtain the final color value of the target pixel according to the vertical coordinate components. This interpolation process uses weighted averaging to make each target pixel's color value integrate the color characteristics of its four surrounding source pixels. After this processing, the original image block is converted into a fixed-size micro image, significantly reducing the subsequent calculation amount while maintaining the color distribution characteristics, providing an optimization basis for the fast extraction of regional color mean values.

[0024] Then, combine the color parameters obtained above with the current timestamp to convert them into a standardized data structure. According to the pre-defined JSON format, assemble the data: write the protocol version number and timestamp in the frame header, record the width and height dimensions of the original video in the resolution field, and fill in the corresponding RGB color values in the region data section. Form a JSON synchronization frame containing complete color spatiotemporal information.

[0025] Finally, after obtaining the JSON synchronization frame containing complete color spatiotemporal information, the system immediately broadcasts it to the connected intermediate layer abstraction module.

[0026] Intermediate layer abstraction module: The intermediate layer abstraction module serves as the data hub of the system, bearing the core functions of JSON synchronization frame conversion and distribution. See Figure 4 . This module continuously receives standardized JSON synchronization frames from the video analysis engine through a subscription mechanism and checks the integrity of the data structure.

[0027] The subscription mechanism is the core method for the module to achieve efficient and flexible data distribution. When the system starts, all downstream protocol bridge modules (such as UDP bridge and Zigbee bridge) will register with the intermediate layer abstraction module, declaring themselves as "subscribers." This establishes a low-coupling publish-subscribe relationship: the intermediate layer abstraction module is the publisher of data, which only broadcasts data without knowing or caring about the specific receivers; each protocol bridge module is a subscriber of data, which only receives the data it cares about without needing to know where the data comes from.

[0028] The module maintains a list of protocol processors internally. When a new JSON synchronization frame arrives, the module performs the following operations in parallel: first, parse the version identifier in the frame header, select the corresponding data extraction strategy according to the version number, and ensure the compatible processing of new and old version data. Then send the JSON synchronization frame to each protocol converter while checking the existence of the extended field.

[0029] Selecting the corresponding data extraction strategy according to the version number means that multiple version parsers are preloaded in the module. For example, if the version is "1.0", the parser only processes the basic RGB color value; if the version is "1.1", the parser will additionally find and parse the optional "color_temperature" (color temperature) and "saturation" (saturation) extended fields. This design ensures that both new and old data formats can be correctly understood, achieving backward compatibility.

[0030] Sending the JSON synchronization frame to each protocol converter means that the module will iterate through the registry and send the data after version parsing to all registered protocol processors at the same time, and each processor will only receive and process the data subset it needs. For example, a basic UDP light strip processor may only use RGB values, while a Matter light fixture processor supporting advanced functions will receive RGB, color temperature, and saturation data at the same time. After the JSON synchronization frame is parsed, the system reads the unique identifier of the target device from the parsed frame data. This identifier is used to query in the device and protocol registry.

[0031] The registry is established by the system during device registration or initialization, and is used to record the communication protocol type and its communication parameters corresponding to each device, including: the used protocol: UDP, Zigbee, Matter, etc., device address, and supportable control fields: RGB, brightness, color temperature, etc.

[0032] When the intermediate layer receives a JSON synchronization frame containing a device identifier, it will first retrieve the device information corresponding to the identifier in the registry. If the query hits, the intermediate layer can determine the protocol type used by the device and filter out the data fields in the parsing result that match the device's capability information. Subsequently, the system only transmits these valid fields to the corresponding protocol processor for subsequent conversion and transmission, thereby ensuring that the instructions are accurately sent to the target device.

[0033] This mechanism enables automatic protocol selection and path decision through the device identifier in the frame. Even if multiple communication protocols exist in the system: UDP light strips, Zigbee lights, Matter lamps, etc., each device will only receive control data belonging to its own protocol, and the situation of "one frame triggering multiple protocol devices to respond simultaneously" will not occur. The registry plays the role of routing core in this process, and it ensures that data can be automatically matched to the correct protocol channel according to the target device, achieving precise control and synchronization communication under cross-protocol systems.

[0034] In the data conversion process, the module uses field mapping mechanism to convert the input RGB color space into the color representation form required by each protocol, such as YUV, HSL, etc. For extended parameters, the module will perform parameter normalization processing, converting color temperature to the specific color temperature range supported by the protocol, and mapping the saturation percentage to the saturation level defined by the protocol. When an unrecognized extended field is detected, the module will retain the original data and add a flag bit to ensure that the data will not be lost during transmission.

[0035] The module realizes seamless collaboration and decoupled expansion of multiple protocols. By maintaining a unified device-protocol registry in the intermediate layer, the system can flexibly access new communication protocols or terminal devices without modifying the upper layer data structure. New protocols only need to define the processor interface and mapping rules in the registration stage, and can immediately access the existing distribution system, thereby greatly reducing the system expansion and maintenance cost. It ensures the accuracy and security of data in cross-protocol environment. Since each synchronization frame contains the target device identifier, the intermediate layer can accurately locate the target device and its protocol according to the registry, avoiding data misbroadcasting and repeated execution, and preventing devices from responding to the same control frame under different communication protocols. The module enhances the compatibility and backward adaptation capability of the system. The introduction of version analysis mechanism and field mapping mechanism enables the system to support multiple versions of JSON data frame structure, realize compatible processing of old version devices, and automatically adapt to new extended fields. The downstream protocol module can selectively receive fields according to its own capabilities, ensuring smooth transition during system upgrade or expansion.

[0036] Multi-protocol bridge module: As a component for realizing cross-device collaboration, the multi-protocol bridge module scans and registers all available protocol processing plugins, forming a complete protocol support capability. Each protocol plugin follows a unified interface specification and is responsible for data encapsulation, connection management, and transmission scheduling for a specific communication protocol.

[0037] When the module receives a standardized JSON data frame from the intermediate abstraction layer, its built-in scheduler performs multiple protocol conversions in parallel.

[0038] For DTLS / UDP protocol processing, the standardized JSON data frame is converted into binary data, then encapsulated, including adding sequence number, timestamp, and CRC check code, which is a 2-byte check code calculated from all bytes of the data packet, and attached at the end. The final encapsulated data packet is sent to the IP port of the target device. This process ensures the security and real-time performance of data transmission.

[0039] In terms of Zigbee / MQTT protocol processing, the plugin maintains a persistent connection with the MQTT broker server. When receiving color data, the plugin converts the RGB value to a device-identified JSON instruction according to the Zigbee2MQTT service topic specification, i.e., an MQTT publish message. Topic specification: Zigbee2MQTT, as a bridge, defines a clear MQTT topic specification, and the system must follow this specification to correctly control the device.

[0040] For Matter protocol support, the plugin builds the complete Matter device discovery, pairing, and control capabilities. When processing color instructions, the plugin first converts the RGB color space to the XY color space used by Matter standards, and then generates cluster commands that comply with Matter specifications by calling the Matter cluster command interface.

[0041] The plugin management mechanism of this module provides complete hot-plug support. The system defines a standard plugin interface, including initialization, data conversion. When new protocol support is needed, developers only need to implement the interface and complete plugin registration, and the system can automatically identify and integrate new protocol processors without restarting. This design enables the system to quickly adapt to emerging communication protocols such as Thread or BLE Mesh.

[0042] Synchronization control module: The synchronization control module is responsible for time synchronization scheduling of encapsulated protocol instructions, see Figure 3 . This module receives protocol ready instruction packets from the bridge module, which already contain converted protocol data, but have not been sent to the physical network.

[0043] The module internally establishes a device delay archive for storing real-time delay data of each terminal device, and maintains real-time delay data of each terminal device through an active probing mechanism. This active probing mechanism sends time synchronization probing packets to each device, records the sending timestamp and parses the reply packet to calculate the accurate network round-trip time.

[0044] Active probing mechanism to calculate round-trip time: This mechanism first obtains the current time T1 from the system high-precision clock source before sending the probing packet, and embeds this timestamp in the probing packet. The probing packet is sent to the target device through the protocol-specific channel, and the device generates a reply packet containing the receiving timestamp immediately after receiving it. When the system receives the reply packet, it reads the high-precision clock again to record the time T2. The calculation formula for round-trip time is: RTT=(T2-T1), to ensure measurement accuracy, this mechanism retains the last 10 valid measurement data, removes abnormal values caused by network jitter, and takes the arithmetic mean as the final RTT.

[0045] Based on real-time measurement data, the module uses Kalman filtering algorithm to build a delay prediction model. This model describes the delay trend through a state transition equation, and fuses the latest measurement value through an observation update equation to achieve accurate prediction of future delay for each device. By accurately predicting the delay, the instructions are sent "in advance", so that all devices execute the command at the same time in the future, thereby reducing the perceived delay of the user.

[0046] Kalman filter algorithm for time compensation: first, establish the state space model of device delay. Define the real network delay of each device as a state variable , and the round-trip time RTT measured periodically as an observation variable . When the system is initialized, an independent state vector is assigned to each device, and the initial delay value and the initial estimated covariance are set, for example: the initial delay value , and the initial estimated covariance .

[0047] When new RTT measurement data arrives, the algorithm performs a prediction-update cycle.

[0048] Prediction stage: state prediction , covariance prediction , where is the process noise variance.

[0049] Update stage: Kalman gain calculation , state update , covariance update , where is the observation noise variance.

[0050] Get the network delay predicted by the Kalman filter .

[0051] The synchronization control module also integrates a frame rate stabilization mechanism, which ensures smooth output of instructions through a multi-level control architecture. The core of this mechanism consists of an instruction cache pool, which uses a three-level ring buffer to achieve dynamic data buffering. The first level buffer receives the original JSON instructions from the synchronization scheduler, the second level buffer sorts the instructions by priority, and the third level buffer serves as a send-ready queue. When the system detects that the input frame rate exceeds the processing capacity, it starts an adaptive frame discarding algorithm that retains the latest instructions based on timestamp freshness, ensuring that the system always processes the most time-sensitive data.

[0052] In specific implementation, the frame rate stabilization mechanism integrated by the synchronization control module realizes smooth control of instruction flow through a three-level ring buffer architecture. This architecture consists of a receiving buffer, a sorting buffer, and a sending buffer, each implemented as a fixed-capacity ring queue.

[0053] The receiving buffer serves as the system data entry point, with a design capacity of, for example, 30 frames, receiving instructions from the synchronization scheduler. When the video scene changes rapidly causing a burst of instructions, this buffer can completely absorb the instantaneous data flood. The buffer monitor will real-time statistics of the queue filling state. When the filling rate exceeds the threshold, the system will send a traffic warning signal to the sorting buffer, and the threshold can be set to 80%.

[0054] The sorting buffer, as the center of instruction scheduling, is designed with a capacity of, for example, 20 frames, and is equipped with a dedicated priority sorting engine. The engine processes instructions according to the target presentation timestamps, delay sensitivity, and preset priority rules contained in the instructions.

[0055] Delay sensitivity: The system obtains the predicted delay values of each target device in real time through the Kalman filter. For those devices with extremely large delay prediction values, their instructions are more likely to be discarded in priority when overloaded. Because even if sent, these instructions are likely to miss the target synchronization time due to excessive delay, losing timeliness. By actively identifying and discarding these low-expected-efficiency instructions, the system can concentrate limited computing resources and network bandwidth on ensuring the instruction flow of devices with stable connections and low delays.

[0056] Priority rules: Global color change instructions are marked as the highest priority, regional color adjustment instructions are marked as medium priority, and subtle color correction instructions are marked as low priority.

[0057] Sorting rules of the sorting buffer: The system first sets a delay tolerance threshold, for example, 80 milliseconds. All device instructions with predicted delay values exceeding the threshold are separately marked and grouped into a "high delay group". Then, based on the preliminary grouping, all instructions are classified according to the preset priority rules. Finally, the instructions are finely sorted according to the target presentation timestamps contained in the instructions. Within the same priority group, for example, instructions with the same "medium priority", instructions with earlier target presentation times are arranged in a more forward position, ensuring that the most urgent instructions are scheduled in priority.

[0058] For example, assume there are four instructions in the sorting buffer: instruction A (target device predicted delay 45 ms, normal group, highest priority global color instruction, target timestamp 10020 ms), instruction B (target device predicted delay 120 ms, high delay risk group, medium priority main area instruction, target timestamp 10010 ms), instruction C (target device predicted delay 120 ms, high delay risk group, highest priority global color instruction, target timestamp 10025 ms), and instruction D (target device predicted delay 35 ms, normal group, medium priority main area instruction, target timestamp 10005 ms). After multi-level sorting rule processing, the final scheduling order from high to low is determined as: instruction C -> instruction A -> instruction B -> instruction D. The specific decision logic is as follows: instruction C is placed in the first priority for scheduling because it belongs to the highest priority, although it is in the high delay risk group and the target time is the latest, but its business importance is the highest; instruction A is followed because it is also of the highest priority and in the normal group for reliable transmission; instructions B and D are both of medium priority, but instruction B is in the high delay risk group, so the system chooses to send instruction D which is reliable although it is later, so instruction B is placed before instruction D, and the system will try to send instruction B which is at risk first. This sorting result reflects the system's comprehensive trade-off between business importance, scheduling urgency, and transmission reliability in complex situations.

[0059] The sending buffer, as the final sending platform, has a design capacity of, for example, 10 frames. The buffer uses a time wheel algorithm to manage the instructions to be sent, and each instruction slot corresponds to a specific sending time window. Time wheel algorithm: similar to a ring dial, each dial scale represents a future time window, such as 1 millisecond. The system pre-stores instructions that need to be sent within the time window in the corresponding scale slot. When the system clock pointer rotates to this scale, all instructions in the slot are processed in batches. A separate time checking thread is maintained to scan the time wheel at a fixed granularity, which can be set to 1 millisecond. When the sending time of an instruction is detected, it is immediately submitted to the corresponding protocol transmission module.

[0060] When the system is continuously in a high load state and the receiving buffer fill rate exceeds the threshold, the adaptive frame discarding algorithm is automatically activated, and the threshold can be set to 95%. The algorithm first filters the instructions to be discarded in the low priority section of the sorting buffer according to the timestamp freshness. Timestamp freshness: the larger the timestamp value, the newer the instruction, and the closer the color information to the current displayed picture content. The system retains the latest few frames of instructions, and the rest of the instructions generated earlier are marked as discardable. At the same time, the algorithm dynamically calculates the current system load and processing capacity, and adjusts the aggressiveness of the discarding strategy accordingly.

[0061] Dynamic calculation of current system load and processing capacity: The system continuously monitors system key performance indicators, including receive buffer fill rate, CPU occupancy rate and input instruction frame rate, and obtains real-time load value L by calculation. Real-time load value L: L = fill rate weight × (receive buffer fill rate) + CPU occupancy rate weight × (CPU occupancy rate). Wherein the receive buffer fill rate is obtained by the ratio of the current buffer occupancy and the total capacity, and the CPU occupancy rate is directly obtained from the system kernel in real time. The fill rate weight and CPU occupancy rate weight can be set to 0.6 and 0.4 respectively. The system preset processing capacity threshold is T, which can be set to 85%. When L > T is detected, the algorithm starts a hierarchical processing strategy according to the size of the overload degree Δ = L-T. Set each level interval, for example: the mild overload interval can be set to 5%-15%, the moderate overload interval can be set to 15%-30%, and the severe overload threshold can be set to 30% or more.

[0062] When Δ is in the mild overload interval, the system starts the basic discard mode. This mode only operates in the low priority section of the sorting buffer, and after arranging in reverse order according to the timestamp, the last 20% of the old instruction frames are discarded. At the same time, the system maintains the complete color calculation function of the core area and the secondary area, and only reduces the processing granularity to reduce the load.

[0063] When Δ reaches the moderate overload interval, the system enables the enhanced discard mode. In this mode, the algorithm expands the discard range to medium priority instructions, reduces the color update frequency of the secondary area while maintaining the complete function of the core area.

[0064] When Δ exceeds the severe overload threshold, the system triggers the protective discard mode. This mode immediately suspends the color calculation tasks of all non-critical areas, and concentrates the computing resources on processing the synchronization instructions of the core visual area. At the same time, the algorithm implements an aggressive cleaning strategy in the sorting buffer, only retaining a few frames of high priority instructions with the latest timestamp, for example 3 frames, to ensure the real-time performance of the core light synchronization.

[0065] The triggering frequency of the active probing mechanism described above is not fixed during system operation, but is affected by the state of the buffer. When the number of instructions in the third-level sending buffer, i.e., the sending ready queue, continues to decrease, and the capacity ratio is below the threshold, for example, 20%, the system sends a "buffer is about to be empty" state signal to the delay probing engine. This signal triggers the probing engine to immediately increase the delay probing frequency for all associated target devices, for example, from the regular 1 per second to 3 per second. This is because the sending buffer approaching empty indicates that the next batch of instructions will soon enter the scheduling link, and the system needs to immediately obtain the latest and most accurate network delay data in order to calculate the accurate sending time for subsequent instructions. Conversely, when the first-level receiving buffer is filled with too many incoming instructions, and its fill rate exceeds the threshold, for example, 80%, the system determines that it is in a high internal processing load state. At this time, in order to avoid the probing task and the instruction processing task competing for limited computing resources and network bandwidth, the system temporarily reduces the probing frequency, and this adjustment aims to avoid wasting valuable system resources.

[0066] For example: when the system is running normally, active probing is performed at a fixed frequency of 1 per second. At this time, if the number of instructions in the third-level sending buffer decreases rapidly due to continuous sending, and its capacity ratio falls below the threshold of 20%, the buffer manager will immediately send a high-priority "buffer is about to be empty" state signal to the delay probing engine. After receiving this signal, the probing engine quickly increases the delay probing frequency for all associated target devices from the regular 1 per second to 3 per second. The reason for this measure is that the sending buffer approaching empty state indicates that the next batch of instructions will soon enter the scheduling link, and the system must immediately obtain the latest and most accurate network delay data in order to calculate the accurate sending time for subsequent instructions, avoiding scheduling deviation due to the use of outdated delay information.

[0067] Conversely, when the first-level receiving buffer is filled with too many incoming instructions due to a sudden surge, and its fill rate exceeds the set threshold of 80%, the system determines that it is in a high internal processing load state. In this case, in order to prioritize the limited computing resources and network bandwidth for the core task of instruction processing, the system temporarily reduces the frequency of active probing from 1 per second to 1 per 3 seconds. Through this dynamic frequency reduction strategy, the system effectively reduces the occupation of valuable resources by the probing task, thereby ensuring the smooth and stable flow of instruction processing under high load conditions.

[0068] In the final scheduling stage of the instructions, the system adopts a comprehensive delay prediction strategy. The system monitors the queue state of the first-level receiving buffer in real time and quantizes it as a key delay compensation item. Specifically, the scheduler continuously obtains the current number of instructions in the first-level receiving buffer and dynamically estimates an expected buffer queuing delay by combining the system's recent average instruction processing speed, for example, calculated from the number of instructions processed in the past 100 milliseconds. For example, if there are 15 frames of instructions waiting to be processed in the current buffer and the system's processing speed is 150 frames per second, the estimated buffer queuing delay is 15 ÷ 150 × 1000 = 100 milliseconds. Subsequently, the system calculates the total delay for final scheduling using the following formula: final scheduling delay = network delay predicted by Kalman filter + buffer queuing delay. For example, the Kalman filter predicts that the network delay of device A is 50 milliseconds, and the current estimated buffer queuing delay is 100 milliseconds, so the total delay used to calculate the sending time is 150 milliseconds. This mechanism takes the degree of data processing congestion in the system into account when considering the global synchronization time, allowing the system to consider both the external network environment and the internal buffer processing load when "giving advance" to instructions, thereby significantly improving the overall accuracy of instruction synchronization scheduling. Subsequently, forward-looking instruction scheduling is implemented. For each video frame, the instruction sending time is calculated based on the final scheduling delay of each device , where is the instruction sending time, the actual sending time of the instruction calculated by the system based on the target time and the predicted delay; is the target presentation time, representing the absolute timestamp of the expected execution of the instruction on the terminal device; is the device total delay, representing the predicted total time required for the instruction to be sent from the system to the device for execution. In the instruction scheduling stage, the module establishes a sending queue based on the presentation time and sends data according to the calculated instruction sending time. For high-delay devices, the system will send instructions into the sending buffer in advance; for low-delay devices, the sending timing will be appropriately delayed. Through this differentiated scheduling strategy, all protocol instructions are ensured to arrive at the terminal device within the target time window.

[0069] The above embodiments can be realized wholly or partially by software, hardware, firmware, or any other combination. When realized by software, the above embodiments can be realized wholly or partially in the form of a computer program product.

[0070] ​Those skilled in the art can understand that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software manner depends on the specific application of the technical solution and the constraints of the invention. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0071] In addition, each functional module in each embodiment of the present application can be integrated in one processing module, or each module can exist physically alone, or two or more modules can be integrated in one module.

[0072] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any modification or replacement within the technical scope disclosed by the present application can be easily thought by those skilled in the art, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0073] Finally: the above is merely preferred embodiments of the present application, and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A low latency multi-protocol intelligent lighting synchronization system based on a unified abstraction layer, characterized in that, The method comprises the following steps: a video analysis engine module acquires original video frames and encapsulates them into standardized JSON synchronization frames; an intermediate layer abstraction module receives the JSON synchronization frames and performs data distribution; a multi-protocol bridge module converts the unified JSON instructions into instructions in the format specific to each protocol; a synchronization control module controls the synchronized sending of instructions by predicting the delay through Kalman filtering and combining a three-level ring buffer with forward-looking scheduling. The synchronization control module receives instructions from the multi-protocol bridge module, acquires network round-trip time data for terminal devices through an active probing mechanism, and uses a Kalman filtering algorithm to construct a delay prediction model based on the network round-trip time data to predict the future network delay of the terminal devices. The instructions are input to a three-level buffer for processing. The three-level buffer includes a receiving buffer, a sorting buffer, and a sending buffer connected in sequence. In the sorting buffer, the instructions are sorted based on their target presentation timestamps, priorities, and future network delays of the corresponding terminal devices. The trigger frequency of the active probing mechanism is dynamically adjusted according to the load state of the three-level ring buffer. Finally, the buffer queuing delay is calculated based on the load state of the receiving buffer, and the final sending time of the instructions is obtained by combining the future network delay of the terminal devices. The instructions are sent from the sending buffer to the corresponding terminal devices at the final sending time. The trigger frequency of the active probing mechanism is dynamically adjusted according to the state of the three-level ring buffer system: when the capacity ratio of the sending buffer is below a first threshold, the trigger frequency of the active probing mechanism is increased; when the fill rate of the receiving buffer is above a second threshold, the trigger frequency of the active probing mechanism is decreased. In the sorting buffer, the instructions are sorted according to a preset delay tolerance threshold, which marks instructions with predicted network delays exceeding the delay tolerance threshold as high-delay-risk instructions. The instructions are classified according to the instruction priority rules, which include: global color change instructions have the highest priority, regional color adjustment instructions have medium priority, and subtle color correction instructions have low priority. Within the same priority group, the instructions are sorted according to the target presentation timestamps. The method further includes an adaptive frame discarding step: when the system load exceeds the processing capacity threshold, some instructions in the sorting buffer are discarded according to the freshness of their timestamps, priorities, and delay sensitivity, following a preset strategy.

2. The low latency multi-protocol intelligent lighting synchronization system based on a unified abstraction layer according to claim 1, characterized in that, The adaptive frame discarding step starts different discarding modes according to the real-time load value of the system: in mild overload, only low-priority instructions are discarded; in moderate overload, some medium-priority instructions are discarded; in severe overload, only a certain number of high-priority instructions with the latest timestamps are retained.

3. The low latency multi-protocol intelligent lighting synchronization system based on unified abstraction layer according to claim 1, characterized in that, The video analysis engine module performs the following operations: acquires original video frames and performs downsampling processing; calculates the average color value of the entire frame using a global color equalization algorithm; extracts the representative color of the user's preset region of interest using an ROI region mapping algorithm; and encapsulates the global average color value and the regional representative color together with the timestamp into a standardized JSON synchronization frame.

4. The low latency multi-protocol intelligent lighting synchronization system based on a unified abstraction layer of claim 1, wherein, ​ 5. A low latency multi-protocol intelligent lighting synchronization system based on a unified abstraction layer according to claim 4, characterized in that, ​ 6. The low latency multi-protocol intelligent lighting synchronization system based on a unified abstraction layer of claim 1, wherein, ​ 7. The low latency multi-protocol intelligent lighting synchronization system based on unified abstraction layer according to claim 1, characterized in that, The intermediate layer abstract module performs the following operations: receiving the JSON synchronous frame from the video parsing engine module through the publish-subscribe mechanism; performing version parsing on the received JSON synchronous frame, and selecting a corresponding data extraction strategy according to a version number; Based on the device-protocol registry, the parsed data field is distributed to the protocol bridge module.

Images