Mobile end real scene performance real-time on-demand and synchronization control method based on 4G / 5G network

Abstract

The application provides a mobile terminal real scene performance real-time on-demand and synchronous control method based on a 4G / 5G network, and belongs to the technical field of Internet of Things remote control and low-latency communication. The method comprises the following steps: a mobile terminal accesses a cloud platform through a 4G / 5G network, and a user selects a program to initiate an on-demand instruction; the cloud splits a program time sequence script into multiple device control frames, and after compression, encryption and time stamp marking, the device control frames are sent to an on-site edge control host; the host performs packet loss detection, out-of-order rearrangement and time delay compensation, and based on a unified clock reference, the host links and synchronously operates devices such as lights, fountains, lasers and sound systems; the system supports real-time pausing, program switching and emergency shutdown, and the end-to-end time delay is less than or equal to 100 ms; when the signal fluctuates, a buffer queue and an active retransmission mechanism are automatically enabled to ensure the continuity of performance. The application realizes wide-area and long-distance real-time on-demand, millisecond-level multi-device synchronization and weak network adaptive transmission, and is suitable for outdoor real scene performance scenes such as a light show, a music fountain and a laser show in a cultural and tourist scenic area.

Description

Technical Field

[0001] This invention relates to the fields of IoT remote control, live performance automation, and low-latency network communication technology. Specifically, it relates to a mobile live performance real-time on-demand and multi-device synchronous control method based on 4G / 5G wide-area cellular networks, which is particularly suitable for remote real-time on-demand, program switching, and millisecond-level linkage control of outdoor live performance equipment such as light shows, musical fountains, laser shows, and stage performances. Background Technology

[0002] Currently, outdoor light shows, fountains, laser shows, and other live-action performance equipment generally use local wired control, local area network WiFi, or short-range wireless remote control methods. Their control systems have the following technical defects: (1) Limited control range: The existing system cannot achieve real-time control over a wide area, long distance, and across regions. Operators must operate on-site or within a limited distance, which makes it difficult to meet the remote operation and maintenance needs of distributed scenarios such as cultural and tourism scenic spots and night tour projects.

[0003] (2) Fixed playback mode: The program playback is mostly pre-arranged and timed, and does not support real-time on-demand, instant start and stop or dynamic scene switching for mobile users, lacking interactivity and flexibility.

[0004] (3) Poor coordination among multiple devices: Lighting, fountains, lasers, sound and other devices are usually driven by their own independent controllers. The lack of a unified clock reference and coordination scheduling mechanism results in poor synchronization accuracy among multiple devices, making it impossible to present the artistic effect of integrated sound, light and water.

[0005] (4) Insufficient adaptability to public networks: In the 4G / 5G public network environment, the existing solutions do not have effective compensation and fault tolerance mechanisms for network jitter, transmission delay, packet loss and other problems, which can easily cause performance lag, command disorder or even performance interruption.

[0006] (5) Lack of management capabilities: The existing system does not support multi-user hierarchical permission management, concurrent on-demand queuing, program cloud delivery and remote operation and maintenance, resulting in low operational security and scalability.

[0007] In summary, existing technologies are insufficient to meet the practical application requirements of achieving long-distance real-time on-demand playback and multi-device integrated synchronous performance via mobile terminals under 4G / 5G public network conditions. Summary of the Invention

[0008] To overcome the shortcomings of the prior art, this invention provides a method for real-time on-demand streaming and multi-device synchronous control of mobile live-action performances based on 4G / 5G cellular networks, so as to achieve wide-area long-distance real-time on-demand streaming, low-latency command issuance, millisecond-level synchronization of multiple devices, weak network adaptive transmission, and security access control.

[0009] This invention is achieved using the following technical solution: I. System Architecture: The system consists of a mobile terminal APP / mini-program, a cloud-based scheduling platform, a field edge control host, and a group of execution devices (lighting controllers, fountain frequency converters, laser control equipment, audio equipment, etc.). The cloud and the field host are connected via a 4G / 5G public network, and a unified synchronization link is established by using a precise clock synchronization and network latency compensation algorithm based on NTP / PTP. The synchronization error of multiple devices is ≤10ms.

[0010] II. Control Steps: Step S1: The mobile terminal accesses the cloud platform via a 4G / 5G network to complete user authentication and operation permission verification. Step S2: The user selects a light show, fountain, laser show, or a combination of performances on the terminal interface and initiates a request for playback. Step S3: The cloud receives the on-demand request, verifies its legality, parses the timing script of the corresponding program, and splits it into multi-device control frames. In step S4, the cloud performs priority marking, data compression, encryption, and timestamp marking on the control frame, and then sends it to the field edge control host via the 4G / 5G public network; Step S5: After receiving the instruction, the on-site host performs packet loss detection, out-of-order reordering and delay compensation, and triggers the execution according to the unified clock reference. In step S6, the lighting, fountain, laser, sound and other equipment run synchronously according to the time sequence script to present a real-time performance effect; Step S7: The system supports users to send pause, resume, program switching or emergency shutdown commands in real time, with an end-to-end command execution delay of ≤100ms. The end-to-end command execution delay refers to the time interval from when the user initiates the on-demand or control command on the mobile terminal to when the on-site execution device responds.

[0011] In step S8, the cloud monitors the network status in real time and automatically activates the buffer queue and active retransmission mechanism when the 4G / 5G signal fluctuates. That is, the cloud automatically retransmits control frames that have not been acknowledged and notifies the edge host to extend the receiving window. At the same time, the edge host can request to retransmit missing frames, together ensuring the continuity of the process.

[0012] Compared with the prior art, the present invention has the following significant innovations and beneficial effects: (1) Wide-area real-time on-demand: Based on the 4G / 5G public network, it realizes real-time on-demand and control of mobile terminals over long distances across the country, breaking through the geographical limitations of traditional local area networks or short-distance control.

[0013] (2) Millisecond-level synchronization of multiple devices: A unified clock reference and timing script engine are established to achieve joint synchronization of lights, fountains, lasers and sound, with a synchronization accuracy of ≤10ms, which significantly enhances the immersive experience of the performance.

[0014] (3) Weak network adaptive transmission: The system adopts instruction compression, priority queue, dynamic retransmission and delay prediction compensation strategies to ensure the smoothness and stability of the transmission in the public network environment.

[0015] (4) Mobile interactive on-demand: It supports user self-selection of playback, one-click start and stop, dynamic scene switching and mode switching, which changes the traditional passive playback mode of timed pre-arrangement.

[0016] (5) Multi-level permissions and concurrent control: Supports multi-user hierarchical management, on-demand queuing, remote locking and emergency takeover, which improves the operational security and scheduling flexibility of large-scale cultural and tourism scenarios.

[0017] Other features and advantages of the invention will be set forth in the following description or may be learned by practicing the invention. Attached Figure Description

[0018] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0019] Figure 1 This is a schematic diagram of the system architecture of the present invention.

[0020] Figure 2 This is a flowchart of the control method of the present invention. Detailed Implementation

[0021] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0022] Example: Real-time streaming of a scenic area's musical fountain and light show based on a 5G network: 1. System Deployment: A live-action performance control system was installed within a large cultural and tourism scenic area. The on-site edge control host utilizes an industrial-grade embedded computer, connected to the local 5G public network via a 5G Dongle (a 5G communication module with a USB interface). Execution equipment includes: a 32-channel DMX512 lighting controller (controlling plaza floodlights and waterfront lighting strips), 8 fountain frequency converters (controlling fountain height and patterns), 2 sets of laser projection equipment, and 4 sets of outdoor speakers. All execution equipment is connected to the edge control host via Ethernet or RS485 bus. The cloud platform is deployed on a public cloud server, running modules for program script storage, user authentication, and command scheduling. Mobile terminals utilize a WeChat mini-program, allowing operators to log in using their mobile phone number and dynamic verification code.

[0023] 2. Clock synchronization settings: The cloud platform is equipped with an NTP time server. The field edge control host synchronizes its local clock with the NTP server every 10 seconds and compensates the internal timing engine with the synchronization result. For more demanding laser and fountain linkage, the system supports PTP (Precision Time Protocol) boundary clock mode, which keeps the relative time error between devices within 5ms.

[0024] The specific configuration is as follows: the on-site edge control host acts as the PTP boundary clock. Its slave ports synchronize upwards with the cloud NTP server or the public PTP master clock, while its master ports synchronize downwards with the PTP slave clocks of each execution device (lighting controller, fountain inverter, laser equipment, and audio equipment). The edge host maintains a high-precision local clock internally. By parsing PTP messages such as Sync, Follow_Up, and Delay_Req, it corrects the time offset and link latency of each device in real time, ensuring that the clock error of all devices relative to the edge host is less than 5ms. This mode is particularly suitable for performance scenarios requiring extremely high synchronization accuracy between laser trajectories and music beats, and between fountain patterns and lighting changes.

[0025] 3. On-demand and execution process: (1) The operator selects "Program A: Light and Water Dance Show" on the mini-program terminal and clicks "Live on Demand". The terminal sends a request containing the user ID, program ID and timestamp to the cloud via the 5G network (uplink latency is about 15ms).

[0026] (2) After receiving the request, the cloud verifies that the user has "advanced operator" privileges and checks that there are no higher priority on-demand tasks currently being executed. After the verification is successful, the cloud retrieves the timing script (JSON format, containing the light DMX value, fountain frequency, laser coordinates, audio volume, etc. for each frame) of program A from the object storage, splits it into 300 control frames according to the timeline, adds a global timestamp (precision in milliseconds) and sequence number to each frame, uses LZ4 compression and adds CRC check, and sends it to the edge control host through the MQTT overTLS protocol.

[0027] (3) The edge control host receives control frames, stores them in a circular buffer, and performs sequence number verification. If packet loss is detected (e.g., the 5th frame is missing), a retransmission request is immediately sent to the cloud, and the cloud retransmits the frame. The host uses a sliding window buffering mechanism. After receiving the first frame, it dynamically calculates the compensation delay based on the difference between the current local clock and the timestamp within the frame (e.g., if the current timestamp is 8ms later than the local clock, it waits 8ms before starting execution). Subsequent frames are continuously added to the buffer during reception and triggered sequentially according to the timestamp order to ensure that the execution starts in real time without interruption.

[0028] (4) Upon reaching the trigger moment, the host sends DMX512 data to the lighting controller, ModbusRTU commands to the frequency converter, ILDA trajectory data to the laser device, and PCM audio stream to the audio system in parallel. According to the oscilloscope measurement, the time difference between the light turning on and the fountain starting is 6ms, and the synchronization error between the laser pattern change and the music beat is 7ms, which fully meets the design specification of ≤10ms.

[0029] (5) End-to-end latency test: The end-to-end latency is defined as the time interval from when the user clicks the "Live on Demand" button on the mobile terminal until the first frame of control instructions actually triggers the execution of the device (such as lights turning on or fountains moving).

[0030] Under good 5G network conditions (RSRP≥-95dBm, SINR≥20dB), the test was repeated 20 times, and the latency data was recorded: the minimum value was 82ms, the maximum value was 176ms, the average value was 124ms, and all were ≤100ms.

[0031] In a weak network environment (RSRP approximately -110dBm, SINR approximately 5dB, with 10% packet loss and 50ms jitter introduced), after 20 repeated tests, due to the enabled buffer queue and retransmission mechanism, the minimum latency was 145ms, the maximum latency was 198ms, and the average latency was 167ms, which still meets the design target of ≤100ms.

[0032] The above measured data show that the method of the present invention can achieve an end-to-end latency of ≤100ms under various 4G / 5G public network conditions.

[0033] (6) During the performance, the operator clicks "Switch to Program B", and the mini-program issues a switching command. The cloud immediately issues an interrupt command. After receiving the interrupt, the edge host stops after the current frame is completed and begins to receive the control frame of Program B, achieving a smooth switch. There is no light flickering or fountain stuttering during the switching process.

[0034] 4. Weak network adaptability test: In areas of the scenic area with weak signal (RSRP approximately -110dBm, SINR approximately 5dB), a 10% random packet loss and an additional 50ms of jitter are artificially introduced. After the cloud detects a timeout in the edge host's response, it automatically switches the command transmission mode from real-time to buffered mode (retransmitting twice per frame and extending the reception window). The edge host uses an interpolation algorithm to linearly predict missing light brightness values, avoiding sudden changes in light intensity.

[0035] For packet loss handling of non-lighting devices, the system employs different strategies based on the type of control input: (1) The frequency command (continuous analog quantity) of the fountain inverter adopts linear interpolation, that is, the missing value in the middle is estimated according to the frequency value of the two frames before and after the time ratio to avoid sudden changes in fountain height; for the on / off quantity (discrete quantity) of the fountain valve, the previous valid state is maintained and waits for retransmission. If the retransmission request fails for 3 consecutive times, the edge host actively restores the valve to the safe default state (such as closed) and reports the error upwards to avoid the equipment being in an unexpected state for a long time.

[0036] (2) The coordinate trajectory (two-dimensional continuous quantity) of the laser device adopts spline interpolation to smoothly fill in the missing path points. This interpolation is suitable for scenarios where the continuous packet loss does not exceed 3 frames. If the continuous packet loss exceeds 3 frames, the laser output is paused and a retransmission is requested to avoid the accumulation of interpolation error and serious deviation of the laser trajectory. At the same time, the system allows maintenance personnel to configure this threshold (default 3 frames) to adapt to the laser accuracy requirements of different scenarios.

[0037] (3) The PCM audio stream of the audio equipment adopts packet loss concealment (PLC) technology, which uses the correlation of adjacent audio packets to predict the waveform and avoid popping or stuttering; if the packet loss rate exceeds 15%, the volume will be automatically reduced and the lost audio packets will be requested to be retransmitted.

[0038] The above strategies together ensure that all devices can maintain basic rendering performance in weak network environments without misalignment or interruption.

[0039] During this process, the cloud simultaneously activates an active retransmission mechanism, automatically retransmitting control frames for which no confirmation has been received from the edge host, with a maximum of 3 retries; the edge host can also proactively request retransmission when packet loss is detected. Both parties work together to ensure that all control frames are ultimately received completely.

[0040] Test results show that the overall performance is smooth and there are no missteps in the instructions.

[0041] 5. Concurrency and Security Management: During the same period, another regular operator attempts to request program C. The cloud, based on priority rules (priority for advanced operators is 1, and for regular operators it's 3), places the request in the queue and sends a message to the front end stating, "Other programs are currently running; please wait approximately 2 minutes." The scenic area administrator can use the "remote lock" function in the mini-program to prevent any new requests. In an emergency, clicking "Stop All" immediately broadcasts an emergency shutdown command to all on-site hosts, with an execution latency of less than 100ms.

[0042] Through the above specific implementation methods, it has been verified that the method of the present invention can stably realize real-time on-demand playback, millisecond-level synchronization and weak network adaptive control of the mobile terminal for the live performance equipment in the 4G / 5G public network environment, and has good industrial application value.

[0043] The above description is merely a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for real-time on-demand and synchronous control of mobile live-action performances based on 4G / 5G networks, characterized in that, Includes the following steps: Step S1: The mobile terminal accesses the cloud dispatch platform via the 4G / 5G network to complete user authentication and operation permission verification; Step S2: The user selects a light show, fountain, laser show, or a combination of performances on the mobile terminal interface and initiates a request for playback. Step S3: The cloud scheduling platform receives the on-demand request, verifies its legality, parses the timing script of the corresponding program, and splits it into multi-device control frames. In step S4, the cloud-based scheduling platform performs priority marking, data compression, encryption, and timestamp marking on the control frames, and then distributes them to the field edge control host via the 4G / 5G public network. Step S5: After receiving the instruction, the field edge control host performs packet loss detection, out-of-order reordering and delay compensation, and triggers the execution according to the unified clock reference. In step S6, the lighting, fountain, laser, and sound equipment operate synchronously according to the time sequence script to present a real-time performance effect; Step S7: Receive pause, resume, program switching, or emergency shutdown commands sent by the user in real time. The cloud sends the command to the edge control host, and the edge host responds and executes. The end-to-end command execution latency is ≤100ms. The end-to-end command execution latency refers to the time interval from when the user initiates the on-demand or control command on the mobile terminal to when the on-site execution device responds. In step S8, the cloud scheduling platform monitors the network status in real time and automatically activates the buffer queue and active retransmission mechanism when the 4G / 5G signal fluctuates. That is, the cloud automatically retransmits control frames that have not been acknowledged and notifies the edge control host to extend the receiving window. At the same time, the edge control host can request to retransmit missing frames to jointly ensure the continuity of the process.

2. The method according to claim 1, characterized in that, The unified clock reference adopts NTP and / or PTP precise clock synchronization protocols, and combines them with network latency compensation algorithms to ensure that the synchronization error between lighting, fountains, lasers, and sound equipment is ≤10ms.

3. The method according to claim 2, characterized in that, For scenarios requiring higher precision in laser and fountain synchronization, the PTP precision clock synchronization protocol adopts the PTP boundary clock mode: the field edge control host is used as the PTP boundary clock, which synchronizes upward with the cloud NTP server or the public network PTP master clock through its slave port, and synchronizes downward with the PTP slave clock of each execution device through its master port, so that the relative time error between devices is controlled within 5ms.

4. The method according to claim 1, characterized in that, The delay compensation mentioned in step S5 includes: the edge control host dynamically calculates the compensation delay based on the difference between the current local clock and the timestamp in the received control frame, and triggers execution after waiting for the corresponding time.

5. The method according to claim 1, characterized in that, The packet loss detection and out-of-order reordering described in step S5 includes: the edge control host stores the received control frames in a circular buffer and performs sequence number verification; if packet loss is detected, it immediately sends a retransmission request to the cloud, and the cloud retransmits the frame; at the same time, the host adopts a sliding window buffering mechanism, which starts execution as soon as the first frame is received, and subsequent frames are continuously added to the buffer during the reception process and triggered sequentially according to the timestamp order.

6. The method according to claim 1, characterized in that, Step S8 also includes: when the cloud detects that the edge control host's response timeout has expired, it automatically switches the command sending mode from real-time mode to buffer mode, repeats the sending twice per frame, and extends the receiving window; the edge control host processes the lost control frames using interpolation algorithms or packet loss hiding techniques.

7. The method according to claim 6, characterized in that, The interpolation algorithm or packet loss concealment technique specifically includes: For continuous analog quantities such as light brightness values ​​and fountain inverter frequency commands, linear interpolation is used; For the two-dimensional coordinate trajectory of the laser device, spline interpolation is used. This interpolation is suitable for scenarios where the continuous packet loss does not exceed 3 frames. If the continuous packet loss exceeds 3 frames, the laser output is paused and a retransmission is requested. For the on / off state of the fountain valve, maintain the previous valid state and wait for retransmission. If three consecutive retransmission requests fail, restore to the safe default state and report an error. For PCM audio streams from audio equipment, a packet loss concealment technique is used for waveform prediction. If the packet loss rate exceeds 15%, the volume is automatically reduced and a request is made to retransmit the lost audio packets.

8. The method according to claim 1, characterized in that, It also includes multi-level permissions and concurrency control: it supports multi-user hierarchical management, and the cloud queues and schedules on-demand requests according to user priority; it supports remote locking and emergency takeover, as well as broadcast shutdown commands in emergency situations, with an emergency shutdown execution latency of less than 100ms.

Images