A 3D interactive control system for LED cloud screens

Abstract

The present invention discloses a 3D interactive control system for an LED cloud screen, which relates to the field of cloud screen interactive control technology. The system includes: a content input module that receives multi-source signal content from the outside and converts it into an SDS stream; a 3D rendering and processing module that performs image generation and effect optimization, generating a final display image step by step; a display output module that converts the final display image into a data frame that can be directly loaded and displayed by the LED cloud screen; a monitoring and maintenance module that performs real-time perception, fault prediction, and remote management of the operating status of the LED cloud screen system, and ultimately outputs system status information and maintenance response instructions; and an interactive control module that parses and responds to control event data in the SDS stream, converting structured control events into standardized system control instructions. The present invention improves the operational stability and interactive performance quality of the cloud screen system and reduces the failure rate during long-term use.

Description

Technical Field

[0001] The present invention relates to the field of cloud screen interactive control technology, and in particular to an LED cloud screen 3D interactive control system. Background Art

[0002] With the rapid development of LED display technology and multimedia rendering capabilities, interactive display systems based on large LED screens have been widely used in digital exhibition halls, immersive demonstrations, cultural tourism scenarios, and other fields. Existing systems mostly use pre-rendered content and split-screen splicing mechanisms, triggering content and switching states through local input devices. Some systems have rudimentary environmental monitoring and temperature control capabilities and support basic remote maintenance operations, generally meeting the needs of general display and simple interaction.

[0003] However, the current LED cloud screen system generally has three technical bottlenecks: at the interaction level, multi-channel input methods have not yet achieved fusion analysis, making it difficult to support complex and diverse user behavior input; at the rendering and display level, the three-dimensional image processing and multi-perspective image synthesis capabilities are insufficient, and there are edge distortions and visual faults after splicing, affecting the overall immersive experience; at the operation control level, the system lacks closed-loop monitoring of the image processing link and display terminal, the operation status data is scattered, and the maintenance response is delayed, resulting in a high failure rate and decreased brightness consistency during long-term operation, which seriously restricts the system stability and maintainability.

[0004] In response to the above problems, the present invention provides a 3D interactive control system for LED cloud screens, constructs a data processing link with SDS stream as the core, and realizes content-driven real-time image generation and response output through multi-modal input recognition technology, GPU three-dimensional rendering mechanism, multi-perspective and light field image synthesis strategy; at the same time, FPGA image drive, HDR color mapping and PTP synchronization mechanism are introduced to ensure consistent display of multi-screen images. Summary of the Invention

[0005] In response to the above problems, the present invention provides a 3D interactive control system for an LED cloud screen to solve the problems in the prior art of poor display consistency, unstable interactive response and high failure rate of LED cloud screens during long-term operation.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: an LED cloud screen 3D interactive control system, comprising the following modules:

[0007] The content input module converts multi-source signal content into SDS streams through a multi-protocol parser, unified frame format encapsulation logic, and control instruction parsing mechanism;

[0008] The 3D rendering and processing module uses GPU-accelerated rendering engines, multi-view synthesis, and light field rendering technologies to perform 3D scene rendering and physical rendering optimization on the rendering frame data in the SDS stream, and gradually generates the final display image.

[0009] The display output module uses image slicing algorithm, FPGA image driving technology, PTP clock synchronization mechanism and HDR mapping algorithm to perform split-screen reconstruction and image enhancement processing on the final display image data to obtain structured LED loading frame data and full-screen refresh synchronization signal;

[0010] The monitoring and maintenance module uses multi-source sensor monitoring, status diagnosis algorithms, and cloud-based operation and maintenance mechanisms to perform closed-loop control of LED sub-screens, image processing links, and environmental parameters, ultimately outputting system status information and maintenance response instructions.

[0011] The interactive control module converts structured control events into standardized system control instructions through input recognition algorithms, event modeling mechanisms and system instruction mapping mechanisms, driving image content changes, mode switching and interface feedback.

[0012] The content input module includes a network streaming media input unit, a local media playback unit and an interactive instruction input unit;

[0013] The network streaming media input unit decapsulates the remote real-time video stream through a multi-protocol parser to obtain a compressed video stream, and then uses the GPU accelerated decoding component to decode the compressed video stream in real time and output standard frame image data;

[0014] The local media playback unit uses a built-in multimedia parser and video codec library to read local multimedia files, extract key frame image data, and output image frame formats consistent with those of the network streaming media input unit;

[0015] The interactive instruction input unit is used to receive interactive instruction information, use the multimodal interactive parser to extract the user intention in the interactive instruction information, and generate structured control events.

[0016] The 3D rendering and processing module includes a rendering engine unit, a multi-view synthesis unit and a light field rendering unit;

[0017] The rendering engine unit uses the GPU accelerated rendering engine to process the rendering frame data in the SDS stream in real time to generate a 3D image, uses physical rendering technology to process the noise effect in the 3D image, and outputs level 1 rendering image data;

[0018] The multi-view synthesis unit processes the first-level rendered image data through a perspective conversion algorithm, calculates the image content at different viewpoints, and generates second-level image data adapted to different viewpoints. The calculation process is based on the depth information of the scene and uses a stereo vision algorithm to generate image frames at different viewpoints.

[0019] The light field rendering unit is used to perform light field reconstruction and image enhancement processing on the level 2 image data. By sampling light information from multiple viewpoints, it synthesizes 3D image frames with a sense of depth and depth of field, generates level 3 rendered image data, and obtains the final display image;

[0020] During the processing, a four-dimensional light field model is constructed based on multi-view images. ,in represents the image plane coordinates, Represents the light angle coordinates, and uses this model to perform pixel-level light resampling and depth of field control algorithm processing.

[0021] The display output module includes a multi-screen splicing control unit, an LED drive output unit, a synchronization control unit, and an HDR mapping and brightness balancing unit;

[0022] The multi-screen splicing control unit uses image slicing algorithms and spatial mapping logic to partition the final display image data. It divides the entire frame image into multiple sub-blocks based on the preset splicing matrix, and eliminates image distortion and splicing errors through geometric correction and edge compensation algorithms to obtain regional image data that matches the layout of each LED sub-screen.

[0023] The HDR mapping and brightness balancing unit remaps the brightness and color information in the regional image data through HDR tone mapping and dynamic range compression algorithms. It uses a brightness consistency analysis model to adjust the output parameters of each region based on the physical response differences of the LED sub-screens to generate optimized image data.

[0024] The LED driver output unit converts the pixel format of the optimized image data and encapsulates the underlying control signals through the FPGA image driver component to generate LED loading frame data that meets the loading requirements of the LED controller;

[0025] The synchronization control unit schedules and controls the refresh process of the LED loading frame data, generates a global refresh synchronization signal, uses the system master clock as a reference, and performs dynamic phase correction in combination with the sub-screen response delay.

[0026] The monitoring and maintenance module includes a sensing and adaptive control unit, a fault diagnosis unit and a remote management unit;

[0027] The sensing and adaptive control unit monitors the operating environment of each LED sub-screen and image processing module in real time through an environmental sensor array, obtains multi-dimensional operating data, uses control algorithms to perform threshold judgment and trend analysis on the multi-dimensional operating data, adjusts fan speed, LED brightness and power supply voltage, and outputs the adjusted parameters as the current state parameters;

[0028] The fault diagnosis unit uses a joint diagnosis algorithm based on a rule base and a machine learning model to perform pattern recognition and anomaly detection on current state parameters to obtain diagnostic labels and alarm events;

[0029] The remote management unit establishes a data exchange channel with the cloud platform based on the MQTT IoT communication protocol. It regularly receives the current status parameters output by the sensing and adaptive control unit and the diagnostic tags and alarm events provided by the fault diagnosis unit, and sends them to the cloud as uploaded data. It also provides an interactive interface for operation and maintenance personnel through a web visualization interface.

[0030] Based on the uploaded status parameters and fault logs, the remote management unit combines with the cloud-based operation and maintenance strategy model to generate system maintenance reports, control strategy adjustment instructions, and upgrade scheduling commands as the response basis for the remote maintenance strategy, and uniformly encapsulates the above processing results into system status information and maintenance response instructions.

[0031] The interactive control module includes a multimodal input recognition unit, an event modeling unit and a logic control unit:

[0032] The multimodal input recognition unit processes voice signals, body images, and remote input through three technical paths: voice recognition, gesture recognition, and mobile device communication. It extracts the original interaction intent and aggregates it to generate the original interaction event stream:

[0033] The user's voice signal is collected through the microphone, and the locally deployed speech recognition model is used to decode the signal and extract keywords, recognize the control command, and output the voice event;

[0034] The user's skeleton posture image is acquired through a depth camera, and the Open Pose posture recognition algorithm is used to perform time series analysis on the key point coordinates, identify gestures, and output somatosensory events.

[0035] By establishing a mobile device communication interface, the interactive command data from the user's mobile phone is received and parsed into remote control events;

[0036] The event modeling unit uses the instruction abstraction algorithm to perform structured modeling on the original interactive event stream, identify the event type, target object, and parameter information, and complete the state mapping between events and instructions based on the current system context state to generate a standard interactive instruction set;

[0037] The logic control unit, based on the state machine control model, combines the system's current operating status, task priority, and instruction execution logic to perform logical analysis, conflict judgment, and path distribution processing on the standard interactive instruction set; through the built-in conflict detection and response scheduling mechanism, it identifies and resolves priority conflicts and execution order issues between instructions; and outputs the scheduling results as image control signals and system feedback control signals.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] By setting up a monitoring and maintenance module, the present invention constructs a multi-dimensional environmental perception and feedback mechanism, which can monitor the operating environment parameters of the LED sub-screen and image processing module in real time, and dynamically adjust the fan speed, brightness and voltage based on trend analysis, thereby effectively avoiding system failures caused by local overheating, power supply fluctuations and other problems, and significantly improving the long-term operation stability of the cloud screen system under complex or high-load conditions.

[0040] The present invention introduces tone remapping, dynamic range compression and sub-screen brightness difference correction mechanisms through the HDR mapping and brightness balancing unit established in the display output module, effectively solving the common problems of color inconsistency and unnatural edge transitions in multi-screen splicing systems.

[0041] The present invention builds a consistency model based on the physical response characteristics of each sub-screen, realizes fine adjustment at the pixel level, and fundamentally improves the image consistency and visual comfort of large-scale LED cloud screens under long-term use.

[0042] The present invention builds a complete interactive control module that supports voice, somatosensory movements, and mobile terminal input channels, and cooperates with event modeling and state control logic to achieve efficient response to multimodal interactive behaviors and dynamic content updates. With the cooperation of multi-view rendering and light field enhancement, it can realize real-time 3D image changes and system feedback based on audience behavior, greatly improving the interactive response speed and immersiveness.

[0043] In terms of fault prediction and remote maintenance, the present invention has a joint diagnosis algorithm based on a rule base and a machine learning model, which can accurately identify trend faults and hidden anomalies, and combine with the MQTT protocol to realize remote data upload and maintenance strategy issuance, forming an intelligent self-diagnosis and closed-loop operation and maintenance system. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. It is understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 It is a system architecture diagram of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but is merely for selected embodiments of the present invention.

[0047] Please refer to Figure 1 , Figure 1 This is an architecture diagram of an LED cloud screen 3D interactive control system provided by an embodiment of the present invention, including the following modules:

[0048] The content input module is used to receive multi-source signal content from the outside, and convert the multi-source signal content into SDS streams through a multi-protocol parser, unified frame format encapsulation logic and control instruction parsing mechanism, and transmit them to the 3D rendering and processing module and the interactive control module respectively; the multi-source signal content includes remote real-time video streams, local multimedia files and interactive instruction information.

[0049] Among them, the SDS stream includes two types of data: the first type is rendering frame data, which is a unified frame image sequence output by the video decoder for 3D rendering and splicing control; the second type is control event data, which is a structured control command output by the interactive input parser, including instruction type, target object and execution parameters.

[0050] The content input module includes a network streaming media input unit, a local media playback unit and an interactive instruction input unit;

[0051] The network streaming media input unit decapsulates the remote real-time video stream through a multi-protocol parser to obtain a compressed video stream; then the GPU-accelerated decoding component decodes the compressed video stream in real time and outputs standard frame image data.

[0052] The local media playback unit reads local multimedia files through the built-in multimedia parser and video codec library, extracts key frame image data, and outputs image frame format consistent with the network streaming media input unit.

[0053] The interactive instruction input unit is used to receive interactive instruction information, use the multimodal interactive parser to extract the user intention in the interactive instruction information, and generate structured control events.

[0054] It should be noted that the frame image data output by the network streaming input unit and the local media playback unit together constitute the rendering frame data, and the structured instructions output by the interactive instruction input unit constitute the control event data.

[0055] The 3D rendering and processing module is used for image generation and effect optimization. Through the GPU-accelerated rendering engine, multi-view synthesis and light field rendering technology, it performs three-dimensional scene rendering and physical rendering optimization on the rendering frame data in the SDS stream, and generates the final display image step by step.

[0056] The 3D rendering and processing module includes a rendering engine unit, a multi-view synthesis unit, and a light field rendering unit;

[0057] The rendering engine unit uses the GPU accelerated rendering engine to process the rendering frame data in the SDS stream in real time to generate a three-dimensional image, uses physical rendering technology to process the noise effect in the three-dimensional image, and outputs level 1 rendering image data. The noise effect includes physical phenomena such as lighting, shadows, and reflections.

[0058] The multi-view synthesis unit processes the level 1 rendered image data through a perspective conversion algorithm, calculates the image content under different viewpoints, and generates level 2 image data adapted to different perspectives. The calculation process is based on the depth information of the scene and generates image frames under different viewpoints through a stereo vision algorithm.

[0059] The light field rendering unit is used to perform light field reconstruction and image enhancement processing on the level 2 image data. By sampling light information from multiple viewpoints, it synthesizes 3D image frames with a sense of depth and depth of field, generates level 3 rendered image data, and obtains the final display image;

[0060] During the processing, a four-dimensional light field model is first constructed based on multi-view images. ,in, represents the image plane coordinates, Represents the light angle coordinates. This model is used to perform pixel-level light resampling and depth of field control algorithm processing to obtain higher-fidelity lighting simulation effects and a stronger sense of spatial three-dimensionality.

[0061] The display output module is used to convert the final display image into a data frame that can be directly loaded and displayed on the LED cloud screen. Through the image slicing algorithm, FPGA image driving technology, PTP clock synchronization mechanism and HDR mapping algorithm, the final display image data is reconstructed and enhanced to obtain structured LED loading frame data and full-screen refresh synchronization signal.

[0062] It should be noted that the rendering engine scheduling and image synthesis process includes four control mechanisms: dynamic balancing mechanism for rendering load, persistent caching and fallback mechanism for rendering parameters, perspective consistency verification and adaptive correction mechanism, and light field data redundancy compensation strategy;

[0063] The rendering load dynamic balancing mechanism monitors GPU resource usage and rendering task load in real time. When abnormal rendering calculation density is detected, it automatically adjusts rendering thread allocation and task queue sorting to ensure a stable and smooth image generation process.

[0064] Rendering parameter persistent caching and fallback mechanism: all key rendering parameters of 3D models, such as lighting, materials, and projection, will be configured and cached during initial rendering, and will be regularly verified and automatically rolled back during operation to avoid changes in picture style or distortion due to parameter drift after long-term operation, ensuring display consistency.

[0065] Perspective consistency verification and adaptive correction mechanism. During the multi-perspective synthesis stage, the system has built-in consistency comparison of the output image frames. Through image similarity detection and deep scene analysis, it automatically adjusts the rendering angle and color distribution under abnormal perspectives to avoid image drift, deformation and other problems caused by perspective synthesis errors.

[0066] The light field data redundancy compensation strategy addresses the problems of viewpoint image loss or noise accumulation that may occur during long-term use. The light field rendering unit introduces redundant sampling channels when constructing the four-dimensional light field model. It reconstructs and compensates low-quality images through pixel-level interpolation and spatial reconstruction technology, enhancing the stability and anti-degradation ability of image quality.

[0067] The display output module includes a multi-screen splicing control unit, an LED drive output unit, a synchronization control unit, and an HDR mapping and brightness balancing unit;

[0068] The multi-screen splicing control unit partitions the final display image data through image slicing algorithm and spatial mapping logic, divides the entire frame image into multiple sub-blocks according to the preset splicing matrix, and eliminates image distortion and splicing errors through geometric correction and edge compensation algorithms to obtain regional image data that matches the layout of each LED sub-screen.

[0069] The HDR mapping and brightness balancing unit remaps the brightness and color information in the regional image data through HDR tone mapping and dynamic range compression algorithms to enhance the image contrast and detail level. At the same time, it adopts a brightness consistency analysis model to adjust the output parameters of each area based on the physical response differences of the LED sub-screens to generate optimized image data with uniform color and balanced brightness.

[0070] The LED driver output unit, through the FPGA image driver component, performs pixel format conversion and underlying control signal encapsulation on the optimized image data to generate LED loading frame data that meets the loading requirements of the LED controller. The processing process includes row and column latch logic, current drive parameter injection and grayscale control.

[0071] The synchronization control unit schedules and controls the refresh process of the LED loading frame data, generates a global refresh synchronization signal, uses the system master clock as a reference, and performs dynamic phase correction based on the sub-screen response delay to avoid screen tearing, screen flashing, and refresh inconsistency. The generated synchronization signal is used to unify the frame switching time of all LED sub-screens to ensure that the loaded frame data is displayed synchronously on each sub-screen.

[0072] It should be noted that the display output module follows the following operations during image processing and screen driving: coordinate tolerance and boundary verification logic are integrated into the splicing and mapping process, and regional tolerance tables and verification items are introduced during image slicing and spatial mapping. When some sub-screens are abnormal, image compensation and adjustment can be automatically performed to avoid splicing misalignment and boundary defects.

[0073] A dynamic balancing self-calibration strategy is introduced during the brightness mapping process. By periodically sampling the color and brightness output values ​​of each sub-screen and comparing them with the target brightness model, the deviation value is dynamically corrected. This process can effectively suppress color drift and brightness unevenness caused by LED aging or component differences.

[0074] The image loading and refresh process uses a dual-buffer structure and grayscale control algorithm. When loading LED pixel-level data, the parallel cache structure ensures that the data is completely written before driving the output. At the same time, the grayscale transition amplitude between frames is controlled to reduce current impact, reduce image flicker, and extend LED life.

[0075] In terms of full-screen synchronization, the frame timing recording and response delay dynamic correction strategy are combined. The synchronization control unit calculates the response delay of each sub-screen in real time based on the received main control synchronization clock. By fine-tuning the frame switching timing and introducing buffered transition frames, it ensures that the screen achieves unified frame synchronization under multi-channel driving conditions, preventing visual tearing and refresh offset.

[0076] The monitoring and maintenance module is used for real-time perception of the operating status of the LED cloud screen system, fault prediction and remote management. Through multi-source sensor monitoring, status diagnosis algorithm and cloud operation and maintenance mechanism, it performs closed-loop control of the LED sub-screen, image processing link and environmental parameters, and finally outputs system status information and maintenance response instructions.

[0077] The monitoring and maintenance module includes a sensing and adaptive control unit, a fault diagnosis unit, and a remote management unit;

[0078] The sensing and adaptive control unit monitors the operating environment of each LED sub-screen and image processing module in real time through an environmental sensor array to obtain multi-dimensional operating data. It uses a control algorithm to perform threshold judgment and trend analysis on the multi-dimensional operating data, dynamically adjusts the fan speed, LED brightness and power supply voltage, and outputs the adjusted parameters as the current state parameters; the multi-dimensional operating data includes temperature, voltage, humidity and ambient light.

[0079] The fault diagnosis unit uses a joint diagnosis algorithm based on a rule base and a machine learning model to perform pattern recognition and anomaly detection on current state parameters to obtain diagnostic labels and alarm events;

[0080] The diagnostic process involves using a rule base to identify fixed-threshold anomalies, using a trained model to identify nonlinear fluctuation characteristics, determining the presence of trending faults and hidden defects, and obtaining diagnostic results. These results are used to trigger local maintenance responses and are simultaneously pushed to a remote management unit to generate cloud-based maintenance recommendations.

[0081] Among them, fixed threshold type anomalies include excessive temperature and abnormal voltage, and local maintenance responses include control measures such as forced temperature limit, channel switching, and power cut-off.

[0082] The remote management unit establishes a data exchange channel with the cloud platform based on the MQTT IoT communication protocol. It regularly receives the current status parameters output by the sensing and adaptive control unit and the diagnostic tags and alarm events provided by the fault diagnosis unit, and sends them to the cloud as uploaded data.

[0083] Provides an interactive interface for operation and maintenance personnel through a Web-based visualization interface, supports the access of remote control commands and the execution of software update tasks, and realizes centralized status monitoring of multi-point cloud screens, fault event location, and remote intervention operations;

[0084] Based on the uploaded status parameters and fault logs, the remote management unit combines with the cloud-based operation and maintenance strategy model to generate system maintenance reports, control strategy adjustment instructions and upgrade scheduling commands as the response basis for the remote maintenance strategy. The above processing results are uniformly encapsulated as system status information and maintenance response instructions, and returned to the system master control node for closed-loop scheduling and control execution.

[0085] The interactive control module is used to parse and respond to the control event data in the SDS stream. Through the input recognition algorithm, event modeling mechanism and system instruction mapping mechanism, it converts structured control events into standardized system control instructions, drives image content changes, mode switching and interface feedback, and realizes interactive response between users and the LED cloud screen system.

[0086] The interactive control module includes a multimodal input recognition unit, an event modeling unit, and a logic control unit:

[0087] The multimodal input recognition unit processes voice signals, body images, and remote input through three technical paths: voice recognition, gesture recognition, and mobile device communication. It extracts the original interaction intent and aggregates it to generate the original interaction event stream:

[0088] The user's voice signal is collected through the microphone, and the locally deployed speech recognition model is used to decode the signal and extract keywords, recognize control commands, and output voice events.

[0089] The user's skeleton posture image is obtained through the depth camera, and the Open Pose posture recognition algorithm is used to perform time series analysis on the coordinates of key points, identify gestures such as waving, clicking, and sliding, and output somatosensory events.

[0090] Establish a mobile device communication interface to access interactive command data from the user's mobile phone, including sliding tracks, answer selections, virtual button clicks and other operations, and interpret them as remote control events.

[0091] The above three types of recognition results are uniformly converted into a structured format and fused into the original interaction event stream as the input for subsequent event logic processing.

[0092] The event modeling unit uses the instruction abstraction algorithm to perform structured modeling on the original interactive event stream, identify the event type, target object, and parameter information, and complete the state mapping between events and instructions based on the current system context state to generate a standard interactive instruction set;

[0093] Among them, event types include switching, zooming and pausing, target objects include specific models and screen areas, and accompanying parameters include zoom ratio and switching target ID.

[0094] The logic control unit, based on the state machine control model, combines the system's current operating status, task priority, and instruction execution logic to perform logic analysis, conflict judgment, and path distribution on the standard interactive instruction set;

[0095] Through the built-in conflict detection and response scheduling mechanism, priority conflicts and execution order issues between instructions are identified and resolved; the scheduling results are output as image control signals and system feedback control signals.

[0096] It should be noted that in the long-term operation of the interactive control module, in order to ensure the stability of interactive recognition and the consistency of system response, the internal integration of multimodal input confidence fusion mechanism and dynamic threshold adjustment strategy: in the speech recognition channel, the system will adaptively adjust the speech feature extraction threshold according to the real-time environmental noise level, effectively reducing the recognition error rate; in the gesture recognition channel, the system improves the stability of gesture recognition through trajectory continuity judgment and interference pattern elimination methods; in the mobile terminal control channel, the system introduces a command confirmation mechanism and redundant inclusive processing strategy to avoid command loss due to communication interruption or interference.

[0097] In addition, the interactive control module also supports periodic performance self-inspection and parameter self-correction mechanisms to ensure that multi-source input always maintains reliable recognition capabilities and response consistency in complex environments, thereby improving the continuity and accuracy of the overall interactive experience.

[0098] The control event data received by this module is generated by the content input module, and contains the interactive command content corresponding to the user's voice, body movements and mobile terminal input, including the control action type, target component and parameter value, etc.; after parsing, the output result is the control execution instruction, which is transmitted to the execution modules such as the 3D rendering and processing module and the display output module.

[0099] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A 3D interactive control system for LED cloud screen, characterized in that: Includes the following modules: The content input module converts multi-source signal content into SDS streams through a multi-protocol parser, unified frame format encapsulation logic, and control instruction parsing mechanism; The 3D rendering and processing module uses GPU-accelerated rendering engines, multi-view synthesis, and light field rendering technologies to perform 3D scene rendering and physical rendering optimization on the rendering frame data in the SDS stream, and gradually generates the final display image. The display output module uses image slicing algorithm, FPGA image driving technology, PTP clock synchronization mechanism and HDR mapping algorithm to perform split-screen reconstruction and image enhancement processing on the final display image data to obtain structured LED loading frame data and full-screen refresh synchronization signal; The monitoring and maintenance module uses multi-source sensor monitoring, status diagnosis algorithms, and cloud-based operation and maintenance mechanisms to perform closed-loop control of LED sub-screens, image processing links, and environmental parameters, ultimately outputting system status information and maintenance response instructions. The interactive control module converts structured control events into standardized system control instructions through input recognition algorithms, event modeling mechanisms and system instruction mapping mechanisms, driving image content changes, mode switching and interface feedback.

2. The LED cloud screen 3D interactive control system according to claim 1, characterized in that: The content input module includes a network streaming media input unit, a local media playback unit and an interactive instruction input unit; The network streaming media input unit decapsulates the remote real-time video stream through a multi-protocol parser to obtain a compressed video stream, and then uses the GPU accelerated decoding component to decode the compressed video stream in real time and output standard frame image data; The local media playback unit uses a built-in multimedia parser and video codec library to read local multimedia files, extract key frame image data, and output image frame formats consistent with those of the network streaming media input unit; The interactive instruction input unit is used to receive interactive instruction information, use the multimodal interactive parser to extract the user intention in the interactive instruction information, and generate structured control events.

3. The LED cloud screen 3D interactive control system according to claim 1, characterized in that: The 3D rendering and processing module includes a rendering engine unit, a multi-view synthesis unit and a light field rendering unit; The rendering engine unit uses the GPU accelerated rendering engine to process the rendering frame data in the SDS stream in real time to generate a 3D image, uses physical rendering technology to process the noise effect in the 3D image, and outputs level 1 rendering image data; The multi-view synthesis unit processes the first-level rendered image data through a perspective conversion algorithm, calculates the image content at different viewpoints, and generates second-level image data adapted to different viewpoints. The calculation process is based on the depth information of the scene and uses a stereo vision algorithm to generate image frames at different viewpoints. The light field rendering unit is used to perform light field reconstruction and image enhancement processing on the level 2 image data. By sampling light information from multiple viewpoints, it synthesizes 3D image frames with a sense of depth and depth of field, generates level 3 rendered image data, and obtains the final display image; During the processing, a four-dimensional light field model is constructed based on multi-view images. ,in represents the image plane coordinates, Represents the light angle coordinates, and uses this model to perform pixel-level light resampling and depth of field control algorithm processing.

4. The LED cloud screen 3D interactive control system according to claim 1, characterized in that: The display output module includes a multi-screen splicing control unit, an LED drive output unit, a synchronization control unit, and an HDR mapping and brightness balancing unit; The multi-screen splicing control unit uses image slicing algorithms and spatial mapping logic to partition the final display image data. It divides the entire frame image into multiple sub-blocks based on the preset splicing matrix, and eliminates image distortion and splicing errors through geometric correction and edge compensation algorithms to obtain regional image data that matches the layout of each LED sub-screen. The HDR mapping and brightness balancing unit remaps the brightness and color information in the regional image data through HDR tone mapping and dynamic range compression algorithms. It uses a brightness consistency analysis model to adjust the output parameters of each region based on the physical response differences of the LED sub-screens to generate optimized image data. The LED driver output unit converts the pixel format of the optimized image data and encapsulates the underlying control signals through the FPGA image driver component to generate LED loading frame data that meets the loading requirements of the LED controller; The synchronization control unit schedules and controls the refresh process of the LED loading frame data, generates a global refresh synchronization signal, uses the system master clock as a reference, and performs dynamic phase correction in combination with the sub-screen response delay.

5. The LED cloud screen 3D interactive control system according to claim 1, characterized in that: The monitoring and maintenance module includes a sensing and adaptive control unit, a fault diagnosis unit and a remote management unit; The sensing and adaptive control unit monitors the operating environment of each LED sub-screen and image processing module in real time through an environmental sensor array, obtains multi-dimensional operating data, uses control algorithms to perform threshold judgment and trend analysis on the multi-dimensional operating data, adjusts fan speed, LED brightness and power supply voltage, and outputs the adjusted parameters as the current state parameters; The fault diagnosis unit uses a joint diagnosis algorithm based on a rule base and a machine learning model to perform pattern recognition and anomaly detection on current state parameters to obtain diagnostic labels and alarm events; The remote management unit establishes a data exchange channel with the cloud platform based on the MQTT IoT communication protocol. It regularly receives the current status parameters output by the sensing and adaptive control unit and the diagnostic tags and alarm events provided by the fault diagnosis unit, and sends them to the cloud as uploaded data. It also provides an interactive interface for operation and maintenance personnel through a web visualization interface. Based on the uploaded status parameters and fault logs, the remote management unit combines with the cloud-based operation and maintenance strategy model to generate system maintenance reports, control strategy adjustment instructions, and upgrade scheduling commands as the response basis for the remote maintenance strategy, and uniformly encapsulates the above processing results into system status information and maintenance response instructions.

6. The LED cloud screen 3D interactive control system according to claim 1, characterized in that: The interactive control module includes a multimodal input recognition unit, an event modeling unit and a logic control unit: The multimodal input recognition unit processes voice signals, body images, and remote input through three technical paths: voice recognition, gesture recognition, and mobile device communication. It extracts the original interaction intent and aggregates it to generate the original interaction event stream: The user's voice signal is collected through the microphone, and the locally deployed speech recognition model is used to decode the signal and extract keywords, recognize the control command, and output the voice event; The user's skeleton posture image is acquired through a depth camera, and the Open Pose posture recognition algorithm is used to perform time series analysis on the key point coordinates, identify gestures, and output somatosensory events. By establishing a mobile device communication interface, the interactive command data from the user's mobile phone is accessed and parsed into remote control events; The event modeling unit uses the instruction abstraction algorithm to perform structured modeling on the original interactive event stream, identify the event type, target object, and parameter information, and complete the state mapping between events and instructions based on the current system context state to generate a standard interactive instruction set; The logic control unit, based on the state machine control model, combines the system's current operating status, task priority, and instruction execution logic to perform logic analysis, conflict judgment, and path distribution on the standard interactive instruction set; Through the built-in conflict detection and response scheduling mechanism, priority conflicts and execution order issues between instructions are identified and resolved; the scheduling results are output as image control signals and system feedback control signals.

Images