LED screen display control method and system and LED screen

Abstract

The invention relates to the technical field of display control, in particular to an LED screen display control method and system and an LED screen. The method comprises the following steps: acquiring multi-modal stage sensing data and LED screen real-time display data, and performing stage performance environment perception according to the multi-modal stage sensing data and the LED screen real-time display data to obtain stage environment perception data; performing flow scene coding prediction based on the stage environment perception data to obtain renderable scene factors; obtaining display content data of the LED screen, and analyzing a pre-output picture of the LED screen to obtain visual pre-output data of the LED screen; performing multi-modal stage dynamic characteristic reasoning according to the LED screen visual pre-output data and the renderable scene factors to obtain a stage visual dynamic output structure diagram; and performing real-time heterogeneous driving rendering on the stage visual dynamic output structure diagram to obtain a millisecond LED rendering diagram block data set. According to the invention, the display response precision and the picture adaptation capability of the LED screen can be improved.

Description

Technical Field

[0001] The present invention relates to the field of display control technology, and in particular to a LED screen display control method, system and LED screen. Background Art

[0002] LED (light-emitting diode) display screens have been widely used in advertising media, stage performances, traffic instructions, sports events and other fields due to their advantages of high brightness, low power consumption, wide viewing angle and long service life. Especially in outdoor high-brightness environments, LED display screens have gradually replaced traditional LCD screens and projection systems with their good environmental adaptability and strong visual impact. In order to achieve richer, more dynamic and accurate picture presentation, the display control method of LED screens has also been continuously evolving, from the initial static dot matrix control to today's high-resolution multi-channel dynamic drive control.

[0003] Existing LED display control methods are mostly based on the data time-sharing transmission mechanism between the main control system and the display module, and often use synchronous or asynchronous methods to achieve frame-by-frame refresh of image information. In the synchronous control method, the display content needs to keep the timing consistent with the external video source in real time, which is suitable for large-scale real-time image transmission; while in the asynchronous control method, the preset content is cached by the control card to achieve independent playback away from the main control system.

[0004] Existing methods generally rely on fixed refresh rates and data transmission bandwidths. In high-resolution, large-area LED screens, delays, freezes, or image tearing are very likely to occur, especially in synchronous control, making it difficult to meet the needs of high-speed image transitions or real-time interactive responses. Secondly, because traditional control strategies mostly use point-by-point or line-by-line driving methods, and the refresh rate is not fully decoupled from the data stream, there are problems of insufficient precision in image dynamic range adjustment, color restoration, and grayscale layering control, which limits the image quality expression. Summary of the invention

[0005] Based on this, it is necessary for the present invention to provide an LED screen display control method, system and LED screen to solve at least one of the above technical problems.

[0006] To achieve the above object, a LED screen display control method comprises the following steps: Step S1: acquiring multimodal stage sensor data and LED screen real-time display data, and performing stage performance environment perception according to the multimodal stage sensor data and LED screen real-time display data to obtain stage environment perception data; Step S2: Based on the stage environment perception data, continuous modeling of stage visual structure-illumination changes is performed to obtain a performance flow scene model; flow scene coding prediction is performed on the performance flow scene model to obtain a renderable scene factor; Step S3: Obtain the LED screen display content data, and analyze the pre-output image of the LED screen based on the LED screen display content data and the stage environment perception data to obtain the visual pre-output data of the LED screen; Step S4: Perform multi-modal stage dynamic feature reasoning based on the visual pre-output data of the LED screen and the renderable scene factors to obtain the stage visual dynamic output structure diagram; Step S5: Perform real-time heterogeneous-driven rendering on the stage visual dynamic output structure diagram to obtain a millisecond-level LED rendering tile dataset, and transmit it to the LED screen control platform to execute the LED screen display rendering task.

[0007] Optionally, Step S1 is specifically as follows: Step S11: Obtain multi-modal stage sensing data and the real-time display data of the LED screen, and perform spatio-temporal alignment and noise suppression processing on the multi-modal stage sensing data to generate calibrated multi-modal sensing data; Step S12: Perform frame sequence decomposition and color gamut mapping analysis on the real-time display data of the LED screen to extract the display metadata features; Step S13: Perform cross-modal fusion analysis on the calibrated multi-modal sensing data and the display metadata features to obtain the modal feature fusion data; Step S14: Use the sensing nodes of the modal feature fusion data as graph nodes and the modal co-variation rate as the edge weight to construct a stage environment perception graph, and perform feature weighting and aggregation to generate a multi-modal co-perception graph; Step S15: Perform semantic clustering and region segmentation on the multi-modal co-perception graph, identify the stage visual hot spots and the active areas of actor interaction, and combine the currently obtained real-time display frame of the LED screen for local feature back-projection comparison to generate the stage environment perception data.

[0008] Optionally, Step S13 is specifically as follows: Step S131: Perform modal dimension structure mapping processing on the calibrated multi-modal sensing data to generate a standardized modal structure dataset; Step S132: Perform timestamp extension and channel layering processing on the display metadata features to generate a time-enhanced display content semantic vector group; Step S133: Perform spatio-temporal position-level alignment on the standardized modal structure dataset and the time-enhanced display content semantic vector group, and perform semantic association matching between modalities to obtain a preliminary cross-modal matching matrix; Step S134: Perform multi-head attention weight fusion coding based on the preliminary cross-modal matching matrix, and perform modal co-feature reconstruction to generate a multi-channel fusion tensor map; Step S135: Perform channel compression and redundant feature pruning on the multi-channel fusion tensor map to generate the modal feature fusion data.

[0009] Optionally, step S15 is specifically as follows: Step S151: Set the frequency domain decomposition dimension to 64, perform spectral feature decomposition on the graph nodes and edge weights in the multi-modal collaborative perception graph, perform spectral density estimation on the modal co-variation frequencies of the graph nodes, and select the top 10 high-energy spectral features with an energy cumulative proportion exceeding 85% to construct the high-frequency feature subspace data of the graph spectrum; Step S152: Perform graph node clustering based on the high-frequency feature subspace data of the graph spectrum, and perform semantic category division on the graph node clustering results to obtain visual active hot spot area label sensing nodes and interactive hot spot clustering area label sensing nodes, thereby generating semantic clustering label data; Step S153: Perform regional-level segmentation and reconstruction on the multi-modal collaborative perception graph according to the semantic clustering label data, form regional sub-graphs with graph nodes of the same clustering label, and set the edge weight truncation threshold to 0.35 for edge sparsification processing to generate a regional recognition graph spectrum; Step S154: Real-time collect the current display frame of the LED screen, construct a visual feature template based on the key color gamut blocks and brightness gradient structure of the current display frame of the LED screen, and perform multi-dimensional feature matching through the attribute vectors of each region node in the regional recognition graph spectrum and the visual template, construct a 5D attribute matching function, set the feature similarity threshold to 0.7 for inverse feature mapping, and generate an inverse projection region association matrix; Step S155: Perform joint decoupling analysis on the regional recognition graph spectrum and the inverse projection region association matrix, set the dimension of the decoupling mapping matrix to 64, use the regional mutual information gain greater than 0.1 as an effective fusion index, extract the cross-overlapping regions and calculate the visual flow focus degree, set the regions with an active frequency ≥ 8 Hz as high-interaction density regions for annotation, and set the regions with a visual concentration ≥ 0.85 as high-visual attention regions for annotation to obtain the stage environment perception data.

[0010] Optionally, the continuous modeling of the stage visual structure - light change in step S2 is specifically as follows: Extract multi-period frames of the stage area image modality based on the stage environment perception data, and use the multi-period frames of the stage area image modality to construct an inter-frame motion reference index to generate an initial stage time-series image set; Perform structured light separation processing on the initial stage time-series image set, decompose it into a structural brightness component and a reflected color component, and calculate the light gradient sequence diagram to generate light change structure data; Based on the light change structure data and the inter-frame motion reference index, construct a stage space continuum model; Perform dynamic modeling on the changes in light intensity, direction, and color temperature on the time axis in the stage space continuum model to obtain a time-series light response tensor; Perform principal component analysis and low-rank feature reconstruction on the temporal light response tensor, model the change trajectory of the main light mode within consecutive time periods in the low-rank feature reconstruction result, and obtain a set of light evolution vectors; Fuse and match the stage space continuum model with the set of light evolution vectors, and reconstruct the dynamic stage visual structure field based on the joint distribution of the spatial position points and time points in the fusion and matching result to generate a performance flow scene model.

[0011] Optionally, the flow scene encoding prediction in step S2 is specifically as follows: Perform regional-level temporal graph slicing on the performance flow scene model, execute equal-time interval data dissection, and establish a cross-frame position correspondence index to generate a set of scene time slice sequences; Extract motion vectors and discriminate significant regions from the set of scene time slice sequences, construct multi-scale optical flow maps and regional attention distribution maps, and thus generate a dynamic region encoding index; Based on the dynamic region encoding index, construct a flow scene encoding network, jointly model the semantic evolution trend and structural continuity in the flow scene encoding network, and output a predicted encoding vector flow; Perform clustering compression and channel reduction processing on the predicted encoding vector flow, retain the main components of dynamic changes, remove redundant modal channels and perform normalization processing to generate a dynamic encoding feature map; Perform dynamic texture enhancement on the dynamic encoding feature map, and model the heat gray value and content sensitivity of the dynamic texture enhancement result to generate a renderable scene factor.

[0012] Optionally, the analysis of the pre-output image of the LED screen in step S3 is specifically as follows: Perform frame-by-frame timestamp synchronization and brightness space normalization processing on the LED screen display content data, and at the same time construct a display content frame sequence index structure to generate a standardized LED display frame data set; Extract frame-by-frame image perception features from the standardized LED display frame data set to generate a multi-dimensional frame feature data group; Set the alignment window length to 10 frames, perform spatio-temporal alignment and co-perception projection on the standardized LED display frame data set and the multi-dimensional frame feature data group, construct a stage-LED fusion feature map, and perform significant re-weighting of the frame content to generate an environment-constrained frame feature map; Perform forward visual prediction on the environment-constrained frame feature map to predict the motion trend, texture change and brightness adjustment amplitude of the screen frame at the next moment, and generate an LED frame prediction offset map; Perform cross-frame fusion comparison between the LED frame prediction offset map and the environment-constrained frame feature map, set the fusion residual threshold to 0.15, screen out pixel blocks below the fusion residual threshold, and perform layer splicing in combination with regions with a structural similarity > 0.75 to generate the LED screen visual pre-output data.

[0013] Optionally, step S5 is specifically as follows: Step S51: Perform tile division and boundary ambiguity evaluation on the stage visual dynamic output structure diagram to obtain a graph structure region coding matrix; Step S52: Based on the graph structure region coding matrix, construct a heterogeneous graph-driven atlas, and perform graph-hardware adaptation reconstruction by integrating the current control interface state, pixel arrangement matrix, and refresh channel configuration file of the LED screen obtained in real time to generate a rendering drive mapping graph; Step S53: Using the rendering drive mapping graph as input, combine the current frame buffer load, GPU computing power, and IO scheduling frequency parameters of the LED screen control platform to construct a real-time frame-level load balancing model, and dynamically generate a millisecond-level rendering priority configuration table; Step S54: According to the millisecond-level rendering priority configuration table, perform multi-channel image rendering on each high-priority graph structure region in the stage visual dynamic output structure diagram, construct a real-time image, and output a millisecond-level LED rendering tile dataset; Step S55: Perform DMA channel compression optimization and timing packetization processing on the LED rendering tile dataset, match the protocol bandwidth and cache frame refresh rate of the LED screen control platform, and construct an LED control signal stream; Step S56: Send the LED control signal stream to the LED screen control platform in real time to drive the LED screen to complete the per-frame display task of the rendering content.

[0014] Based on the collaborative analysis of multi-modal sensing data and LED display data, the present invention can achieve refined perception of the stage environment and high-real-time rendering control. Compared with the existing LED control methods based on static refresh and fixed driving, it has higher response accuracy and picture adaptation ability. By performing spatio-temporal alignment, noise suppression, and structural mapping on the multi-modal sensing data, and fusing and analyzing it with the real-time frame content of the LED screen, not only the integrity and reliability of environmental information acquisition are enhanced, but also the subsequent hotspot area recognition has stronger context adaptability. By introducing semantic clustering and region segmentation, the dependence on a single sensing modality is effectively reduced, and the recognition accuracy of the actor activity area and the visual hotspot area is improved. In terms of parameter settings, for example, the frequency domain decomposition dimension is set to 64, which can ensure the full extraction of high-frequency components sensitive to changing frequencies, facilitating stable modeling in complex stage change environments; the edge weight truncation threshold is set to 0.35, effectively filtering low-correlation edges and enhancing the spatial connectivity and expression clarity of the regional subgraphs; while the merging process of fragmented subgraphs with an area less than 5% can reduce the interference of pseudo hotspots in visual analysis. Through the methods of separating the lighting structure and constructing the inter-frame reference, the lighting evolution process can be accurately modeled, capable of adapting to the situation of frequent lighting movement and drastic scene changes on the stage. The generated dynamic visual structure field provides a more stable temporal reference. The low-rank reconstruction process of the lighting response tensor avoids redundant calculations, improves the model operation efficiency, and the extracted lighting evolution vector group enhances the expression ability for different lighting transition scenarios. In terms of parameters, principal component analysis is used in temporal modeling, which not only retains the main lighting change trends but also compresses invalid features, improving the stability and prediction accuracy of lighting simulation. Through frame timestamp alignment and brightness normalization processing, the precise synchronization of the visual reasoning process and the LED frame display state is ensured. By constructing a stage-LED fusion feature map and introducing a saliency reweighting strategy, the prediction accuracy of high-concern areas in future display frames is significantly improved, and the fusion residual threshold is set to 0.15 to ensure that only highly consistent layers participate in the output, enhancing the consistency between the final rendered content and the actual stage state. Areas with a structural similarity greater than 0.75 participate in layer splicing, effectively avoiding image tearing or blurring caused by low-quality matches. By fusing the LED visual pre-output and the performance scene factors, the reasoning results can take into account both the physical state of the stage and the LED image logic, improving the real-time response ability and spatial distribution rationality of the final visual output structure. Through the tile division and graph-hardware adaptation and reconstruction mechanism, the high-complexity visual structure is reasonably mapped to the LED control hardware environment, avoiding rendering jams caused by hardware resource bottlenecks. The rendering priority configuration table is dynamically adjusted in combination with the frame buffer load, GPU capabilities, and IO frequency of the LED platform, which can effectively optimize the rendering fluency of key areas and enhance the visual experience of the audience in the main viewing area.Millisecond-level tile rendering ensures smooth picture transitions. DMA compression and timing packetization precisely match the transmission bandwidth and refresh rhythm, effectively preventing image tearing and latency, and enhancing the overall display quality. Additionally, the decoupled mapping matrix is introduced with a dimension of 64, which balances computational complexity and real-time performance while ensuring that the dimension of the matching function fully expresses the cross-feature structure. The mutual information gain threshold is set to 0.1 to effectively filter out irrelevant or weakly correlated region pairs, ensuring that the finally extracted regions have significant information fusion value. An active frequency ≥8Hz is used to characterize regions with frequent interactions in a short period, ensuring that the recognition of high-interaction regions is not interfered by sporadic events, while a visual concentration ≥0.85 increases the discrimination intensity of high-visual attention regions, making the LED image rendering results more in line with the stage visual focus distribution, thus achieving efficient linkage between the real scene and digital display. The overall method significantly improves the accuracy, detail expressiveness, and audience immersion of visual presentation while taking into account the real-time performance of LED rendering and the adaptability to stage changes.

[0015] Optionally, this specification also provides an LED screen display control system, characterized in that it is used to execute the LED screen display control method as described above. The LED screen display control system includes: An environmental perception module, configured to obtain multi-modal stage sensing data and real-time LED screen display data, and perform stage performance environment perception based on the multi-modal stage sensing data and real-time LED screen display data to obtain stage environment perception data; A flowing scene modeling module, configured to continuously model the stage visual structure-lighting changes based on the stage environment perception data to obtain a performance flowing scene model; perform flowing scene encoding prediction on the performance flowing scene model to obtain renderable scene factors; A pre-output picture analysis module, configured to obtain LED screen display content data, and analyze the LED screen pre-output picture based on the LED screen display content data and stage environment perception data to obtain LED screen visual pre-output data; A dynamic feature inference module, configured to perform multi-modal stage dynamic feature inference based on the LED screen visual pre-output data and renderable scene factors to obtain a stage visual dynamic output structure diagram; A heterogeneous rendering module, configured to perform real-time heterogeneous-driven rendering on the stage visual dynamic output structure diagram to obtain a millisecond-level LED rendering tile dataset, and transmit it to the LED screen control platform to execute the LED screen display rendering task.

[0016] The LED screen display control system of the present invention can implement any LED screen display control method of the present invention. It is a medium for coordinating the operations and signal transmissions between various modules to complete the LED screen display control method. The internal modules of the system cooperate with each other, thereby improving the LED screen display response accuracy and picture adaptation ability.

[0017] Optionally, this specification also provides an LED screen, including an LED screen main body, a power supply unit, and an electrical control unit. Among them, the power supply unit is installed inside the LED screen main body, the electrical control unit is electrically connected to the power supply unit, the electrical control unit is used to charge the LED screen main body and control the LED screen main body, and the electrical control unit is used to execute the LED screen display control method described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objects, and advantages of the present invention will become more apparent: Figure 1 It is a schematic flow chart of the steps of the LED screen display control method of the present invention; Figure 2 It is a schematic detailed flow chart of step S1 in the present invention; The realization of the object of the present invention, functional features, and advantages will be further described with reference to the embodiments and the drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The technical method of the present invention will be clearly and completely described below with reference to the drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0020] In addition, the drawings are only schematic diagrams of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor methods and / or microcontroller methods.

[0021] It should be understood that although the terms "first", "second", etc. may be used here to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, the first unit can be called the second unit, and similarly the second unit can be called the first unit. The term "and / or" used here includes any and all combinations of one or more of the listed related items.

[0022] To achieve the above object, please refer to Figures 1 to 2The present invention provides a method for controlling LED screen display, the method comprising the following steps: Step S1: acquiring multimodal stage sensor data and LED screen real-time display data, and performing stage performance environment perception according to the multimodal stage sensor data and LED screen real-time display data to obtain stage environment perception data; In this embodiment, in the initialization stage of the stage performance, real-time perception data is collected by deploying infrared motion capture cameras, pressure sensing carpets, array microphones, ambient light meters and other multimodal sensing devices around the stage and on the ceiling. The sampling frequency of the equipment is uniformly set to 30Hz, and the timestamp alignment error is controlled within ±5ms through the synchronization controller. At the same time, the current frame display pixel matrix and brightness signal stream are synchronously obtained from the LED screen control platform. After the above data is filtered, formatted and numerically normalized by the edge computing node, a structured data table containing the stage lighting status, character position trajectory, audio energy density and LED screen visual output is generated, and a collaborative perception map between sensor nodes is constructed through a graph modeling method, and a spatial-temporal fusion modeling is performed on the local environmental changes and the evolution of the performance status, and finally the stage environment perception data containing the current stage brightness distribution, regional dynamic level and character activity score is output.

[0023] Step S2: Based on the stage environment perception data, continuous modeling of stage visual structure-illumination changes is performed to obtain a performance flow scene model; flow scene coding prediction is performed on the performance flow scene model to obtain a renderable scene factor; In this embodiment, based on the stage environment perception data, the LED screen visual interaction area and the character activity hotspot within the boundary of the stage structure grid are selected, and the stage area is grid discretized according to the set spatial granularity of 20cm. For each discrete grid area, the average brightness difference change rate and the illumination direction change angle range between adjacent frames are calculated, and the evolution trajectory within the continuous time period Δt=2 seconds is recorded, and the continuous frame structure feature stream is generated through the structure-illumination mapping rule. The structural feature stream is then sliced ​​with 5 frames of equal time steps, and the brightness disturbance and deformation trend caused by the main character action in each time slice are extracted. The redundant frames are compressed in combination with the regional attention distribution data to establish a lightweight temporal structure description model. According to the description model, the most drastic changes in the dynamic scene are further extracted and channel mapping is performed to generate a scene coding index, identify the visual dominant factors and the main components of the deformation trend, and finally generate a renderable scene factor for subsequent rendering reasoning.

[0024] Step S3: obtaining LED screen display content data, and analyzing the LED screen pre-output picture based on the LED screen display content data and the stage environment perception data to obtain LED screen visual pre-output data; In this embodiment, the frame-by-frame display data stream is extracted in real time from the control interface of the LED screen control platform. The frame rate is set to 60fps to be consistent with the display refresh rate, and the content format is uniformly the YUV420 compressed color space format. For each frame of the image, after first performing brightness channel normalization processing and edge high-pass filtering enhancement, it is synchronized to the current stage environment perception data according to the timestamp. A 3-second sliding window index structure is constructed in the frame sequence to extract key frames and transition frames and mark the high-frequency change regions. Subsequently, according to the extracted image features, a multi-dimensional feature vector is constructed for each frame, including eight types of parameters such as hue saturation distribution, texture continuity index, and regional brightness gradient. Through cross-modal projection mapping technology, the frame feature vector is jointly matched and analyzed with the visual focus points in the stage area to predict the visual focus offset and potential visual conflict regions of the next frame, and the construction of the LED screen visual pre-output data is completed.

[0025] Step S4: Perform multi-modal stage dynamic feature inference based on the LED screen visual pre-output data and the renderable scene factors to obtain the stage visual dynamic output structure diagram; In this embodiment, after obtaining the LED screen visual pre-output data and the renderable scene factors, the two are aligned in the structural dimension, and a high-dimensional stage feature response map is constructed through tensor fusion. Each node in the map represents the visual contribution and dynamic activity of a certain area in the current scene. Taking the high-dimensional stage feature response map as the input, the main visual path of the character, the secondary light influence response trajectory, and the regional priority are derived in sequence based on the rule set, and the visual conversion weight of each area is calculated by combining the visual rhythm adjustment factor (set to 0.8). Subsequently, according to the collaborative activity degree and visual association degree between regions, a stage visual dynamic output structure diagram is constructed, including the display intensity, color transition, and dynamic texture change trend of each LED sub-block in the next frame cycle.

[0026] Step S5: Perform real-time heterogeneous-driven rendering on the stage visual dynamic output structure diagram to obtain a millisecond-level LED rendering tile data set, and transmit it to the LED screen control platform to execute the LED screen display rendering task.

[0027] In this embodiment, the stage visual dynamic output structure diagram is input into the LED rendering control module, and in this module, the regional-level image construction task is executed according to the LED screen hardware configuration parameters (such as resolution 1920×1080, and the rendering delay per frame is controlled within 15 ms). First, based on the regional dynamic indicators defined in the structure diagram, the local texture library and color fusion function are called to construct the next-frame rendering tile data. Each tile contains a texture direction tensor, a light and shadow transition path, and a color gamut adjustment factor, and the brightness is adjusted based on the LED drive current intensity model. Subsequently, all tiles are mapped to the corresponding sub-module buffer according to the physical distribution of the LED screen for millisecond-level image loading. Finally, the rendering tile data is sent to the LED screen control platform through a high-speed serial control bus (refresh period 16 ms) to ensure that the visual output is highly synchronized with the live stage performance, and finally, a frame of dynamic LED screen display task is completed.

[0028] Optionally, step S1 is specifically as follows: Step S11: Obtain multi-modal stage sensing data and the real-time display data of the LED screen, and perform spatio-temporal alignment and noise suppression processing on the multi-modal stage sensing data to generate calibrated multi-modal sensing data; In this embodiment, a multi-modal perception unit is formed by deploying an infrared motion capture camera (model OptiTrack Prime 13, frame rate 120 Hz), an array microphone (array spacing 0.5 m), a contact pressure-sensitive carpet (sensitivity 0.2 N), and an ambient illuminance sensor (response frequency 25 Hz). The synchronization acquisition timestamp of each type of sensor is uniformly set to take 10 ms as the step size, and the sliding window is 5 seconds to synchronize the data timestamps of different modalities. The data with a time drift exceeding 0.5 seconds is corrected by linear interpolation. At the same time, the current frame display pixel matrix and brightness signal stream are synchronously obtained from the LED screen control platform. Subsequently, the median filtering method is used to perform noise suppression processing on the infrared and pressure sensor data, and the outlier recognition threshold is set to ±3σ to filter out the burst signal interference, and finally, a calibrated multi-modal sensing data set with time alignment and consistent spatial distribution is generated.

[0029] Step S12: Perform frame sequence decomposition and color gamut mapping analysis on the real-time display data of the LED screen to extract the display metadata features; In this embodiment, based on the current frame display pixel matrix and brightness signal stream obtained from the LED control platform access port, each frame of image is decomposed and structurally analyzed. The decomposition frame rate is 30 fps. After decomposition, the main color gamut blocks are extracted in the YUV color space. The HSB color gamut mapping function is used to perform the main component conversion on the color information, and the mapping analysis accuracy is set to 12 bits to maintain the integrity of color details. By calculating the brightness gradient change rate of the image area, the dynamic range and light and dark distribution of each frame content are encoded. At the same time, the color main axis and edge intensity data in the image metadata are extracted to form a display metadata feature set that can be used for subsequent multimodal correlation analysis.

[0030] Step S13: Perform cross-modal fusion analysis on the calibrated multimodal sensing data and display metadata features to obtain modal feature fusion data; In this embodiment, the calibrated multimodal sensing data and display metadata features are subjected to cross-domain mapping processing. Their spatial positions are respectively unified in coordinates. The stage plane space is aligned with an accuracy of 0.1 m, and a modal response mapping relationship is constructed based on the cooperation degree between the signal intensity change rate and the color change rate. The mapping weight is set to 0.6:0.4 to define the coupling degree between the sensing intensity and the display feature. After time series alignment, key points with consistent change trends within a 5-second window are extracted, and the modal feature fusion data including spatial position, modal change rate, and image response factor is integrated and generated for building the basis for identifying dynamic behaviors in the stage area.

[0031] Step S14: Use the sensing nodes of the modal feature fusion data as graph nodes and the modal co-variation rate as edge weights to construct a stage environment perception graph, and perform feature weighting and aggregation to generate a multimodal collaborative perception graph; In this embodiment, each sensing node in the modal feature fusion data is set as a node element in the spatial graph, and the modal co-variation rate is used as the weight between the edges, which is defined as the joint feature change amplitude between adjacent two frames of nodes. Edges with a co-variation rate between 0.1 and 0.8 are considered to be effectively associated. An undirected weighted graph structure is constructed, where the node attributes include eight-dimensional vector features such as action amplitude, visual saliency, and audio intensity, and the edge attributes represent the collaborative relationship between nodes. Subsequently, a local aggregation operation is performed on each node in the graph. The adjacent node features are weighted and accumulated in a way that the convolution window size is 3, and the weight is linearly scaled from the node-to-node co-variation rate to [0.2, 1.0]. The main component direction is extracted from the aggregated node feature vectors, and the node embedding representation is constructed based on the main component change trend. Finally, a multimodal collaborative perception graph with the ability to aggregate perception features within the region is generated.

[0032] Step S15: Perform semantic clustering and regional segmentation on the multi-modal collaborative perception map, identify the stage visual hotspots and the active areas of actor interaction, and generate stage environment perception data by combining the local feature back-projection comparison with the currently acquired display frame of the LED screen.

[0033] In this embodiment, semantic relationship measurement is performed based on the aggregated feature vectors of each node in the multi-modal collaborative perception map. Preliminary semantic clustering of the map nodes is carried out using the modal category weight distribution. The number of clustering clusters is set to k = 6, and region labels such as visual focus, action intensity, and sound density are divided. Spatial consistency correction is performed on the clustering results, and redundant small clusters are eliminated by analyzing the density and connectivity of the node spatial distribution, and regions with an average number of intra-cluster nodes ≥ 10 are retained. At the same time, in combination with the currently acquired frame image of the LED screen, pixel block reverse mapping operation is performed on the clustering region, and the color mutation points in the LED frame are spatially compared with the clustering boundary in the collaborative perception map to determine whether the LED screen image transformation responds to actor behavior or lighting field changes, thereby completing the semantic association reconstruction of the stage region. The finally output stage environment perception data includes the spatial position, time activation degree, and multi-modal perception weight distribution of each visual hotspot region and the active area of actor interaction.

[0034] Optionally, step S13 is specifically: Step S131: Perform modal dimension structure mapping processing on the calibrated multi-modal sensing data to generate a standardized modal structure data set; In this embodiment, for the multi-modal stage sensing data that has undergone spatio-temporal alignment and noise suppression, modal categories are divided according to its physical source and signal type, including four categories: infrared temperature field, ambient illuminance, stage pressure distribution, and actor position trajectory. By projecting the two-dimensional spatial data of different modalities into a unified three-dimensional structure tensor framework, a grid unit with a spatial resolution of 50 cm × 50 cm is set, and the corresponding time series window slides for 2 seconds. The modal time axis dimension is constructed by indexing according to the serial number. The linear normalization method is used to uniformly map the numerical values of each modality into the range [0, 1], and Z-score normalization is used to eliminate the cross-modal scale difference. Finally, a standardized modal structure data set including three dimensions of space, time, and modality is constructed to open up the dimension path for subsequent cross-modal fusion.

[0035] Step S132: Perform timestamp extension and channel layering processing on the display metadata features to generate a time-enhanced display content semantic vector group; In this embodiment, the frame sequence data in the LED display metadata features is extended in the time dimension. The extension window is set to a group of every 5 frames, and three types of information, namely brightness, color, and pixel position, are extracted separately in a channel manner. First-order and second-order difference operations are performed on each channel within each group of frame data to calculate the brightness change amplitude, color gradient drift, and pixel displacement trend. The difference weight ratio is set to 0.7:0.2:0.1 to strengthen the dominance of brightness changes. The extracted difference results are expanded in the channel dimension to form a multi-layer semantic representation. The length of each layer of vectors is controlled within 128 dimensions. A time-enhanced display content semantic vector group is constructed through feature splicing. This vector group has the dynamic perception ability across time frames and retains the color change characteristics of the original image.

[0036] Step S133: Align the normalized modal structure data set and the time-enhanced display content semantic vector group at the spatio-temporal position level, perform semantic association matching between modalities, and obtain a preliminary cross-modal matching matrix. In this embodiment, the spatial nodes in the normalized modal structure data set and the frame channel data in the time-enhanced display content semantic vector group are index-matched one-to-one or one-to-many according to timestamps and spatial coordinates. The matching tolerance threshold is set to ±0.1 seconds in time and ±0.25 meters in spatial range. Matching relationships that exceed the threshold are automatically excluded. Under the condition of meeting the time-space alignment, the weighted fusion value of the Euclidean distance and cosine similarity between different modal features is calculated, and the fusion weights are 0.6 and 0.4. A preliminary cross-modal matching matrix is constructed. This matrix is organized in units of frames and identifies the semantic connection strength between modalities in a time series manner, providing a mapping weight basis for the subsequent feature fusion process.

[0037] Step S134: Perform multi-head attention weight fusion encoding based on the preliminary cross-modal matching matrix, and perform modal collaborative feature reconstruction to generate a multi-channel fusion tensor map. In this embodiment, based on the preliminary cross-modal matching matrix, the modal matching relationships corresponding to each moment are assigned to multiple attention channels. Each channel focuses on the feature associations of different modal combinations. The number of channels is set to 8, and a weight sharing mechanism is adopted between channels to reduce redundancy. Within each channel, the modal pairs with high similarity are strengthened through weighted aggregation, and the aggregation results are stacked along the channel direction to form tensor fragments. After overall splicing, a multi-channel fusion tensor map is constructed. The dimension of this map is uniformly N×H×W×C, where N is the number of time steps, H and W are the spatial grid dimensions, and C is the number of fusion channels. The channel feature consistency is maintained and the reconstruction expression ability is improved through linear transformation and normalization.

[0038] Step S135: Perform channel compression and redundant feature pruning on the multi-channel fusion tensor map to generate modal feature fusion data.

[0039] In this embodiment, the multi-channel fusion tensor atlas is compressed in the channel dimension. The variance information density of each channel is calculated first, and the low variance channel threshold is set to 0.03. Channels below this value are considered redundant and eliminated. At the same time, the retained channels are screened for average activation intensity, and secondary channels whose average responses are lower than the global mean of 0.2 are screened out. After pruning, the remaining channels are batch normalized to stabilize the distribution range of each channel output to between [0,1], and the channel arrangement order is reordered in combination with the principal component response direction. The final output is modal feature fusion data with a channel number less than 16. The fusion result has high expression density and cross-modal semantic adaptation capabilities, which is suitable for subsequent visual-driven modeling processes.

[0040] Optionally, step S15 is specifically: Step S151: setting the frequency domain decomposition dimension to 64, performing spectral feature decomposition on the graph nodes and edge weights in the multimodal collaborative perception graph, performing spectral density estimation on the modal collaborative change frequency of the graph nodes, and selecting the top 10 high-energy spectral features with a cumulative energy share of more than 85% to construct the graph high-frequency feature subspace data; In this embodiment, for the constructed multimodal collaborative perception graph, the spectral decomposition dimension is set to 64, and the Laplace feature matrix is ​​used to represent the spatial structural relationship and edge weight changes between nodes. On this basis, the spectral domain transformation is performed to extract the frequency domain response characteristics of the modal collaborative changes between nodes. The frequency domain components of each node are obtained by fast Fourier expansion, and the frequency energy density is estimated. The top 10 main frequency features with a cumulative energy share of more than 85% are selected as the high-frequency response basis based on the cumulative value of the energy distribution. These main frequency components are used as the high-frequency feature subspace representation of the graph to form a spectral domain feature set for characterizing visual interaction and spatial activation areas, and are output in the form of high-frequency feature tensors, retaining the high response amplitude and position variation trend within the time window to support subsequent visual semantic focus analysis.

[0041] Step S152: clustering graph nodes based on the high-frequency feature subspace data of the graph, and dividing the graph node clustering results into semantic categories to obtain label sensor nodes of visually active hotspot areas and label sensor nodes of interactive hotspot clustering areas, thereby generating semantic clustering label data; In this embodiment, by using the main frequency response values of each node in the high-frequency feature subspace, nodes with similar adjacent response patterns are grouped into one category through a spatial aggregation method based on spectral similarity, and the number of category divisions does not exceed 10 categories. Semantic relationship analysis is performed on each category of graph nodes obtained, and preliminary labels for visually active hotspots or interactive hotspot areas are assigned according to their source modality, spatial position, and frequency response characteristics. The label categories include the main activity area of the actor, the area with light strobing, the area where the audience's gazes meet, etc. Using the central node of each category of clustering label as the identification node, the propagation edges in its surrounding neighborhood are expanded layer by layer and the label consistency is verified. Finally, a clustering label data set is formed, and the output format is the node ID-semantic category pair, providing semantic support for the next step of region segmentation and local visual interaction determination.

[0042] Step S153: According to the semantic clustering label data, perform regional-level segmentation and reconstruction on the multi-modal collaborative perception graph. Combine graph nodes with the same clustering label to form a regional sub-graph, and set the edge weight truncation threshold to 0.35 for edge sparsification processing to generate a regional recognition graph; In this embodiment, based on the clustering label data, graph nodes with the same label are recombined to form multiple sub-region graphs, and the edge weight truncation threshold is set to 0.35 to remove non-strongly connected edges with sparse connections. The mean vector of node features is statistically calculated for each sub-graph respectively, a spatial adjacency matrix is constructed, the connectivity score is calculated based on the node spatial distance and edge connection strength, and isolated fragment graphs with an area less than 5% of the total graph area are merged into adjacent connected regions to improve regional integrity. The generated regional recognition graph after merging includes a regional boundary node set, regional central node attributes, local feature means, and spatial connectivity graph metrics. This structure can be used as the basic spatial perception representation for subsequent visual mapping comparison and regional-level rendering matching.

[0043] Step S154: Real-time collect the current display frame of the LED screen, construct a visual feature template based on the key color gamut blocks and brightness gradient structure of the current display frame of the LED screen, and perform multi-dimensional feature matching through the attribute vector of each region node in the regional recognition graph and the visual template. Construct a 5D attribute matching function, set the feature similarity threshold to 0.7 for reverse feature mapping, and generate an inverse projection region association matrix; In this embodiment, the current display content of the LED screen is collected through the LED screen platform at a frequency of 60 frames per second, and the key color gamut block information of the current frame image is extracted, including the RGB histogram peak of the main color area, the local brightness change gradient structure, and the distribution of high-saturation color block areas, to construct a visual feature template composed of a color channel, a brightness channel, a texture channel, a position channel, and a shape contour channel. For each node in the regional recognition map, its five-dimensional attribute vector is extracted and a multi-dimensional similarity measure is performed with the above visual template. The output value of the matching function is defined as the scoring of the node feature matching degree. Nodes with a matching score higher than 0.7 are regarded as relevant, and a corresponding back-projection area association matrix is generated to represent the mapping relationship between the stage space perception nodes and the current visual output.

[0044] Step S155: Perform a joint decoupling analysis on the regional recognition map and the back-projection area association matrix. Set the dimension of the decoupling mapping matrix to 64, use the regional mutual information gain greater than 0.1 as an effective fusion index, extract the cross-overlapping areas and calculate the visual flow focus. Set the areas with an active frequency ≥ 8 Hz as high-interaction density areas for annotation, and the areas with a visual concentration ≥ 0.85 as high-visual attention areas for annotation to obtain the stage environment perception data.

[0045] In this embodiment, the region recognition map is jointly analyzed with the back-projection region correlation matrix. First, a mapping matrix with a dimension of 64 is constructed to represent the cross-information relationship of each stage region in the multi-dimensional space between different data sources. The mutual information value is calculated for each pair of regions in this mapping matrix, and the region combinations with a mutual information gain greater than 0.1 are extracted as the basis for the effective fusion of information between regions. For the effectively fused region combinations, the visual flow focus index within unit time is calculated. The visual flow focus is obtained based on the change of the LED screen brightness focus, the overlap frequency of the actor's position trajectory and the stage light focus point. The calculation of this index is based on visual timing data, recording the number of coincidence events of each visual element in the spatial coordinates within a single frame or consecutive frames. When the coincidence frequency exceeds 8 times per second, that is, the active frequency ≥ 8 Hz, this region is marked as a high interaction density region. At the same time, to determine the high visual attention region, the current visual template needs to be introduced. The current visual template is a reference visual feature set constructed based on the actual output image of the LED screen and the inference result of the stage dynamic visual features in the previous rendering cycle. It reflects the comprehensive image expression of indicators such as the prediction of the audience's gaze direction, the central field of view distribution on the stage, and the expected visual flow trajectory. The image feature vector of each candidate region is compared with the current visual template to calculate the visual concentration degree, that is, the matching degree between this region and the visual template. When the matching degree (such as the structural similarity) ≥ 0.85, it is determined as a high visual attention region. Finally, the stage environment perception data is output in the form of a structured data set, including the positions, categories, and their feature statistics of the above-mentioned marked high interaction density regions and high visual attention regions, and provides a basic input interface for path inference, LED layer control, and visual focus scheduling in subsequent steps.

[0046] Particularly importantly, step S152 is specifically as follows: Standardize the data in the high-frequency feature subspace of the map to generate a spectral feature standardization matrix; In this embodiment, the extracted high-frequency feature subspace data of the map is input into the standardization process, and the Z-score normalization method is used to normalize each frequency domain feature vector to ensure that different spectral components express the co-variation trend within the same scale. The standardization range is set with a mean of 0 and a standard deviation of 1, and the extremely high fluctuation frequency bands are truncated, with a maximum threshold of plus or minus 3 times the standard deviation, to suppress the influence of individual node mutation values on the overall clustering. The processed spectral feature standardization matrix maintains the original frequency domain structure and has a stable horizontal comparison ability, which can be used for subsequent similarity calculation and clustering operations. The matrix format is the node index and the corresponding high-frequency standardized value sequence, which is convenient for directly embedding into the construction of the graph structure relationship.

[0047] Construct a modal collaborative similarity graph between nodes based on the spectral feature normalization matrix, and calculate the spectral correlation degree of graph nodes using the modal collaborative similarity graph between nodes to obtain the spectral correlation degree data of graph nodes; In this embodiment, based on the eigenvector distance between each graph node in the spectral feature normalization matrix, a cosine similarity metric method is used to construct a modal collaborative similarity graph between nodes, the edge weight is set as the similarity value, and the edge connection threshold is set to 0.6, and only the strongly associated edges with higher similarity are retained. After construction, calculate the average correlation degree of each node with its adjacent nodes in the high-frequency spectral space as the spectral correlation degree score, and at the same time introduce a local aggregation effect adjustment factor to reflect the core degree of spectral collaboration of the node in its clustering neighborhood. The output spectral correlation degree data structure of graph nodes is the node ID and its corresponding spectral collaboration value, which can be used as the input basis for the next-stage clustering criterion.

[0048] Use the spectral correlation degree data of graph nodes to construct a modal spectral clustering model, and set the optimal clustering cluster number range as [4, 12] for silhouette coefficient optimization to generate modal spectral clustering label data; In this embodiment, according to the spectral correlation degree data of graph nodes, by gradually searching the grouping structure under different numbers of clustering clusters, the search range of the number of clustering clusters is set from 4 to 12. For each clustering result, calculate the corresponding silhouette coefficient score to evaluate the intra-group compactness and inter-group difference, and finally select the number of clustering clusters that maximizes the average silhouette coefficient as the final clustering structure. Under this structure, all nodes are assigned to the corresponding modal spectral clusters, and the output is modal spectral clustering label data, which includes the clustering number of each node and its relative position in the clustering, and has good spectral structure interpretability and inter-modal collaborative linkage performance.

[0049] Conduct semantic reasoning based on the modal spectral clustering label data, implement node semantic weighting, and generate a preliminary semantic classification result; In this embodiment, the obtained modal spectral clustering label data is used for the preliminary reasoning of the semantic structure. First, for each clustering cluster, extract the average coordinate of the spatial position of the nodes it contains, the proportion of the corresponding modal types (such as the response distribution of action sensors, acoustic sensors, and light sensors), and the average value of the high-frequency spectral response amplitude of the nodes, which are respectively denoted as 、 、 . On this basis, set the weighting coefficients to 0.4, 0.3, and 0.3 respectively, and calculate the semantic comprehensive score of each clustering: ; where, It means to normalize each feature to the interval [0, 1] to eliminate the influence of dimension. To identify the semantic attributes of clustering, the stage area is divided into a central area and a peripheral area. The average coordinates of the nodes in the central area are usually concentrated in the middle of the stage (for example, the X and Y coordinates fall within 60% of the center of the area), while the clustering in the peripheral area is distributed in the outer periphery. At the same time, for the clustering with an action modality or sound response intensity higher than 0.6, it is judged to have strong interactivity; for the clustering with a large deviation amplitude of light or color response, it is judged to have high visual attention. Therefore, if a certain clustering cluster has high spatial concentration, is mainly composed of visual modalities, and has an obvious high-frequency spectrum response, then according to its high 、 high (biased towards vision), high value, it is determined as "visual active hot spot area clustering"; on the contrary, if its spatial coordinates are biased towards the edge of the stage and it has a significant response in sound or action modalities, it more conforms to the feature standard of "interactive hot spot edge perception clustering". The classification labels are not hard-divided by thresholds, but are automatically mapped and generated after the above feature combinations fall into the preset semantic type intervals, and finally a structured preliminary semantic classification result is output as the basis for subsequent clustering refinement and hot spot recognition.

[0050] Perform stability re-scoring and redundant cluster merging on the preliminary semantic classification results, and divide them into visual active hot spot area label sensing nodes and interactive hot spot clustering area label sensing nodes, and finally generate semantic clustering label data.

[0051] In this embodiment, multiple rounds of re-scoring operations are performed on the preliminary semantic classification results. By comparing the semantic stability scores of nodes in the same clustering under multiple time windows, the clustering labels with a scoring stability higher than 0.75 are retained. At the same time, for the clustering clusters with a similarity higher than 0.9, merge processing is performed to eliminate redundancy. After merging, recalculate the central nodes and boundary contours of each class, and divide them into two categories according to semantic features, namely visual active hot spot areas and interactive hot spot clustering areas, and re-confirm their spatial coherence and modal consistency within each class to eliminate stray nodes. The finally output semantic clustering label data is a stable label structure after optimization, which is used for subsequent area recognition and visual hot spot annotation processing.

[0052] Optionally, the continuous modeling of the stage visual structure - light change in step S2 is specifically as follows: Extract multi-period frames of the stage area image modality based on the stage environment perception data, and use the multi-period frames of the stage area image modality to construct an inter-frame motion reference index to generate an initial stage time-series image set; In this embodiment, among the acquired stage environment perception data, spatial indices containing high interaction density regions and high visual attention regions are selected, and the index screening condition is set as an active frequency not lower than 8 Hz and a visual concentration greater than 0.85. Taking the spatial coordinates of these high-frequency region nodes as the extraction range, an image frame set within the corresponding time period is extracted from the image modality channel. Each high-frequency region extracts no less than 30 frames, and the time span is controlled within 15 seconds to ensure that the multi-modal stage image data covers a complete dynamic cycle. The frame data is rearranged according to the time axis and the spatial source is recorded, forming a multi-period frame of the stage area image modality for subsequent modeling, and a motion reference index between frames is synchronously constructed, thereby generating an initial stage time-series image set.

[0053] Perform structured light separation processing on the initial stage time-series image set, decompose it into a structural brightness component and a reflected color component, and calculate a light gradient sequence map to generate light change structure data; In this embodiment, each frame of the multi-period image modality frame is subjected to brightness layering processing. Based on the reflection edge constraint model, the original image is decomposed into a structural brightness component and a reflected color component, and a brightness gradient sequence map is constructed on the time axis. The gradient sequence is demarcated by an average brightness change between frames exceeding 12%, generating a high light change section mask. The superimposed result of the mask is used as the light change structure data, and a spatial brightness change heat map with a resolution not lower than 128×128 is output as the input for subsequent spatial modeling.

[0054] Based on the light change structure data and the inter-frame motion reference index, construct a stage space continuum model; In this embodiment, the time tag corresponding to each pixel point in the light change structure data is bound to the spatial coordinates in the stage environment perception data, and mapped through the spatial boundary information of each region in the region recognition map. A three-dimensional spatial lattice distribution (x, y, z) is constructed and the time dimension t is added to form a four-dimensional coordinate system. Each spatial point is set with a resolution of 0.25 m, and combined with the 5D attribute matching result defined in the back-projection region association matrix, the missing region features are complemented, thereby completing the construction of the spatial continuous light distribution field and obtaining the stage space continuum model.

[0055] Dynamically model the changes in light intensity, direction, and color temperature on the time axis in the stage space continuum model to obtain a time-series light response tensor; In this embodiment, based on the stage space continuum model, a sequence of light intensity is established along the time dimension at each space point, and the main light direction (described in polar coordinates), average brightness (unit: Lux), and main color temperature (Kelvin) are recorded respectively. The sequence length is set to 30, and the time interval does not exceed 0.2 seconds. The three-dimensional light sequences constructed by all space points are combined into a tensor, and the tensor dimension is set to [N×30×3], where N is the total number of space sampling points, and the stage temporal light response tensor is obtained.

[0056] Perform principal component analysis and low-rank feature reconstruction on the temporal light response tensor, and model the change trajectory of the main light mode within a continuous time period in the low-rank feature reconstruction result to obtain a group of light evolution vectors; In this embodiment, the temporal light response tensor is expanded into a two-dimensional representation, principal components are extracted for each group of light intensity-direction-color temperature triples, the cumulative energy retention ratio is set to 90%, and low-rank compression is performed on the high-order redundant components. Smooth the trajectory of the main component at each space point in the low-rank reconstruction result, extract the change trend in the continuous paragraph, and describe the migration path of the main light mode in the form of a five-dimensional vector to obtain a group of light evolution vectors, representing the light dynamic trend over time at different positions on the stage.

[0057] Fuse and match the stage space continuum model with the group of light evolution vectors, and reconstruct the dynamic stage visual structure field based on the joint distribution of the space position points and time points in the fusion matching result to generate a performance flow scene model.

[0058] In this embodiment, the time series of the light evolution vector is used as the main linear reference. The position of each vector in the stage space continuum model is matched according to its spatial index. The vector space overlap threshold is set to 0.65, and a five-frame sliding window is used for vector trajectory fitting. The regional points with the absolute value of the change gradient exceeding 15% during the fitting process are marked as dynamic response centers. The spatio-temporal coordinates of all dynamic response centers are aggregated to construct a spatial thermal response map under light illumination, and the thermal response map is mapped to specific stage partitions according to the spatial division boundaries in the regional recognition map. For each spatial partition, the distribution density and change direction of the internal dynamic response centers are analyzed. The lower limit of the response density is set to 0.25 per m², and the directional consistency rate is set to not less than 80% as the identification condition for the effective motion trend area. The section that meets the conditions is defined as the main visual migration trajectory band. Further, with the main visual migration trajectory band as the skeleton, the edge of the trajectory change range in adjacent time periods is traced and the temporal density map is superimposed to generate the stage dynamic activity field spectrum per second. Next, the temporal synchronization construction of the dynamic stage visual structure field is carried out. The total span of the time axis is set as the current performance time period, and the unit resolution is 0.1 second. The entire time axis is equally spaced and segmented. For each time point, the light evolution vector, dynamic response center, and trajectory density information within the range of ±0.05 seconds near this moment are extracted. Average fitting is performed within this range to eliminate mutation interference, and a spatial distribution map corresponding to this time point is generated based on the fitting result. This processing flow is iteratively executed frame by frame, and the spatial activity characteristics of each 0.1-second time segment are gradually stitched together to form a complete temporal visual field. At the same time, the brightness center, color temperature main peak, and flow direction main axis information are marked in each frame map to form a spatio-temporal coupled dynamic stage visual structure sequence. In the output model, the average brightness, dominant color temperature, and main migration direction information of each spatio-temporal unit are retained, and high-frequency visual response tags are added as the basis for subsequent LED projection adaptation. Finally, a performance flow scene model including visual hot spot flow direction, regional visual density, and migration trend information is formed.

[0059] Optionally, the flow scene encoding prediction in step S2 is specifically as follows: Perform regional-level temporal map slicing processing on the performance flow scene model, execute equal-time interval data dissection, and establish a cross-frame position correspondence index to generate a set of scene time slice sequences; In this embodiment, after completing the spatio-temporal joint construction of the performance flow scene model, each second is selected as the basic time sequence unit, and the slice step size is set to 0.2 seconds. An equal-time-interval slicing operation is performed on the entire scene time sequence diagram to ensure that the generated diagram sequence has the time integrity and inter-frame coherence covering the complete performance cycle. During the slicing process, the regional resolution of each frame of the stage area diagram is set to 512×512 pixels, and the spatial coordinates and visual heat values of each tile at different time points are recorded based on the time-axis reconstruction structure constructed in advance. Through the spatial trajectory backtracking technology, a spatial position index is constructed for the feature regions in all adjacent sliced diagrams, and the historical changes of the same region are continuously encoded to obtain a cross-frame consistent scene time slice sequence set, providing a structural basis for subsequent dynamic region extraction and coding analysis.

[0060] Extract the motion vectors and distinguish the significant regions from the scene time slice sequence set to construct multi-scale optical flow diagrams and regional attention distribution diagrams, thereby generating dynamic region coding indexes; In this embodiment, for the constructed scene time slice sequence set, the displacement relationship between each pair of adjacent frames is extracted. Triple-scale windows are respectively set to 8×8, 16×16, and 32×32 pixels to calculate the difference amount of the optical flow distribution at the regional block level, and the regions with an average velocity gradient exceeding the threshold of 0.4 are marked as the initial motion significant regions. Combining the color gradient change directions in the illumination evolution vector group, calculate the response amplitudes of each significant region in the luminance channel and the color temperature channel, and construct a regional attention response diagram based on this. Further, integrate the motion consistency scores and attention response stability indexes of the same region at each scale, and determine it as a highly credible dynamic region according to the set aggregation criteria (such as continuous motion exists in all three scales and the variance of attention change is lower than 0.05). On this basis, the optical flow direction vector (θ), average velocity value (v), offset vectors of the color channels (ΔR, ΔG, ΔB), and the coherence degree with the surrounding region boundaries (calculated through the structure tensor) of the region are encoded into a dynamic region index node. Finally, these nodes are organized into a set of dynamic region coding index structures, and each element in the structure contains parameters such as the source scale identifier, motion intensity level, attention priority level, and time sequence stability score, serving as the key input data for subsequent rendering scheduling and content matching.

[0061] Based on the dynamic region coding index, construct a flow scene coding network, jointly model the semantic evolution trend and structural continuity in the flow scene coding network, and output a predicted coding vector flow; In this embodiment, relying on the dynamic region coding index structure, temporal state embedding is introduced into the constructed flowing scene coding network. By setting 3 seconds as the time window, sequential construction of the structure is performed on the continuous region coding within the window. The region position migration trend, the significance fluctuation curve, and the brightness center change curve are simultaneously input into the constructed coding process, and a jump tolerance of 10% is set for the continuous features therein. By performing trend comparison and structure reconstruction on the state sequence of each region within the time window, parametric modeling of the semantic evolution trend is carried out, and at the same time, changes in spatial connectivity and texture continuity in adjacent time periods are monitored, obtaining a predicted coding vector flow jointly composed of four dimensions: region number, dynamic score, trend direction, and visual heat, which is used to express the upcoming visual change trend.

[0062] Clustering compression and channel reduction processing are performed on the predicted coding vector flow, retaining the main components of dynamic changes, removing redundant modal channels and performing normalization processing to generate a dynamic coding feature map; In this embodiment, clustering compression processing is performed on the predicted coding vector flow according to the semantic channels after generation. First, it is classified by region number, and then the main variation direction of each class is calculated, and the first 2 main direction vectors are retained as representative features. A variation coefficient threshold of 0.2 is set for the remaining channels, and the modal channels below this threshold are uniformly marked as low contribution channels and removed. At the same time, normalization processing is performed on the retained channels, normalizing the dynamic components to the interval [-1, 1] to ensure that the contributions of each modal component to the final map are balanced. The finally output dynamic coding feature map is maintained within 64 channels, including five main types of features: motion direction, heat change, color offset main axis, region jump frequency, and spatial connectivity score, which is convenient for subsequent use in visual texture rendering and interactive area decision-making.

[0063] Dynamic texture enhancement is performed on the dynamic coding feature map, and heat gray value and content sensitivity modeling are performed on the results of the dynamic texture enhancement to generate a renderable scene factor.

[0064] In this embodiment, based on the dynamically encoded feature map, it is set that the texture enhancement reference library is composed of typical stage materials, including 10 types of lighting texture types, 12 color temperature gradient templates, and 8 pattern dynamic structure templates. Guided by the visual heat distribution, for each area, a texture template with a matching degree greater than 0.85 is selected for structural inlay enhancement, and the brightness gradient direction and local color mapping relationship are adjusted synchronously to enhance the visual expressiveness. The heat gray value of the area is evaluated on the enhancement result, and the weighted scores are calculated respectively based on the color dynamic degree, the structural dynamic degree, and the texture contrast. It is set that the comprehensive score greater than 0.75 is a high heat gray value area. At the same time, a content sensitivity model is established for each enhanced area, and combined with the previous audience interaction hotspots and visual concentration mapping, the texture detail retention level of the visually sensitive areas is improved. Finally, a renderable scene factor data set is generated, which includes the texture type encoding, enhancement level, dynamic feature weight, and rendering priority level of each area, and is used to drive the real-time image scheduling and rendering control of the LED screen in the next stage.

[0065] Especially importantly, the heat gray value and content sensitivity modeling are specifically as follows: Perform multi-scale texture level expansion on the dynamically encoded feature map, extract the texture direction consistency, regional edge strength, and brightness change gradient, calculate the intra-frame texture complexity score, and obtain the preliminary texture saliency score map; In this embodiment, after the dynamically encoded feature map is constructed, image regions are multi-scale expanded using tiles with three resolution levels of 128×128, 256×256, and 512×512 respectively to ensure capturing texture detail features at different spatial levels. Calculate the local texture direction consistency index for the tiles at each scale, extract the edge gradient histogram, and measure the maximum amplitude and frequency response value of the brightness change gradient. Among them, the direction consistency uses a standard deviation less than 0.3 as the determination threshold for the texture convergence area, and the edge strength uses the Sobel operator response value greater than 60 as the basis for the significant boundary. After weighted combination of the above three indicators, an intra-frame texture complexity score is constructed, and the scoring standard is set as a continuous value in the range of [0,1], and all scores are mapped to the gray intensity to generate the preliminary texture saliency score map. This map is used to express the density of structural changes at different scales in each frame and its potential contribution degree to the visual focus effect.

[0066] Based on the preliminary texture saliency score map, and combined with the active frequency of the high interaction density area and the visual concentration of the high visual attention area, establish a regional content sensitive weight matrix, thereby generating the content sensitivity map; In this embodiment, after normalizing the active frequency of the high interaction density region and the visual concentration of the high visual attention region to [0, 1], the texture score of each pixel is extracted from the preliminary texture saliency score map. By fusing the above three metrics, the content sensitivity calculation method for each position point is as follows: using the active frequency as the main weight term (weight 0.5), with the visual concentration and texture saliency each accounting for 0.25, and forming the initial content sensitivity value for each position through weighted summation. Subsequently, to enhance spatial continuity and the overall regional perception level, within each 64×64 pixel region, the average value and standard deviation of the initial content sensitivity values of all pixels within the region are calculated, and the variance of the texture scores within the region is introduced as a discriminant factor to construct a regional content sensitivity weight matrix. If the average content sensitivity within the region is greater than 0.6 and the variance of the texture scores is less than 0.15, it is judged as a "stable high-sensitivity region"; conversely, if the sensitive average is lower than 0.3 or the texture variance is greater than 0.3, it is marked as a "low signal-to-noise region", and a downscaling coefficient is assigned in the matrix. Finally, the weight coefficient of each region is multiplied by the initial sensitivity value of the pixels in that region to construct the overall content sensitivity map. This map retains regions with dense user behavior, stable line-of-sight attention, and rich texture features, while suppressing regions with random visual distribution or large texture change noise, serving as the basic input for constructing the subsequent rendering priority heat map.

[0067] The preliminary texture saliency score map and the content sensitivity map are fused pixel by pixel with weights to construct a rendering priority heat map; In this embodiment, a pixel-by-pixel fusion strategy is adopted to fuse the preliminary texture saliency score map and the content sensitivity map. At the pixel level, the proportion of the texture score in the fusion formula is set to 0.45, and the proportion of the content sensitivity is set to 0.55, ensuring that the finally generated rendering priority heat map is more inclined to retain regions with high perceptual value. The fusion value of each pixel is normalized and presented as a heat gray value in the rendering priority heat map, with the range defined between [0, 1]. To enhance the contrast between regions, a local maximum enhancement mechanism is introduced after the heat map is generated. For regions with a heat value higher than 0.75, local weighted amplification processing is performed to increase their selection probability in subsequent feature tensor processing. The generated rendering priority heat map is output in the form of a two-dimensional floating-point matrix, serving as the basis for extracting the main rendering components, and a regional position index is generated synchronously for subsequent spatial mapping and image rendering operations to call.

[0068] Based on the spatial positions of the regions in the rendering priority heat map where the heat gray value exceeds the preset heat gray value threshold, the channel tensors in the corresponding dynamic encoding feature map are activated and extracted and channel-scaled to output the main rendering component encoding vector set; In this embodiment, after the construction of the rendering priority heat map is completed, according to the regional spatial position index where the heat gray value is greater than the heat gray value threshold of 0.7, the corresponding tile channel tensor in the dynamic coding feature map is located. An activation extraction operation is performed on the selected regional channel tensor, strong response channels are extracted and local feature tensor compression is carried out, and the compression ratio is set to retain the first 5 response channels. The response intensity of each channel tensor is evaluated by adding the variance multiplied by the weight to the channel mean, and on this basis, the channel scaling operation is completed, standardizing the response amplitude range to [0,1] to eliminate the performance differences between channels. After compression and scaling, the coding vectors of all high-heat regions are summarized into the main rendering component coding vector set, and each vector contains attributes such as channel number, response intensity, time sequence position, and spatial position index, which are used to accurately describe the dominant content characteristics of each region in the rendering dimension.

[0069] Combined with the main rendering component coding vector set and the dynamic region coding index for spatial position mapping, a renderable scene factor is obtained.

[0070] In this embodiment, based on the main rendering component coding vector set, combined with the position trajectory and saliency weight information in the dynamic region coding index, regional-level spatial position mapping is performed. The mapping operation follows the coordinate tolerance with a spatial accuracy limit within 4 pixels, performs weighted merging on multiple coding vectors in the same region, and calculates the comprehensive rendering weight based on the response intensity and dynamic activity level. The merged vector group contains four core indicators: texture complexity weight, interaction priority level, brightness direction change trend, and dynamic saliency score, which constitute the final renderable scene factor. This factor is output in the JSON structure format, supporting subsequent key rendering scheduling for specific regions by the LED screen control system.

[0071] Optionally, the analysis of the pre-output image of the LED screen in step S3 is specifically as follows: Perform frame-by-frame timestamp synchronization and brightness spatial normalization processing on the LED screen display content data, and at the same time construct a display content frame sequence index structure to generate a standardized LED display frame data set; In this embodiment, the LED screen display content data in the LED screen display content data is extracted. By setting the time step to 0.1 second, the timestamps of each frame are calibrated to ensure that each display frame is processed in the actual time sequence. Subsequently, for the brightness information of each frame, the brightness values are normalized to the range of [0, 1] using a normalization algorithm to eliminate the influence of external light changes, ensure the consistency of the brightness contrast of each display frame, and enhance the consistency of visual perception. After completing the timestamp synchronization and brightness space normalization, a frame sequence index structure is created to record the timestamp, normalized brightness data, and their corresponding display content of each frame for subsequent timing analysis. Through these processes, a standardized LED display frame dataset is obtained, providing a unified and consistent data input for the subsequent steps.

[0072] Extract the per-frame image perception features from the standardized LED display frame dataset to generate a multi-dimensional frame feature data group; In this embodiment, for each frame in the standardized LED display frame dataset, first, image processing methods are used to extract multi-dimensional image features, including edge detection, color histogram analysis, texture analysis, and the extraction of local feature points (such as SIFT or ORB). During this process, by setting the region division of each frame image into an 8×8 grid, features such as the corresponding color mean, texture roughness, and edge contrast are extracted within each grid region. For each frame, the extracted feature information is stored in the multi-dimensional frame feature data group. These feature data include the spatial information, texture distribution, and visual focus area of the image, comprehensively characterizing the perception features of the LED screen display content. To improve the stability and accuracy of the features, preliminary dimensionality reduction processing can be performed on the image features to remove redundant features and ensure the balance between computational efficiency and processing accuracy.

[0073] Set the alignment window length to 10 frames, perform spatio-temporal alignment and co-perception projection on the standardized LED display frame dataset and the multi-dimensional frame feature data group, construct a stage-LED fusion feature map, and perform significant re-weighting of the frame content to generate an environment-constrained frame feature map; In this embodiment, the alignment window length is set to 10 frames, that is, the alignment process is performed within a 10-frame time window. During the alignment process, using the timestamp information of each frame of data, the normalized LED display frame data is temporally matched with the multi-dimensional frame feature data to ensure that the image features of each frame are accurately docked with the corresponding time information. Subsequently, through collaborative perception projection, the image features and time series information are fused to obtain a stage-LED fusion feature map. This fusion feature map comprehensively considers the dynamic changes and spatial features of each frame of the image within the time series, and enhances the significant regions (such as color changes, object contours, etc.) through a reweighting strategy, thereby generating an environment-constrained frame feature map. This process optimizes the perception of visual content, enabling the display content to be adjusted according to the current environmental conditions (such as external light, audience gaze direction, etc.).

[0074] Perform forward visual prediction on the environment-constrained frame feature map to predict the motion trend, texture change, and brightness adjustment amplitude of the screen frame at the next moment, and generate an LED frame prediction offset map; In this embodiment, based on the environment-constrained frame feature map, forward visual prediction is performed to infer the change trend of the LED screen display content at the next moment. First, the visual content of the next frame is inferred by combining the motion trend, texture change, and brightness adjustment information of the current frame. During this process, the optical flow method is used to calculate the motion vector of the image to evaluate the spatial change of the screen content. By analyzing the texture change and brightness adjustment amplitude of the current frame, the light and shadow effect and color tone change of the display content at the next time point are predicted. The prediction result generates an LED frame prediction offset map, which contains the offset amount, texture change, and brightness adjustment amplitude of each pixel point in the time dimension, providing data support for subsequent inter-frame fusion.

[0075] Perform cross-frame fusion comparison between the LED frame prediction offset map and the environment-constrained frame feature map, set the fusion residual threshold to 0.15, screen out the pixel blocks below the fusion residual threshold, and combine the regions with a structural similarity > 0.75 for layer splicing to generate the LED screen visual pre-output data.

[0076] In this embodiment, when performing cross-frame fusion comparison between the LED frame prediction offset map and the environmental constraint frame feature map, first set the fusion residual threshold to 0.15, and calculate the pixel value difference at the corresponding positions of the LED frame prediction offset map and the environmental constraint frame feature map, that is, the residual. If the residual value of a certain area is greater than 0.15, it is considered that significant changes have occurred in this area in the time dimension, so key processing is required. These areas usually represent areas of motion change, texture change, or brightness adjustment. Thus, the areas where the residuals of the two images at the corresponding pixel blocks are greater than this threshold are screened out. Then, calculate the structural similarity of the image areas. To further optimize the fusion effect, use the structural similarity index (SSIM) to calculate the structural similarity between the LED frame prediction offset map and the environmental constraint frame feature map. SSIM measures the similarity between two images in terms of brightness, contrast, and structure. Set the area with a structural similarity greater than 0.75 as the high-similarity area, indicating that the similarity of this part of the area between the two images is high and can be seamlessly docked visually. After screening out the high-similarity areas, layer splicing will be performed. Specifically, perform pixel-by-pixel weighted fusion on the image data of these areas and fine-tune it in combination with the predicted offset values. This means not only splicing the pixel blocks in the high-similarity areas, but also making spatial adjustments according to the offset amounts in the LED frame prediction offset map, so as to ensure the accurate docking of the two images in terms of spatial position and avoid phenomena such as image misalignment or content inconsistency. Finally, generate the visual pre-output data of the LED screen. After splicing is completed, merge these fused areas into the current display frame to generate the final visual pre-output data of the LED screen. These output data will be used as the reference frame for the next moment's display to ensure smooth and natural transitions between consecutive frames of the display content.

[0077] Optionally, step S5 is specifically as follows: Step S51: Perform tile division and boundary ambiguity evaluation on the stage visual dynamic output structure diagram to obtain the graph structure area coding matrix; In this embodiment, tile division and boundary ambiguity evaluation are performed on the stage visual dynamic output structure diagram. Multiscale analysis is carried out on the stage visual dynamic output structure diagram. Based on the visual contribution and dynamic activity in the stage feature response map, fine-grained division is performed within each region. Specifically, first, regional segmentation is performed on the multiscale features of the stage visual output diagram to ensure accurate division of each region. When dividing tiles, the boundary ambiguity of each region is evaluated, and the color difference and texture change between adjacent regions are calculated. The boundary ambiguity is judged by calculating the gray gradient and texture complexity of the region. If the change gradient of this region is lower than the set threshold (for example, the gray change is less than 0.2), then this region is determined as a boundary fuzzy region. If the gradient is greater than the threshold, it is determined as a boundary clear region. This evaluation result will form a graph structure region coding matrix, which contains the spatial position, visual contribution degree, and dynamic activity of the region, providing important data support for subsequent graph construction.

[0078] Step S52: Based on the graph structure region coding matrix, construct a heterogeneous graph-driven atlas, and fuse the currently obtained LED screen control interface status, pixel arrangement matrix, and refresh channel configuration file for graph-hardware adaptation reconstruction to generate a rendering driver mapping graph; In this embodiment, based on the graph structure region coding matrix, start constructing a heterogeneous graph-driven atlas. First, combine the real-time status data obtained from the LED screen control interface with the graph structure region coding matrix. These data include the control interface status of the LED screen (such as display mode, brightness adjustment, color space configuration), pixel arrangement matrix (for example, the resolution is 1920×1080), and refresh channel configuration file (such as 60Hz refresh rate per second). These data ensure that the output of the atlas meets the display requirements of the hardware. Then, perform hardware adaptation on each tile according to these input data to ensure that the rendering requirements of each region can match the actual display capabilities of the LED screen. For example, in regions with higher resolution, more display details will be added, while in low-resolution regions, redundant data will be reduced, thereby improving the display efficiency. Finally, a rendering driver mapping graph is generated, which maps the adaptation relationship between the stage visual region and the LED screen hardware resources and provides data support for subsequent rendering priority configuration.

[0079] Step S53: Using the rendering driver mapping graph as the input, combine the current frame buffer load of the LED screen control platform, GPU computing power, and IO scheduling frequency parameters to construct a real-time frame-level load balancing model, and dynamically generate a millisecond-level rendering priority configuration table; In this embodiment, a real-time frame-level load balancing model is constructed by combining the rendering driver mapping diagram, the current frame buffer load of the LED screen control platform, the GPU computing power, and the I / O scheduling frequency parameters. The process of constructing the real-time frame-level load balancing model dynamically adjusts the rendering order and resource allocation of the LED screen display content by integrating multiple key parameters. First, real-time data is obtained from the LED screen control platform, specifically including the current frame buffer load (for example, the cache usage rate is 70%), the GPU computing power (such as the GPU processing power is 1.5 TFLOPS), and the I / O scheduling frequency (for example, the I / O scheduling frequency is 200 times per second). Based on the evaluation of the visual contribution and dynamic activity of each region in the rendering driver mapping diagram, the rendering tasks of different regions are prioritized. High-priority regions usually include regions with large dynamic changes or strong visual requirements (such as the area where the actor's position changes or the lighting is adjusted), and low-priority regions may be static backgrounds or regions with small changes. According to the GPU computing power and I / O frequency, the high-priority regions are matched with stronger computing resources (such as regions with stronger GPU capabilities) to ensure that the high-priority regions can obtain sufficient computing power and bandwidth resources. For low-priority regions, resource reduction processing is performed to reduce computing and bandwidth allocation to reduce the load. By real-time monitoring the frame buffer load, the utilization of hardware resources can be detected. For example, when the GPU load approaches the maximum computing power, the rendering complexity of the high-load regions will be automatically reduced, or the rendering accuracy will be reduced to avoid GPU overload. At the same time, the I / O frequency and bandwidth allocation will also be dynamically adjusted according to the changes in the real-time data stream to ensure the effective transmission of data. After completing the resource allocation and load adjustment, a frame-level load balancing configuration table is generated based on the above information, which defines the allocation of GPU and I / O resources during each rendering time period. This configuration table includes the rendering priority, computing resource requirements, and data transmission requirements of each region, ensuring that each rendering task can be completed efficiently and on time.

[0080] Step S54: According to the millisecond-level rendering priority configuration table, perform multi-channel image rendering on each high-priority graph structure region in the stage visual dynamic output structure diagram, construct a real-time picture, and output a millisecond-level LED rendering tile data set; In this embodiment, according to the generated millisecond-level rendering priority configuration table, multi-channel image rendering is performed on each high-priority graphic structure area in the stage visual dynamic output structure diagram. Processing each area one by one according to the rendering priority configuration table to ensure that the rendering effect of high-priority areas is more delicate. For example, areas with higher visual contribution and dynamic activity (such as the stage foreground area) will be rendered preferentially, while the rendering accuracy of the background area with lower visual contribution can be appropriately reduced. The rendering of each area adopts a multi-channel rendering method, separately processing different color channels, brightness channels, and texture channels to construct a real-time picture. During the rendering process, the rendering strategy of each area is dynamically adjusted according to the status of hardware resources, and finally a millisecond-level LED rendering tile data set is output to ensure high-quality image rendering in an extremely short time.

[0081] Step S55: Perform DMA channel compression optimization and timing packetization processing on the LED rendering tile data set to match the protocol bandwidth and cache frame refresh rate of the LED screen control platform, and construct an LED control signal stream; In this embodiment, DMA channel compression optimization and timing packetization processing are performed on the LED rendering tile data set. First, the system performs compression optimization on the rendering tiles according to the protocol bandwidth of the LED screen (for example, the maximum bandwidth is 5 Gbps) and the cache frame refresh rate (for example, 60 frames per second refresh rate). During the compression optimization process, different compression strategies are selected according to the size and complexity of different tiles. For example, for high-resolution areas, a more efficient compression algorithm is used to reduce the consumption of transmission bandwidth. Then, the compressed tile data is sorted according to the refresh channel priority, and a signal stream frame header is constructed according to the communication protocol format of the LED screen control platform (such as SPI, LVDS, or a dedicated serial interface), including tile number, timestamp, channel identifier, and data length fields, to ensure that the signal stream can be correctly decoded on the control platform. Subsequently, each tile data is encoded into a byte sequence corresponding to the control platform, and a synchronization identifier and error detection check code (such as CRC check) are inserted to generate a complete control signal frame (LED control signal stream). This process also considers the LED screen cache mechanism and parallel refresh strategy, marks the channel ownership of each channel tile signal stream, and performs interleaved timing scheduling on the bus to form a DMA packetized data stream supporting multi-channel concurrent transmission. Finally, the control signal stream is transmitted to the control platform buffer through a high-speed interface (such as the FPGA DMA write channel).

[0082] Step S56: Send the LED control signal stream to the LED screen control platform in real time to drive the LED screen to complete the task of displaying each frame of the rendered content.

[0083] In this embodiment, the generated LED control signal stream is sent to the LED screen control platform in real time. First, the signal stream is divided into multiple time frames to ensure that each frame is transmitted in real time with millisecond-level precision. To ensure smooth data transmission, the size and transmission order of each time frame are dynamically adjusted according to the frame buffer and transmission bandwidth of the LED screen control platform. For example, if there is less remaining space in the buffer of the LED screen, the size of the time frame will be adjusted to reduce the amount of data transmitted in a single frame to avoid buffer overflow. At the same time, ensure that each time frame strictly follows the preset time sequence to ensure that each tile is displayed within the specified time to complete the frame-by-frame display task. Finally, the LED screen is driven by the LED screen control platform to complete the frame-by-frame display task of the stage visual content.

[0084] Optionally, this specification also provides an LED screen display control system, which is characterized in that it is used to execute the LED screen display control method as described above. The LED screen display control system includes: An environment perception module, which is used to obtain multi-modal stage sensing data and real-time LED screen display data, and perform stage performance environment perception based on the multi-modal stage sensing data and real-time LED screen display data to obtain stage environment perception data; A flowing scene modeling module, which is used to perform continuous modeling of the stage visual structure-light change based on the stage environment perception data to obtain a performance flowing scene model; perform flowing scene coding prediction on the performance flowing scene model to obtain a renderable scene factor; A pre-output picture analysis module, which is used to obtain LED screen display content data, and analyze the LED screen pre-output picture based on the LED screen display content data and stage environment perception data to obtain LED screen visual pre-output data; A dynamic feature inference module, which is used to perform multi-modal stage dynamic feature inference based on the LED screen visual pre-output data and the renderable scene factor to obtain a stage visual dynamic output structure diagram; A heterogeneous rendering module, which is used to perform real-time heterogeneous drive rendering on the stage visual dynamic output structure diagram to obtain a millisecond-level LED rendering tile data set, and transmit it to the LED screen control platform to execute the LED screen display rendering task.

[0085] Optionally, this specification also provides an LED screen, which includes an LED screen main body, a power supply unit, and an electrical control unit. Among them, the power supply unit is installed inside the LED screen main body, the electrical control unit is electrically connected to the power supply unit, and the electrical control unit is used to charge the LED screen main body and control the LED screen main body. The electrical control unit is used to execute the LED screen display control method as described above.

[0086] Therefore, from any perspective, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Accordingly, all changes that fall within the meaning and scope of the equivalent elements of the application documents are intended to be embraced within the present invention.

[0087] The above description is only a specific implementation manner of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features invented herein.

Claims

1. A LED screen display control method, characterized in that: The following steps are involved: Step S1: acquiring multimodal stage sensor data and LED screen real-time display data, and performing stage performance environment perception according to the multimodal stage sensor data and LED screen real-time display data to obtain stage environment perception data; Step S2: Based on the stage environment perception data, continuous modeling of stage visual structure-illumination changes is performed to obtain a performance flow scene model; flow scene coding prediction is performed on the performance flow scene model to obtain a renderable scene factor; Step S3: obtaining LED screen display content data, and analyzing the LED screen pre-output picture based on the LED screen display content data and the stage environment perception data to obtain LED screen visual pre-output data; Step S4: performing multimodal stage dynamic feature inference according to the LED screen visual pre-output data and the renderable scene factor to obtain a stage visual dynamic output structure diagram; Step S5: Perform real-time heterogeneous drive rendering on the stage visual dynamic output structure diagram to obtain a millisecond-level LED rendering block data set, and transmit it to the LED screen control platform to execute the LED screen display rendering task.

2. The LED screen display control method according to claim 1, characterized in that: Step S1 is specifically as follows: Step S11: acquiring multimodal stage sensor data and LED screen real-time display data, and performing spatiotemporal alignment and noise suppression processing on the multimodal stage sensor data to generate calibrated multimodal sensor data; Step S12: performing frame sequence decomposition and color gamut mapping analysis on the real-time display data of the LED screen to extract display metadata features; Step S13: performing cross-modal fusion analysis on the calibration multi-modal sensor data and the display metadata features, thereby obtaining modal feature fusion data; Step S14: constructing a stage environment perception map with the sensor nodes of the modal feature fusion data as graph nodes and the modal collaborative change rate as edge weights, and performing feature weighting and aggregation to generate a multi-modal collaborative perception map; Step S15: semantic clustering and regional segmentation are performed on the multimodal collaborative perception map to identify the stage visual hot spots and the areas where actors interact actively. Local feature back-projection comparison is performed in combination with the current display frame of the LED screen obtained in real time to generate stage environment perception data.

3. The LED screen display control method according to claim 2, characterized in that: Step S13 is specifically as follows: Step S131: performing modal dimension structure mapping processing on the calibrated multimodal sensing data to generate a standardized modal structure data set; Step S132: performing timestamp extension and channel layering processing on display metadata features to generate a time-enhanced display content semantic vector group; Step S133: performing spatiotemporal position-level alignment on the standardized modal structure dataset and the time-enhanced display content semantic vector group, performing semantic association matching between modalities, and obtaining a preliminary cross-modal matching matrix; Step S134: performing multi-head attention weight fusion encoding based on the preliminary cross-modal matching matrix, and reconstructing modal collaborative features to generate a multi-channel fusion tensor atlas; Step S135: Perform channel compression and redundant feature pruning on the multi-channel fusion tensor atlas to generate modal feature fusion data.

4. The LED screen display control method according to claim 2, characterized in that: Step S15 is specifically as follows: Step S151: setting the frequency domain decomposition dimension to 64, performing spectral feature decomposition on the graph nodes and edge weights in the multimodal collaborative perception graph, performing spectral density estimation on the modal collaborative change frequency of the graph nodes, and selecting the top 10 high-energy spectral features with a cumulative energy share of more than 85% to construct the graph high-frequency feature subspace data; Step S152: clustering graph nodes based on the high-frequency feature subspace data of the graph, and dividing the graph node clustering results into semantic categories to obtain label sensor nodes of visually active hotspot areas and label sensor nodes of interactive hotspot clustering areas, thereby generating semantic clustering label data; Step S153: Performing regional segmentation and reconstruction on the multimodal collaborative perception graph according to the semantic clustering label data, grouping the nodes of the same clustering label graph into regional subgraphs, and setting the edge weight cutoff threshold to 0.35 for edge sparse processing to generate a regional recognition graph; Step S154: real-time acquisition of the current display frame of the LED screen, building a visual feature template based on the key color gamut block and brightness gradient structure of the current display frame of the LED screen, and performing multi-dimensional feature matching with the visual template through the attribute vector of each regional node in the regional recognition map, building a 5-dimensional attribute matching function, setting the feature similarity threshold to 0.7 for reverse feature mapping, and generating a back-projected regional association matrix; Step S155: Perform a joint decoupling analysis on the region recognition map and the back-projected region association matrix, set the decoupling mapping matrix dimension to 64, use the regional mutual information gain greater than 0.1 as the effective fusion index, extract the cross-overlapping areas and calculate the visual flow focus, set the areas with active frequency ≥ 8 Hz as high interaction density areas for labeling, and set the areas with visual concentration ≥ 0.85 as high visual attention areas for labeling, and obtain the stage environment perception data.

5. The LED screen display control method according to claim 1, characterized in that: The stage visual structure-illumination change continuous modeling in step S2 is specifically as follows: Extracting multi-period frames of stage area image modality based on stage environment perception data, and using the multi-period frames of stage area image modality to construct inter-frame motion reference indexes, and generating an initial stage time-series image set; Perform structured illumination separation on the initial stage time-series image set, decompose it into structural brightness component and reflection color component, calculate the illumination gradient sequence diagram, and generate illumination change structure data; Based on the illumination change structure data and inter-frame motion reference index, a stage space continuum model is constructed; Dynamically model the changes in light intensity, direction and color temperature on the time axis of the stage space continuum model to obtain the temporal light response tensor; Perform principal component analysis and low-rank feature reconstruction on the time series illumination response tensor, model the illumination main mode change trajectory in the continuous time period in the low-rank feature reconstruction result, and obtain the illumination evolution vector group; The stage space continuum model is fused and matched with the lighting evolution vector group, and the dynamic stage visual structure field is reconstructed according to the joint distribution of spatial position points and time points of the fusion and matching results to generate a performance flow scene model.

6. The LED screen display control method according to claim 1, characterized in that: The flow scene coding prediction in step S2 is specifically as follows: Perform regional-level time-series graph slicing on the performance flow scene model, perform equal-time interval data segmentation, and establish cross-frame position corresponding indexes to generate scene time slice sequence sets; Extract motion vectors and identify salient regions from the scene time slice sequence set, construct multi-scale optical flow maps and regional attention distribution maps, and thus generate dynamic region coding indexes; Based on the dynamic region coding index, a flow scene coding network is constructed, the semantic evolution trend and structural continuity in the flow scene coding network are jointly modeled, and the prediction coding vector flow is output; The prediction coding vector stream is clustered and compressed and channel-reduced to retain the dynamically changing principal components, remove redundant modal channels and perform standardization to generate a dynamic coding feature map. Dynamic texture enhancement is performed on the dynamic coding feature map, and the thermal grayscale value and content sensitivity of the dynamic texture enhancement result are modeled to generate a renderable scene factor.

7. The LED screen display control method according to claim 6, characterized in that: The analysis LED screen pre-output picture in step S3 is specifically: Perform frame-by-frame timestamp synchronization and brightness space normalization on the LED screen display content data, and build a display content frame sequence index structure to generate a standardized LED display frame data set; Extract frame-by-frame image perception features from the standardized LED display frame data set to generate a multi-dimensional frame feature data set; The alignment window length is set to 10 frames, and the standardized LED display frame data set and the multi-dimensional frame feature data set are spatiotemporally aligned and collaboratively perceived to construct a stage-LED fusion feature map. The frame content saliency is re-weighted to generate an environmental constraint frame feature map. Perform forward visual prediction on the environmental constraint frame feature map to predict the motion trend, texture change and brightness adjustment amplitude of the screen frame at the next moment, and generate an LED frame prediction offset map; The LED frame prediction offset map and the environmental constraint frame feature map are fused and compared across frames. The fusion residual threshold is set to 0.15, the pixel blocks below the fusion residual threshold are screened out, and the layers are stitched in the areas with structural similarity > 0.75 to generate the LED screen visual pre-output data.

8. The LED screen display control method according to claim 1, characterized in that: Step S5 is specifically as follows: Step S51: performing block division and boundary fuzziness evaluation on the stage visual dynamic output structure graph to obtain a graph structure region coding matrix; Step S52: construct a heterogeneous graph drive map based on the graph structure region coding matrix, and fuse the current control interface state of the LED screen, the pixel arrangement matrix and the refresh channel configuration file obtained in real time to perform graph-hardware adaptation reconstruction to generate a rendering drive mapping map; Step S53: Taking the rendering driver map as input, combining the current frame buffer load, GPU computing power and IO scheduling frequency parameters of the LED screen control platform, constructing a real-time frame-level load balancing model, and dynamically generating a millisecond-level rendering priority configuration table; Step S54: performing multi-channel image rendering on each high-priority graph structure area in the stage visual dynamic output structure diagram according to the millisecond rendering priority configuration table, constructing a real-time image, and outputting a millisecond-level LED rendering block data set; Step S55: performing DMA channel compression optimization and timing packet processing on the LED rendering tile data set, matching the protocol bandwidth and cache frame refresh rate of the LED screen control platform, and constructing an LED control signal flow; Step S56: Send the LED control signal stream to the LED screen control platform in real time to drive the LED screen to complete the frame-by-frame display task of the rendered content.

9. A LED screen display control system, characterized in that: Used to execute the LED screen display control method as claimed in claim 1, the LED screen display control system comprises: The environment perception module is used to obtain multi-modal stage sensor data and LED screen real-time display data, and to perceive the stage performance environment based on the multi-modal stage sensor data and LED screen real-time display data to obtain stage environment perception data; The flow scene modeling module is used to continuously model the stage visual structure-illumination changes based on the stage environment perception data to obtain the performance flow scene model; the flow scene coding prediction is performed on the performance flow scene model to obtain the renderable scene factor; The pre-output image analysis module is used to obtain the LED screen display content data, and analyze the LED screen pre-output image based on the LED screen display content data and the stage environment perception data to obtain the LED screen visual pre-output data; The dynamic feature inference module is used to perform multimodal stage dynamic feature inference based on the LED screen visual pre-output data and renderable scene factors to obtain the stage visual dynamic output structure diagram; The heterogeneous rendering module is used to perform real-time heterogeneous drive rendering of the stage visual dynamic output structure diagram, obtain millisecond-level LED rendering block data set, and transmit it to the LED screen control platform to execute the LED screen display rendering task.

10. An LED screen, characterized in that: It includes an LED screen main body, a power supply unit and an electrical control unit, wherein the power supply unit is installed inside the LED screen main body, the electrical control unit is electrically connected to the power supply unit, the electrical control unit is used to charge and control the LED screen main body, and the electrical control unit is used to execute the LED screen display control method as described in any one of claims 1-8.

Images