A smart control integrated system for commercial optoelectronic display devices
By employing a cloud-edge-device distributed architecture and multi-agent reinforcement learning, the problem of heterogeneous device scheduling and energy efficiency coordination in large-scale networking of commercial optoelectronic display equipment has been solved, achieving low-cost expansion, low latency, and high-efficiency energy management.
Patent Information
- Application Number
- CN202611250254.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-18
- Publication Date
- 2026-11-17
AI Technical Summary
In large-scale networking applications of commercial optoelectronic display equipment, there are problems such as the inability to uniformly and intelligently schedule heterogeneous devices from multiple brands, the lack of a globally optimal energy efficiency collaborative control mechanism in large-scale display networks, and irreconcilable contradictions between edge-side artificial intelligence computing power constraints and control accuracy, and overall system power consumption.
It adopts a three-layer distributed architecture consisting of cloud layer, edge layer and terminal layer, combined with device capability description file registration mechanism, multi-agent reinforcement learning and power consumption-aware hierarchical artificial intelligence inference scheduling, to achieve unified intelligent access, global optimal energy efficiency collaborative allocation and power consumption-aware hierarchical scheduling of heterogeneous devices across brands.
It enables low-cost access to heterogeneous devices from multiple brands, reduces system expansion costs, lowers overall energy consumption by 20-35%, and stabilizes inference latency within 50 milliseconds, ensuring a positive gain in control accuracy and system energy-saving benefits.
Smart Images

Figure CN123123293A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and Internet of Things technology, specifically relating to an intelligent control system for commercial optoelectronic display equipment. It can be applied to large-scale networking scenarios for unified access, global energy efficiency collaborative optimization, and power consumption perception-based hierarchical intelligent scheduling of multi-brand heterogeneous display devices such as outdoor LED large screens, LCD splicing screens, and OLED irregular-shaped screens. Background Technology
[0002] The control system of commercial optoelectronic display equipment typically consists of a display controller, a data receiving module, broadcast control software, and a network communication layer. The display controller is responsible for encoding video signals and transmitting them to the receiving module through each manufacturer's proprietary protocols, driving the display module to complete the image presentation. At the network management level, existing systems typically use centralized broadcast control software to push content and configure basic parameters for each display terminal via a local area network or the Internet. The software and hardware of each brand are deeply bound together, and the communication protocols are not publicly disclosed, forming a closed ecosystem. In the field of energy-saving control, existing technical solutions include timed brightness adjustment based on a real-time clock and ambient light compensation dimming based on a single threshold trigger of a photosensitive sensor. At the level of artificial intelligence applications, reinforcement learning and its multi-agent extension have achieved significant application results in fields such as industrial control and smart grids. The hierarchical inference architecture that combines edge computing and cloud computing has also been successfully implemented in scenarios such as autonomous driving and industrial quality inspection. The above technologies constitute the algorithmic theoretical basis of this invention.
[0003] The closest existing technical solution to this invention is a multi-screen linkage display control system based on a cloud-based broadcast control platform. This system consists of a cloud-based broadcast control server, display terminals of various brands, and brand-specific protocol adaptation modules. This solution has the following three core technical defects: First, because each manufacturer uses closed proprietary protocols, the existing solution must maintain a separate hard-coded protocol adaptation module for each brand. When a new brand device is connected, the adaptation layer needs to be redeveloped. The system expansion cost increases linearly with the number of connected brands, and it cannot execute unified intelligent scheduling commands among heterogeneous devices. Second, the existing energy-saving control solution only relies on static time rules or single sensor threshold triggering, and cannot comprehensively perceive multi-dimensional contextual information such as display content characteristics, audience density, and dynamic load of the power grid. Each device independently dims to form a local suboptimal solution, and the overall display network energy consumption has significant redundancy. Third, under the condition of limited computing power on the device side, deploying a high-precision artificial intelligence model will introduce excessively high inference latency and its own energy consumption, while over-compressing the model will lead to a sharp drop in control accuracy. At present, the industry lacks an integrated framework that can simultaneously ensure control accuracy and overall system energy efficiency under the constraint of computing power on the device side. Summary of the Invention
[0004] The technical problem to be solved by this invention is: in the application scenario of large-scale networking of commercial optoelectronic display equipment, there are problems such as the inability to uniformly and intelligently schedule multi-brand heterogeneous devices, the lack of a global optimal energy efficiency collaborative control mechanism in large-scale display networks, and the irreconcilable contradictions between edge-side artificial intelligence computing power constraints and control accuracy and overall system power consumption. The invention provides an intelligent control integration system that can realize unified intelligent access of cross-brand heterogeneous devices, global optimal energy efficiency collaborative allocation, and power consumption-aware hierarchical artificial intelligence reasoning scheduling.
[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:
[0006] A smart control integration system for commercial optoelectronic display equipment adopts a three-layer distributed architecture consisting of a cloud layer, an edge layer, and an end-side layer. The cloud layer includes a large model inference server, a multi-agent reinforcement learning centralized training module, and a fault prediction module. The edge layer includes a semantic middleware module, a multi-agent reinforcement learning distributed execution strategy module, an edge artificial intelligence perception fusion module, and an artificial intelligence energy consumption constraint monitoring module. The semantic middleware module includes a semantic receiving layer, an intent parsing engine, a device abstraction layer, and a protocol adaptation layer. The device abstraction layer maintains a device capability description file registry. The protocol adaptation layer contains private protocol driver plugins corresponding to each brand. The end-side layer includes multi-brand display equipment and the on-chip system controller containing a neural network processor, a field-programmable gate array module, and environmental sensors mounted on each device.
[0007] A smart control method for commercial optoelectronic display equipment, applied to the aforementioned system, includes steps for receiving and verifying intent commands, classifying intents and filling slots, querying and downgrading equipment capabilities, generating and issuing operation primitive sequences, coordinating global energy efficiency optimization, and allocating hierarchical artificial intelligence reasoning tasks.
[0008] The beneficial effects of this invention are as follows:
[0009] 1. This invention utilizes a device capability description file registration mechanism and a device abstraction layer generic primitive mapping mechanism. When a new brand device is connected, it is only necessary to add the corresponding device capability description file to the device abstraction layer registry and deploy the corresponding brand protocol driver plugin. No modification to the cloud control logic is required. This reduces the cost of connecting and expanding multi-brand heterogeneous display devices from linearly increasing with the number of brands to a fixed registration operation, greatly improving the system's scalability.
[0010] 2. This invention utilizes a global energy efficiency collaborative optimization mechanism based on multi-agent reinforcement learning to comprehensively leverage multi-dimensional information such as the average image level of the displayed content, ambient light intensity, anonymized audience density estimation, real-time power consumption of the device, and grid load factor. Under the premise of satisfying visual comfort constraints, compared with the traditional static time-based dimming scheme, the overall energy consumption of the entire network can be reduced by 20% to 35%.
[0011] 3. The cloud-edge-device layered AI inference scheduling and AI energy consumption constraint feedback loop of this invention ensure that the real-time dimming inference latency on the device side is stable within 50 milliseconds. At the same time, in high-load grid scenarios, by dynamically reducing the sampling rate and inference frequency, the total power consumption of the AI sensing module does not exceed the preset upper limit of the system's energy-saving benefits, thus achieving a positive gain in energy-saving benefits.
[0012] 4. This invention utilizes a field-programmable gate array (FPGA) hardware pipeline parallel accumulation tree structure to calculate the average image level value while processing video frame rate conversion, achieving real-time feature extraction with zero additional computing power overhead and a frame processing delay of no more than sixteen milliseconds. Attached Figure Description
[0013] Figure 1 This is a data flow diagram of the three-layer overall architecture of the present invention;
[0014] Figure 2 This is a flowchart of the semantic middleware instruction processing of the present invention;
[0015] Figure 3 This is a schematic diagram of the closed-loop design of multi-agent energy efficiency optimization and grid energy consumption constraints of the present invention. Detailed Implementation
[0016] The following is in conjunction with the appendix Figure 1-3 The specific embodiments of the present invention will be further described below. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0017] I. System Overall Architecture
[0018] The intelligent control integration system for commercial optoelectronic display equipment proposed in this invention adopts a three-layer distributed architecture consisting of a cloud layer, an edge layer, and a terminal layer. Each layer has a clear function and close cooperation, forming a complete closed-loop control system.
[0019] The cloud layer is deployed in public clouds or private data centers, undertaking the tasks with the highest computing power requirements. The large model inference server is responsible for receiving high-level semantic control instructions input by operators (such as reducing the brightness of outdoor screens to a level suitable for nighttime viewing), understanding and parsing them into structured intent instruction packages, and then sending them to the edge layer via message queues. The multi-agent reinforcement learning centralized training module is responsible for completing the centralized training of energy efficiency optimization strategies offline or in near real-time. After training, the quantized and compressed distributed strategy parameters are sent to the corresponding edge nodes. The fault prediction module uses long short-term memory networks to perform equipment health status assessment and predictive maintenance analysis based on the historical operating data of each display device. The fault prediction results are allowed to be delayed by no more than ten minutes. The cloud layer does not communicate directly with the display terminal, but only exchanges intent instructions and strategy parameters with the edge layer through secure hypertext transfer protocol or message queue telemetry transmission protocol.
[0020] The edge layer, deployed on the broadcast control server of each display network node, is the core control hub of this invention. It integrates four core modules: semantic middleware module, multi-agent reinforcement learning distributed execution strategy module, edge artificial intelligence perception fusion module, and artificial intelligence energy consumption constraint monitoring module. The edge layer is responsible for converting cloud intents into device operation primitives in real time, executing global energy efficiency coordination decisions, and dynamically managing the power consumption budget of artificial intelligence inference.
[0021] The edge layer refers to the on-chip system main controller of each display device, which contains a neural network processor. It only runs a quantized and compressed ultra-lightweight model to perform real-time dimming (inference latency not exceeding 50 milliseconds) and basic degradation control for device self-protection (response latency not exceeding 100 milliseconds). The edge layer is also equipped with an illuminance sensor, millimeter-wave radar and field-programmable gate array module for local feature acquisition and frame synchronization hardware calculation of average image level values.
[0022] II. Core Module 1: Cross-Brand Heterogeneous Protocol Semantic Middleware
[0023] The semantic middleware module adopts a four-layer abstract structure, consisting of a semantic receiving layer, an intent parsing engine layer, a device abstraction layer, and a protocol adaptation layer, from input to output.
[0024] The semantic receiving layer continuously listens to the message queue from the cloud, receiving intent instruction packets encapsulated in a structured data format. The core fields of the intent instruction packet include: intent type field (such as operation type identifiers for adjusting brightness, synchronizing scenes, emergency shutdown, etc.), target device identifier set (a list of universally unique identifiers for the display devices to be controlled), target parameter dictionary (the mapping relationship between parameter names and target parameter values), priority field (an integer ranging from one to five, where five is the highest priority, and high-priority intents can preempt the currently executing low-priority operation primitive sequence), and constraint field (such as execution constraints such as the brightness change rate not exceeding a specified value per second). The semantic receiving layer performs format validity verification on each received intent instruction packet according to a predefined format pattern. If the verification fails, an error code is returned to the cloud and the packet is discarded without further processing.
[0025] The intent parsing engine receives the verified intent instruction packet, inputs the intent type field into a quantized and compressed lightweight intent classifier (based on a small bidirectional encoder representation model with approximately 4.4 million parameters, and a single inference latency of approximately eight milliseconds on a neural network processor), and outputs a standard operation type code and a confidence value. The default decision threshold for the confidence value is set to 0.85: if the confidence value is lower than this threshold, the intent is marked as an ambiguous intent and a process of requesting secondary confirmation from the cloud is triggered; if the confidence value is not lower than this threshold, the standard operation type code, the target parameter dictionary, and the constraint condition field are encapsulated together into a structured intent object and transferred to the device abstraction layer for processing.
[0026] The device abstraction layer is the core innovation layer of the semantic middleware. It maintains a device capability description file registry. Each connected display device must register a device capability description file in the registry. This file records: the device's unique identifier (general unique identification code format), brand identifier, display technology type (LED, LCD, OLED), and the set of supported general operation primitives. In the set of general operation primitives, each operation primitive includes a primitive name (such as setting brightness, synchronously adjusting contrast, etc.), parameter type, parameter range, and a function pointer to the brand driver function corresponding to the protocol adaptation layer.
[0027] After receiving the structured intent object, the device abstraction layer iterates through each device identifier in the target device identifier set, queries its device capability description file, checks whether the general primitive corresponding to the standard operation type code is in the device's support list, and adds the device to the capability mismatch device list and generates a degradation operation scheme (e.g., downgrading synchronous contrast adjustment to an approximate replacement of setting brightness) for the supported device. For supported devices, the corresponding protocol adaptation layer function pointer is extracted from the device capability description file.
[0028] During the operation primitive sequence generation stage, considering the parameter rate of change constraints in the comprehensive constraint field, a single intent is expanded into an operation primitive execution sequence. For continuous parameter adjustments with rate constraints, the intermediate step sequence is generated using the following interpolation method: Let the current parameter value be... The target parameter value is The upper limit of the rate of change of the parameter per unit time specified by the constraint is The time interval between two adjacent steps is The number of interpolation steps required for:
[0029] ;
[0030] No. intermediate parameter values of the step for:
[0031] ;
[0032] in This indicates the rounding up operation. The current parameter values of the device. The target parameter value required by the intention, To transfer parameters from Gradient to Number of interpolation steps required This is the upper limit of the parameter change rate specified by the constraint conditions. The time interval (in seconds) between two adjacent operation primitives.
[0033] The protocol adaptation layer contains proprietary protocol driver plugins for each brand, which exist as dynamically loaded libraries. For each primitive in the primitive sequence, the corresponding device's protocol adaptation function is called to serialize the parameter values into a proprietary binary frame. This frame is then sent to the target device at the corresponding timestamp through the corresponding network channel (supporting Transmission Control Protocol, User Datagram Protocol, RS485 bus, etc., depending on the device interface type). After transmission, the layer waits for a response confirmation from the device. If no confirmation is received within the timeout threshold (default 500 milliseconds), a retransmission mechanism is triggered. If the preset retransmission limit (default three times) is exceeded, the device is marked as having a communication error and the edge alarm module is notified.
[0034] When a new brand display device needs to be connected to the system, only the corresponding device capability description file needs to be added to the device abstraction layer registry, and the corresponding brand's driver plugin needs to be deployed in the protocol adaptation layer. There is no need to modify the cloud control logic or intent parsing engine, which reduces the cost of connecting heterogeneous devices to a fixed registration operation and eliminates the bottleneck of having to redevelop the adaptation module for each new brand connected in the traditional solution.
[0035] III. Core Module 2: Global Dynamic Energy Efficiency Cooperative Optimization Based on Multi-Agent Reinforcement Learning
[0036] This invention models distributed display networks as cooperative multi-agent Markov games, with the system having a total of There are 1 display devices, each corresponding to an independent intelligent agent, and the set of intelligent agents is denoted as . All agents share a global reward function, and the learning objective is to maximize the global cumulative discount reward.
[0037] ;
[0038] in For the joint strategy of all agents, For intelligent agents The strategy function, This is a discount factor (default value 0.99), used to weigh the importance of current rewards against future rewards. The total length of the time steps in a single control decision cycle. for The global system state at any given moment. for The coordinated actions of all intelligent agents at any given moment.
[0039] Each agent The local observation vector is defined as:
[0040] ;
[0041] in for Time device The average image level of the currently playing content, ranging from zero to one, is calculated in real time by the edge-side field-programmable gate array hardware. for Time device The normalized value of the local illuminance sensor reading ranges from zero to one. for Time device The estimated density of the anonymized audience at the front, ranging from zero to one, is calculated using millimeter-wave radar. for Time device The normalized value of the real-time power consumption reading. for Time device The current brightness setting ranges from zero to one. for The grid load factor, which is shared at all times, ranges from zero to one. The closer it is to one, the closer the grid is to full load. It is a globally shared observation component.
[0042] Each agent's action space is a set of discrete actions, corresponding to the direction and magnitude of brightness adjustment:
[0043] ;
[0044] in This is the single-step large brightness adjustment range, with a default value of 0.15 (i.e., 15% of the brightness). This is a small, single-step brightness adjustment range, with a default value of 0.05 (i.e., 5% of the brightness). An action value of 0 indicates that the current brightness remains unchanged.
[0045] The globally shared reward function is designed as a multi-objective weighted form:
[0046] ;
[0047] Each sub-item is defined as follows:
[0048] ;
[0049] ;
[0050] ;
[0051] in , , These are the weighting coefficients for energy saving rate rewards, visual quality violation penalties, and grid load penalties, respectively, satisfying... It can be configured and adjusted by the operators. for Energy saving bonus relative to reference baseline power consumption at any given time. For equipment Reference power consumption at rated brightness setting As a penalty for breach of visual experience quality, when the estimated visual comfort value of the device is lower than the preset acceptable threshold lower limit. Punishment will be imposed at that time. The visual comfort estimation function is calculated based on the current brightness, average image level of the content, and ambient illuminance, and is provided by a pre-trained lightweight evaluation model. This is an additional power consumption penalty term under high grid load scenarios, related to the grid load factor. Proportional.
[0052] Training employs a multi-agent proximal policy optimization algorithm, conducted under a centralized training and distributed execution paradigm. The training phase takes place in a digital twin simulation environment of a display network built upon historical operational data. Initialization... The network consists of a distributed execution network (each network takes the local observation vector of the corresponding device as input and outputs the action probability distribution) and a centralized evaluation network (takes the global state obtained by concatenating the local observation vectors of all agents as input and outputs a global state value estimate). The training hyperparameters are set as follows: proximal policy optimization truncation parameter. The default value is 0.2, and the discount factor is... The default value of 0.99 is used for the smoothing parameter of the generalized dominance estimation. Set the default value to 0.95 and apply the network learning rate. Pick Evaluation of network learning rate Take 1×10 -3 .
[0053] The core update mechanism of training is as follows: For each agent, the importance sampling ratio of the current policy to the old policy is calculated, a near-policy optimization truncation loss function is constructed to limit the magnitude of a single policy update and prevent policy collapse. At the same time, the generalized advantage estimate is calculated by centrally evaluating the value estimate of the network output to guide the execution of network parameter updates. When the average relative change rate of global reward for one hundred consecutive training rounds is lower than the convergence threshold (default 0.01), the training is considered to have converged and training is stopped.
[0054] After convergence, Each execution network is quantized and compressed (quantization accuracy loss is controlled within one percent), and then exported and deployed to the multi-agent reinforcement learning distributed execution strategy module of the corresponding edge node. In the distributed execution phase, the distributed strategy unit of each agent reads the local observation vector with a fixed control cycle of five seconds by default, infers and outputs the brightness adjustment action, and sends the brightness adjustment instruction to the corresponding display device through the device abstraction layer of the semantic middleware. At the same time, each edge node gathers the execution trajectory data of this cycle to the cloud through the asynchronous reporting channel for the cloud to perform periodic online fine-tuning to maintain the model's adaptability to scene drift.
[0055] IV. Core Module 3: Power-Aware Cloud-Edge-Device Layered AI Inference Scheduling and Energy Constraint Feedback Loop
[0056] According to the real-time requirements of the control task, this invention allocates the artificial intelligence inference task to three layers: the edge layer neural network processor undertakes the real-time dimming task, runs a quantized and compressed lightweight execution network (model size less than 5 megabytes), and the inference latency does not exceed 50 milliseconds; the edge layer central processor implements device self-protection basic degradation control with a rule engine, and the response latency does not exceed 100 milliseconds; the edge layer field-programmable gate array module undertakes the frame synchronization hardware calculation of the average image level value, and the frame processing latency does not exceed 16 milliseconds (based on a time sequence of 60 frames per second); the edge layer undertakes the cross-device perception feature fusion and multi-agent reinforcement learning global coordination task, runs a quantized and compressed distributed policy processor (model size less than 50 megabytes), and the decision cycle does not exceed 5 seconds; the cloud layer undertakes the centralized training and policy update of multi-agent reinforcement learning (non-real-time, periodic execution) as well as fault prediction and health status assessment.
[0057] The hardware parallel extraction method for the average image level using a field-programmable gate array is as follows: For a resolution of... The average image level value of a single frame of an image is defined as:
[0058] ;
[0059] in The width of the image frame in pixels. The height of the image frame in pixels. , , Each pixel The grayscale values for the red, green, and blue channels range from zero to 255. As a normalization factor, the average image level value is normalized to the range of zero to one. The field-programmable gate array (FPGA) performs the accumulation calculation of the pixel grayscale value of the whole frame in parallel while the video signal is undergoing frame rate conversion processing through pipeline and parallel accumulation tree structure. The average image level value of the current frame is provided to the on-chip system master controller for reading in the form of register interface, without occupying the additional computing power of the on-chip system, thus realizing real-time average image level feature extraction with zero additional computing power overhead.
[0060] The implementation steps of the AI energy consumption constraint feedback loop are as follows:
[0061] First, regarding the acquisition of power grid load signals, the AI-powered energy consumption constraint monitoring module at the edge node obtains the current power grid load factor through the power grid dispatch interface at a fixed period of sixty seconds by default. (Normalized to the zero-to-one range), Meanwhile, based on historical power grid load time series data, the cloud uses a long short-term memory prediction model to predict the average power grid load within the default 15-minute time window in the future, and sends the prediction results to the edge layer for forward-looking adjustment.
[0062] Second, the current energy consumption of the AI perception module is estimated, and the total power consumption of the edge AI perception fusion module is estimated in real time at the edge layer, using a power consumption proxy model based on sampling rate and inference frequency:
[0063] ;
[0064] in for Estimated total power consumption (in watts) of the edge AI perception module. for Time sensor data sampling rate (unit: Hertz). Energy consumption per sensor sampling operation (unit: joules per operation). for Time-based strategy inference frequency (unit: times per second). Energy consumption per unit of strategic reasoning (unit: joules per time).
[0065] Third, the constraint-triggered decision will adjust the current grid load factor. With the two preset segmented trigger thresholds (Default 0.7) and (Default 0.9) Comparison, execute three-level constraint strategy: when Below When entering normal mode, the sensor operates at the preset upper limit of the sampling rate, the strategy unit operates at the preset upper limit of the inference frequency, and the artificial intelligence perception module operates at full precision. Not less than And lower than When the system enters constraint mode, the sensor sampling rate is reduced to the upper limit of the sampling rate multiplied by a preset first reduction factor (default 0.6), and the strategy inference frequency is reduced to the upper limit of the inference frequency multiplied by a preset second reduction factor (default 0.5). The actual number of inference iterations is reduced by reusing the inference results from the previous step. Not less than When the system enters emergency constraint mode, it reduces the sensor sampling rate to a preset lower limit and downgrades the multi-agent reinforcement learning strategy engine to a rule engine. It then directly looks up the table to execute the preset conservative energy-saving strategy, reducing the power consumption of artificial intelligence inference to an acceptable level.
[0066] Fourth, the adjustment effect is verified. After the constraint adjustment command is issued, the actual AI energy consumption is re-collected and verified after a default observation time window of 120 seconds. If the actual AI energy consumption still exceeds the preset upper limit (calculated with 30% of the system's energy saving benefit as the upper limit), the sampling rate is further reduced to the lower limit, and the abnormal log is reported to the cloud for strategy parameter review to ensure that the energy saving benefit is positive.
[0067] V. Anonymous Perception of Audience Density
[0068] The millimeter-wave radar module configured on the edge layer processes the Doppler frequency shift and phase difference information of the millimeter-wave signal to output only the number of people in the area in front of the display device and the audience density estimation information. It does not generate or store any facial images or personal identity information. The original signal data does not leave the edge layer, so as to avoid privacy data compliance risks and meet the data security requirements of commercial public places.
[0069] VI. Overall System Control Closed-Loop Process
[0070] The complete control closed-loop process of the system is as follows: The cloud-based large model inference server parses the high-level semantic control instructions from the operators, generates structured intent instruction packages, and sends them to the edge layer semantic receiving layer through a message queue. The edge layer intent parsing engine completes format verification, intent classification, and slot filling. The device abstraction layer queries the capability description files of each target device. The protocol adaptation layer serializes the operation primitives into brand-specific binary frames and sends them to each display device. The edge layer display devices perform brightness adjustment operations. The field-programmable gate array extracts the average image level value in real time. The environmental sensor collects illuminance and audience density data. The on-chip system reads power consumption data. The above information is reported to the edge layer. The edge layer multi-agent reinforcement learning distributed execution strategy module integrates the local observation vectors of each device and calculates the globally coordinated brightness adjustment action with a default five-second control cycle. The brightness adjustment instruction is sent through the semantic middleware. The artificial intelligence energy consumption constraint monitoring module synchronously reads the power grid load coefficient, determines whether the constraint mode is triggered, and adjusts the sampling rate and inference frequency accordingly. Each edge node reports the trajectory data of the current cycle to the cloud. The cloud performs multi-agent reinforcement learning fine-tuning and fault prediction updates. The new strategy parameters are sent to the edge layer, forming a complete closed loop.
[0071] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, any modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope defined by the claims of the present invention.
Claims
1. An intelligent control integrated system for commercial optoelectronic display devices, characterized in that, Including cloud layer, edge layer, and endpoint layer; The cloud layer includes a large model inference server, a multi-agent reinforcement learning centralized training module, and a fault prediction module; The edge layer includes a semantic middleware module, a multi-agent reinforcement learning distributed execution strategy module, an edge AI perception fusion module, and an AI energy consumption constraint monitoring module. The semantic middleware module sequentially includes a semantic receiving layer, an intent parsing engine, a device abstraction layer, and a protocol adaptation layer. The device abstraction layer maintains a device capability description file registry. Each device capability description file records the unique device identifier, brand identifier, display technology type, and supported set of general operation primitives for the corresponding display device. The protocol adaptation layer contains private protocol driver plugins for each brand. The semantic receiving layer receives structured intent instruction packets from the cloud layer and performs legality verification. The intent parsing engine classifies the verified intent instruction packets to obtain standard operation type codes. The device abstraction layer queries the device capabilities, and the protocol adaptation layer serializes the general operation primitives into brand-specific binary frames and sends them to the target display device through the corresponding network channel. The end-side layer includes multi-brand display devices and on-chip system controllers containing neural network processors, field-programmable gate array modules, and environmental sensors on each device; The multi-agent reinforcement learning distributed execution strategy module is deployed on edge nodes after completing strategy training based on the centralized training distributed execution paradigm. It reads the local observation vectors of each device at a fixed control cycle and infers and outputs brightness adjustment actions. The brightness adjustment instructions are sent to the corresponding display devices through the semantic middleware module. The AI energy consumption constraint monitoring module is used to obtain the current power grid load factor and dynamically adjust the sensor sampling rate and strategy inference frequency of the edge AI perception fusion module based on the comparison result between the power grid load factor and the preset segmented trigger threshold.
2. The system of claim 1, wherein, Each operation primitive in the general operation primitive set includes a primitive name, parameter type, parameter range, and a function pointer to the corresponding brand driver function of the protocol adaptation layer. When a new brand display device is connected, it is only necessary to add the corresponding device capability description file to the registry of the device abstraction layer and deploy the corresponding brand driver plugin in the protocol adaptation layer, without modifying the cloud layer control logic.
3. The system of claim 1, wherein, The edge AI perception fusion module is used to fuse the average image level value, ambient light value, anonymized audience density estimate, real-time power consumption reading, current brightness setting value, and globally shared power grid load factor of each display device. The multi-agent reinforcement learning distributed execution strategy module models each display device as a cooperative agent. The local observation vector of each agent is composed of the six components. The action space of each agent is a set of discrete brightness adjustment actions. The global shared reward function is composed of a weighted combination of energy saving rate reward, visual experience quality violation penalty, and grid high load additional power consumption penalty. The sum of the three weight coefficients is one.
4. The system of claim 1, wherein, The AI-powered energy consumption constraint monitoring module has a first trigger threshold and a second trigger threshold, where the second trigger threshold is higher than the first trigger threshold. It then executes a three-tiered constraint strategy based on the grid load factor. When the power grid load factor is lower than the first trigger threshold, the edge AI perception fusion module operates at a preset upper limit of sampling rate and a preset upper limit of inference frequency; When the grid load factor is not lower than the first trigger threshold and is lower than the second trigger threshold, the sensor sampling rate is reduced to the product of the preset sampling rate upper limit and the preset first reduction coefficient, and the strategy inference frequency is reduced to the product of the preset inference frequency upper limit and the preset second reduction coefficient. When the grid load factor is not lower than the second trigger threshold, the sensor sampling rate is reduced to a preset sampling rate lower limit, and the multi-agent reinforcement learning distributed execution strategy module is downgraded to a rule engine to execute a preset conservative energy-saving strategy.
5. The system of claim 1, wherein, The field-programmable gate array module uses a hardware pipeline and a parallel accumulator tree structure to calculate the average image level of the current frame in parallel while processing video frame rate conversion. The calculation result is then provided to the on-chip system main controller for reading via a register interface, achieving real-time average image level feature extraction with zero additional computing power overhead and a frame processing delay of no more than sixteen milliseconds.
6. The system of claim 1, wherein, The environmental sensor includes a millimeter-wave radar module. The millimeter-wave radar module processes the Doppler frequency shift and phase difference information of the millimeter-wave signal to output only the human body count information and audience density estimation information in the area in front of the display device. The original signal data does not leave the end-side layer and does not generate any face images / personal identity information.
7. A method for intelligent control of a commercial electro-optical display device, applied to the system of claim 1, characterized in that, include: Intent command reception and verification: The semantic receiving layer continuously listens to the message queue from the cloud layer, performs legality verification on the received intent command packets according to the predefined format pattern, and returns an error code to the cloud layer and discards the packet if the verification fails. If the verification is successful, the intent type field, target device identifier set, target parameter field, priority field and constraint condition field are extracted. Intent classification and slot filling: The extracted intent type field is input into the lightweight intent classifier of the intent parsing engine, and the standard operation type code and confidence value are output. If the confidence value is lower than the preset confidence threshold, a process of requesting secondary confirmation from the cloud layer is triggered. If the confidence value is not lower than the preset confidence threshold, the standard operation type code, target parameter dictionary and constraint field are encapsulated into a structured intent object. Device capability query and degradation processing: Traverse each device identifier in the target device identifier set, query the corresponding device capability description file in the device abstraction layer registry, check whether the general primitive corresponding to the standard operation type code is in the support list of the device, generate a degradation operation scheme for devices that do not support the primitive, and extract the corresponding protocol adaptation layer function pointer for devices that support the primitive. Operation primitive sequence generation and distribution: Based on the parameter gradient rate limit in the comprehensive constraint field, a single intent is expanded into an operation primitive execution sequence. For each primitive in the sequence, the corresponding device's protocol adaptation function is called to serialize the parameter value into a brand-specific binary frame and send it to the target device through the network channel. If no device response confirmation is received within the preset timeout threshold, a retransmission mechanism is triggered. After exceeding the preset retransmission limit, the device is marked as a communication anomaly and reported to the edge alarm module.
8. The method of claim 7, wherein, It also includes a global energy efficiency synergistic optimization step: During the centralized training phase in the cloud layer, in the digital twin simulation environment of the display network constructed based on historical operational data, a preset number of collection steps are run with the current joint strategy. The local observation vectors, actions, and global reward signals of each agent are collected and stored in the experience replay buffer. The time-series difference residuals and generalized advantage estimates for each step are calculated using the centralized evaluation network. Small batches of data are randomly sampled from the buffer. The truncated objective function is optimized using the proximal strategy to update the network parameters of each agent. The parameters of the centralized evaluation network are updated using the mean squared error loss. When the average relative change rate of the global reward for a preset number of consecutive training rounds is lower than the preset convergence threshold, convergence is determined and training is stopped. After convergence, each execution network is quantized and compressed, and then deployed to the corresponding edge nodes. The distributed policyr corresponding to each agent reads the local observation vector and infers the brightness adjustment action at a fixed control cycle, and sends it to the corresponding display device through the semantic middleware module. Each edge node reports the execution trajectory data of this cycle to the cloud layer for periodic online fine-tuning by the cloud layer.
9. The method of claim 7, wherein, It also includes the hierarchical reasoning task allocation step: The real-time dimming inference task is assigned to the neural network processor of the edge layer for execution, with an inference latency of no more than 50 milliseconds. The device self-protection basic degradation control task is assigned to the central processor of the edge layer for execution by the rule engine, with a response latency of no more than 100 milliseconds. The cross-device perception feature fusion and multi-agent reinforcement learning global coordination task is assigned to the edge layer for execution, with a decision cycle of no more than 5 seconds. The fault prediction and health status assessment task, as well as the multi-agent reinforcement learning centralized training and policy update task, are assigned to the cloud layer for execution.
10. The method of claim 8, wherein, In the global reward function, the energy saving rate reward is measured by the deviation of the ratio of the sum of the real-time power consumption of all devices in the network to the sum of the reference baseline power consumption. The visual experience quality violation penalty is generated when the estimated visual comfort value of each device is lower than the lower limit of the preset acceptable visual comfort threshold. The grid high load additional power consumption penalty is measured by the product of the current grid load factor and the sum of the real-time power consumption of all devices in the network. The weight coefficients of the three items can be configured by the operators and adjusted to meet the constraint that the sum of the three items is one.
