Landscape lighting collaborative control method and system based on multi-agent reinforcement learning

Abstract

This invention belongs to the field of artificial intelligence technology, specifically relating to a collaborative control method and system for landscape lighting based on multi-agent reinforcement learning. The proposed solution constructs a light environment quality assessment model that extracts objective light environment quality indicators and adaptively fuses them with preference alignment, providing explicit and quantifiable reward signals for multi-agent reinforcement learning. Secondly, by constructing a three-dimensional decoupled hierarchical credit allocation mechanism encompassing spatial contribution, chromaticity coordination, and temporal coordination, the global reward signal is precisely decomposed to each lighting zone, ensuring that the credit obtained by each zone matches its physical contribution. Compared to existing centralized deep learning solutions, this embodiment solves the technical problems of unquantifiable environmental quality assessment and the inability to accurately distinguish contributions from multiple zones, enabling multi-agent reinforcement learning to be effectively applied to the adaptive collaborative control of large-scale landscape lighting, improving the overall quality of lighting effects while reducing system energy consumption.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a collaborative control method and system for landscape lighting based on multi-agent reinforcement learning. Background Technology

[0002] Landscape lighting is an important component of urban nightscape construction, widely used in urban landmarks, riverside landscape belts, large venues, bridges, and gardens. With the advancement of smart city construction, landscape lighting systems are evolving from single-point independent control to large-scale coordinated control, placing higher demands on the intelligence and adaptability of the control system.

[0003] Currently, there have been attempts to introduce artificial intelligence technology into landscape lighting control within the industry. Chinese patent CN121772077B discloses a deep learning-based adaptive adjustment method and system for urban landscape lighting. This method employs a centralized single-model architecture, using sensor data as input and directly mapping the output lighting control parameters through an end-to-end neural network. However, this approach has the following drawbacks: First, the centralized architecture experiences state space expansion when dealing with dozens to hundreds of lighting zones, and is susceptible to single-point failures. Second, the quality judgment of lighting effects is implicitly encoded in the neural network, without outputting any explicit, quantifiable quality assessment signals. This makes it impossible to determine the actual effect of each control adjustment and to provide the necessary reward signals for reinforcement learning. Third, the single model does not involve multi-decision-making unit collaboration and lacks the ability to distinguish the differentiated contributions of each lighting zone.

[0004] In landscape lighting engineering practice, due to differences in spatial location, visual role and functional attributes, the contribution of each lighting zone to the overall lighting effect varies significantly. Furthermore, there is a spatial coupling relationship between zones in terms of hue, brightness and dynamic rhythm. Objectively, this requires each zone to maintain the flexibility of independent decision-making while also possessing the consistency of overall coordination. The aforementioned characteristics make it possible to apply the distributed decision-making paradigm of Multi-Agent Reinforcement Learning (MARL) in landscape lighting engineering. However, introducing existing MARL methods into landscape lighting scenarios presents two problems: First, existing MARL methods rely on predefined reward functions, while landscape lighting scenarios lack computational environmental quality assessment methods and cannot obtain real-time, differentiable reward signals, making training impossible. Second, existing MARL credit allocation methods, such as Value Decomposition Network (VDN) and Q-Mixing, only perform pure numerical contribution decomposition and do not encode physical dimension information unique to the lighting field, such as spatial coupling, color coordination, and dynamic rhythm. In large-scale collaboration of dozens to hundreds of partitions, serious contribution confusion and training degradation problems easily occur. Summary of the Invention

[0005] This invention addresses the problems existing in the prior art by providing a collaborative control method and system for landscape lighting based on multi-agent reinforcement learning, effectively solving the problems of the prior art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A collaborative control method for landscape lighting based on multi-agent reinforcement learning includes: Based on real-time acquired landscape lighting scene images, a comprehensive quality score of the current lighting effect is calculated through a pre-constructed light environment quality assessment model, wherein the change in the comprehensive quality score is used as the global reward signal for multi-agent reinforcement learning. The landscape lighting area is divided into several lighting zones, each corresponding to a reinforcement learning agent. Each agent outputs lighting control actions with continuous values ​​through a policy network based on its local observations. A three-dimensional decoupled hierarchical credit allocation mechanism is adopted to decompose the global reward signal into individual credit values ​​of each agent; Based on the assigned individual credit value, each agent aims to maximize the long-term cumulative comprehensive quality reward and uses a proximate policy optimization algorithm to update its own policy network. Based on the updated policy network, each agent continuously outputs lighting control actions according to real-time local observations. Through the coordination of the control actions of each agent, the landscape lighting system is jointly driven to achieve real-time adaptive collaborative control.

[0007] Furthermore, the light environment quality assessment model specifically performs the following processing steps: Based on the landscape lighting scene image, objective light environment quality indicators are extracted; wherein, the objective light environment quality indicators include illuminance uniformity, glare index, color temperature consistency, brightness contrast, color rendering index, and visual focus matching degree. Based on the extracted objective light environment quality indicators, an adaptive weighted fusion is performed through a learnable quality fusion network to obtain a comprehensive quality score. Specifically, the learnable quality fusion network performs the following processing steps: extracting scene context-aware features from the objective light environment quality indicators; generating fusion weights for each indicator based on the extracted scene context-aware features; mapping each indicator to a sub-score; and weighting and summing the sub-scores based on the fusion weights to obtain the comprehensive quality score.

[0008] Furthermore, the three-dimensional decoupled hierarchical credit allocation mechanism specifically performs the following processing steps: Based on the spatial location embedding of each lighting zone, the spatial contribution credit reflecting the visual focus weight of each zone is calculated. Based on the deviation of the chromaticity vector of each lighting zone from the global dominant hue, the chromaticity harmony credit, which reflects the degree of hue harmony, is calculated. Based on the similarity between the temporal embedding of the action sequences of each lighting zone and the global prosodic embedding, temporal collaborative credit reflecting the degree of synchronization of dynamic changes is calculated. The spatial contribution credit, chromaticity coordination credit, and temporal coordination credit are fused into a comprehensive credit weight for each agent through a learnable fusion layer, and then global normalization is performed.

[0009] Furthermore, the calculation of the spatial contribution credit is specifically as follows: Construct location embedding vectors for each lighting zone; wherein, the location embedding vectors are composed of the zone's three-dimensional geometric coordinates, architectural semantic labels, and viewpoint visibility scores; Spatial attention scores between partitions are calculated using a multi-head attention network; wherein, the spatial attention score represents the visual attention intensity of one partition to another. The spatial contribution credit of a given partition is obtained by normalizing the sum of the spatial attention scores of all partitions that pay attention to it.

[0010] Furthermore, the calculation of the chromaticity coordination credit is specifically as follows: The chromaticity vectors of each lighting zone are mapped to a perceptually uniform color space to obtain a set of chromaticity vectors; The mapped chroma vector set is clustered, and the cluster center with the largest sample size is extracted as the global dominant chroma center; Calculate the color difference between the chromaticity vector of each partition and the global dominant hue center; The color difference values ​​are mapped to colorimetric consistency credits using a Gaussian decay function.

[0011] Furthermore, the calculation of the temporal collaborative credit is specifically as follows: A short-time Fourier transform is performed on the global brightness time-series signal to extract the main frequency component and phase information, which are then used to construct a global prosodic embedding vector. Based on the action sequence of each lighting zone, a temporal embedding vector is obtained through gated cyclic unit encoding; The dot product similarity between the temporal embedding vector and the global prosodic embedding vector is calculated, and temporal collaborative credit is obtained by mapping with the sigmoid function.

[0012] Furthermore, the proximal policy optimization algorithm employs a multi-agent proximal policy optimization algorithm, which directly incorporates the individual credit values ​​assigned to each agent into the advantage estimation during the policy network update process. in, Indicates the first i Credit-weighted advantage estimate of an agent Indicates assignment to the first i The individual credit value of an intelligent agent. Indicates time step t Global rewards As a discount factor, and These represent the value estimates of the current state and the next state by the centralized commentator, respectively.

[0013] A landscape lighting collaborative control system based on multi-agent reinforcement learning includes: The light environment quality assessment module is configured to: calculate the comprehensive quality score of the current lighting effect based on real-time acquired landscape lighting scene images through a pre-built light environment quality assessment model, wherein the change in the comprehensive quality score is used as the global reward signal for multi-agent reinforcement learning; The multi-agent policy module is configured as follows: the landscape lighting area is divided into several lighting zones, each lighting zone corresponds to a reinforcement learning agent, wherein each agent outputs a lighting control action with continuous values ​​through the policy network based on its local observations. The three-dimensional credit allocation module is configured to use a three-dimensional decoupled hierarchical credit allocation mechanism to decompose the global reward signal into individual credit values ​​for each agent. The policy update module is configured as follows: based on the assigned individual credit value, each agent updates its own policy network by using a proximal policy optimization algorithm with the goal of maximizing the long-term cumulative comprehensive quality reward. The collaborative control module is configured such that each agent continuously outputs lighting control actions based on the updated policy network and real-time local observations. Through the coordination of the control actions of each agent, the landscape lighting system is jointly driven to achieve real-time adaptive collaborative control.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are as follows: (1) The present invention provides a landscape lighting collaborative control method and system based on multi-agent reinforcement learning. The scheme constructs a two-level cascaded light environment quality assessment model that extracts objective light environment quality indicators and adaptively fuses preferences, providing explicit and quantifiable reward signals for multi-agent reinforcement learning. Secondly, by constructing a three-dimensional decoupled hierarchical credit allocation mechanism that includes three physical dimensions: spatial contribution, color coordination, and temporal coordination, the global reward signal is accurately decomposed to each lighting zone, so that the credit obtained by each zone matches its physical contribution. Compared with the existing centralized deep learning scheme, the scheme in this embodiment solves the technical problems of unquantifiable environmental quality assessment and inaccurate differentiation of contributions from multiple zones, enabling multi-agent reinforcement learning to be effectively applied to the adaptive collaborative control of large-scale landscape lighting, improving the overall quality of lighting effects while reducing system operating energy consumption.

[0015] (2) The light environment quality assessment model in the scheme of the present invention adopts a two-level cascaded architecture. First, it extracts multiple quantifiable physical indicators based on national standards or International Commission on Illumination standards from real-world images. Then, it performs adaptive weighted fusion based on scene context through a learnable fusion network to obtain a comprehensive quality score. On the one hand, each indicator has a clear physical definition and a recognized calculation formula, which can be independently verified by standard measurement equipment, ensuring the objectivity and traceability of the assessment results. On the other hand, the model is trained on paired preference-labeled data through a learnable fusion network, enabling the model to learn reasonable trade-off strategies for each indicator under different scene types. Through the above two-level architecture, the functional positioning of AI is transformed from "replacing human subjective aesthetic judgment" to "learning multi-objective trade-off strategies based on objective physical quantities", making the assessment results interpretable. That is, every change in the comprehensive score can be traced back to the numerical change of specific indicators.

[0016] (3) The solution described in this invention constructs a three-dimensional decoupled hierarchical credit allocation mechanism, which encodes the spatial coupling relationship, color coordination rules, and dynamic prosodic features in the lighting field into the credit allocation calculation graph of multi-agent reinforcement learning (MARL). This solves the shortcomings of existing methods such as VDN and QMIX, which only perform pure numerical decomposition and cannot capture multi-dimensional semantic coordination. In the spatial dimension, through position embedding and multi-head attention mechanisms, the visual focus area receives a higher contribution weight; in the chromaticity dimension, based on global dominant color clustering and standard color difference calculation, the color-coordinated partitions receive positive credit incentives; in the temporal dimension, through global prosodic extraction and similarity matching of partition temporal encoding, the dynamically synchronized partitions receive collaborative credit.

[0017] (4) The scheme described in this invention adopts a centralized training and distributed execution paradigm of multi-agent reinforcement learning. Each agent can independently output control actions by relying only on local observations during the execution phase, without the need for global information interaction, which fundamentally eliminates the risk of single point of failure in centralized architecture. At the same time, when communication is interrupted and global collaborative information is unavailable, each agent can automatically switch to an independent strategy mode based on local observations to maintain reasonable operation, and seamlessly switch back to the global collaborative mode after communication is restored. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below: Figure 1 This is a flowchart of the landscape lighting collaborative control method based on multi-agent reinforcement learning described in this embodiment of the invention. Figure 2 This is a flowchart illustrating the three-dimensional decoupled hierarchical credit allocation mechanism described in this embodiment of the invention. Figure 3 This is a flowchart illustrating the calculation process of spatial contribution credit as described in this embodiment of the invention. Figure 4 This is a flowchart illustrating the calculation of colorimetric coordination credit as described in this embodiment of the invention. Figure 5 This is a flowchart illustrating the calculation of time-series collaborative credit as described in an embodiment of the present invention; Figure 6 This is a schematic diagram of the landscape lighting collaborative control system based on multi-agent reinforcement learning as described in an embodiment of the present invention. Detailed Implementation

[0019] To better understand the above-mentioned objectives, features and advantages of the present invention, the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0020] Numerous specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways than those described herein, and therefore the invention is not limited to the specific embodiments disclosed in the following specification.

[0021] Example 1 The intelligent diagnostic method for cable busbars based on multimodal data fusion described in Embodiment 1 will be explained in detail below with reference to the accompanying drawings.

[0022] In one or more embodiments, such as Figure 1 As shown, this embodiment provides a landscape lighting cooperative control method based on multi-agent reinforcement learning, including: Step S100: Based on the real-time acquired landscape lighting scene images, calculate the comprehensive quality score of the current lighting effect through a pre-built light environment quality assessment model, wherein the change in the comprehensive quality score is used as the global reward signal for multi-agent reinforcement learning; Step S200: Divide the landscape lighting area into several lighting zones, each lighting zone corresponding to a reinforcement learning agent. Each agent outputs a lighting control action with continuous values ​​through a policy network based on its local observations. Step S300: A three-dimensional decoupled hierarchical credit allocation mechanism is adopted to decompose the global reward signal into individual credit values ​​of each agent; Step S400: Based on the assigned individual credit value, each agent updates its policy network using a proximal policy optimization algorithm with the goal of maximizing the long-term cumulative comprehensive quality reward. Step S500: Based on the updated policy network, each agent continuously outputs lighting control actions according to real-time local observations. Through the coordination of the control actions of each agent, the landscape lighting system is jointly driven to achieve real-time adaptive collaborative control.

[0023] In one or more embodiments, the landscape lighting collaborative control method based on multi-agent reinforcement learning described in this embodiment is specifically divided into a training phase and an online control phase. In the training phase, a light environment quality assessment model is pre-built and the multi-agent policy network is centrally trained. In the online control phase, the trained model and policy network are used to drive each lighting zone to achieve adaptive collaborative control based on real-time collected landscape lighting scene images.

[0024] In specific implementation, step S100 involves calculating the comprehensive quality score of the current lighting effect based on real-time acquired landscape lighting scene images using a pre-built light environment quality assessment model, specifically executing the following processing steps: Step S101: Construction of the light environment quality assessment model The light environment quality assessment model adopts a two-level cascaded structure, namely, two processing stages: objective light environment quality index extraction and learnable quality fusion. Level 1: Extraction of Objective Light Environment Quality Indicators Based on real-time acquired landscape lighting scene images I The following objective light environment quality indicators are extracted through six parallel indicator extraction branches: (1) Illuminance uniformity According to GB 50034-2013 "Standard for Lighting Design of Buildings" and CIE 140:2019, illuminance uniformity is defined as the ratio of the minimum illuminance to the average illuminance in a scene, and its expression is: in, The minimum illuminance in a landscape lighting scene image. The average illuminance in the scene. The closer the value is to 1, the more uniform the illuminance distribution.

[0025] In practice, a convolutional regression network based on the ResNet-18 backbone network (pre-trained on ImageNet-1K with the last fully connected classification layer removed) estimates the pixel-level relative illumination distribution from the input image, then calculates the minimum and average illumination, and outputs the illumination uniformity. .

[0026] It should be noted that during the training of the aforementioned branch network, an imaging luminance meter was used to perform pixel-level illuminance calibration on some scenes in the data preparation stage to establish a mapping relationship between image RGB values ​​and absolute illuminance values. The mean squared error loss function was used during training.

[0027] (2) Glare Index According to CIE 117:1995 "Uncomfortable Glare" and GB / Z 26212-2010, the Unified Glare Rating (UGR) is defined as follows: in, Background brightness, For the first i The brightness of each light source For the first i The solid angle of the light source relative to the observation point For the first i Guth position index for each light source. This represents the total number of light sources.

[0028] In the specific implementation, the same ResNet-18 backbone network as the illumination uniformity branch is adopted. The regression head outputs the UGR estimate, which is constrained to the range of [10,30]. During training, the true UGR is calculated as a supervision signal using the brightness, solid angle and position index information of the light source area labeled by the imaging luminance meter.

[0029] (3) Color temperature consistency According to ANSI C78.377 "Colorimetric Specification for Solid State Lighting Products", color temperature consistency is defined as the standard deviation of the correlated color temperature of each lighting zone, and its expression is: in, Indicates the first i Correlated color temperature of each lighting zone The arithmetic mean of the correlated color temperatures for all lighting zones. The total number of lighting zones, The smaller the value, the more consistent the overall color tone.

[0030] In practice, the average chromaticity coordinates of each partition are extracted from the input image. These coordinates are then converted into correlated color temperature (CCT) values ​​using the McCamy approximation formula. The CCT standard deviation of all partitions is then calculated as the color temperature consistency. .

[0031] (4) Brightness contrast According to CIE 95:1992 "Contrast Ratio and Visibility", the luminance contrast ratio is defined as: in, The average brightness of the focal area. The average brightness of the background area. A larger value indicates a more prominent visual focus.

[0032] In practical implementation, the brightness contrast extraction network includes a scene semantic segmentation subnetwork and a brightness statistics subnetwork. The semantic segmentation subnetwork uses the DeepLabV3+ architecture (with a ResNet-50 backbone network) to segment the input image into focal regions (e.g., building facades, water feature axes) and background regions (e.g., sky, surrounding buildings, greenery). The brightness statistics subnetwork calculates the average pixel brightness values ​​for each of the two regions and outputs the brightness contrast. .

[0033] (5) Color rendering index According to CIE 13.3:1995, "Methods for measuring and specifying the color rendering properties of light sources," the color rendering index is defined as: in, This represents the average color difference value of eight standard color samples under illumination from the light source to be tested and the reference light source. A higher value indicates better color rendering performance of the light source.

[0034] In practice, the color rendering index is determined based on the chromatographic shift analysis of the illuminated surface material in the image, combined with prior knowledge of the spectral power distribution of the light source. The estimate.

[0035] (6) Visual focus matching degree Visual focus matching degree is defined as follows: in, To create a mask for the actual high-brightness regions extracted from the input image by segmenting using a brightness threshold, the brightness threshold is set to 85% of the image's maximum brightness. A mask for the design intent focal area, obtained from a building information model or lighting design drawings and projected onto an image coordinate system. This represents the area of ​​the region (i.e., the pixel count). and These represent the intersection and union operations of sets, respectively. The closer the value is to 1, the better the actual center of gravity of the lighting matches the design intent.

[0036] In practice, after the above six branches are processed in parallel, a 6-dimensional objective environmental quality index vector is output: Level 2: Learnable Quality Fusion The 6-dimensional index vector output from the first level As the input for the second stage: it is first mapped to a 64-dimensional embedding space through linear projection, and learnable positional encoding is added. Then, it passes through two layers of standard Transformer encoders (each layer contains an 8-head self-attention network and a 2048-dimensional feedforward network, with GELU activation function) to extract scene context-aware feature representations. .

[0037] At the output of the Transformer encoder, two sets of task heads are connected in parallel: (a) Context-aware weight generator It consists of two fully connected layers (64→32→6), and the output is normalized by Softmax to obtain the context-aware weights of each indicator. Different scene types (such as building facade scenes vs. garden scenes) correspond to different indicator weight distributions, which are automatically learned by the model based on the input images.

[0038] (b) Indicator scoring mapping module It consists of six independent 64→32→1 fully connected subnetworks, each of which is responsible for mapping an index to a sub-score in the [0,1] interval. .

[0039] The final comprehensive light environment quality score is obtained by weighted summation: in, For the first i Context-aware weights for objective environmental quality indicators, satisfying , For the first i The original values ​​of the objective environmental quality indicators, For the first i The non-linear scoring mapping function corresponding to each indicator. A higher score indicates a better overall light environment quality.

[0040] Step S102: Training the light environment quality assessment model It should be noted that the training objective of the light environment quality assessment model in this embodiment is not to regress a true value of an absolute quality score (objectively, such a true value does not exist), but to make the ranking of the model's output scores consistent with the ranking of professional lighting designers' preferences through preference alignment learning.

[0041] In the scheme described in this embodiment, the training data for the model is constructed in the following way: High-definition real-scene images were collected during the lighting commissioning phase of past large-scale landscape lighting projects. These images were then paired and annotated with preferences by professional designers with over five years of experience in landscape lighting design. For example: Given two real-world images A and B of the same scene with different lighting configurations, label them as "The overall lighting quality of scheme A is better than that of scheme B", "Scheme B is better than scheme A", or "No significant difference".

[0042] 5,000 to 10,000 pairs were selected as the training set, covering multiple landscape lighting scenarios. At the same time, parameters such as color temperature, brightness distribution, and contrast were parametrically perturbed in the digital twin environment and then re-rendered to generate enhanced training sample pairs.

[0043] The training loss function uses a Bradley-Terry preference probability model combined with RankNet loss: in, Let A be the probability that the overall quality score of option A is better than that of option B, as predicted by the model. , These are the comprehensive light environment quality scores output by the light environment quality assessment model for images A and B, respectively. The labeled preference probability is given when option A is of higher quality than option B. When option B is better than option A When there is no significant difference .

[0044] Step S103: Real-time comprehensive quality score calculation and reward signal generation In multi-agent reinforcement learning and online control processes, for time steps t Real-world images collected and the previous time step The images are input into a pre-trained light environment quality assessment model to obtain a comprehensive quality score. ( )and The global reward signal is defined as the increment of the overall quality score: in, For time step The global reward signal, and These represent the combined light environment quality scores for the current time step and the previous time step, respectively.

[0045] The reason why the scheme described in this embodiment uses an incremental form as a reward is that: incremental rewards encourage agents to take actions to improve lighting quality and obtain positive rewards, while suppressing actions to reduce lighting quality and obtaining negative rewards; the incremental form is not sensitive to the systematic bias of the absolute score of the light environment quality assessment model; at the same time, it can avoid the strategy "saturation" effect after the absolute score reaches a high level.

[0046] In one or more embodiments, step S200 involves dividing the landscape lighting area into several lighting zones, each corresponding to a reinforcement learning agent; specifically, it includes the following processing steps: (1) Lighting zoning The landscape lighting area is divided according to spatial function and visual structure. N Lighting zones (in typical scenarios) N(Select 50 to 300), the division of zones is based on, but is not limited to: the building structure boundaries (main facade, podium, tower crown, etc.) in the BIM model, the geometric boundaries of roads and squares, the scope of water bodies and bridges, the regional distribution of garden landscapes, etc., and each lighting zone is configured with a reinforcement learning agent.

[0047] (2) Agent design Each agent ( i (e.g., 1, 2, ..., N) maintains its own state, action space, and local observations, where: The status (i.e., local observation) includes the current lighting parameters of the partition (brightness percentage, correlated color temperature, RGBW channel values), electrical parameters (real-time current, power factor, drive power supply temperature), and the partition's position code in the scene's three-dimensional space.

[0048] The motion space is defined as a continuous motion vector, including dimming increment, color temperature shift, red component increment, green component increment, and blue component increment.

[0049] Local observations include the complete state of the partition itself and the state of adjacent partitions in space.

[0050] Each agent Maintain an independent policy network The lighting control action is based on the continuous value of the local observation output, and the specific structure and training process of the policy network are explained in detail in step S400.

[0051] In one or more embodiments, such as Figure 2 As shown, step S300 involves using a three-dimensional decoupled hierarchical credit allocation mechanism to decompose the global reward signal into individual credit values ​​for each agent; specifically, it includes the following processing steps: The core of this step lies in the global reward signal. Decomposed into individual credit values ​​matching the contributions of each agent. The abstract concept of "lighting synergy" is engineered into three independently computable and jointly optimizable physical dimensions through the aforementioned three-dimensional decoupled hierarchical credit allocation mechanism. Specifically, this includes the following steps: Step S301: Spatial contribution credit calculation (e.g.) Figure 3 (As shown) Space contributes credit Reflecting the partition i Visual weight in space—the zoning area in the visual focal zone naturally contributes more to the overall lighting effect than the edge area.

[0052] First, construct the position embedding vectors of each agent. It is obtained by splicing the following three parts: three-dimensional geometric coordinates (world coordinates of the center of the partition bounding box, normalized to [-1,1]), architectural semantic tags (representing space types in 8-dimensional one-hot encoding: main facade, podium, tower crown, plaza, water body, etc.), and viewpoint visibility score (visibility ratio calculated from K preset main viewing points through line of sight analysis, 1-dimensional scalar). Then, establish multi-head attention computation: embed the positions of all agents into a set. Learnable Q (query), K (key), and V (value) projection matrices respectively , , (Head count h=4) is mapped to the query, key, and value space.

[0053] The spatial attention score matrix is ​​represented as follows: in, Represents intelligent agents i The query vector, Represents intelligent agents j The key vector, =16 indicates the dimension of the query / key vector. Indicates partition j For partitions i Visual attention intensity.

[0054] The spatial contribution credit of each partition is the normalized value of the sum of the attention weights of all partitions to it: Among them, molecules Indicates all partitions to partitions i The sum of attention levels, denominator This represents the maximum sum of attention levels across all partitions. It is used to normalize the credit value to the [0,1] range.

[0055] Optionally, when a saliency map of the current real-world image is available (extracted via the Itti-Koch saliency detection algorithm or a lightweight saliency detection network), partitions are extracted. i Significance mean of the corresponding region Accuracy is further enhanced through weighted fusion: in, Indicates the first i The significance mean of each partition in the corresponding region of the significance map. These are the weighting coefficients.

[0056] Step S302: Colorimetric Coordination Credit Calculation (e.g.) Figure 4 (As shown) Color Coordination Credit Reflecting the partition i The degree of harmony between the color tone and the overall main color tone.

[0057] First, collect the chromaticity vectors of all agents. For RGBW lighting fixtures, map each channel value to the perceptually uniform CIELAB color space using a standard conversion process, resulting in: Then, for the set Clustering using Gaussian Mixture Model (GMM) (number of clusters) =3), select the cluster with the largest sample size as the global primary color.

[0058] Calculate the CIEDE2000 color difference between the chromaticity vector of each partition and the global primary hue center: in, This is the color difference calculation function for CIEDE2000. As the center of the overall main color scheme, The smaller the value, the more partitioned the partition. i The closer the color tone is to the overall main color tone, the better.

[0059] Chromaticity coordination credits are obtained by mapping using a Gaussian decay function: in, The initial value is set to 1 for the learnable temperature parameter. This represents the mean of the chromaticity variance across all partitions. ,when When the zone color scheme perfectly matches the main color scheme, the maximum value of 1 is used.

[0060] For applications with more complex color relationships, Gaussian attenuation can be replaced with a learnable color coordination matrix scheme: in, For learnable color coordination matrix, For learnable bias terms, This is the sigmoid function.

[0061] Step S303: Time-series collaborative credit calculation (e.g.) Figure 5 (As shown) Time-series collaborative credit Reflecting the partition i The degree of synchronization between the rhythm of the lighting changes and the overall dynamic rhythm.

[0062] First, from the global brightness timing signal Extract the rhythmic reference from the average brightness of all partitions at each time step. Perform a 16-point Short-Time Fourier Transform (STFT) on the brightness sequence of the past K=16 time steps to extract the dominant frequency. Main phase and the proportion of main frequency energy This constitutes a 3D prosody reference vector. .

[0063] Secondly, for each partition i Past K-step action sequence Encoding is performed using a gated recurrent unit (GRU), and the last hidden state is taken as the temporal embedding. .

[0064] Will Global prosodic embeddings are encoded using a two-layer fully connected network FFN (32→64→32, GELU activation). .

[0065] Finally, temporal collaborative credit is calculated using dot product similarity: in, For partition i, the action sequence time embedding vector. For global prosody embedding vectors, For temperature parameters, This is the sigmoid function.

[0066] Step S304: 3D credit fusion and normalization processing The credit scores from the three dimensions are concatenated into a 3-dimensional vector, linearly fused through a learnable fusion layer, and then globally normalized after Sigmoid activation. in, To be a learnable fusion weight matrix, For learnable bias terms, It is a concatenated vector of credit values ​​across three dimensions. This refers to the initial credit weights after merging, which are not yet normalized. This represents the final individual credit score after global normalization, satisfying... .

[0067] It should be noted here that all learnable parameters in step S300 (including learnable Q, K, and V projection matrices) , , Learnable temperature parameters You could learn about color coordination matrices. The GRU and FFN parameters of the timing branch, and the fusion layer and As a centralized network of commentators The components of the policy network update process in step S400 are jointly optimized end-to-end by minimizing the global temporal difference (TD) error.

[0068] In one or more embodiments, step S400 involves each agent updating its policy network based on the assigned individual credit value, aiming to maximize the long-term cumulative comprehensive quality reward; specifically, this includes the following processing steps: Step S401: Training parameter configuration As shown in Table 1, the training parameters are configured as follows: Table 1 Training parameters Step S402: Training Process Initialization: Load the pre-trained and qualified light environment quality assessment model, and initialize the policy network of N agents according to the lighting zoning scheme. ( ) and centralized commentator networks ; Initialize all learnable parameters in step S300 Initialize the experience replay buffer B to empty; For each training round (M rounds in total): (a) Reset the environment to its initial scene state (Typically, this is a baseline lighting scheme for design), rendering generates an initial real-world image. The initial comprehensive quality score is calculated by calling the light environment quality assessment model. .

[0069] (b) For each time step : Various intelligent agents Parallel execution: acquiring local observations Policy Network Output action mean and standard deviation From Gaussian distribution During the action sampling; Combined actions Update the scene lighting parameters and render a new real-world image. ; Calculate using the light environment quality assessment model Global rewards ; Execute step S300 of the three-dimensional credit allocation process and input the global state. and joint actions Output the individual credit value of each agent. ; Storage Track To buffer B, where, For local observation of each intelligent agent A set of.

[0070] (c) After each loop round, sample mini-batch from buffer B to perform policy updates.

[0071] First, calculate the credit-weighted advantage estimate: in, Represents intelligent agents The credit-weighted advantage estimate, This indicates the allocation of the first step in step S300. i A smart agent Individual credit score, Indicates time step t Global rewards As a discount factor, and These represent the value estimates of the current state and the next state by the centralized commentator, respectively.

[0072] Then, each intelligent agent Minimize the MAPPO (Multi-Agent Proximal Policy Optimization) pruning loss for policy network updates: in, This represents the probability ratio between the old and new strategies, i.e., the current strategy. Compared to the old strategy before the update The probability ratio for the same observation-action pair For trimming parameters, The function is used to constrain the probability ratio within the interval [0.8, 1.2]. This represents the function that takes the minimum value. This represents the overall expectation operator, which is the average of the expression within the parentheses over all possible samples.

[0073] The commentator network updates by minimizing the mean squared error loss: in, For the commentator network on the state Value prediction This represents the time difference (TD) target value.

[0074] All learnable parameters in step S300 Joint update: All learnable parameters pass Gradient backpropagation is performed synchronously, and the gradient flow path is: TD error → centralized commentator. →Credit Fusion Layer→Credit branch parameters in each dimension. Through joint optimization, a credit allocation strategy that minimizes the prediction error of the global value function is automatically learned.

[0075] (d) Clear the buffer and execute the next loop round.

[0076] Step S403: Convergence Judgment Calculate the average overall quality score for the most recent 10 rounds every 50 rounds. and standard deviation .when And two consecutive judgments When the increase in all values ​​is less than 0.002, the training is considered to have converged and training is terminated.

[0077] In one or more embodiments, step S500 involves each agent continuously outputting lighting control actions based on the updated policy network and real-time local observations. Through the coordination of these control actions, the landscape lighting system is jointly driven to achieve real-time adaptive collaborative control. Specifically, this includes the following processing steps: After training convergence in step S400, the updated policy networks of each agent are loaded into the computing unit of the lighting control system. During the online control phase, each agent continuously outputs lighting control actions based on the updated policy network from step S400 and real-time local observations. These control actions are then distributed to the corresponding lighting fixtures in the lighting zones via the lighting control protocol, collectively driving the landscape lighting system to achieve real-time adaptive collaborative control.

[0078] It should be noted that the operation in the online control phase is the same as the operation of action generation and environment interaction in the time step loop of the training phase in step S400. The only difference is that the output of the policy network directly takes the mean of the action instead of sampling from the distribution, and trajectory storage and gradient update are no longer performed.

[0079] In one or more embodiments, when external information required for global collaborative control (such as global reward signals, credit allocation weights, or state information of other agents) becomes unavailable due to communication failure, each agent automatically switches to an independent policy mode based on local observation to operate autonomously; when communication of the external information is restored, each agent automatically switches back to the global collaborative mode.

[0080] Example 2 In one or more embodiments, corresponding to the method described in Embodiment 1, such as Figure 6 As shown, this embodiment provides a landscape lighting collaborative control system based on multi-agent reinforcement learning, including: The light environment quality assessment module is configured to: calculate the comprehensive quality score of the current lighting effect based on real-time acquired landscape lighting scene images through a pre-built light environment quality assessment model, wherein the change in the comprehensive quality score is used as the global reward signal for multi-agent reinforcement learning; The multi-agent policy module is configured as follows: the landscape lighting area is divided into several lighting zones, each lighting zone corresponds to a reinforcement learning agent, wherein each agent outputs a lighting control action with continuous values ​​through the policy network based on its local observations. The three-dimensional credit allocation module is configured to use a three-dimensional decoupled hierarchical credit allocation mechanism to decompose the global reward signal into individual credit values ​​for each agent. The policy update module is configured as follows: based on the assigned individual credit value, each agent updates its own policy network by using a proximal policy optimization algorithm with the goal of maximizing the long-term cumulative comprehensive quality reward. The collaborative control module is configured such that each agent continuously outputs lighting control actions based on the updated policy network and real-time local observations. Through the coordination of the control actions of each agent, the landscape lighting system is jointly driven to achieve real-time adaptive collaborative control.

[0081] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications or equivalent changes made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A landscape lighting cooperative control method based on multi-agent reinforcement learning, characterized in that, include: Based on real-time acquired landscape lighting scene images, a comprehensive quality score of the current lighting effect is calculated through a pre-constructed light environment quality assessment model, wherein the change in the comprehensive quality score is used as the global reward signal for multi-agent reinforcement learning. The landscape lighting area is divided into several lighting zones, each corresponding to a reinforcement learning agent. Each agent outputs lighting control actions with continuous values ​​through a policy network based on its local observations. A three-dimensional decoupled hierarchical credit allocation mechanism is adopted to decompose the global reward signal into individual credit values ​​of each agent; Based on the assigned individual credit value, each agent aims to maximize the long-term cumulative comprehensive quality reward and uses a proximate policy optimization algorithm to update its own policy network. Based on the updated policy network, each agent continuously outputs lighting control actions according to real-time local observations. Through the coordination of the control actions of each agent, the landscape lighting system is jointly driven to achieve real-time adaptive collaborative control.

2. The landscape lighting cooperative control method based on multi-agent reinforcement learning as described in claim 1, characterized in that, The light environment quality assessment model specifically performs the following processing steps: Based on the landscape lighting scene image, objective light environment quality indicators are extracted; wherein, the objective light environment quality indicators include illuminance uniformity, glare index, color temperature consistency, brightness contrast, color rendering index, and visual focus matching degree. Based on the extracted objective light environment quality indicators, an adaptive weighted fusion is performed through a learnable quality fusion network to obtain a comprehensive quality score. Specifically, the learnable quality fusion network performs the following processing steps: extracting scene context-aware features from the objective light environment quality indicators; generating fusion weights for each indicator based on the extracted scene context-aware features; mapping each indicator to a sub-score; and weighting and summing the sub-scores based on the fusion weights to obtain the comprehensive quality score.

3. The landscape lighting cooperative control method based on multi-agent reinforcement learning as described in claim 1, characterized in that, The three-dimensional decoupled hierarchical credit allocation mechanism specifically performs the following processing steps: Based on the spatial location embedding of each lighting zone, the spatial contribution credit reflecting the visual focus weight of each zone is calculated. Based on the deviation of the chromaticity vector of each lighting zone from the global dominant hue, the chromaticity harmony credit, which reflects the degree of hue harmony, is calculated. Based on the similarity between the temporal embedding of the action sequences of each lighting zone and the global prosodic embedding, temporal collaborative credit reflecting the degree of synchronization of dynamic changes is calculated. The spatial contribution credit, chromaticity coordination credit, and temporal coordination credit are fused into a comprehensive credit weight for each agent through a learnable fusion layer, and then global normalization is performed.

4. The landscape lighting cooperative control method based on multi-agent reinforcement learning as described in claim 3, characterized in that, The calculation of the spatial contribution credit is as follows: Construct location embedding vectors for each lighting zone; wherein, the location embedding vectors are composed of the zone's three-dimensional geometric coordinates, architectural semantic labels, and viewpoint visibility scores; Spatial attention scores between partitions are calculated using a multi-head attention network; wherein, the spatial attention score represents the visual attention intensity of one partition to another. The spatial contribution credit of a given partition is obtained by normalizing the sum of the spatial attention scores of all partitions that pay attention to it.

5. The landscape lighting cooperative control method based on multi-agent reinforcement learning as described in claim 3, characterized in that, The calculation of the chromaticity coordination credit is as follows: The chromaticity vectors of each lighting zone are mapped to a perceptually uniform color space to obtain a set of chromaticity vectors; The mapped chroma vector set is clustered, and the cluster center with the largest sample size is extracted as the global dominant chroma center; Calculate the color difference between the chromaticity vector of each partition and the global dominant hue center; The color difference values ​​are mapped to colorimetric consistency credits using a Gaussian decay function.

6. The landscape lighting cooperative control method based on multi-agent reinforcement learning as described in claim 3, characterized in that, The calculation of the time-series collaborative credit is specifically as follows: A short-time Fourier transform is performed on the global brightness time-series signal to extract the main frequency component and phase information, which are then used to construct a global prosodic embedding vector. Based on the action sequence of each lighting zone, a temporal embedding vector is obtained through gated cyclic unit encoding; The dot product similarity between the temporal embedding vector and the global prosodic embedding vector is calculated, and temporal collaborative credit is obtained by mapping with the sigmoid function.

7. The landscape lighting cooperative control method based on multi-agent reinforcement learning as described in claim 1, characterized in that, The aforementioned proximal policy optimization algorithm employs a multi-agent proximal policy optimization algorithm, which directly incorporates the individual credit values ​​assigned to each agent into the advantage estimation during the policy network update process. in, Indicates the first i Credit-weighted advantage estimate of an agent Indicates assignment to the first i The individual credit value of an intelligent agent. Indicates time step t Global rewards As a discount factor, and These represent the value estimates of the current state and the next state by the centralized commentator, respectively.

8. A landscape lighting collaborative control system based on multi-agent reinforcement learning, characterized in that, include: The light environment quality assessment module is configured to: calculate the comprehensive quality score of the current lighting effect based on real-time acquired landscape lighting scene images through a pre-built light environment quality assessment model, wherein the change in the comprehensive quality score is used as the global reward signal for multi-agent reinforcement learning; The multi-agent policy module is configured as follows: the landscape lighting area is divided into several lighting zones, each lighting zone corresponds to a reinforcement learning agent, wherein each agent outputs a lighting control action with continuous values ​​through the policy network based on its local observations. The three-dimensional credit allocation module is configured to use a three-dimensional decoupled hierarchical credit allocation mechanism to decompose the global reward signal into individual credit values ​​for each agent. The policy update module is configured as follows: based on the assigned individual credit value, each agent updates its own policy network by using a proximal policy optimization algorithm with the goal of maximizing the long-term cumulative comprehensive quality reward. The collaborative control module is configured such that each agent continuously outputs lighting control actions based on the updated policy network and real-time local observations. Through the coordination of the control actions of each agent, the landscape lighting system is jointly driven to achieve real-time adaptive collaborative control.

Images