A smart lighting energy-saving method and system based on multimodal data fusion

Abstract

The present invention relates to a smart lighting energy-saving method and system based on multimodal data fusion. The present invention includes periodically acquiring a decision trajectory data set; merging the experience pool of the previous cycle with the newly collected decision trajectory data set of the current cycle according to a set ratio, and screening according to priority weights; training to obtain candidate strategies; evaluating the candidate strategies under a test scenario set; adding the strategies to a strategy library when their comprehensive scores meet the admission conditions; when the strategy capacity in the strategy library reaches a predetermined value, eliminating the strategies with the lowest comprehensive scores based on the Pareto front; constructing a strategy fitness function and establishing a feedback-weight mapping model for obtaining a prediction score; establishing a loss function gradient based on the feedback-weight mapping model, and performing iterative calculation of sub-item weights based on the loss function gradient to obtain the sub-item weights after iterative calculation. The present invention improves the long-term energy-saving efficiency and user experience of the lighting system.

Description

Technical Field

[0001] The present invention relates to the field of smart city and Internet of Things technologies, and in particular to a smart lighting energy-saving method and system based on multimodal data fusion. Background Art

[0002] With the rapid development of smart cities and the Internet of Things (IoT) technologies, the demand for intelligent, energy-efficient, and dynamically controllable lighting management in urban, park, and commercial areas is becoming increasingly urgent. Currently, lighting systems are widely used in a variety of scenarios, including urban roads, parks, ecological parks, commercial complexes, pedestrian streets, residential areas, campuses, and special scenarios (such as large-scale events, emergency rescue, and temporary lighting during disasters). Relevant management parties, including city managers, property management companies, park managers, and commercial operators, generally hope to achieve energy conservation and consumption reduction, improve lighting quality, enhance safety, and further promote the development of green and low-carbon cities through more efficient and intelligent lighting methods.

[0003] However, traditional lighting systems, especially those in urban roads, parks, and commercial areas, typically use simple timer-based sensors (such as photosensors) for control. While these systems achieve a certain degree of lighting automation, they generally suffer from the following shortcomings:

[0004] Rigid strategies lack flexibility; most systems rely on preset schedules or fixed sensor thresholds and are unable to adapt to real-time environmental conditions, dynamic crowd and vehicle traffic, etc. For example, in some areas, street lights remain fully illuminated for extended periods of time in the early morning hours, even when there are no pedestrians or vehicles passing through, wasting energy.

[0005] The data source is single and the perception is insufficient. Traditional solutions often rely solely on light sensors or infrared sensors, which cannot obtain more comprehensive scene information and make precise control difficult. For example, using only light intensity cannot distinguish whether the brightness is insufficient due to rain or the need for lighting at night.

[0006] Low energy efficiency and lack of optimization; in terms of multi-objective balance (such as energy consumption, comfort, and safety), traditional systems lack optimization capabilities and are prone to the problem of "either too bright or too dark", which not only wastes energy but also reduces the user experience.

[0007] Some research and products have attempted to incorporate data from multiple sensors (such as temperature, humidity, and video surveillance) for optimization, but these solutions have the following technical drawbacks:

[0008] The depth of data fusion is insufficient; data from multiple sensors are often processed independently, lacking cross-sensor collaborative analysis and decision-making capabilities.

[0009] The control strategy lacks the ability to evolve; most systems use rule-based strategies, which are difficult to adapt to changing environments, and the optimization effect decreases after long-term operation.

[0010] Lack of modeling of dynamic correlations between the environment and the crowd; factors such as peak traffic flow, traffic congestion, and bad weather have a direct impact on lighting needs, but existing solutions often do not introduce these correlation modeling and dynamic predictions.

[0011] Insufficient communication and collaboration: Traditional lighting control systems are mostly centralized and decentralized solutions, lacking efficient and low-latency communication mechanisms between nodes, making it difficult to achieve coordinated control and data sharing at the lamp pole level.

[0012] The ability to deal with failures and emergencies is limited; the existing system can usually only passively accept central instructions, lacks self-healing capabilities and active compensation mechanisms, and when some nodes fail, it will directly affect the lighting effect of the entire area. Summary of the Invention

[0013] To this end, the present invention provides a smart lighting energy-saving method and system based on multimodal data fusion to improve the long-term energy-saving efficiency and user experience of the lighting system.

[0014] To solve the above technical problems, the present invention provides a smart lighting energy-saving method based on multimodal data fusion, comprising:

[0015] S1. Periodically acquiring a decision trajectory dataset; wherein the decision trajectory dataset includes a state space, an action space, and a reward; the state space includes multimodal environmental data such as light intensity, traffic volume, pedestrian density, weather information, and positioning information; the action space includes lighting actions such as brightness adjustment and on / off states; and the reward includes energy saving effect, comfort, and safety;

[0016] S2. Add the decision trajectory dataset of the first cycle to the experience pool as a sample; in subsequent cycles, merge the experience pool of the previous cycle with the newly collected decision trajectory dataset of the current cycle according to a set ratio, and filter them according to priority weight;

[0017] S3. Use samples in the experience pool to perform strategy training, establish an objective function for the best candidate strategy, and train the candidate strategy based on the objective function. A safety constraint coefficient is introduced into the objective optimization function, and a PPO loss function is introduced during the training process to prevent excessive updates to strategy parameters through probability ratio clipping.

[0018] S4. Evaluate the candidate strategies under the test scenario set to obtain multi-dimensional evaluation indicators including energy efficiency, safety, and user subjective comfort, and obtain a comprehensive strategy score based on the multi-dimensional evaluation indicators and corresponding sub-item weights;

[0019] S5. In response to the comprehensive score of the strategy meeting the admission condition, the strategy is added to the strategy library; when the strategy capacity in the strategy library reaches a predetermined value, the scores of the candidate strategies meeting the admission condition are compared with the existing strategies in the strategy library based on the Pareto front, and the strategies with the lowest comprehensive scores are eliminated;

[0020] S6. Construct a fitness function of the strategy based on the weight of the score of each indicator in the multi-dimensional evaluation indicator; establish a feedback-weight mapping model for obtaining a predicted score based on the sub-item weights and the corresponding feedback data; establish a loss function gradient based on the feedback-weight mapping model, and perform iterative calculation of the sub-item weights based on the loss function gradient to obtain the sub-item weights after iterative calculation; based on the sub-item weights after iterative calculation, recalculate the corresponding comprehensive score of the strategy according to the fitness function, and compare it with the score of the current strategy; if it is better than the score of the current strategy, dynamically update the sub-item weights of the current strategy.

[0021] In one embodiment of the present invention, in a subsequent cycle, the experience pool of the previous cycle and the newly collected decision trajectory dataset of the current cycle are merged according to a set ratio, including:

[0022] The merger is done as follows:

[0023] ,

[0024] in, Indicates that the data set is collected in the first cycle;

[0025] The dataset collected for the tth period;

[0026] a is the historical data retention rate, which is 0.7, indicating that 70% of the old data is retained and 30% of the new data is added;

[0027] To merge operations by priority;

[0028] After the tth cycle ends, and The merged experience pool.

[0029] In one embodiment of the present invention, the priority weight is expressed as:

[0030] ,

[0031] in: The trajectory length normalization factor normalizes trajectories of different lengths to ensure that long and short trajectories are comparable;

[0032] For the trajectory;

[0033] For the trajectory length;

[0034] k For the moment;

[0035] is a constant used to adjust the weight and decay rate;

[0036] The value prediction bias measures the degree of surprise predicted by the current value function;

[0037] for k Instant rewards at all times;

[0038] is the discount factor, set to 0.99, which measures the importance of future rewards;

[0039] Is the policy value function, which means that under the policy π, from the state s The expected return from starting and following the strategy;

[0040] Average state characteristics this week;

[0041] For the trajectory middle k The state vector at time .

[0042] In one embodiment of the present invention, the objective function is expressed as:

[0043] ,

[0044] in,

[0045] ,

[0046] in, The parameters are hour, Maximum, expect to get an optimal candidate strategy, parameters and corresponding strategies ; For unknown parameters, substitute them into all experience pools for training, hoping to find an optimal parameter;

[0047] A data set that stores the current experience pool with historical trajectories;

[0048] is the safety constraint factor;

[0049] is a safety constraint item;

[0050] The difference between the maximum immediate reward and the minimum immediate reward;

[0051] and are the immediate rewards at time k and time k-1 respectively.

[0052] In one embodiment of the present invention, the PPO loss function is expressed as:

[0053] ,

[0054] in,

[0055] ,

[0056] ,

[0057] in, is a constant;

[0058] For the trimming function; Python interception function, interception Zhongzai Data within the range will be discarded if it is not within the range;

[0059] are the parameters of the strategy;

[0060] The parameter is strategy when

[0061] are the last updated policy parameters;

[0062] For receiving s Output Action a The probability distribution of

[0063] For value prediction bias;

[0064] is the discount factor;

[0065] is a constant.

[0066] In one embodiment of the present invention, the candidate strategies are evaluated under a set of test scenarios to obtain multi-dimensional evaluation indicators including energy efficiency, safety, and user subjective comfort. Based on the multi-dimensional evaluation indicators and the corresponding sub-item weights, a comprehensive strategy score is obtained, including:

[0067] In the test scene set C =Evaluate on {rainy night, sunny day, foggy day, peak};

[0068] The comprehensive strategy score is expressed as:

[0069] ,

[0070] in, is the scene weight, satisfying , the system gives weight to each scene according to its importance; It is the collection of weights of each sub-item in each scenario;

[0071] is the energy efficiency sub-item weight; The weight of the safety sub-item; The weight of the user's subjective comfort items;

[0072] Energy efficiency is expressed as:

[0073] ,

[0074] in, For Total energy consumption in the scenario;

[0075] c For the test scene set C A scene in

[0076] For The baseline strategy in the scenario is the total energy consumption when the system power consumption is the maximum;

[0077] Security is expressed as:

[0078] ,

[0079] in, For the scene c The number of controls under

[0080] For the scene c The number of times the brightness exceeds the safety threshold;

[0081] The fewer the number of violations, the closer the safety score is to 1, which is the best;

[0082] User subjective comfort is expressed as:

[0083] ,

[0084] in, For the first i User ratings; 1-5 points;

[0085] N users were rated in scenario c;

[0086] It reflects the average user rating in scenario c and the user's subjective comfort.

[0087] In one embodiment of the present invention, in response to the comprehensive score of the strategy meeting the admission condition, the strategy is added to the strategy library; when the strategy capacity in the strategy library reaches a predetermined value, based on the Pareto front, the scores of the candidate strategies that meet the admission condition are compared with the existing strategies in the strategy library, and the strategies with the lowest comprehensive scores are eliminated; including:

[0088] The admission conditions are expressed as follows:

[0089] ,

[0090] in, 、 are the mean and standard deviation of the scores of the strategy library respectively;

[0091] For strategy A collection of

[0092] If the admission conditions are met, add a policy:

[0093] ,

[0094] in, The parameters are Strategy at the time A collection of

[0095] Based on the Pareto frontier, elimination strategy:

[0096] ,

[0097] in, The process of comparing the scores of candidate strategies for the admission conditions with the existing strategies in the strategy library and eliminating the strategies with the lowest scores;

[0098] The strategy you want to introduce;

[0099] When satisfied:

[0100] ,

[0101] in, It is the lowest score among the original strategies. When the strategy is lower than the minimum score, it is eliminated.

[0102] In one embodiment of the present invention, the fitness function is expressed as:

[0103] ,

[0104] ,

[0105] in, , ;

[0106] is the user comfort index;

[0107] is the total energy consumption when executing the current strategy;

[0108] is the minimum total energy consumption in the strategy set;

[0109] is the maximum total energy consumption in the strategy set;

[0110] The number of violations of this policy;

[0111] is the total number of violations;

[0112] Score the comfort level of the i-th person among N people when applying the strategy.

[0113] In one embodiment of the present invention, based on each of the sub-item weights and the corresponding feedback data, a feedback-weight mapping model for obtaining a predicted score is established; based on the feedback-weight mapping model, a loss function gradient is established; based on the loss function gradient, the sub-item weights are iteratively calculated to obtain the sub-item weights after iterative calculation; based on the iteratively calculated sub-item weights, according to the fitness function, the corresponding strategy comprehensive score is recalculated and compared with the score of the current strategy; if the score is better than the score of the current strategy, the sub-item weights of the current strategy are dynamically updated, including:

[0114] Establish a feedback-weight mapping model:

[0115] ,

[0116] in, Score the predictions;

[0117] The weight data of each item in the current execution strategy for this trajectory;

[0118] is the energy efficiency sub-item weight;

[0119] The weight of the safety sub-item;

[0120] The weight of the user's subjective comfort items;

[0121] For feedback data;

[0122] is the theoretical energy efficiency upper limit for this location and time period;

[0123] is the safety risk factor based on the environment;

[0124] comfort preferences of user groups;

[0125] Establish the loss function gradient:

[0126] ,

[0127] in,

[0128] ,

[0129] in, is the weight of each sub-indicator;

[0130] is a constant;

[0131] , Indicates that the function is a gradient operator function;

[0132] Actual ratings for user feedback;

[0133] is the prediction score, calculated by the formula;

[0134] It is a constant, the minimum value of the item weight set by the system;

[0135] is the current item weight.

[0136] The present invention also provides a smart lighting energy-saving system based on multimodal data fusion, comprising:

[0137] A data acquisition module, configured to periodically acquire a decision trajectory dataset; wherein the decision trajectory dataset includes a state space, an action space, and rewards; the state space includes multimodal environmental data such as light intensity, vehicle flow, pedestrian density, weather information, and positioning information; the action space includes lighting actions such as brightness adjustment and on / off states; and the rewards include energy saving effects, comfort, and safety;

[0138] An experience pool update module is used to add the decision trajectory dataset of the first cycle to the experience pool as a sample; in subsequent cycles, the experience pool of the previous cycle and the newly collected decision trajectory dataset of the current cycle are merged according to a set ratio and filtered according to priority weight;

[0139] The strategy training module is used to perform strategy training using samples in the experience pool, establish the objective function of the best candidate strategy, and train the candidate strategy based on the objective function. The safety constraint coefficient is introduced into the objective optimization function, and the PPO loss function is introduced during the training process to prevent the strategy parameters from being updated too much through probability ratio clipping;

[0140] A strategy comprehensive score acquisition module is used to evaluate the candidate strategy under a set of test scenarios to obtain multi-dimensional evaluation indicators including energy efficiency, safety, and user subjective comfort, and to obtain a strategy comprehensive score based on the multi-dimensional evaluation indicators and corresponding sub-item weights;

[0141] a strategy library update module, configured to add a strategy to the strategy library in response to the strategy's comprehensive score meeting the admission criteria; and when the strategy capacity in the strategy library reaches a predetermined value, compare the scores of candidate strategies that meet the admission criteria with the existing strategies in the strategy library based on the Pareto front, and eliminate the strategy with the lowest comprehensive score;

[0142] The sub-item weight update module is used to construct the fitness function of the strategy according to the weight of the score of each indicator in the multi-dimensional evaluation indicator; based on each sub-item weight and the corresponding feedback data, a feedback-weight mapping model for obtaining the predicted score is established; based on the feedback-weight mapping model, a loss function gradient is established, and based on the loss function gradient, the sub-item weight is iteratively calculated to obtain the sub-item weight after iterative calculation; based on the sub-item weight after iterative calculation, according to the fitness function, the corresponding comprehensive score of the strategy is recalculated, and compared with the score of the current strategy. If it is better than the score of the current strategy, the sub-item weight of the current strategy is dynamically updated.

[0143] The above technical solution of the present invention has the following advantages over the prior art:

[0144] The present invention describes a smart lighting energy-saving method and system based on multimodal data fusion, which significantly improves the comprehensive perception capability of complex application scenarios by integrating multimodal environmental data such as light intensity, vehicle flow, pedestrian density, weather information, and positioning information. It adopts a decision trajectory experience pool and a multi-dimensional priority weight screening mechanism to achieve dynamic optimization and adaptive update of the policy library. In the policy optimization process, it simultaneously introduces multi-objective constraints such as energy efficiency, comfort, and safety to comprehensively balance various indicators. It innovatively introduces a safety constraint coefficient into the objective function and uses the PPO loss function to prevent drastic fluctuations in the policy parameters, ensuring the safety and reliability of the lighting effect and the subjective comfort of the user. It introduces reinforcement learning and evolutionary algorithms, which can continuously evolve and iterate the control strategy based on historical operating data and user feedback, thereby dynamically adapting to the ever-changing environment and pedestrian conditions, and improving the long-term energy-saving efficiency and user experience of the lighting system. BRIEF DESCRIPTION OF THE DRAWINGS

[0145] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

[0146] Figure 1 This is a flowchart of an intelligent lighting energy-saving method based on multimodal data fusion in Example 1.

[0147] Figure 2 This is a schematic diagram of a smart lighting energy-saving system architecture based on multimodal data fusion in Example 2.

[0148] Figure 3 This is a schematic diagram of a smart lighting energy-saving system communication based on multimodal data fusion in Example 2. DETAILED DESCRIPTION

[0149] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0150] Example 1

[0151] Reference Figure 1 As shown, this embodiment provides a smart lighting energy-saving method based on multimodal data fusion, including the following steps:

[0152] S1. Periodically acquire a decision trajectory dataset; the decision trajectory dataset includes a state space, an action space, and rewards. The state space includes multimodal environmental data such as light intensity, traffic flow, pedestrian density, weather information, and positioning information. The action space includes brightness adjustment and lighting actions such as on / off states. The rewards include energy saving effects, comfort, and safety. The specific decision trajectory dataset is represented as follows:

[0153] ,

[0154] in: The decision trajectory dataset collected for week t (empirical dataset);

[0155] is the number of trajectories in the tth period, which depends on the number of interactions between the agent and the environment in the tth period;

[0156] A trajectory identifies a complete sequence of actions performed by the agent in the environment, starting from the state and ending in the terminal state;

[0157] End identifier, there is no and No further action will be performed;

[0158] ,s is the state, S is the set of all states (lighting, traffic, pedestrian flow, weather, etc.);

[0159] ,Normalized action space; A is the set of all actions;

[0160] , reward function; identifies the reward function of the tth cycle;

[0161] are the state, action, and reward function at the 0th cycle respectively.

[0162] S2. Add the decision trajectory dataset of the first cycle to the experience pool as a sample; in subsequent cycles, merge the experience pool of the previous cycle with the newly collected decision trajectory dataset of the current cycle according to a set ratio, and filter them according to priority weight.

[0163] Specifically, the merging is performed by the following formula:

[0164] ,

[0165] in, Indicates that the data set is collected in the first cycle;

[0166] The dataset collected for the tth period;

[0167] a is the historical data retention rate, which is 0.7, indicating that 70% of the old data is retained and 30% of the new data is added;

[0168] To merge operations by priority;

[0169] After the tth cycle ends, and The merged experience pool.

[0170] Specifically, the priority weight is expressed as:

[0171] ,

[0172] in: The trajectory length normalization factor normalizes trajectories of different lengths to ensure that long and short trajectories are comparable;

[0173] For the trajectory;

[0174] For the trajectory length;

[0175] k For the moment;

[0176] is a constant used to adjust the weight and decay rate;

[0177] The value prediction bias measures the degree of surprise predicted by the current value function;

[0178] for k Instant rewards at all times;

[0179] is the discount factor, set to 0.99, which measures the importance of future rewards;

[0180] Is the policy value function, which means that under the policy π, from the state s The expected return from starting and following the strategy;

[0181] Average state characteristics this week;

[0182] For the trajectory middle k The state vector at time .

[0183] S3. Use samples in the experience pool to perform strategy training, establish the objective function of the best candidate strategy, and train the candidate strategy based on the objective function. A safety constraint coefficient is introduced into the objective optimization function, and a PPO loss function is introduced during the training process. Probability ratio clipping is used to prevent the strategy parameters from being updated too much.

[0184] Specifically, the objective function is expressed as:

[0185] ,

[0186] in,

[0187] ,

[0188] in, The parameters are hour, Maximum, expect to get an optimal candidate strategy, parameters and corresponding strategies ; For unknown parameters, substitute them into all experience pools for training, hoping to find an optimal parameter;

[0189] A data set that stores the current experience pool with historical trajectories;

[0190] is the safety constraint factor;

[0191] is a safety constraint item;

[0192] The difference between the maximum immediate reward and the minimum immediate reward;

[0193] and are the immediate rewards at time k and time k-1 respectively.

[0194] Specifically, the PPO loss function is expressed as:

[0195] ,

[0196] in,

[0197] ,

[0198] ,

[0199] in, is a constant;

[0200] For the trimming function; Python interception function, interception Zhongzai Data within the range will be discarded if it is not within the range;

[0201] are the parameters of the strategy;

[0202] The parameter is strategy when

[0203] are the last updated policy parameters;

[0204] For receiving s Output Action a The probability distribution of

[0205] For value prediction bias;

[0206] is the discount factor;

[0207] is a constant.

[0208] S4. Evaluate the candidate strategies under a set of test scenarios to obtain multi-dimensional evaluation indicators including energy efficiency, safety, and user subjective comfort. Obtain a comprehensive strategy score based on the multi-dimensional evaluation indicators and corresponding sub-item weights.

[0209] In the test scene set C =Evaluate on {rainy night, sunny day, foggy day, peak};

[0210] The comprehensive strategy score is expressed as:

[0211] ,

[0212] in, is the scene weight, satisfying , the system gives weight to each scene according to its importance; It is the collection of weights of each sub-item in each scenario;

[0213] is the energy efficiency sub-item weight; The weight of the safety sub-item; The weight of the user's subjective comfort items;

[0214] Energy efficiency is expressed as:

[0215] ,

[0216] in, For Total energy consumption in the scenario;

[0217] c For the test scene set C A scene in

[0218] For The baseline strategy in the scenario is the total energy consumption when the system power consumption is the maximum;

[0219] Security is expressed as:

[0220] ,

[0221] in, For the scene c The number of controls under

[0222] For the scene c The number of times the brightness exceeds the safety threshold;

[0223] The fewer the number of violations, the closer the safety score is to 1, which is the best;

[0224] User subjective comfort is expressed as:

[0225] ,

[0226] in, For the first i User ratings; 1-5 points;

[0227] N users were rated in scenario c;

[0228] It reflects the average user rating in scenario c and the user's subjective comfort.

[0229] S5. In response to the comprehensive score of the strategy meeting the admission conditions, the strategy is added to the strategy library; when the strategy capacity in the strategy library reaches a predetermined value, based on the Pareto front, the scores of the candidate strategies that meet the admission conditions are compared with the existing strategies in the strategy library, and the strategies with the lowest comprehensive scores are eliminated.

[0230] Specifically, the admission conditions are expressed as:

[0231] ,

[0232] in, 、 are the mean and standard deviation of the scores of the strategy library respectively;

[0233] For strategy A collection of

[0234] If the admission conditions are met, add a policy:

[0235] ,

[0236] in, The parameters are Strategy at the time A collection of

[0237] Based on the Pareto frontier, elimination strategy:

[0238] ,

[0239] in, The process of comparing the scores of candidate strategies for the admission conditions with the existing strategies in the strategy library and eliminating the strategies with the lowest scores;

[0240] The strategy you want to introduce;

[0241] When satisfied:

[0242] ,

[0243] in, It is the lowest score among the original strategies. When the strategy is lower than the minimum score, it is eliminated.

[0244] S6. Construct a fitness function of the strategy based on the weight of the score of each indicator in the multi-dimensional evaluation indicator; establish a feedback-weight mapping model for obtaining a predicted score based on the sub-item weights and the corresponding feedback data; establish a loss function gradient based on the feedback-weight mapping model, and perform iterative calculation of the sub-item weights based on the loss function gradient to obtain the sub-item weights after iterative calculation; based on the sub-item weights after iterative calculation, recalculate the corresponding comprehensive score of the strategy according to the fitness function, and compare it with the score of the current strategy; if it is better than the score of the current strategy, dynamically update the sub-item weights of the current strategy.

[0245] Specifically, the fitness function is expressed as:

[0246] ,

[0247] ,

[0248] in, , ;

[0249] is the user comfort index;

[0250] is the total energy consumption when executing the current strategy;

[0251] is the minimum total energy consumption in the strategy set;

[0252] is the maximum total energy consumption in the strategy set;

[0253] The number of violations of this policy;

[0254] is the total number of violations;

[0255] Score the comfort level of the i-th person among N people when applying the strategy.

[0256] Establish a feedback-weight mapping model:

[0257] ,

[0258] in, Score the predictions;

[0259] The weight data of each item in the current execution strategy for this trajectory;

[0260] is the energy efficiency sub-item weight;

[0261] The weight of the safety sub-item;

[0262] The weight of the user's subjective comfort items;

[0263] For feedback data;

[0264] is the theoretical energy efficiency upper limit for this location and time period;

[0265] is the safety risk factor based on the environment;

[0266] comfort preferences of user groups;

[0267] Establish the loss function gradient:

[0268] ,

[0269] in,

[0270] ,

[0271] in, is the weight of each sub-indicator;

[0272] is a constant;

[0273] , Indicates that the function is a gradient operator function;

[0274] Actual ratings for user feedback;

[0275] is the prediction score, calculated by the formula;

[0276] It is a constant, the minimum value of the item weight set by the system;

[0277] is the current item weight.

[0278] Based on the item weights after iterative calculation , through the fitness function , recalculate the corresponding comprehensive score of the strategy and compare it with the score of the current strategy. If it is better than the score of the current strategy, dynamically update the sub-item weights of the current strategy.

[0279] Example 2

[0280] Based on the same inventive concept, this embodiment provides a smart lighting energy-saving system based on multimodal data fusion. The principle of solving the problem is similar to that of the smart lighting energy-saving method based on multimodal data fusion, and the repeated parts will not be repeated.

[0281] This embodiment provides a smart lighting energy-saving system based on multimodal data fusion, including:

[0282] A data acquisition module, configured to periodically acquire a decision trajectory dataset; wherein the decision trajectory dataset includes a state space, an action space, and rewards; the state space includes multimodal environmental data such as light intensity, vehicle flow, pedestrian density, weather information, and positioning information; the action space includes lighting actions such as brightness adjustment and on / off states; and the rewards include energy saving effects, comfort, and safety;

[0283] An experience pool update module is used to add the decision trajectory dataset of the first cycle to the experience pool as a sample; in subsequent cycles, the experience pool of the previous cycle and the newly collected decision trajectory dataset of the current cycle are merged according to a set ratio and filtered according to priority weight;

[0284] The strategy training module is used to perform strategy training using samples in the experience pool, establish the objective function of the best candidate strategy, and train the candidate strategy based on the objective function. The safety constraint coefficient is introduced into the objective optimization function, and the PPO loss function is introduced during the training process to prevent the strategy parameters from being updated too much through probability ratio clipping;

[0285] A strategy comprehensive score acquisition module is used to evaluate the candidate strategy under a set of test scenarios to obtain multi-dimensional evaluation indicators including energy efficiency, safety, and user subjective comfort, and to obtain a strategy comprehensive score based on the multi-dimensional evaluation indicators and corresponding sub-item weights;

[0286] a strategy library update module, configured to add a strategy to the strategy library in response to the strategy's comprehensive score meeting the admission criteria; and when the strategy capacity in the strategy library reaches a predetermined value, compare the scores of candidate strategies that meet the admission criteria with the existing strategies in the strategy library based on the Pareto front, and eliminate the strategy with the lowest comprehensive score;

[0287] The sub-item weight update module is used to construct the fitness function of the strategy according to the weight of the score of each indicator in the multi-dimensional evaluation indicator; based on each sub-item weight and the corresponding feedback data, a feedback-weight mapping model for obtaining the predicted score is established; based on the feedback-weight mapping model, a loss function gradient is established, and based on the loss function gradient, the sub-item weight is iteratively calculated to obtain the sub-item weight after iterative calculation; based on the sub-item weight after iterative calculation, according to the fitness function, the corresponding comprehensive score of the strategy is recalculated, and compared with the score of the current strategy. If it is better than the score of the current strategy, the sub-item weight of the current strategy is dynamically updated.

[0288] It should be noted that, referring to Figure 2 As shown in the figure, multimodal environmental data is collected and transmitted in real time through the data acquisition and transmission module (data acquisition module) installed on the light pole, which includes: light sensor, human presence sensor, environmental sensor, motion sensor, camera, Beidou positioning module and star flash module. The functions of each sensor are as follows:

[0289] Light sensor: used to collect ambient brightness data;

[0290] Human presence sensor and motion sensor: used to detect the relative distance between the human body and the light pole, the direction of travel and the speed;

[0291] Camera: used for regional detection (such as pedestrian flow and crowd density) and providing monitoring data;

[0292] Environmental sensor: used to collect environmental parameters such as temperature, humidity, and weather;

[0293] Beidou positioning module: used to achieve precise positioning of lighting poles and provide data support for cluster division and regional collaboration;

[0294] Star Flash Module: Realizes short-range automatic networking communication and multi-node data sharing of smart lamp poles.

[0295] The environmental information collected in real time by these sensors is uploaded to the control layer via the data transmission module. After undergoing data processing processes such as data cleaning, normalization, and feature extraction, it is stored on the cloud platform. Based on this preprocessed multimodal data, the system utilizes the modules integrated into the control layer (from the experience pool update module to the sub-item weight update module) to automatically adjust and optimize the brightness of the light poles in the area and the energy-saving mode of the functional modules, thereby achieving optimal lighting comfort and energy consumption.

[0296] In addition, refer to Figure 3 As shown in the figure, the system divides the controlled area into multiple clusters (such as roads, parks, and commercial areas) based on the geographical environment and functional characteristics of urban streets. The smart lighting terminals within each cluster are connected to the cloud platform and communicate via ad hoc networks using Starlight modules. Through inter-module Starlight ad hoc networking and data sharing, sensor data and control status are exchanged, enabling collaborative decision-making on lighting strategies. The system supports lighting module brightness adjustment, on / off control, and low-power mode control, achieving efficient energy conservation and intelligent management.

[0297] Reference Figure 2 As shown, the data processing and control process includes: data collection and preprocessing: cleaning, normalizing, aggregating and extracting features from multi-source raw data, and storing the processing results in the management cloud platform to achieve unified management of historical data. In addition, the system also includes:

[0298] Control layer: The control layer includes central control services and local control services (edge ​​computing gateways). The central control service issues control policies via the IoT protocol. The local control service dynamically adjusts lighting status and equipment operating parameters based on real-time sensor data using the method described in Example 1, achieving local and global coordinated control and optimization.

[0299] User display layer: provides users with a visual management interface that supports functions such as device distribution, area division, device control, energy-saving status information, and real-time energy consumption monitoring to facilitate management and decision-making.

[0300] Specifically, the system uses motion sensors and human presence sensors to calculate a person's direction and speed in real time, and uses Star Flash communication channels to synchronize data with surrounding light poles. Star Flash technology is a short-range, high-speed, low-latency, and highly reliable wireless communication technology that supports automatic discovery, routing, and ad hoc network communication between terminals, significantly improving the responsiveness and data collaboration capabilities of smart lighting systems.

[0301] In actual operation, the system can input relevant data into the strategy operation engine based on the real-time collection of human travel speed, human flow and ambient brightness, and calculate and issue the light pole brightness coefficient and lighting time. For example, when a person is detected approaching, the system calculates the time T of the pedestrian to reach the next lighting node based on the distance S and speed V. arrival :T arrival = S / V, and regional coordinated dimming is achieved through environmental data. The light sensor triggers basic lighting thresholds (e.g., automatic lighting when ambient light falls below 50 Lux). The camera uses the YOLO algorithm to automatically identify crowd density and grade lighting. The Beidou module accurately clusters lighting nodes.

[0302] The system also supports spatiotemporal coordinated control, such as:

[0303] Phase synchronization: The brightness of adjacent lamp poles changes gradually to avoid sudden changes in lighting (e.g., from 100% to 20% in 3 seconds);

[0304] Fault self-healing: When a node fails, the brightness of adjacent nodes is automatically increased to compensate (compensation formula):

[0305] ΔB=B 失效 / N 相邻节点，

[0306] ΔB, brightness compensation value, the brightness of the surrounding adjacent nodes needs to be increased;

[0307] B 失效 ,The original brightness of the failed node is the brightness when the failed node is working normally;

[0308] N 相邻节点， The number of valid adjacent nodes, which can be used to compensate for the brightness of failed nodes;

[0309] Environmental Adaptation: In high temperature and high humidity environments, the brightness is reduced to 80% to reduce device heat; in rainy and foggy weather, the brightness is increased to 120% to enhance road safety.

[0310] Therefore, when any smart lighting terminal in the cluster loses network connection with the platform, as long as the device's Star Flash module and the cluster's self-organizing network function are normal, data can be uploaded through other terminal upload channels; when the environment changes, it will automatically notify the nearby location (next) node to execute the strategy, forming a linkage.

[0311] In the Star Flash ad hoc network communication, the link quality evaluation (LQ) is calculated based on the weighted normalization of the received signal strength (RSSI), bit error rate (BER) and throughput (TP):

[0312] ,

[0313] Normalization function definition:

[0314] ,

[0315] x is the received signal strength (RSSI), bit error rate (BER), or throughput (TP);

[0316] are the minimum and maximum values ​​of each parameter.

[0317] The routing score S comprehensively considers the link quality and node residual energy:

[0318] ,

[0319] Among them, E is the remaining energy of the node, and the total energy consumption of the path is calculated as:

[0320] ,

[0321] , is the total energy consumed by the path;

[0322] , is the energy required for node i to send data;

[0323] , is the energy required for node i to receive data;

[0324] Under the adaptive channel selection model, the interference level is calculated as follows:

[0325] ,

[0326] in, is the total interference of channel c;

[0327] N is the set of neighbor nodes;

[0328] is the interference contribution of neighbor node i on channel c;

[0329] For nodes i The weight is inversely proportional to the distance.

[0330] Optimal channel selection function:

[0331] ,

[0332] in, The node set with the smallest total interference among adjacent link nodes providing brightness compensation is the optimal path node set.

[0333] Through the above-mentioned Star Flash self-organizing network model, the system can automatically select the best link quality, optimal route and channel to achieve efficient and reliable data transmission and strategy coordination, providing a solid communication and perception foundation for the smart lighting energy-saving system.

[0334] In summary, the system first divides the area into clusters based on geographic environment and functional characteristics. Lighting nodes (such as light poles) within each cluster communicate via a self-organizing network using the Star Flash module, enabling efficient sharing and synchronization of sensor data and control status. During data collection and processing, the system not only integrates multi-source information such as light intensity, human presence, movement direction and speed, and crowd density captured by cameras, but also incorporates precise location information provided by the Beidou positioning module, providing comprehensive and accurate data input for the subsequent evolutionary algorithm.

[0335] To generate control strategies, the system employs a multi-objective optimization approach, comprehensively weighing factors such as energy consumption, user comfort, and environmental impact to establish an optimal control strategy algorithm, effectively avoiding issues such as excessive lighting and blind spot lighting. Compared to traditional fixed lighting strategies, this system dynamically responds to changes in the environment and traffic flow, achieving significant energy savings of 45% to 80%.

[0336] The system also incorporates a predictive lighting mechanism. By analyzing a person's direction and speed, it predicts when a pedestrian will reach the next lighting node and activates the corresponding light pole in advance, creating a seamless lighting experience. The system also features fault recovery capabilities. If a node fails, adjacent nodes automatically increase their brightness to compensate, ensuring continuous road lighting and driving safety.

[0337] Starflash wireless communication technology not only enables efficient networking and communication between lighting nodes, but also supports automatic node discovery, intelligent routing, link quality assessment, and adaptive channel selection. This short-range, high-speed, low-latency wireless communication method significantly improves data synchronization efficiency and system response speed within the cluster.

[0338] At the same time, the system supports the continuous collection and analysis of historical data and user feedback (such as in-app ratings, etc.). The strategy library will be automatically updated and optimized every week based on actual operating data and user experience, to achieve continuous evolution and adaptive iteration of control strategies, and continuously improve the overall energy saving and intelligence level of the system.

[0339] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0340] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0341] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0342] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0343] Finally, it should be noted that the above specific implementation methods are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to examples, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A smart lighting energy-saving method based on multimodal data fusion, characterized in that: include: S1. Periodically acquiring a decision trajectory dataset; wherein the decision trajectory dataset includes a state space, an action space, and a reward; the state space includes multimodal environmental data such as light intensity, traffic volume, pedestrian density, weather information, and positioning information; the action space includes lighting actions such as brightness adjustment and on / off states; and the reward includes energy saving effect, comfort, and safety; S2. Add the decision trajectory dataset of the first cycle to the experience pool as a sample; in subsequent cycles, merge the experience pool of the previous cycle with the newly collected decision trajectory dataset of the current cycle according to a set ratio, and filter them according to priority weight; S3. Use samples in the experience pool to perform strategy training, establish an objective function for the best candidate strategy, and train the candidate strategy based on the objective function. A safety constraint coefficient is introduced into the objective function, and a PPO loss function is introduced during the training process to prevent excessive updates to strategy parameters through probability ratio clipping. S4. Evaluate the candidate strategies under the test scenario set to obtain multi-dimensional evaluation indicators including energy efficiency, safety, and user subjective comfort, and obtain a comprehensive strategy score based on the multi-dimensional evaluation indicators and corresponding sub-item weights; S5. In response to the comprehensive score of the strategy meeting the admission condition, the strategy is added to the strategy library; when the strategy capacity in the strategy library reaches a predetermined value, the scores of the candidate strategies meeting the admission condition are compared with the existing strategies in the strategy library based on the Pareto front, and the strategies with the lowest comprehensive scores are eliminated; S6. Construct a fitness function of the strategy based on the weight of the score of each indicator in the multi-dimensional evaluation indicator; establish a feedback-weight mapping model for obtaining a predicted score based on the sub-item weights and the corresponding feedback data; establish a loss function gradient based on the feedback-weight mapping model, and perform iterative calculation of the sub-item weights based on the loss function gradient to obtain the sub-item weights after iterative calculation; based on the sub-item weights after iterative calculation, recalculate the corresponding comprehensive score of the strategy according to the fitness function, and compare it with the score of the current strategy; if it is better than the score of the current strategy, dynamically update the sub-item weights of the current strategy.

2. The intelligent lighting energy-saving method based on multimodal data fusion according to claim 1 is characterized in that: In subsequent cycles, the experience pool of the previous cycle is merged with the newly collected decision trajectory dataset of this cycle according to the set ratio, including: The merger is done by: ， in, Indicates that the data set is collected in the first cycle; The dataset collected for the tth period; a is the historical data retention rate, which is 0.7, indicating that 70% of the old data is retained and 30% of the new data is added; To merge operations by priority; After the tth cycle ends, and The merged experience pool.

3. The intelligent lighting energy-saving method based on multimodal data fusion according to claim 1 is characterized in that: The priority weight is expressed as: ， in: The trajectory length normalization factor is used to normalize trajectories of different lengths to ensure that long and short trajectories are comparable. For the trajectory; T For the trajectory length; k For the moment; is a constant used to adjust the weight and decay rate; The value prediction bias measures the degree of surprise predicted by the current value function; for k Instant rewards at all times; is the discount factor, set to 0.99, which measures the importance of future rewards; Is the policy value function, which means that under the policy π, from the state s The expected return from starting and following the strategy; Average state characteristics this week; For the trajectory middle k The state vector at time .

4. The intelligent lighting energy-saving method based on multimodal data fusion according to claim 1 is characterized in that: The objective function is expressed as: ， in, ， in, The parameters are hour, Maximum, expect to get an optimal candidate strategy, parameters and corresponding strategies ; For unknown parameters, substitute them into all experience pools for training, hoping to find an optimal parameter; A data set that stores the current experience pool with historical trajectories; is the safety constraint factor; is a safety constraint item; The difference between the maximum immediate reward and the minimum immediate reward; and are the immediate rewards at time k and time k-1 respectively.

5. The intelligent lighting energy-saving method based on multimodal data fusion according to claim 1 is characterized in that: The PPO loss function is expressed as: ， in, ， in, is a constant; For the trimming function; Python interception function, interception Zhongzai Data in range; Those outside this range will be discarded; are the parameters of the strategy; The parameter is strategy when are the last updated policy parameters; For receiving s Output Action a The probability distribution of For value prediction bias; is the discount factor; is a constant.

6. The intelligent lighting energy-saving method based on multimodal data fusion according to claim 1 is characterized in that: The candidate strategies are evaluated under a set of test scenarios to obtain multi-dimensional evaluation indicators including energy efficiency, safety, and user subjective comfort. Based on the multi-dimensional evaluation indicators and the corresponding sub-item weights, a comprehensive strategy score is obtained, including: In the test scene set C =Evaluate on {rainy night, sunny day, foggy day, peak}; The comprehensive strategy score is expressed as: ， in, is the scene weight, satisfying , the system gives weight to each scene according to its importance; It is the collection of weights of each sub-item in each scenario; is the energy efficiency sub-item weight; The weight of the safety sub-item; The weight of the user's subjective comfort items; Energy efficiency is expressed as: ， in, For Total energy consumption in the scenario; c For the test scene set C A scene in For The baseline strategy in the scenario is the total energy consumption when the system power consumption is the maximum; Security is expressed as: ， in, For the scene c The number of controls under For the scene c The number of times the brightness exceeds the safety threshold; The fewer the number of violations, the closer the safety score is to 1, which is the best; User subjective comfort is expressed as: ， in, For the first i User rating; 1-5 points; N users were rated in scenario c; It reflects the average user rating in scenario c and the user's subjective comfort.

7. The intelligent lighting energy-saving method based on multimodal data fusion according to claim 6 is characterized in that: In response to the comprehensive score of the strategy meeting the admission conditions, it is added to the strategy library; when the strategy capacity in the strategy library reaches a predetermined value, based on the Pareto front, the scores of the candidate strategies that meet the admission conditions are compared with the existing strategies in the strategy library, and the strategies with the lowest comprehensive scores are eliminated; including: The admission conditions are expressed as follows: ， in, 、 are the mean and standard deviation of the scores of the strategy library respectively; is the set of strategies π; If the admission conditions are met, add a policy: ， in, The parameters are Strategy at the time A collection of Based on the Pareto frontier, elimination strategy: ， in, The process of comparing the scores of candidate strategies for the admission conditions with the existing strategies in the strategy library and eliminating the strategies with the lowest scores; The strategy you want to introduce; When satisfied: ， in, It is the lowest score among the original strategies. When the strategy is lower than the minimum score, it is eliminated.

8. The intelligent lighting energy-saving method based on multimodal data fusion according to claim 1 is characterized in that: The fitness function is expressed as: ， in, , ; is the user comfort index; is the total energy consumption when executing the current strategy; is the minimum total energy consumption in the strategy set; is the maximum total energy consumption in the strategy set; The number of violations of this policy; is the total number of violations; Score the comfort level of the i-th person among N people when applying the strategy.

9. The intelligent lighting energy-saving method based on multimodal data fusion according to claim 1, characterized in that: Based on each of the sub-item weights and the corresponding feedback data, a feedback-weight mapping model for obtaining a predicted score is established; based on the feedback-weight mapping model, a loss function gradient is established; based on the loss function gradient, the sub-item weights are iteratively calculated to obtain the sub-item weights after iterative calculation; based on the iteratively calculated sub-item weights, according to the fitness function, the corresponding strategy comprehensive score is recalculated and compared with the score of the current strategy; if the score is better than the score of the current strategy, the sub-item weights of the current strategy are dynamically updated, including: Establish a feedback-weight mapping model: ， in, Score the predictions; The weight data of each item in the current execution strategy for this trajectory; is the energy efficiency sub-item weight; The weight of the safety sub-item; The weight of the user's subjective comfort items; For feedback data; is the theoretical energy efficiency upper limit for this location and time period; is the safety risk factor based on the environment; comfort preferences of user groups; Establish the loss function gradient: ， in, , in, is the weight of each sub-indicator; 、 is a constant; , Indicates that the function is a gradient operator function; Actual ratings for user feedback; is the prediction score, calculated by the formula; It is a constant, the minimum value of the item weight set by the system; is the current item weight.

10. A smart lighting energy-saving system based on multimodal data fusion, characterized in that: include: A data acquisition module, configured to periodically acquire a decision trajectory dataset; wherein the decision trajectory dataset includes a state space, an action space, and rewards; the state space includes multimodal environmental data such as light intensity, vehicle flow, pedestrian density, weather information, and positioning information; the action space includes lighting actions such as brightness adjustment and on / off states; and the rewards include energy saving effects, comfort, and safety; An experience pool update module is used to add the decision trajectory dataset of the first cycle to the experience pool as a sample; in subsequent cycles, the experience pool of the previous cycle and the newly collected decision trajectory dataset of the current cycle are merged according to a set ratio and filtered according to priority weight; The strategy training module is used to perform strategy training using samples in the experience pool, establish an objective function for the best candidate strategy, and train the candidate strategy based on the objective function. The safety constraint coefficient is introduced into the objective function, and the PPO loss function is introduced during the training process to prevent the strategy parameters from being updated too much through probability ratio clipping; A strategy comprehensive score acquisition module is used to evaluate the candidate strategy under a set of test scenarios to obtain multi-dimensional evaluation indicators including energy efficiency, safety, and user subjective comfort, and to obtain a strategy comprehensive score based on the multi-dimensional evaluation indicators and corresponding sub-item weights; a strategy library update module, configured to add a strategy to the strategy library in response to the strategy's comprehensive score meeting the admission criteria; and when the strategy capacity in the strategy library reaches a predetermined value, compare the scores of candidate strategies that meet the admission criteria with the existing strategies in the strategy library based on the Pareto front, and eliminate the strategy with the lowest comprehensive score; The sub-item weight update module is used to construct the fitness function of the strategy according to the weight of the score of each indicator in the multi-dimensional evaluation indicator; based on each sub-item weight and the corresponding feedback data, a feedback-weight mapping model for obtaining the predicted score is established; based on the feedback-weight mapping model, a loss function gradient is established, and based on the loss function gradient, the sub-item weight is iteratively calculated to obtain the sub-item weight after iterative calculation; based on the sub-item weight after iterative calculation, according to the fitness function, the corresponding comprehensive score of the strategy is recalculated, and compared with the score of the current strategy. If it is better than the score of the current strategy, the sub-item weight of the current strategy is dynamically updated.

Images