AI street lamp adaptive control method and system based on multi-modal sensor fusion
The AI-based adaptive control method for streetlights, which integrates multimodal sensors, calculates connection weights and Moran's index using power and environmental perception data. This enables real-time monitoring and accurate detection of streetlight faults, solving the problem of misjudgment caused by environmental interference in traditional methods and improving the stability of streetlight operation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-06
- Publication Date
- 2026-04-28
AI Technical Summary
Traditional single-sensor fault detection methods are susceptible to environmental interference, leading to inaccurate fault detection in smart streetlights and increasing ineffective maintenance.
An AI-based adaptive control method for streetlights employs multimodal sensor fusion. By collecting streetlight power and environmental perception data, calculating the connection weight matrix and Moran's index, a target cluster is generated. The method monitors the fluctuation of the Moran's index sequence in real time, identifies faults, and issues early warnings.
This improved the accuracy of street light fault detection, reduced ineffective maintenance, and ensured the stable operation of street lights.
Smart Images
Figure CN121935783A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of adaptive street light control, and in particular to an AI-based adaptive street light control method and system based on multimodal sensor fusion. Background Technology
[0002] With the development of artificial intelligence, streetlights are becoming increasingly intelligent and smart. The brightness of streetlights can be adjusted based on whether someone is passing by, and the voltage and current of smart streetlights will change accordingly. Traditional fault detection often relies on single sensors (such as current sensors monitoring the circuit and light sensors determining the lighting status), which are easily affected by environmental interference: for example, strong light reflection on rainy days may cause light sensors to misjudge "daytime," causing the streetlights to turn off prematurely; voltage fluctuations may cause current sensors to falsely report "circuit faults," increasing unnecessary maintenance. These traditional detection methods are no longer suitable for fault monitoring of smart streetlights, and since fault detection is crucial for streetlight control, multimodal AI-based streetlight fault detection is an urgent problem to be solved. Summary of the Invention
[0003] In order to accurately detect and control street light malfunctions in a timely manner, this application provides an AI-based adaptive control method and system for street lights based on multimodal sensor fusion.
[0004] In a first aspect, this application provides an AI street light adaptive control method based on multimodal sensor fusion, comprising:
[0005] Collect street light power data and environmental sensing data;
[0006] The period of the street light power data is calculated and divided into periods to generate multiple target periods.
[0007] Calculate the connection weights between the power data of any two streetlights in each target period based on the environmental perception data, and generate the connection weight matrix between all streetlights.
[0008] Based on the connection weight matrix and the power data of the streetlights at each time step, the Moran index corresponding to each time step is calculated.
[0009] The Moran index is divided based on a preset division standard to generate corresponding target clusters;
[0010] Collect real-time street light power data and real-time environmental perception data, and calculate the real-time Moran index sequence of the real-time street light power data based on the real-time environmental perception data according to the target cluster where the real-time street light power data is located;
[0011] If the fluctuation value of the real-time Moran index sequence is greater than the preset fluctuation threshold or the Moran index is less than the preset threshold, a fault is determined and an early warning is issued for maintenance.
[0012] Optionally, the environmental perception data collected includes:
[0013] Collect image data within the sensing range of the streetlights;
[0014] The image data is identified and detected based on the image detection model to obtain the traffic flow, vehicle speed, and number of pedestrians passing the streetlights as environmental perception data.
[0015] Optionally, the calculation of the connection weights includes:
[0016]
[0017] in, This represents the connection weight between the x-th street light and the y-th street light. Represents an exponential function. This represents the mean of the environmental perception data for the x-th street light at the ith time. Let represent the mean of the environmental perception data for the y-th streetlight at the ith time. Represented as a constant, the distance between the environmental perception data of the x-th street light and the y-th street light is mapped to the connection weight of the two street lights through a Gaussian kernel function.
[0018] Optionally, the calculation of the connection weights further includes:
[0019]
[0020] in, This represents the connection weight between the x-th street light and the y-th street light. This represents a vector composed of the mean values of the environmental perception data for the x-th street light. Let represent the vector composed of the mean values of the environmental perception data for the y-th street light. Let represent the magnitude of the vector composed of the mean values of the environmental perception data of the x-th street light. Let represent the magnitude of the vector composed of the mean values of the environmental perception data of the y-th street light. This represents the inner product of two vectors.
[0021] Optionally, when calculating the connection weights, the transfer entropy of the i-th environmental perception data of the x-th streetlight to the i-th environmental perception data of the y-th streetlight is calculated, including:
[0022]
[0023] in, Let represent the transfer entropy of the i-th environmental perception data of the x-th street light to the i-th environmental perception data of the y-th street light. This represents the value of the i-th environmental perception data for the y-th streetlight at time t. This indicates that the environmental perception data of the y-th street light before time t is... The value at each moment, This indicates that the environmental perception data of the x-th street light and the ith street light before time t... The value at each moment, This represents conditional entropy.
[0024] Optionally, the connection weight between streetlight x and streetlight y is calculated based on the transfer entropy of the different environmental perception data of streetlight x and streetlight y. The calculation formula is as follows:
[0025]
[0026] in, This represents the connection weight between the x-th street light and the y-th street light. Represents the normalization function. Let represent the transfer entropy of the i-th environmental perception data of the x-th street light to the i-th environmental perception data of the y-th street light.
[0027] Optional,
[0028] in, This represents the connection weight between the x-th street light and the y-th street light. This represents the physical distance between the x-th street light and the y-th street light. Represents the normalization function. Let represent the transfer entropy of the i-th environmental perception data of the x-th street light to the i-th environmental perception data of the y-th street light.
[0029] Secondly, this application provides an AI street light adaptive control system based on multimodal sensor fusion, comprising:
[0030] The data acquisition module is used to collect street light power data and environmental sensing data.
[0031] The period division module is used to calculate the period of the street light power data and divide it into periods to generate multiple target periods.
[0032] The weight calculation module is used to calculate the connection weight between the power data of each street lamp in each target period for any two street lamps based on the environmental perception data, and to generate the connection weight matrix between all street lamps.
[0033] The index calculation module is used to calculate the Moran index corresponding to each time step based on the connection weight matrix and the power data of the street lamp at each time step.
[0034] The sequence partitioning module is used to partition the Moran index based on a preset partitioning standard to generate corresponding target clusters;
[0035] The real-time computing module is used to collect real-time street light power data and real-time environmental perception data, and to calculate the real-time Moran index sequence of the real-time street light power data based on the real-time environmental perception data according to the target cluster where the real-time street light power data is located.
[0036] The fault determination module is used to determine that a fault has occurred and issue an early warning for maintenance if the fluctuation value of the real-time Moran index sequence is greater than a preset fluctuation threshold or the Moran index is less than the preset threshold.
[0037] This application has the following technical advantages:
[0038] Streetlights are categorized using historical data. The spatial similarity between streetlights within a cluster is used to determine if any streetlights are faulty. During categorization, a connection matrix is constructed based on the similarity of environmental perception data between streetlights. The spatial similarity of each cluster is calculated based on the connection matrix and the electrical parameters of the streetlights. The categorization is complete when the stability of the spatial similarity of each cluster and the overall spatial similarity meets the requirements. Based on the categorization results, the real-time spatial similarity sequence of each cluster is calculated. A fault alarm is issued when the spatial similarity fluctuation exceeds a set threshold or falls below a set threshold, prompting staff to conduct maintenance. Attached Figure Description
[0039] Figure 1 This is a flowchart illustrating steps S1-S7 of the AI street light adaptive control method based on multimodal sensor fusion in this application. Detailed Implementation
[0040] This application discloses an AI street light adaptive control method based on multimodal sensor fusion, referring to... Figure 1 ,include:
[0041] S1: Collect street light power data and environmental perception data.
[0042] In one embodiment of this application, the street light power data includes various data related to the street light power system. Sensors can be used to collect power data such as voltage, current, and power of the street light, and cameras installed on the street light can be used to collect images within the sensing range of the street light.
[0043] The perception range refers to the area within which the camera can scan and monitor the captured images. The images can be input into an image detection model to obtain data such as traffic flow, vehicle speed, and number of pedestrians passing by the streetlight. The image detection model is deepsort, which can count vehicles and pedestrians. The vehicle speed is obtained by mapping the distance the vehicle moves in the image to the ratio of the actual distance and time.
[0044] S2: Calculate the period of street light power data and divide it into periods to generate multiple target periods.
[0045] In one embodiment of this application, the autocorrelation function of the street light data is calculated, and the period of the street light data is obtained when the autocorrelation function is at its maximum. Alternatively, a Fourier transform is performed on the street light data, and the reciprocal of the dominant frequency is the period of the street light data.
[0046] As an example, the autocorrelation function R(τ) is a measure of the similarity between a time series x(t) and itself at different time delays τ. For different time delays τ, the autocorrelation value R(τ) is calculated. The first significant peak in the autocorrelation function R(τ) corresponds to the period of the street light data. Periodicity can be visually observed by plotting the autocorrelation function. The autocorrelation function of periodic data typically shows significant peaks at periodic intervals.
[0047] S3: Calculate the connection weights between the power data of any two streetlights in each target period based on the environmental perception data, and generate the connection weight matrix between all streetlights.
[0048] In one embodiment of this application, the Moran's index of each street light data under each target period is calculated based on environmental perception. In street light fault detection, the illuminance of smart street lights changes with the environmental data they perceive; that is, voltage, current, power, etc., change according to the environmental perception data. Traditional street light fault detection models may identify this change as a fault. To improve the accuracy of street light fault identification, the Moran's index between street lights is calculated based on the environmental perception data of the street lights. The specific calculation process is as follows:
[0049] First, calculate the adjacency matrix between streetlights based on environmental perception data. For any two streetlights, the connection weight between them is calculated using the following formula: First, calculate the mean of the environmental perception data for each streetlight within one period, and then calculate the connection weight based on the mean of the environmental perception data:
[0050]
[0051] in, This represents the connection weight between the x-th street light and the y-th street light. Represents an exponential function. This represents the mean of the environmental perception data for the x-th street light at the ith time. Let represent the mean of the environmental perception data for the y-th streetlight at the ith time. This is represented as a constant. The distance between the environmental perception data of two streetlights is mapped to the connection weight between the two streetlights using a Gaussian kernel function. The closer the environmental perception data of two streetlights are, the greater the connection weight between them.
[0052] In addition, another method for calculating connection weights is as follows:
[0053]
[0054] in, This represents the connection weight between the x-th street light and the y-th street light. This represents a vector composed of the mean values of the environmental perception data for the x-th street light. Let represent the vector composed of the mean values of the environmental perception data for the y-th street light. Let represent the magnitude of the vector composed of the mean values of the environmental perception data of the x-th street light. Let represent the magnitude of the vector composed of the mean values of the environmental perception data of the y-th street light. This represents the inner product of two vectors. The greater the similarity of the environmental perception data between two streetlights, the greater the weight of the connection between them.
[0055] It should be noted that the two methods for calculating the connection weight of streetlights mentioned above consider the similarity of environmental perception data between two streetlights, but do not consider the causal relationship between their environmental perception data. In actual road scenarios, the environmental perception data between streetlights is related to the direction of traffic flow. The environmental perception data of streetlights upstream in the traffic flow will precede the environmental perception data of streetlights downstream in the traffic flow. Therefore, the causal relationship of environmental data can be considered when calculating the connection weight between the two. The specific calculation formula is as follows:
[0056]
[0057] in, Let represent the transfer entropy of the i-th environmental perception data of the x-th street light to the i-th environmental perception data of the y-th street light. This represents the value of the i-th environmental perception data for the y-th streetlight at time t. This indicates that the environmental perception data of the y-th street light before time t is... The value at each moment, This indicates that the environmental perception data of the x-th street light and the ith street light before time t... The value at each moment, This represents conditional entropy. Transition entropy reflects the increase in determinism of the environmental perception data of streetlight y after the introduction of environmental perception data of streetlight x. The larger the value, the greater the influence of streetlight x on the environmental perception data of streetlight y.
[0058] Then, the connection weight between the two streetlights is calculated based on the transfer entropy of their different environmental perception data. The specific calculation formula is as follows:
[0059]
[0060] in, This represents the connection weight between the x-th street light and the y-th street light. Represents the normalization function. Let represent the transfer entropy of the i-th environmental perception data of the x-th street light to the i-th environmental perception data of the y-th street light.
[0061] It should be noted that the greater the transfer entropy of streetlight x to streetlight y, the greater the influence of streetlight x on streetlight y, and the greater the connection weight of streetlight x to streetlight y. Transfer entropy is directional; generally, the transfer entropy of environmental perception data from upstream streetlights to downstream streetlights in the traffic flow is larger, resulting in heavier connection weights; conversely, the transfer entropy of environmental perception data from downstream streetlights to upstream streetlights is smaller, resulting in lighter connection weights. Compared to the connection weights mentioned above that do not consider direction, this method involves a larger computational load and can be used when the number of streetlights is less than 500.
[0062] In another embodiment of this application, the connection weight between two streetlights is related not only to environmental perception data but also to the distance between the streetlights. The specific calculation formula is as follows:
[0063]
[0064] in, This represents the connection weight between the x-th street light and the y-th street light. This represents the physical distance between the x-th street light and the y-th street light. Represents the normalization function. Let represent the transfer entropy of the i-th environmental perception data of the x-th street light to the i-th environmental perception data of the y-th street light.
[0065] S4: Calculate the Moran index corresponding to each time step based on the connection weight matrix and the power data of the streetlights at each time step.
[0066] In one embodiment of this application, a connection weight matrix between all streetlights is obtained based on the connection weights between the streetlights, and the Moran index at each time moment is calculated based on the connection weight matrix and the electrical data of the streetlights at each time moment.
[0067] Moran's index (greater than 0) indicates positive spatial autocorrelation, suggesting spatial clustering of eigenvalues. High values cluster near high values (hot spots), and low values cluster near low values (cold spots), which is the expected pattern of normal traffic flow (continuous traffic flow). A Moran's index close to 0 indicates no spatial autocorrelation, with eigenvalues randomly distributed spatially, potentially indicating anomalies. A Moran's index less than 0 indicates negative spatial autocorrelation, suggesting spatial dispersion of eigenvalues. High values are surrounded by low values, and vice versa, which is usually a highly unusual signal.
[0068] S5: Divide the Moran index based on the preset division criteria to generate the corresponding target cluster.
[0069] It should be noted that the preset division standard is a pre-set standard for dividing streetlights into different clusters according to the Moran index.
[0070] In one embodiment of this application, the optimal partitioning result is obtained when the Moran index of each cluster is large and the overall Moran index is stable. Overall Moran index stability refers to the variance of all Moran indices. A large Moran index means that the Moran index of each cluster is greater than a preset partitioning standard. Specifically, the partitioning evaluation, i.e., the preset partitioning standard, includes that when the Moran index of each cluster is greater than 0.8 and the variance of the Moran index of all clusters is minimized, the corresponding target cluster is generated. Thus, the partitioning result of the streetlights is obtained.
[0071] S6: Collect real-time street light power data and real-time environmental perception data, and calculate the real-time Moran index sequence of real-time street light power data based on real-time environmental perception data according to the target cluster where the real-time street light power data is located.
[0072] S7: If the fluctuation value of the real-time Moran index sequence is greater than the preset fluctuation threshold or the Moran index is less than the preset threshold, a fault is determined and an early warning is issued for maintenance.
[0073] It should be noted that each data point corresponds to a Moran's index. To obtain a real-time Moran's index sequence, streetlight data from multiple real-time moments can be collected. The preset fluctuation threshold is a pre-set standard for determining a streetlight malfunction when the fluctuation value of the real-time Moran's index sequence exceeds a certain value; conversely, the preset threshold is a pre-set standard for determining a streetlight malfunction when the Moran's index falls below a certain value.
[0074] In one embodiment of this application, real-time street light electrical data is collected. Based on the aforementioned clusters of street lights, the Moran index of each cluster is calculated in real time according to the connection matrix and electrical parameters of all street lights in the cluster. Based on the changes in the Moran index, when the Moran index of a certain cluster fluctuates significantly between different times, that is, when the difference between the Moran index of the later time and the Moran index of the previous time is greater than a preset fluctuation threshold, it is considered that the street lights in the cluster have malfunctioned. The preset fluctuation threshold is 0.2. Or when the Moran index is negative, that is, when the preset threshold is 0.3, it is considered that the street lights in the cluster have malfunctioned, an early warning is issued, and maintenance is carried out.
[0075] As an example, in the real-time Moran's index sequence (0.9, 0.85, 0.8, 0.86, 0.6, 0.6), if the difference between the 4th and 5th times in the sequence is greater than 0.2, it is considered that the fluctuation exceeds the set threshold. Similarly, if the Moran's index is less than 0.3 at any time in the Moran's index sequence, it is considered that the street light is malfunctioning.
[0076] This application also discloses an AI street light adaptive control system based on multimodal sensor fusion, including:
[0077] The data acquisition module is used to collect street light power data and environmental sensing data.
[0078] The period division module is used to calculate the period of street light power data and divide it into periods to generate multiple target periods.
[0079] The weight calculation module is used to calculate the connection weight between the power data of each street lamp in each target period for any two street lamps based on the environmental perception data, and to generate the corresponding connection weight matrix between all street lamps.
[0080] The index calculation module is used to calculate the Moran index corresponding to each time step based on the connection weight matrix and the power data of the street lamp at each time step.
[0081] The sequence partitioning module is used to partition the Moran index based on a preset partitioning standard and generate the corresponding target clusters;
[0082] The real-time computing module is used to collect real-time street light power data and real-time environmental perception data, and to calculate the real-time Moran index sequence of real-time street light power data based on real-time environmental perception data according to the target cluster where the real-time street light power data is located.
[0083] The fault determination module determines that a fault has occurred and issues an early warning for maintenance if the fluctuation value of the real-time Moran index sequence is greater than the preset fluctuation threshold or the Moran index is less than the preset threshold.
[0084] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.
Claims
1. An AI-based adaptive control method for streetlights based on multimodal sensor fusion, characterized in that, include: Collect street light power data and environmental sensing data; The period of the street light power data is calculated and divided into periods to generate multiple target periods. Calculate the connection weights between the power data of any two streetlights in each target period based on the environmental perception data, and generate the connection weight matrix between all streetlights. Based on the connection weight matrix and the power data of the streetlights at each time step, the Moran index corresponding to each time step is calculated. The Moran index is divided based on a preset division standard to generate corresponding target clusters; Collect real-time street light power data and real-time environmental perception data, and calculate the real-time Moran index sequence of the real-time street light power data based on the real-time environmental perception data according to the target cluster where the real-time street light power data is located; If the fluctuation value of the real-time Moran index sequence is greater than the preset fluctuation threshold or the Moran index is less than the preset threshold, a fault is determined and an early warning is issued for maintenance.
2. The AI street light adaptive control method based on multimodal sensor fusion according to claim 1, characterized in that, The environmental perception data collected includes: Collect image data within the sensing range of the streetlights; The image data is identified and detected based on the image detection model to obtain the traffic flow, vehicle speed, and number of pedestrians passing the streetlights as environmental perception data.
3. The AI street light adaptive control method based on multimodal sensor fusion according to claim 1, characterized in that, The calculation of the connection weights includes: ； in, This represents the connection weight between the x-th street light and the y-th street light. Represents an exponential function. This represents the mean of the environmental perception data for the x-th street light at the ith time. Let represent the mean of the environmental perception data for the y-th streetlight at the ith time. Represented as a constant, the distance between the environmental perception data of the x-th street light and the y-th street light is mapped to the connection weight of the two street lights through a Gaussian kernel function.
4. The AI street light adaptive control method based on multimodal sensor fusion according to claim 1, characterized in that, The calculation of the connection weights also includes: ； in, This represents the connection weight between the x-th street light and the y-th street light. This represents a vector composed of the mean values of the environmental perception data for the x-th street light. Let represent the vector composed of the mean values of the environmental perception data for the y-th street light. Let represent the magnitude of the vector composed of the mean values of the environmental perception data of the x-th street light. Let represent the magnitude of the vector composed of the mean values of the environmental perception data of the y-th street light. This represents the inner product of two vectors.
5. The AI street light adaptive control method based on multimodal sensor fusion according to claim 1, characterized in that, When calculating the connection weights, the transfer entropy of the i-th environmental perception data of the x-th streetlight to the i-th environmental perception data of the y-th streetlight is calculated, including: ； in, Let represent the transfer entropy of the i-th environmental perception data of the x-th street light to the i-th environmental perception data of the y-th street light. This represents the value of the i-th environmental perception data for the y-th streetlight at time t. This indicates that the environmental perception data of the y-th street light before time t is... The value at each moment, This indicates that the environmental perception data of the x-th street light and the ith street light before time t... The value at each moment, This represents conditional entropy.
6. The AI street light adaptive control method based on multimodal sensor fusion according to claim 5, characterized in that, Also includes: The connection weight between streetlight x and streetlight y is calculated based on the transfer entropy of different environmental perception data from streetlight x and streetlight y. The calculation formula is as follows: ； in, This represents the connection weight between the x-th street light and the y-th street light. Represents the normalization function. Let represent the transfer entropy of the i-th environmental perception data of the x-th street light to the i-th environmental perception data of the y-th street light.
7. The AI street light adaptive control method based on multimodal sensor fusion according to claim 5, characterized in that, Also includes: ； in, This represents the connection weight between the x-th street light and the y-th street light. This represents the physical distance between the x-th street light and the y-th street light. Represents the normalization function. Let represent the transfer entropy of the i-th environmental perception data of the x-th street light to the i-th environmental perception data of the y-th street light.
8. The AI street light adaptive control system based on multimodal sensor fusion according to claim 1, characterized in that, include: The data acquisition module is used to collect street light power data and environmental sensing data. The period division module is used to calculate the period of the street light power data and divide it into periods to generate multiple target periods. The weight calculation module is used to calculate the connection weight between the power data of each street lamp in each target period for any two street lamps based on the environmental perception data, and to generate the connection weight matrix between all street lamps. The index calculation module is used to calculate the Moran index corresponding to each time step based on the connection weight matrix and the power data of the street lamp at each time step. The sequence partitioning module is used to partition the Moran index based on a preset partitioning standard to generate corresponding target clusters; The real-time computing module is used to collect real-time street light power data and real-time environmental perception data, and to calculate the real-time Moran index sequence of the real-time street light power data based on the real-time environmental perception data according to the target cluster where the real-time street light power data is located. The fault determination module is used to determine that a fault has occurred and issue an early warning for maintenance if the fluctuation value of the real-time Moran index sequence is greater than a preset fluctuation threshold or the Moran index is less than the preset threshold.