A flying robot control method and system for intelligent lighting system
Patent Information
- Application Number
- CN202610144507.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-02-02
- Publication Date
- 2026-10-16
- Estimated Expiration
- 2046-02-02
AI Technical Summary
然而,现有的无人机控制方法无法智能、灵活且精准地应对复杂和多变的巡检任务需求
[0017]The technical solution of this invention provides a method for controlling a flying robot in an intelligent lighting system, comprising: spatially scanning lighting facilities in a target area using a multispectral camera and lidar mounted on the flying robot to acquire three-dimensional point cloud data containing the position, attitude, and spectral characteristics of the lighting facilities; constructing a digital twin model of the lighting facilities based on the three-dimensional point cloud data, wherein the digital twin model includes the spatial coordinates, physical parameters, and real-time operating status of each lighting facility; optimizing the inspection path of the flying robot using a quantum annealing algorithm based on the spatial distribution of the lighting facilities in the digital twin model to generate an adaptive inspection trajectory that covers all lighting facilities and has the lowest energy consumption, wherein the quantum annealing algorithm jointly optimizes flight distance, obstacle avoidance, and lighting facility priority as constraints; controlling the flying robot to fly along the adaptive inspection trajectory, and using the onboard thermal imaging system... Sensors and spectral analyzers perform non-contact detection on various lighting facilities, acquiring multi-dimensional detection data in real time, including thermal distribution images, spectral characteristic curves, and electromagnetic radiation intensity. This multi-dimensional detection data is then encrypted and uploaded to cloud storage via a blockchain network. A deep convolutional neural network is used to fuse and analyze this multi-dimensional detection data, identifying abnormal patterns in the lighting facilities, including abnormal light decay, overheating faults, and flicker defects. Based on the severity of these abnormal patterns, intelligent decision-making instructions are autonomously generated, including fault location, repair priorities, and suggested maintenance solutions. These intelligent decision-making instructions are synchronized to the IoT control platform of the intelligent lighting system, triggering remote dimming, zone control, or fault isolation operations for the lighting facilities. Simultaneously, based on changes in ambient light and user feedback, the next inspection cycle and key monitoring areas of the flying robot are dynamically adjusted, achieving closed-loop optimization management of the lighting system. By utilizing technologies such as flying robots, digital twins, quantum optimization algorithms, and blockchain data management, the intelligent lighting system achieves fully automated, high-precision inspection and intelligent control. This improves detection efficiency and accuracy, enables early warning and precise location of lighting facility faults, reduces operational energy consumption through adaptive path planning, and ensures the traceability and tamper-proof nature of detection data through blockchain technology. Overall, this enhances the reliability, energy efficiency, and intelligent management level of the intelligent lighting system.
Smart Images

Figure CN121957091B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of robotics, and more specifically to a control method and system for a flying robot used in an intelligent lighting system. Background Technology
[0002] Low-altitude unmanned aerial vehicles (UAVs) have become a highly efficient tool in modern infrastructure monitoring and production management. During inspections, UAVs may encounter dynamic environmental changes and various anomalies. However, existing UAV control methods are unable to intelligently, flexibly, and accurately address the complex and ever-changing needs of inspection tasks. Summary of the Invention
[0003] Based on the above-mentioned problems, this invention proposes a flying robot control method and system for intelligent lighting systems. Through this invention, fully automated, high-precision inspection and intelligent management of intelligent lighting systems are realized, which can improve detection efficiency and accuracy, realize early warning and accurate location of lighting facility faults, and improve the overall reliability, energy saving and intelligent management level of intelligent lighting systems.
[0004] In view of this, one aspect of the present invention proposes a method for controlling a flying robot for an intelligent lighting system, comprising: The flying robot uses a multispectral camera and lidar to perform spatial scanning of the lighting facilities in the target area, and obtains three-dimensional point cloud data containing the location, attitude and spectral characteristics of the lighting facilities. Based on the three-dimensional point cloud data, a digital twin model of the lighting facilities is constructed. The digital twin model includes the spatial coordinates, physical parameters and real-time operating status of each lighting facility. Based on the spatial distribution of lighting facilities in the digital twin model, the quantum annealing algorithm is used to optimize the inspection path of the flying robot, generating an adaptive inspection trajectory that covers all lighting facilities and has the lowest energy consumption. The quantum annealing algorithm uses flight distance, obstacle avoidance and lighting facility priority as constraints for joint optimization. The flight robot is controlled to fly along the adaptive inspection trajectory. It performs non-contact inspection of each lighting facility using the thermal imaging sensor and spectral analyzer it is equipped with. It collects multi-dimensional detection data in real time, including thermal distribution images, spectral characteristic curves and electromagnetic radiation intensity, and uploads the multi-dimensional detection data to the cloud for storage via a blockchain network. The multidimensional detection data is fused and analyzed using a deep convolutional neural network to identify abnormal patterns of lighting facilities, including abnormal light decay, overheating faults, and flicker defects. Based on the severity of the abnormal patterns, intelligent decision-making instructions are autonomously generated, including fault location, repair priority, and suggested maintenance solutions. The intelligent decision-making instructions are synchronized to the IoT control platform of the intelligent lighting system, triggering remote dimming, zone control, or fault isolation operations of the lighting facilities. At the same time, based on changes in ambient light and user feedback, the next round of inspection cycle and key monitoring areas of the flying robot are dynamically adjusted to achieve closed-loop optimization management of the lighting system.
[0005] Optionally, a multispectral camera and lidar mounted on a flying robot are used to perform a spatial scan of the lighting facilities in the target area to acquire three-dimensional point cloud data containing the location, attitude, and spectral characteristics of the lighting facilities. A digital twin model of the lighting facilities is then constructed based on the three-dimensional point cloud data. This digital twin model includes the spatial coordinates, physical parameters, and real-time operating status of each lighting facility. The steps include: The flight robot is controlled to hover or fly at low speed to the predetermined detection position, and a multispectral camera and lidar are activated simultaneously to collect data. The multispectral camera captures spectral images of the lighting facility containing visible light, near-infrared and ultraviolet bands, and the lidar emits pulse beams and receives reflected signals to obtain the three-dimensional point cloud coordinates of the surface of the lighting facility. At the same time, the timestamp of each data point and the real-time pose information of the flight robot are recorded. Spatial registration is performed between the 3D point cloud data acquired by the lidar and the spectral images of the lighting facilities collected by the multispectral camera. The correspondence between the point cloud coordinates and the image pixels is established through the feature point matching algorithm. The spectral feature information is mapped to the corresponding 3D point cloud points to form a fused point cloud dataset containing spatial location and spectral attributes. A deep learning-based semantic segmentation algorithm is applied to the fused point cloud dataset to identify and distinguish different objects such as the lighting facility itself, supporting structure, cable connection and surrounding environment. Point cloud clusters belonging to the lighting facility are extracted, and the type, installation posture and physical size parameters of the lighting facility are automatically identified based on the geometric features of the point cloud clusters. The spectral feature data of the extracted point cloud clusters of lighting facilities are analyzed. The luminous intensity, color temperature distribution and light decay of the lighting facilities are determined by spectral curve analysis. The working status of the lighting facilities is identified by combining the temperature information collected by the thermal imaging sensor, including normal operation, abnormal heating, light source aging and power outage fault. The extracted spatial coordinates, physical parameters, and operational status information are integrated into a predefined digital twin model framework. A digital twin instance containing a unique identifier, three-dimensional geometric model, material properties, installation location, operating parameters, and real-time status is created for each lighting facility. Spatial topological relationships and power supply network connections between lighting facilities are also established to form a complete digital twin model of the lighting system.
[0006] Optionally, based on the spatial distribution of lighting facilities in the digital twin model, a quantum annealing algorithm is used to optimize the inspection path of the flying robot, generating an adaptive inspection trajectory that covers all lighting facilities and has the lowest energy consumption. The quantum annealing algorithm includes the following steps for joint optimization: flight distance, obstacle avoidance, and lighting facility priority are used as constraints. The spatial coordinate information of all lighting facilities is extracted from the digital twin model. The center position of each lighting facility is used as a task node. The straight-line flight distance between any two task nodes is calculated as the edge weight. At the same time, priority weights are assigned to each task node according to the operating status and historical failure frequency of the lighting facilities recorded in the digital twin model. A directed inspection task graph containing node priority and edge weights is constructed. Extract the three-dimensional spatial location and boundary information of obstacles in the digital twin model, set a safety buffer for each obstacle, divide the three-dimensional space into a flyable area and a no-fly zone, detect whether each edge in the inspection task map crosses the no-fly zone, generate detour constraints for path edges that cross the no-fly zone, and ensure that candidate trajectories meet spatial safety requirements. The inspection path optimization problem is transformed into an extended form of the traveling salesman problem. A qubit variable is defined to represent the order and path selection of the flying robot's access to each lighting facility. The flight distance minimization target, obstacle avoidance constraints, and priority access requirements of priority lighting facilities are encoded as energy terms of quantum Hamiltonians. The low-energy state corresponds to the superior trajectory scheme with short flight distance, obstacle avoidance, and priority access to high-priority facilities. The initial temperature parameters and annealing rate of the quantum annealing algorithm are set. By simulating the quantum tunneling effect, the system gradually evolves from a high-energy state to a low-energy state. During the annealing process, the system is allowed to accept a state transition with a certain probability of temporarily increasing energy in order to escape the local optimum. After multiple rounds of annealing iterations, the system converges to the stable state with the lowest energy. The quantum bit configuration corresponding to this state represents the optimized inspection path access sequence. The discrete task node access sequence output by the quantum annealing algorithm is converted into a continuous three-dimensional space track. The track is smoothed by Bézier curve or spline interpolation algorithm to meet the dynamic constraints of the flying robot. The generated track is verified to meet the physical limitations such as maximum flight speed, maximum acceleration and minimum turning radius. Track segments that do not meet the constraints are locally adjusted and optimized, and finally an adaptive inspection track is generated.
[0007] Optionally, the steps of controlling the flying robot to fly along the adaptive inspection trajectory, performing non-contact inspection of each lighting facility using onboard thermal imaging sensors and a spectral analyzer, and real-time acquiring multi-dimensional detection data including thermal distribution images, spectral characteristic curves, and electromagnetic radiation intensity, and encrypting and uploading the multi-dimensional detection data to cloud storage via a blockchain network, include: The instructions corresponding to the adaptive inspection trajectory are loaded into the flight control system of the flying robot, and the autonomous navigation mode is activated to make the flying robot fly along the predetermined trajectory. When it approaches the target lighting facility, the flight attitude and hovering position of the flying robot are automatically adjusted according to the spatial pose information of the lighting facility stored in the digital twin model, so that the detection optical axis of the thermal imaging sensor and the spectral analyzer are aligned with the key detection part of the lighting facility, and the best observation angle and detection distance are maintained. After the flying robot reaches the preset stable detection posture, the thermal imaging sensor, the spectrum analyzer and the electromagnetic radiation detector are triggered simultaneously to collect data. The thermal imaging sensor captures a two-dimensional temperature distribution image of the surface of the lighting facility to identify abnormal hot spots. The spectrum analyzer measures the spectral characteristic curve emitted by the lighting facility to evaluate the performance of the light source and the color temperature shift. The electromagnetic radiation detector detects the electromagnetic field strength around the lighting facility to determine the electrical safety status. Each set of detection data is automatically labeled with the collection timestamp, facility identifier and spatial location information. The system performs real-time preprocessing on the acquired raw multidimensional detection data, including noise reduction filtering of thermal distribution images, baseline correction of spectral curves, and interference suppression of electromagnetic radiation signals. It judges whether the data quality meets the analysis requirements by evaluating image sharpness, calculating spectral signal-to-noise ratio, and checking signal integrity. It triggers a re-acquisition command for data that does not meet the quality requirements, ensuring that the acquired detection data has the reliability and accuracy required for subsequent intelligent analysis. The quality-verified multidimensional detection data, along with metadata information, is encapsulated into a standardized data packet. An asymmetric encryption algorithm is used to encrypt the data packet to generate ciphertext data. The hash digest value of the data packet is calculated as a digital fingerprint. The ciphertext data, hash digest, and timestamp information are packaged to construct a blockchain transaction request. The integrity of the data is verified through a smart contract, and the transaction record is written into a distributed ledger to ensure the immutability and traceability of the detection data. The encrypted detection data is uploaded to the cloud storage system. A distributed storage strategy is adopted to distribute data copies across multiple physical nodes to improve data reliability. At the same time, a retrieval structure indexed by lighting facility identifier, collection time and data type is established in the cloud database. A mapping relationship between blockchain transaction hash and cloud storage address is generated to realize the rapid location of detection data and version management of historical data, providing efficient data access services for subsequent data analysis and decision support.
[0008] Optionally, the process involves using a deep convolutional neural network to fuse and analyze the multidimensional detection data, identifying abnormal patterns in the lighting facilities, including abnormal light decay, overheating faults, and flicker defects, and autonomously generating intelligent decision-making instructions that include fault location, repair priority, and suggested maintenance plans based on the severity of the abnormal patterns. This includes: Multidimensional detection data of the target lighting facility is extracted from the cloud storage system. The thermal distribution image is converted into a temperature feature matrix, the spectral feature curve is converted into a spectral intensity vector, and the electromagnetic radiation intensity is converted into a field strength feature sequence. These heterogeneous data are organized according to the time dimension, spatial dimension, and feature dimension to construct a four-dimensional feature tensor. At the same time, the historical detection data and normal operation benchmark data of the target lighting facility are loaded as a comparison reference to provide a standardized input data structure for neural network analysis. The constructed four-dimensional feature tensor is input into a pre-trained deep convolutional neural network model. The spatial pattern features of thermal distribution, the frequency domain features of spectral curves, and the temporal features of electromagnetic radiation are extracted through multiple convolutional and pooling layers. An attention mechanism is used in the middle layer of the network to weight and fuse different modal features, so that the network adaptively focuses on the feature dimension that contributes the most to anomaly recognition. The fused features are mapped to the anomaly pattern classification space through a fully connected layer, and the output includes the classification probability distribution of light decay anomaly, overheating fault, stroboscopic defect and normal state. Based on the classification probability distribution output by the deep convolutional neural network, the category with the highest probability is selected as the current operating status determination result of the lighting facility. When an abnormal state is identified, the activation response map of the intermediate layer of the network is extracted to locate the precise spatial location of the abnormal area on the lighting facility. The confidence index of the abnormal determination is calculated to evaluate the reliability of the diagnosis result. For ambiguous cases with low confidence, a manual review process is triggered or a flying robot is arranged to conduct a second fine inspection to ensure the accuracy and reliability of the abnormal identification. For each identified anomaly pattern, a comprehensive assessment is conducted based on the degree of deviation of the anomaly characteristics, duration, and scope of impact. The severity of the anomaly is divided into four levels: emergency, important, general, and minor. Emergency anomalies indicate that the lighting facilities are at risk of immediate failure or safety hazards. Important anomalies indicate that the facility performance has significantly degraded and requires immediate attention. General anomalies indicate that the facility has potential problems and can be maintained in a planned manner. Minor anomalies indicate that the facility is operating basically normally but requires continuous monitoring. Each anomaly is assigned a quantitative risk score. Based on the anomaly pattern type, severity level, and location information of lighting facilities, the system retrieves matching maintenance plan templates from a pre-built maintenance knowledge base. It automatically generates a detailed maintenance plan that includes a precise fault location description, recommended repair measures, a list of required spare parts, and estimated repair time. According to the risk score and the anomaly distribution of all lighting facilities in the area, a multi-objective optimization algorithm is used to calculate the maintenance priority ranking of each facility. The system generates intelligent decision instructions that include a priority list, repair route suggestions, and resource scheduling plans, and pushes them to operation and maintenance management personnel and on-site maintenance teams through the IoT control platform.
[0009] Optionally, the intelligent decision-making instructions are synchronized to the IoT control platform of the intelligent lighting system to trigger remote dimming, zone control, or fault isolation operations of the lighting facilities. Simultaneously, based on changes in ambient light and user feedback, the next inspection cycle and key monitoring areas of the flying robot are dynamically adjusted to achieve closed-loop optimization management of the lighting system. This includes the following steps: The intelligent decision-making instructions are synchronously transmitted to the IoT control platform of the intelligent lighting system through a secure communication protocol. The platform's decision analysis module extracts fault location information, maintenance priorities, and suggested maintenance plans. It automatically matches the corresponding control strategy according to the type of abnormal mode. Specifically, it generates dimming compensation instructions for light decay abnormalities to maintain the target illuminance, generates power reduction operation or temporary shutdown instructions for overheating faults to prevent equipment damage, generates drive circuit parameter adjustment instructions for flicker defects, and generates fault isolation instructions for emergency faults to cut off the power supply to the facility and activate backup lighting. The IoT control platform sends the parsed control commands to the smart controllers of each lighting facility through a wireless communication network. After receiving the commands, the smart controllers execute corresponding dimming operations, zone switching control, or fault isolation actions. During the execution process, they monitor the operating parameters of the lighting facilities in real time, such as voltage, current, power, and light output. The execution results and equipment status change information are sent back to the IoT control platform through the feedback channel. The platform verifies whether the control commands have been successfully executed and records the operation log. For commands that fail to be executed, an alarm is triggered and manual intervention is arranged. By continuously collecting real-time environmental data such as ambient light intensity, weather conditions, and human activity density through illuminance sensors, human presence sensors, and environmental monitoring equipment deployed in the lighting area, and collecting user demand data such as lighting effect satisfaction evaluations, illuminance adjustment requests, and fault repair information from the user feedback system, these multi-source data are aggregated into the data analysis module of the IoT control platform to construct a comprehensive dataset that includes environmental change trends and user behavior patterns. Based on the collected environmental perception data and user demand data, combined with the current health assessment results of lighting facilities and historical fault statistics, machine learning algorithms are used to analyze the spatiotemporal patterns and environmental correlations of lighting facility faults, identify time periods, areas and environmental conditions prone to faults, dynamically adjust the next round of inspection cycle of the flying robot, appropriately extend the inspection interval for areas with good health and high user satisfaction, shorten the inspection cycle and increase the inspection frequency for key monitoring areas with frequent faults or many user complaints, and update the inspection task priority list and the list of facilities to be monitored. By integrating the results of control command execution, changes in the operating status of lighting facilities, environmental data, and user feedback data to form a closed-loop feedback information flow, the lighting facility status parameters and operating history in the digital twin model are updated. This accumulated data is used to continuously train and optimize the deep convolutional neural network model to improve the accuracy of anomaly identification. At the same time, the parameter configuration of the quantum annealing algorithm is optimized to improve the trajectory planning effect. Through a data-driven continuous learning mechanism, the detection accuracy, control effect, and resource allocation efficiency of the entire intelligent lighting system are continuously improved over time, realizing the evolution from responsive management to predictive management and adaptive optimization management.
[0010] Optionally, the process of building a digital twin model of a lighting facility specifically includes: After adaptive filtering of the acquired 3D point cloud data, a multi-scale feature fusion algorithm is used to extract the key geometric features of the lighting facilities, and the spatial coordinates of each lighting facility are dynamically calibrated using the following formula for correcting the spatial position accuracy of the lighting facilities:
[0011] in, This represents the corrected three-dimensional spatial coordinate vector of the m-th lighting fixture; This represents the original measured coordinate vector of the m-th lighting fixture; The compensation coefficient represents the pose drift of the flying robot; This represents the cumulative pose deviation vector detected by the inertial measurement unit; The corrected weighting coefficients for environmental disturbances; Indicates the total number of reference lighting facilities involved in the calibration; This represents the spatial influence weight of the nth reference lighting fixture on the mth lighting fixture, which is calculated based on the inverse Euclidean distance between the two. The known precise coordinate vector represents the nth reference lighting fixture.
[0012] Optionally, the process of generating adaptive inspection tracks using the quantum annealing algorithm specifically includes: The inspection path optimization problem is mapped to a quantum Hamiltonian minimization problem, and the overall cost of the trajectory scheme is evaluated using the following energy function:
[0013] in, The total energy cost function represents the candidate inspection trajectory; The weighting coefficient represents the cost of flight distance; P represents the total number of waypoints in the inspected flight path. Let f(p) represent the Euclidean distance function between the p-th waypoint and the (p+1)-th waypoint. This represents the spatial position vector of the p-th waypoint; Weighting coefficients representing the cost of obstacle avoidance; Indicates the total number of obstacles within the detection range; This represents the threat level coefficient of the q-th obstacle; This represents the center position vector of the q-th obstacle; This represents a small positive number that prevents the denominator from being zero. Weighting coefficients representing the priority cost of lighting facilities; This represents the maintenance urgency index of the lighting facilities corresponding to the p-th waypoint; This represents the priority decay index, used to control the degree to which higher-priority facilities are accessed first.
[0014] Optionally, the step of controlling the flying robot to fly along the adaptive inspection trajectory and performing non-contact inspection of each lighting facility using onboard thermal imaging sensors and a spectral analyzer, and real-time acquisition of multi-dimensional detection data including thermal distribution images, spectral characteristic curves, and electromagnetic radiation intensity, further includes: Based on the relative pose relationship between the flying robot and the lighting facilities, the acquisition parameters of the thermal imaging sensor and the spectral analyzer are dynamically and adaptively adjusted. The following multi-sensor collaborative acquisition quality assessment formula is used to optimize the data acquisition quality:
[0015] in, This indicates the data quality assessment metrics under the current acquisition status; The weighting coefficients representing perspective factors; This indicates the angle between the sensor's optical axis and the normal to the surface of the lighting fixture; This represents the weighting coefficient of the detection distance factor; This indicates the actual distance between the flying robot and the lighting facility; Indicates the optimal detection distance of the sensor; The tolerance standard deviation represents the distance deviation. The weighting coefficients representing flight stability factors; This represents the real-time vibration velocity vector of the flying robot. This indicates the maximum permissible vibration velocity threshold.
[0016] Another aspect of the present invention provides a flight robot control system for an intelligent lighting system, for executing a flight robot control method for an intelligent lighting system, comprising: a flight robot and a server; The flying robot is configured to: use a multispectral camera and lidar mounted on the flying robot to perform spatial scanning of the lighting facilities in the target area, and acquire three-dimensional point cloud data including the position, attitude and spectral characteristics of the lighting facilities; The server is configured as follows: A digital twin model of the lighting facilities is constructed based on the three-dimensional point cloud data. The digital twin model includes the spatial coordinates, physical parameters, and real-time operating status of each lighting facility. Based on the spatial distribution of lighting facilities in the digital twin model, the quantum annealing algorithm is used to optimize the inspection path of the flying robot, generating an adaptive inspection trajectory that covers all lighting facilities and has the lowest energy consumption. The quantum annealing algorithm uses flight distance, obstacle avoidance and lighting facility priority as constraints for joint optimization. The flight robot is controlled to fly along the adaptive inspection trajectory. It performs non-contact inspection of each lighting facility using the thermal imaging sensor and spectral analyzer it is equipped with. It collects multi-dimensional detection data in real time, including thermal distribution images, spectral characteristic curves and electromagnetic radiation intensity, and uploads the multi-dimensional detection data to the cloud for storage via a blockchain network. The multidimensional detection data is fused and analyzed using a deep convolutional neural network to identify abnormal patterns of lighting facilities, including abnormal light decay, overheating faults, and flicker defects. Based on the severity of the abnormal patterns, intelligent decision-making instructions are autonomously generated, including fault location, repair priority, and suggested maintenance solutions. The intelligent decision-making instructions are synchronized to the IoT control platform of the intelligent lighting system, triggering remote dimming, zone control, or fault isolation operations of the lighting facilities. At the same time, based on changes in ambient light and user feedback, the next round of inspection cycle and key monitoring areas of the flying robot are dynamically adjusted to achieve closed-loop optimization management of the lighting system.
[0017] The technical solution of this invention provides a method for controlling a flying robot in an intelligent lighting system, comprising: spatially scanning lighting facilities in a target area using a multispectral camera and lidar mounted on the flying robot to acquire three-dimensional point cloud data containing the position, attitude, and spectral characteristics of the lighting facilities; constructing a digital twin model of the lighting facilities based on the three-dimensional point cloud data, wherein the digital twin model includes the spatial coordinates, physical parameters, and real-time operating status of each lighting facility; optimizing the inspection path of the flying robot using a quantum annealing algorithm based on the spatial distribution of the lighting facilities in the digital twin model to generate an adaptive inspection trajectory that covers all lighting facilities and has the lowest energy consumption, wherein the quantum annealing algorithm jointly optimizes flight distance, obstacle avoidance, and lighting facility priority as constraints; controlling the flying robot to fly along the adaptive inspection trajectory, and using the onboard thermal imaging system... Sensors and spectral analyzers perform non-contact detection on various lighting facilities, acquiring multi-dimensional detection data in real time, including thermal distribution images, spectral characteristic curves, and electromagnetic radiation intensity. This multi-dimensional detection data is then encrypted and uploaded to cloud storage via a blockchain network. A deep convolutional neural network is used to fuse and analyze this multi-dimensional detection data, identifying abnormal patterns in the lighting facilities, including abnormal light decay, overheating faults, and flicker defects. Based on the severity of these abnormal patterns, intelligent decision-making instructions are autonomously generated, including fault location, repair priorities, and suggested maintenance solutions. These intelligent decision-making instructions are synchronized to the IoT control platform of the intelligent lighting system, triggering remote dimming, zone control, or fault isolation operations for the lighting facilities. Simultaneously, based on changes in ambient light and user feedback, the next inspection cycle and key monitoring areas of the flying robot are dynamically adjusted, achieving closed-loop optimization management of the lighting system. By utilizing technologies such as flying robots, digital twins, quantum optimization algorithms, and blockchain data management, the intelligent lighting system achieves fully automated, high-precision inspection and intelligent control. This improves detection efficiency and accuracy, enables early warning and precise location of lighting facility faults, reduces operational energy consumption through adaptive path planning, and ensures the traceability and tamper-proof nature of detection data through blockchain technology. Overall, this enhances the reliability, energy efficiency, and intelligent management level of the intelligent lighting system. Attached Figure Description
[0018] Figure 1 This is a flowchart of a flying robot control method for an intelligent lighting system provided in one embodiment of the present invention; Figure 2 This is a schematic block diagram of a flying robot control system for an intelligent lighting system provided in one embodiment of the present invention; Figure 3 This is a schematic diagram of a possible form of a flying robot for an intelligent lighting system provided in one embodiment of the present invention; Figure 4This is a structural block diagram of a flying robot for an intelligent lighting system provided in one embodiment of the present invention. Detailed Implementation
[0019] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.
[0020] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.
[0021] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.
[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0023] The following reference Figures 1 to 4 This invention describes a method and system for controlling a flying robot in an intelligent lighting system, provided by some embodiments of the present invention.
[0024] like Figure 1 As shown, one embodiment of the present invention provides a flying robot control method for an intelligent lighting system, comprising: The flying robot uses a multispectral camera and lidar to perform spatial scanning of the lighting facilities in the target area, and obtains three-dimensional point cloud data containing the location, attitude and spectral characteristics of the lighting facilities. Based on the three-dimensional point cloud data, a digital twin model of the lighting facilities is constructed. The digital twin model includes the spatial coordinates, physical parameters and real-time operating status of each lighting facility. In this embodiment of the invention, a soccer ball-shaped quadcopter drone can be used. The drone is equipped with an antenna with adjustable transmission power and directional transmission capabilities to transmit the command for the control device to enter a discoverable state to the smart lighting device, thereby enabling the designated device to enter a discoverable state.
[0025] Based on the spatial distribution of lighting facilities in the digital twin model, the quantum annealing algorithm is used to optimize the inspection path of the flying robot, generating an adaptive inspection trajectory that covers all lighting facilities and has the lowest energy consumption. The quantum annealing algorithm uses flight distance, obstacle avoidance and lighting facility priority as constraints for joint optimization. The flight robot is controlled to fly along the adaptive inspection trajectory. It performs non-contact inspection of each lighting facility using the thermal imaging sensor and spectral analyzer it is equipped with. It collects multi-dimensional detection data in real time, including thermal distribution images, spectral characteristic curves and electromagnetic radiation intensity, and uploads the multi-dimensional detection data to the cloud for storage via a blockchain network. The multidimensional detection data is fused and analyzed using a deep convolutional neural network to identify abnormal patterns of lighting facilities, including abnormal light decay, overheating faults, and flicker defects. Based on the severity of the abnormal patterns, intelligent decision-making instructions are autonomously generated, including fault location, repair priority, and suggested maintenance solutions. The intelligent decision-making instructions are synchronized to the IoT control platform of the intelligent lighting system, triggering remote dimming, zone control, or fault isolation operations of the lighting facilities. At the same time, based on changes in ambient light and user feedback, the next round of inspection cycle and key monitoring areas of the flying robot are dynamically adjusted to achieve closed-loop optimization management of the lighting system.
[0026] The technical solution adopted in this embodiment utilizes technologies such as flying robots, digital twin technology, quantum optimization algorithms, and blockchain data management to achieve fully automated, high-precision inspection and intelligent control of the intelligent lighting system. This improves detection efficiency and accuracy, enables early warning and precise location of lighting facility faults, reduces operational energy consumption through adaptive path planning, and ensures the traceability and tamper-proof nature of detection data through blockchain technology. Overall, it enhances the reliability, energy efficiency, and intelligent management level of the intelligent lighting system.
[0027] In some possible embodiments of the present invention, a multispectral camera and lidar mounted on a flying robot are used to perform spatial scanning of lighting facilities in a target area to acquire three-dimensional point cloud data containing the location, attitude, and spectral characteristics of the lighting facilities. Based on this three-dimensional point cloud data, a digital twin model of the lighting facilities is constructed. This digital twin model includes the spatial coordinates, physical parameters, and real-time operating status of each lighting facility. The steps include: The flight robot is controlled to hover or fly at low speed to the predetermined detection position, and a multispectral camera and lidar are activated simultaneously to collect data. The multispectral camera captures spectral images of the lighting facility containing visible light, near-infrared and ultraviolet bands, and the lidar emits pulse beams and receives reflected signals to obtain the three-dimensional point cloud coordinates of the surface of the lighting facility. At the same time, the timestamp of each data point and the real-time pose information of the flight robot are recorded. Spatial registration is performed between the 3D point cloud data acquired by the lidar and the spectral images of the lighting facilities collected by the multispectral camera. The correspondence between the point cloud coordinates and the image pixels is established through the feature point matching algorithm. The spectral feature information is mapped to the corresponding 3D point cloud points to form a fused point cloud dataset containing spatial location and spectral attributes. A deep learning-based semantic segmentation algorithm is applied to the fused point cloud dataset to identify and distinguish different objects such as the lighting facility itself, supporting structure, cable connection and surrounding environment. Point cloud clusters belonging to the lighting facility are extracted, and the type, installation posture and physical size parameters of the lighting facility are automatically identified based on the geometric features of the point cloud clusters. The spectral feature data of the extracted point cloud clusters of lighting facilities are analyzed. The luminous intensity, color temperature distribution and light decay of the lighting facilities are determined by spectral curve analysis. The working status of the lighting facilities is identified by combining the temperature information collected by the thermal imaging sensor, including normal operation, abnormal heating, light source aging and power outage fault. The extracted spatial coordinates, physical parameters, and operational status information are integrated into a predefined digital twin model framework. A digital twin instance containing a unique identifier, three-dimensional geometric model, material properties, installation location, operating parameters, and real-time status is created for each lighting facility. Spatial topological relationships and power supply network connections between lighting facilities are also established to form a complete digital twin model of the lighting system.
[0028] The solution in this embodiment achieves a high-fidelity mapping of lighting facilities from physical space to digital space through spatiotemporal synchronous acquisition and intelligent fusion processing of multi-source sensor data. It can automatically and efficiently acquire comprehensive information about lighting facilities, including not only precise three-dimensional geometry and spatial location, but also dynamic information such as spectral characteristics and real-time operating status. The constructed digital twin model has high precision, high integrity, and real-time update capability, providing a reliable digital infrastructure for subsequent intelligent inspection, fault diagnosis, and optimized control, and significantly improving the digital management level and intelligent operation and maintenance of the intelligent lighting system.
[0029] In some possible embodiments of the present invention, based on the spatial distribution of lighting facilities in the digital twin model, a quantum annealing algorithm is used to optimize the inspection path of the flying robot, generating an adaptive inspection trajectory that covers all lighting facilities and has the lowest energy consumption. The quantum annealing algorithm includes the step of jointly optimizing flight distance, obstacle avoidance, and lighting facility priority as constraints, comprising: The spatial coordinate information of all lighting facilities is extracted from the digital twin model. The center position of each lighting facility is used as a task node. The straight-line flight distance between any two task nodes is calculated as the edge weight. At the same time, priority weights are assigned to each task node according to the operating status and historical failure frequency of the lighting facilities recorded in the digital twin model. A directed inspection task graph containing node priority and edge weights is constructed. Extract the three-dimensional spatial location and boundary information of obstacles in the digital twin model, set a safety buffer for each obstacle, divide the three-dimensional space into a flyable area and a no-fly zone, detect whether each edge in the inspection task map crosses the no-fly zone, generate detour constraints for path edges that cross the no-fly zone, and ensure that candidate trajectories meet spatial safety requirements. In this step, 3D point cloud data of obstacles such as buildings, trees, utility poles, and support towers are extracted from the obstacle data layer of the digital twin model. A minimum outer envelope is constructed for each obstacle using a convex hull algorithm or bounding box fitting method to characterize its spatial occupancy. Based on the obstacle type and the physical dimensions of the flying robot, a preset safety distance is extended outwards from each envelope to form a 3D safety buffer zone. Rigid obstacles such as buildings have a smaller buffer distance, while dynamic obstacles such as tree branches swaying in the wind have a larger buffer distance. All obstacles and their buffer zones are uniformly marked as no-fly zones in 3D space, and the remaining space is marked as flyable areas. For each straight path edge connecting any two lighting facility task nodes in the inspection task map, the start and end coordinates of the path edge are extracted to construct a 3D line segment. Geometric intersection is determined by traversing the envelopes of all no-fly zones. The method employs a ray casting algorithm or the separating axis theorem to detect whether a line segment intersects with any obstacle's safety buffer zone. Path edges with detected collision risks are marked as non-direct flight paths, and a list of intersecting obstacles is recorded. Path edges without detected collisions are marked as safe direct flight paths. For path edges marked as non-direct flight paths, alternative paths around the obstacle buffer zone are searched in 3D space based on the spatial location and geometry of the intersecting obstacles. The original straight path is decomposed into multiple polyline paths by setting intermediate waypoints around the obstacles, ensuring that each sub-path is within a flyable area and does not cross any no-fly zones. These detour paths are used as alternative flight paths for that path edge. Corresponding detour constraints are added to the constraints of the quantum annealing algorithm, automatically selecting safe detour paths instead of dangerous straight-line crossing paths during the optimization process. This method, through precise 3D spatial modeling and intelligent collision detection, transforms obstacle avoidance from manual experience-based judgment to systematic automatic constraint generation, ensuring the flight safety of planned paths, effectively avoiding collision risks, and improving the reliability and automation level of inspection tasks.
[0030] The inspection path optimization problem is transformed into an extended form of the traveling salesman problem. A qubit variable is defined to represent the order and path selection of the flying robot's access to each lighting facility. The flight distance minimization target, obstacle avoidance constraints, and priority access requirements of priority lighting facilities are encoded as energy terms of quantum Hamiltonians. The low-energy state corresponds to the superior trajectory scheme with short flight distance, obstacle avoidance, and priority access to high-priority facilities. In this step, binary qubit variables are assigned to each lighting facility task node and each possible access location in the inspection task graph. A two-dimensional qubit matrix is used to represent the order in which the flying robot visits each lighting facility, where the row index represents the lighting facility number and the column index represents the access order position. A qubit value of 1 indicates that the corresponding lighting facility is visited at the corresponding order position, and a value of 0 indicates that it is not visited at that position. Simultaneously, qubit variables are assigned to each optional path edge to indicate whether the path is selected, constructing a complete quantum state space containing the access order and path selection. The flight distance minimization objective is encoded as the first energy term of the quantum Hamiltonian, which calculates the cumulative distance of all selected path edges. The obstacle avoidance constraint is encoded as the second energy term. A high-energy penalty is applied to quantum state configurations that select non-direct flight paths without using detour schemes, thus optimizing the flight path. Priority access requirements for high-priority lighting facilities are encoded as a third energy term. An energy penalty is applied to the quantum state configuration of high-priority facilities located later in the access sequence. Simultaneously, a constraint energy term is added to ensure that each lighting facility is accessed exactly once and the access order is consecutively valid. The importance of different optimization objectives is balanced by adjusting the weight coefficients of each energy term. The energy function characteristics of the constructed quantum Hamiltonian are analyzed to clarify the qubit configuration characteristics corresponding to low-energy states: short total length of selected path edges, all paths located within the flyable region, and high-priority lighting facilities arranged at the beginning of the access sequence. A quantitative mapping relationship is established between quantum state energy values and the quality of the flight path scheme. This allows the quantum annealing algorithm to naturally evolve towards low-energy states representing short flight distances, high safety, and reasonable task priorities during iterative optimization, organically unifying the physical optimization process of quantum computing with the actual optimization requirements of the inspection path. This method cleverly transforms complex multi-constraint path optimization into a quantum energy minimization problem through problem mapping, efficiently exploring a large-scale solution space using quantum parallel search capabilities, achieving multi-objective collaborative optimization, and significantly improving solution efficiency and scheme quality.
[0031] The initial temperature parameters and annealing rate of the quantum annealing algorithm are set. By simulating the quantum tunneling effect, the system gradually evolves from a high-energy state to a low-energy state. During the annealing process, the system is allowed to accept a state transition with a certain probability of temporarily increasing energy in order to escape the local optimum. After multiple rounds of annealing iterations, the system converges to the stable state with the lowest energy. The quantum bit configuration corresponding to this state represents the optimized inspection path access sequence. The discrete task node access sequence output by the quantum annealing algorithm is converted into a continuous three-dimensional space track. The track is smoothed by Bézier curve or spline interpolation algorithm to meet the dynamic constraints of the flying robot. The generated track is verified to meet the physical limitations such as maximum flight speed, maximum acceleration and minimum turning radius. Track segments that do not meet the constraints are locally adjusted and optimized, and finally an adaptive inspection track is generated.
[0032] This embodiment fully leverages the global search capability and local optima-breaking advantage of the quantum annealing algorithm in combinatorial optimization problems, enabling it to find near-global optimal inspection trajectory schemes more efficiently under complex constraints. By uniformly mapping the multi-objective optimization problem to a quantum energy space for solution, a synergistic balance between flight energy consumption, safety, and mission priority is achieved. The generated trajectory not only provides comprehensive coverage and optimal path selection but also exhibits excellent obstacle avoidance capabilities and mission execution flexibility, improving inspection efficiency, reducing flight energy consumption, and ensuring flight safety and timely detection of high-priority lighting facilities. This provides an efficient and reliable path planning solution for the intelligent inspection of large-scale lighting facilities.
[0033] In some possible embodiments of the present invention, the steps of controlling the flying robot to fly along the adaptive inspection trajectory, performing non-contact inspection of each lighting facility using onboard thermal imaging sensors and a spectral analyzer, and real-time acquiring multi-dimensional detection data including thermal distribution images, spectral characteristic curves, and electromagnetic radiation intensity, and encrypting and uploading the multi-dimensional detection data to cloud storage via a blockchain network include: The instructions corresponding to the adaptive inspection trajectory are loaded into the flight control system of the flying robot, and the autonomous navigation mode is activated to make the flying robot fly along the predetermined trajectory. When it approaches the target lighting facility, the flight attitude and hovering position of the flying robot are automatically adjusted according to the spatial pose information of the lighting facility stored in the digital twin model, so that the detection optical axis of the thermal imaging sensor and the spectral analyzer are aligned with the key detection part of the lighting facility, and the best observation angle and detection distance are maintained. After the flying robot reaches the preset stable detection posture, the thermal imaging sensor, the spectrum analyzer and the electromagnetic radiation detector are triggered simultaneously to collect data. The thermal imaging sensor captures a two-dimensional temperature distribution image of the surface of the lighting facility to identify abnormal hot spots. The spectrum analyzer measures the spectral characteristic curve emitted by the lighting facility to evaluate the performance of the light source and the color temperature shift. The electromagnetic radiation detector detects the electromagnetic field strength around the lighting facility to determine the electrical safety status. Each set of detection data is automatically labeled with the collection timestamp, facility identifier and spatial location information. The system performs real-time preprocessing on the acquired raw multidimensional detection data, including noise reduction filtering of thermal distribution images, baseline correction of spectral curves, and interference suppression of electromagnetic radiation signals. It judges whether the data quality meets the analysis requirements by evaluating image sharpness, calculating spectral signal-to-noise ratio, and checking signal integrity. It triggers a re-acquisition command for data that does not meet the quality requirements, ensuring that the acquired detection data has the reliability and accuracy required for subsequent intelligent analysis. The quality-verified multidimensional detection data, along with metadata information, is encapsulated into a standardized data packet. An asymmetric encryption algorithm is used to encrypt the data packet to generate ciphertext data. The hash digest value of the data packet is calculated as a digital fingerprint. The ciphertext data, hash digest, and timestamp information are packaged to construct a blockchain transaction request. The integrity of the data is verified through a smart contract, and the transaction record is written into a distributed ledger to ensure the immutability and traceability of the detection data. The encrypted detection data is uploaded to the cloud storage system. A distributed storage strategy is adopted to distribute data copies across multiple physical nodes to improve data reliability. At the same time, a retrieval structure indexed by lighting facility identifier, collection time and data type is established in the cloud database. A mapping relationship between blockchain transaction hash and cloud storage address is generated to realize the rapid location of detection data and version management of historical data, providing efficient data access services for subsequent data analysis and decision support.
[0034] This embodiment achieves comprehensive, accurate, and non-contact monitoring of lighting facility operation status through multi-sensor collaborative detection and intelligent attitude control, avoiding equipment damage and safety risks that may arise from traditional contact-based detection. Real-time data quality assessment and an adaptive re-acquisition mechanism ensure high-quality and high-reliability detection data. The introduction of blockchain technology for data encryption and distributed storage not only guarantees the security and privacy of sensitive detection data but also enables full lifecycle traceability of data through an immutable distributed ledger, effectively preventing data forgery and malicious tampering. This provides a reliable, secure, and efficient data infrastructure for the intelligent operation and maintenance of lighting systems, enhancing the transparency of data management and the overall credibility of intelligent lighting systems.
[0035] In some possible embodiments of the present invention, the step of using a deep convolutional neural network to fuse and analyze the multidimensional detection data to identify abnormal patterns of the lighting facilities, including abnormal light decay, overheating faults, and flicker defects, and autonomously generating intelligent decision instructions containing fault location, repair priority, and suggested maintenance plans based on the severity of the abnormal patterns, includes: Multidimensional detection data of the target lighting facility is extracted from the cloud storage system. The thermal distribution image is converted into a temperature feature matrix, the spectral feature curve is converted into a spectral intensity vector, and the electromagnetic radiation intensity is converted into a field strength feature sequence. These heterogeneous data are organized according to the time dimension, spatial dimension, and feature dimension to construct a four-dimensional feature tensor. At the same time, the historical detection data and normal operation benchmark data of the target lighting facility are loaded as a comparison reference to provide a standardized input data structure for neural network analysis. The constructed four-dimensional feature tensor is input into a pre-trained deep convolutional neural network model. The spatial pattern features of thermal distribution, the frequency domain features of spectral curves, and the temporal features of electromagnetic radiation are extracted through multiple convolutional and pooling layers. An attention mechanism is used in the middle layer of the network to weight and fuse different modal features, so that the network adaptively focuses on the feature dimension that contributes the most to anomaly recognition. The fused features are mapped to the anomaly pattern classification space through a fully connected layer, and the output includes the classification probability distribution of light decay anomaly, overheating fault, stroboscopic defect and normal state. In this step, the constructed four-dimensional feature tensor is decomposed into thermal distribution image sub-tensor, spectral curve sub-tensor, and electromagnetic radiation signal sub-tensor according to modality type. These are then input into three parallel feature extraction branches of a deep convolutional neural network. The thermal distribution branch uses two-dimensional convolutional layers and max-pooling layers to extract the spatial distribution pattern of the temperature field and the location and morphological features of anomalous hotspots. The spectral branch uses one-dimensional convolutional layers and frequency domain transform layers to extract the peak position, spectral linewidth, and frequency component features of the spectral curve. The electromagnetic radiation branch uses temporal convolutional layers and recurrent neural network units to extract the time-varying patterns and periodic fluctuation features of the signal. Each branch independently outputs a high-dimensional feature vector corresponding to its modality. The feature vectors output from the three parallel branches are then input into the multimodal attention fusion module in the middle layer of the network. This module first calculates the attention weights related to the current detection task for each modality feature vector, and then uses the learned attention... The method adaptively evaluates the contribution of thermal distribution features, spectral features, and electromagnetic radiation features to the identification of current lighting facility anomalies, assigning higher weights to features with high contributions and lower weights to features with low contributions. The calculated attention weights are used to perform weighted summation or concatenation operations on the feature vectors of each modality, generating a unified feature representation that integrates multimodal complementary information. This unified feature representation output from the attention fusion module is then input into a fully connected layer for dimensionality reduction and nonlinear transformation. A multi-layer fully connected network maps the high-dimensional fused features to an anomaly pattern classification space, which includes four categories: light decay anomaly, overheating fault, flicker defect, and normal state. A softmax activation function is applied to the network's output layer to normalize the classification scores of each category into a probability distribution. The output represents the probability value of the lighting facility belonging to each anomaly pattern category; the category with the highest probability is the final anomaly identification result. This method extracts multimodal complementary features through a parallel multi-branch network and combines them with an attention mechanism for adaptive fusion, fully exploring the synergistic value of heterogeneous data, significantly improving the accuracy and robustness of anomaly identification, and achieving accurate discrimination of complex fault modes.
[0036] Based on the classification probability distribution output by the deep convolutional neural network, the category with the highest probability is selected as the current operating status determination result of the lighting facility. When an abnormal state is identified, the activation response map of the intermediate layer of the network is extracted to locate the precise spatial location of the abnormal area on the lighting facility. The confidence index of the abnormal determination is calculated to evaluate the reliability of the diagnosis result. For ambiguous cases with low confidence, a manual review process is triggered or a flying robot is arranged to conduct a second fine inspection to ensure the accuracy and reliability of the abnormal identification. In this step, a classification probability distribution including light decay anomalies, overheating faults, flicker defects, and normal states is obtained from the output layer of the deep convolutional neural network. The category with the highest probability value is selected as the current operating state determination result of the lighting facility. When the determination result is an abnormal state, the activation response of the feature map that contributes the most to the abnormal classification in the network's convolutional layer is extracted using class activation mapping technology or gradient weighting method. The high activation response region is back-mapped back to the spatial coordinate system of the original thermal distribution image and spectral data to generate an abnormal thermal map that marks the precise location distribution of abnormal features on the surface of the lighting facility, identifying the specific component locations of hotspots, light decay areas, or flicker sources. Based on the classification probability distribution, the confidence index of the abnormal determination is calculated. The difference between the highest probability value and the second highest probability value is used as a measure of determination certainty. At the same time, the entropy value of the output probability is combined to evaluate the ambiguity of the classification result. The variance of the network's output after multiple forward propagations is calculated to evaluate the stability of the model's prediction. These indicators are combined to form a confidence score. A confidence threshold is set to classify diagnostic results into three levels: high confidence, medium confidence, and low confidence. A high confidence score indicates a reliable diagnosis that can be directly accepted, while a low confidence score indicates uncertainty requiring further verification. For ambiguous cases with a confidence score below the preset threshold, the system automatically triggers a secondary verification process. First, the case and its detection data are pushed to a manual review queue for professional technicians to interpret and judge based on experience. Simultaneously, a secondary detection task instruction is generated for the flying robot, instructing it to return to the location of the lighting facility to re-collect detection data at a closer detection distance, more observation angles, and a higher data sampling rate. The high-quality data obtained from the secondary detection is then input into the neural network for analysis. The final anomaly determination is formed by combining the manual review opinions and the secondary detection results. The case and its true label are added to the training dataset for subsequent model optimization. This method, through quantitative confidence assessment and a multi-level verification mechanism, effectively identifies and processes ambiguous diagnostic cases, avoiding misjudgments and missed judgments, significantly improving the reliability of anomaly identification, and achieving complementary advantages and continuous improvement between intelligent diagnosis and human experience.
[0037] For each identified anomaly pattern, a comprehensive assessment is conducted based on the degree of deviation of the anomaly characteristics, duration, and scope of impact. The severity of the anomaly is divided into four levels: emergency, important, general, and minor. Emergency anomalies indicate that the lighting facilities are at risk of immediate failure or safety hazards. Important anomalies indicate that the facility performance has significantly degraded and requires immediate attention. General anomalies indicate that the facility has potential problems and can be maintained in a planned manner. Minor anomalies indicate that the facility is operating basically normally but requires continuous monitoring. Each anomaly is assigned a quantitative risk score. Based on the anomaly pattern type, severity level, and location information of lighting facilities, the system retrieves matching maintenance plan templates from a pre-built maintenance knowledge base. It automatically generates a detailed maintenance plan that includes a precise fault location description, recommended repair measures, a list of required spare parts, and estimated repair time. According to the risk score and the anomaly distribution of all lighting facilities in the area, a multi-objective optimization algorithm is used to calculate the maintenance priority ranking of each facility. The system generates intelligent decision instructions that include a priority list, repair route suggestions, and resource scheduling plans, and pushes them to operation and maintenance management personnel and on-site maintenance teams through the IoT control platform.
[0038] This embodiment leverages the powerful feature learning and multimodal data fusion capabilities of deep convolutional neural networks to achieve intelligent and automated identification of abnormal patterns in lighting facilities. It accurately captures complex combinations of abnormal features and early signs of failure, significantly improving the accuracy and timeliness of fault diagnosis. Through an end-to-end intelligent decision-making process, the entire process—from anomaly identification and severity assessment to maintenance plan generation—is automated. This not only drastically shortens the response time from problem discovery to solution development but also improves operational efficiency through scientific prioritization and optimized resource allocation. It enables lighting systems to shift from reactive maintenance to predictive maintenance and proactive management, effectively reducing the failure rate of lighting facilities, extending equipment lifespan, and providing strong intelligent decision support for the reliable operation and refined management of intelligent lighting systems.
[0039] In some possible embodiments of the present invention, the steps of synchronizing the intelligent decision-making instructions to the IoT control platform of the intelligent lighting system to trigger remote dimming, zone control, or fault isolation operations of the lighting facilities, and dynamically adjusting the next inspection cycle and key monitoring areas of the flying robot based on changes in ambient light and user feedback, to achieve closed-loop optimization management of the lighting system, include: The intelligent decision-making instructions are synchronously transmitted to the IoT control platform of the intelligent lighting system through a secure communication protocol. The platform's decision analysis module extracts fault location information, maintenance priorities, and suggested maintenance plans. It automatically matches the corresponding control strategy according to the type of abnormal mode. Specifically, it generates dimming compensation instructions for light decay abnormalities to maintain the target illuminance, generates power reduction operation or temporary shutdown instructions for overheating faults to prevent equipment damage, generates drive circuit parameter adjustment instructions for flicker defects, and generates fault isolation instructions for emergency faults to cut off the power supply to the facility and activate backup lighting. The IoT control platform sends the parsed control commands to the smart controllers of each lighting facility through a wireless communication network. After receiving the commands, the smart controllers execute corresponding dimming operations, zone switching control, or fault isolation actions. During the execution process, they monitor the operating parameters of the lighting facilities in real time, such as voltage, current, power, and light output. The execution results and equipment status change information are sent back to the IoT control platform through the feedback channel. The platform verifies whether the control commands have been successfully executed and records the operation log. For commands that fail to be executed, an alarm is triggered and manual intervention is arranged. By continuously collecting real-time environmental data such as ambient light intensity, weather conditions, and human activity density through illuminance sensors, human presence sensors, and environmental monitoring equipment deployed in the lighting area, and collecting user demand data such as lighting effect satisfaction evaluations, illuminance adjustment requests, and fault repair information from the user feedback system, these multi-source data are aggregated into the data analysis module of the IoT control platform to construct a comprehensive dataset that includes environmental change trends and user behavior patterns. Based on the collected environmental perception data and user demand data, combined with the current health assessment results of lighting facilities and historical fault statistics, machine learning algorithms are used to analyze the spatiotemporal patterns and environmental correlations of lighting facility faults, identify time periods, areas and environmental conditions prone to faults, dynamically adjust the next round of inspection cycle of the flying robot, appropriately extend the inspection interval for areas with good health and high user satisfaction, shorten the inspection cycle and increase the inspection frequency for key monitoring areas with frequent faults or many user complaints, and update the inspection task priority list and the list of facilities to be monitored. By integrating the results of control command execution, changes in the operating status of lighting facilities, environmental data, and user feedback data to form a closed-loop feedback information flow, the lighting facility status parameters and operating history in the digital twin model are updated. This accumulated data is used to continuously train and optimize the deep convolutional neural network model to improve the accuracy of anomaly identification. At the same time, the parameter configuration of the quantum annealing algorithm is optimized to improve the trajectory planning effect. Through a data-driven continuous learning mechanism, the detection accuracy, control effect, and resource allocation efficiency of the entire intelligent lighting system are continuously improved over time, realizing the evolution from responsive management to predictive management and adaptive optimization management.
[0040] This embodiment constructs a complete closed-loop management system from anomaly detection, intelligent decision-making, control execution to effect feedback, realizing full-process automation and intelligence in the operation and maintenance management of lighting systems. Through the rapid issuance and execution of real-time control commands, countermeasures can be taken immediately upon the occurrence of a fault, effectively preventing the escalation of the fault and secondary accidents, significantly improving the reliability and safety of the lighting system. Through a dynamic response mechanism based on environmental perception and user needs, the lighting control strategy can adaptively adjust to meet the changing needs of actual usage scenarios, improving user experience and energy efficiency. Through intelligent optimization of inspection strategies and precise resource allocation, operating costs are minimized while ensuring monitoring quality. Through a continuous learning and feedback optimization mechanism, the entire system possesses self-evolution capabilities, continuously improving performance as data accumulates, providing an efficient, reliable, and adaptive closed-loop intelligent operation and maintenance solution for intelligent lighting systems, truly realizing the transformation and upgrading of lighting systems from traditional management to intelligent management.
[0041] In some possible embodiments of the present invention, the process of constructing a digital twin model of a lighting facility specifically includes: After adaptive filtering of the acquired 3D point cloud data, a multi-scale feature fusion algorithm is used to extract the key geometric features of the lighting facilities, and the spatial coordinates of each lighting facility are dynamically calibrated using the following formula for correcting the spatial position accuracy of the lighting facilities:
[0042] in, This represents the corrected three-dimensional spatial coordinate vector of the m-th lighting fixture; This represents the original measured coordinate vector of the m-th lighting fixture; The compensation coefficient represents the pose drift of the flying robot; This represents the cumulative pose deviation vector detected by the inertial measurement unit; The corrected weighting coefficients for environmental disturbances; Indicates the total number of reference lighting facilities involved in the calibration; This represents the spatial influence weight of the nth reference lighting fixture on the mth lighting fixture, which is calculated based on the inverse Euclidean distance between the two. The known precise coordinate vector represents the nth reference lighting fixture.
[0043] In this embodiment, the modified formula effectively eliminates positioning drift and environmental interference during flight by fusing inertial navigation data and multi-reference point spatial constraints, significantly improving the spatial positioning accuracy and geometric consistency of the digital twin model.
[0044] In some possible embodiments of the present invention, the process of generating adaptive inspection tracks using the quantum annealing algorithm specifically includes: The inspection path optimization problem is mapped to a quantum Hamiltonian minimization problem, and the overall cost of the trajectory scheme is evaluated using the following energy function:
[0045] in, The total energy cost function represents the candidate inspection trajectory; The weighting coefficient represents the cost of flight distance; P represents the total number of waypoints in the inspected flight path. Let f(p) represent the Euclidean distance function between the p-th waypoint and the (p+1)-th waypoint. This represents the spatial position vector of the p-th waypoint; Weighting coefficients representing the cost of obstacle avoidance; Indicates the total number of obstacles within the detection range; This represents the threat level coefficient of the q-th obstacle; This represents the center position vector of the q-th obstacle; This represents a small positive number that prevents the denominator from being zero. Weighting coefficients representing the priority cost of lighting facilities; This represents the maintenance urgency index of the lighting facilities corresponding to the p-th waypoint; This represents the priority decay index, used to control the degree to which higher-priority facilities are accessed first.
[0046] In this embodiment, the energy function is optimized through multi-objective weighted optimization. Under the premise of ensuring the integrity of the inspection, it achieves a coordinated balance between minimizing flight distance, safely avoiding obstacles, and prioritizing the detection of high-priority facilities, which effectively improves the inspection efficiency and safety.
[0047] In some possible embodiments of the present invention, the steps of controlling the flying robot to fly along the adaptive inspection trajectory, performing non-contact inspection of each lighting facility using onboard thermal imaging sensors and a spectral analyzer, and real-time acquisition of multi-dimensional detection data including thermal distribution images, spectral characteristic curves, and electromagnetic radiation intensity, further include: Based on the relative pose relationship between the flying robot and the lighting facilities, the acquisition parameters of the thermal imaging sensor and the spectral analyzer are dynamically and adaptively adjusted. The following multi-sensor collaborative acquisition quality assessment formula is used to optimize the data acquisition quality:
[0048] in, This indicates the data quality assessment metrics under the current acquisition status; The weighting coefficients representing perspective factors; This indicates the angle between the sensor's optical axis and the normal to the surface of the lighting fixture; This represents the weighting coefficient of the detection distance factor; This indicates the actual distance between the flying robot and the lighting facility; Indicates the optimal detection distance of the sensor; The tolerance standard deviation represents the distance deviation. The weighting coefficients representing flight stability factors; This represents the real-time vibration velocity vector of the flying robot. This indicates the maximum permissible vibration velocity threshold.
[0049] The quality assessment formula in this embodiment comprehensively considers the impact of observation angle, detection distance and platform stability on data acquisition. By adjusting flight attitude and sensor parameters through real-time feedback, it ensures the acquisition of high-quality and highly reliable multi-dimensional detection data.
[0050] In some possible embodiments of the present invention, the process of anomaly identification and intelligent decision-making in step S4 specifically includes: A spatiotemporal feature tensor is constructed from the fused multidimensional detection data, and the operating status of each lighting facility is quantified using the following dynamic health assessment formula:
[0051] in, This represents the overall health assessment value of the m-th lighting fixture; This represents the total number of feature dimensions in the multidimensional detection data; This represents the importance weight coefficient of the c-th feature dimension; This represents the current detection value of the m-th lighting fixture in the c-th feature dimension; This represents the health baseline value of the m-th lighting fixture in the c-th feature dimension; This represents a small positive number that prevents the denominator from being zero. This represents the time decay coefficient of the c-th feature dimension, reflecting the rate at which the feature degrades over time. This represents the cumulative operating time of the m-th lighting fixture since the last maintenance.
[0052] The health assessment formula in this embodiment achieves accurate quantification of the operating status of lighting facilities and early warning of abnormal trends through the coupled analysis of multidimensional feature deviation measurement and time degradation factor, providing a scientific and reliable basis for intelligent decision-making.
[0053] In some possible embodiments of the present invention, the process of dynamically adjusting the next inspection cycle and key monitoring area of the flying robot includes: Based on historical inspection data and real-time environmental feedback, the following adaptive inspection cycle optimization formula is used to dynamically adjust the next round of inspection interval for the flying robot:
[0054] in, This indicates the optimized time interval for the next round of inspections; Indicates the basic inspection cycle time; This indicates the total number of lighting facilities within the monitored area; This represents the overall health assessment value of the m-th lighting fixture; A sensitivity index indicating the impact of health status on the inspection cycle; The moderating coefficient representing seasonal environmental factors; This indicates the ordinal number of the current date within the year. This indicates the total number of special event types that affect inspection needs. This represents the influence intensity coefficient of the s-th type of special event; This function indicates the occurrence of the s-th type of special event, taking a value of 1 when the event occurs and 0 otherwise.
[0055] The inspection cycle optimization formula in this embodiment integrates the health status of lighting facilities, seasonal environmental changes, and the impact of emergencies, achieving intelligent allocation of inspection resources and adaptive optimization of response speed. This effectively reduces operating costs and energy consumption while ensuring monitoring effectiveness.
[0056] Please refer to Figure 2 Another embodiment of the present invention provides a flight robot control system for an intelligent lighting system, for executing a flight robot control method for an intelligent lighting system, comprising: a flight robot and a server; The flying robot is configured to: use a multispectral camera and lidar mounted on the flying robot to perform spatial scanning of the lighting facilities in the target area, and acquire three-dimensional point cloud data including the position, attitude and spectral characteristics of the lighting facilities; The server is configured as follows: A digital twin model of the lighting facilities is constructed based on the three-dimensional point cloud data. The digital twin model includes the spatial coordinates, physical parameters, and real-time operating status of each lighting facility. Based on the spatial distribution of lighting facilities in the digital twin model, the quantum annealing algorithm is used to optimize the inspection path of the flying robot, generating an adaptive inspection trajectory that covers all lighting facilities and has the lowest energy consumption. The quantum annealing algorithm uses flight distance, obstacle avoidance and lighting facility priority as constraints for joint optimization. The flight robot is controlled to fly along the adaptive inspection trajectory. It performs non-contact inspection of each lighting facility using the thermal imaging sensor and spectral analyzer it is equipped with. It collects multi-dimensional detection data in real time, including thermal distribution images, spectral characteristic curves and electromagnetic radiation intensity, and uploads the multi-dimensional detection data to the cloud for storage via a blockchain network. The multidimensional detection data is fused and analyzed using a deep convolutional neural network to identify abnormal patterns of lighting facilities, including abnormal light decay, overheating faults, and flicker defects. Based on the severity of the abnormal patterns, intelligent decision-making instructions are autonomously generated, including fault location, repair priority, and suggested maintenance solutions. The intelligent decision-making instructions are synchronized to the IoT control platform of the intelligent lighting system, triggering remote dimming, zone control, or fault isolation operations of the lighting facilities. At the same time, based on changes in ambient light and user feedback, the next round of inspection cycle and key monitoring areas of the flying robot are dynamically adjusted to achieve closed-loop optimization management of the lighting system.
[0057] It should be known that, Figure 2 The block diagram of the flight robot control system for the intelligent lighting system shown is for illustrative purposes only, and the number of modules shown does not limit the scope of protection of this invention. The flight robot control system for the intelligent lighting system provided in this embodiment can be used to execute various embodiments of the corresponding flight robot control method for the intelligent lighting system. For specific implementation details, please refer to the descriptions of the respective method embodiments, which will not be repeated here.
[0058] Please refer to Figure 3 and Figure 4 For example, the following describes one possible implementation of the present invention. It should be noted that this implementation is for illustrative purposes only and does not constitute a limitation on the present invention.
[0059] The hardware modules of the flying robot include, but are not limited to: image sensors, video processing chips, LBN communication modules, directional antennas, power management modules, ultrasonic sensors, infrared distance sensors, etc. Among them, the image sensors communicate bidirectionally with the video processing chips via a MIPI interface to transmit the acquired image data in real time; the video processing chips connect to the flight controller via a UART interface to output information such as device position coordinates and recognition results, providing a basis for flight control; the LBN communication module interconnects with the flight controller via a UART interface, receiving power adjustment and networking commands, and providing feedback on the device's communication status; the directional antenna connects to the LBN communication module via an SMA interface, with the antenna pointing vertically upwards; the power management module supplies power to components such as the flight controller (5V), video processing unit (12V), and communication module (3.3V) through a DC-DC conversion circuit; the ultrasonic sensors and infrared distance sensors are directly connected to the ADC interface of the flight controller to provide environmental obstacle and distance data.
[0060] The software modules possessed by flying robots include, but are not limited to: APP interaction module: runs on Android / iOS / Harmony systems, supports room information configuration (room number, number of devices), flight robot control (start / pause / return), real-time display of network configuration progress, and device status query; Flight control module: integrates path planning algorithm (based on AI algorithm, combined with room floor plan to generate optimal flight path), obstacle avoidance algorithm (real-time analysis of ultrasonic data, dynamic adjustment of flight trajectory), and fixed-point hovering algorithm (integrating GPS and inertial navigation data to ensure positioning accuracy within 10cm). AI video recognition module: Based on the YOLOv8 lightweight model, the training set includes the shape features of common smart lighting devices (round / square lampshade, mounting bracket, brand logo), and supports device recognition in indoor low light environments (recognition accuracy ≥98%). LBN communication adapter module: compatible with RW-LBN protocol, supports network command encapsulation (device ID binding, room ID allocation), and dynamic adjustment of transmission power (based on infrared distance sensor data, the power is adjusted to -10dBm when the distance to the device is 10cm to avoid interference). Status feedback module: Collects data such as the flying robot's battery level, device identification results, and network success rate in real time, and feeds it back to the mobile APP via the LBN protocol.
[0061] The core algorithms used by flying robots include, but are not limited to: Equipment recognition algorithm: After the image sensor acquires images of the roof area, the video processing chip performs grayscale and noise reduction on the images, extracts equipment features through the YOLOv8 model, matches them with the built-in equipment feature library, and determines the equipment location coordinates (establishing a two-dimensional coordinate system with the upper left corner of the room as the origin). Fixed-point flight algorithm: The flight controller plans a straight path from the current position to 10cm directly below the device based on the device coordinates output by the AI recognition module and combined with ultrasonic obstacle avoidance data. The hovering accuracy is ±10cm by adjusting the speed of the brushless motor through PID. Network communication algorithm: After approaching the device, the LBN communication module adjusts the directional antenna to align with the device, sends a network start command, and after the device responds, writes the room ID and device number preset by the mobile APP into the device storage unit, and sends a confirmation command after the binding is completed.
[0062] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.
[0063] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0064] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.
[0065] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0066] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0067] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.
[0068] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0069] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
[0070] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can easily conceive of variations or substitutions without departing from the spirit and scope of the present invention, and various modifications and alterations can be made, including combinations of the different functions and implementation steps described above, as well as software and hardware implementation methods, all of which are within the protection scope of the present invention.
Claims
1. A control method for a flying robot used in an intelligent lighting system, characterized in that, include: The flying robot uses a multispectral camera and lidar to perform spatial scanning of the lighting facilities in the target area, and obtains three-dimensional point cloud data containing the location, attitude and spectral characteristics of the lighting facilities. Based on the three-dimensional point cloud data, a digital twin model of the lighting facilities is constructed. The digital twin model includes the spatial coordinates, physical parameters and real-time operating status of each lighting facility. Based on the spatial distribution of lighting facilities in the digital twin model, the quantum annealing algorithm is used to optimize the inspection path of the flying robot, generating an adaptive inspection trajectory that covers all lighting facilities and has the lowest energy consumption. The quantum annealing algorithm uses flight distance, obstacle avoidance and lighting facility priority as constraints for joint optimization. The flight robot is controlled to fly along the adaptive inspection trajectory. It performs non-contact inspection of each lighting facility using the thermal imaging sensor and spectral analyzer it is equipped with. It collects multi-dimensional detection data in real time, including thermal distribution images, spectral characteristic curves and electromagnetic radiation intensity, and uploads the multi-dimensional detection data to the cloud for storage via a blockchain network. The multidimensional detection data is fused and analyzed using a deep convolutional neural network to identify abnormal patterns of lighting facilities, including abnormal light decay, overheating faults, and flicker defects. Based on the severity of the abnormal patterns, intelligent decision-making instructions are autonomously generated, including fault location, repair priority, and suggested maintenance solutions. The intelligent decision-making instructions are synchronized to the IoT control platform of the intelligent lighting system to trigger remote dimming, zone control or fault isolation operations of the lighting facilities. At the same time, based on changes in ambient light and user feedback, the next round of inspection cycle and key monitoring areas of the flying robot are dynamically adjusted to achieve closed-loop optimization management of the lighting system. The process of building a digital twin model of lighting facilities specifically includes: After adaptive filtering of the acquired 3D point cloud data, a multi-scale feature fusion algorithm is used to extract the key geometric features of the lighting facilities, and the spatial coordinates of each lighting facility are dynamically calibrated using the following formula for correcting the spatial position accuracy of the lighting facilities: in, This represents the corrected three-dimensional spatial coordinate vector of the m-th lighting fixture; This represents the original measured coordinate vector of the m-th lighting fixture; The compensation coefficient represents the pose drift of the flying robot; This represents the cumulative pose deviation vector detected by the inertial measurement unit; The corrected weighting coefficients for environmental disturbances; Indicates the total number of reference lighting facilities involved in the calibration; This represents the spatial influence weight of the nth reference lighting fixture on the mth lighting fixture, which is calculated based on the inverse Euclidean distance between the two. The known precise coordinate vector represents the nth reference lighting fixture.
2. The flying robot control method for an intelligent lighting system according to claim 1, characterized in that, The flying robot uses a multispectral camera and lidar to perform a spatial scan of the lighting facilities in the target area, acquiring 3D point cloud data containing the location, attitude, and spectral characteristics of the lighting facilities. Based on this 3D point cloud data, a digital twin model of the lighting facilities is constructed. This digital twin model includes the spatial coordinates, physical parameters, and real-time operating status of each lighting facility. The steps include: The flight robot is controlled to hover or fly at low speed to the predetermined detection position, and a multispectral camera and lidar are activated simultaneously to collect data. The multispectral camera captures spectral images of the lighting facility containing visible light, near-infrared and ultraviolet bands, and the lidar emits pulse beams and receives reflected signals to obtain the three-dimensional point cloud coordinates of the surface of the lighting facility. At the same time, the timestamp of each data point and the real-time pose information of the flight robot are recorded. Spatial registration is performed between the 3D point cloud data acquired by the lidar and the spectral images of the lighting facilities collected by the multispectral camera. The correspondence between the point cloud coordinates and the image pixels is established through the feature point matching algorithm. The spectral feature information is mapped to the corresponding 3D point cloud points to form a fused point cloud dataset containing spatial location and spectral attributes. A deep learning-based semantic segmentation algorithm is applied to the fused point cloud dataset to identify and distinguish different objects such as the lighting facility itself, supporting structure, cable connection and surrounding environment. Point cloud clusters belonging to the lighting facility are extracted, and the type, installation posture and physical size parameters of the lighting facility are automatically identified based on the geometric features of the point cloud clusters. The spectral feature data of the extracted point cloud clusters of lighting facilities are analyzed. The luminous intensity, color temperature distribution and light decay of the lighting facilities are determined by spectral curve analysis. The working status of the lighting facilities is identified by combining the temperature information collected by the thermal imaging sensor, including normal operation, abnormal heating, light source aging and power outage fault. The extracted spatial coordinates, physical parameters, and operational status information are integrated into a predefined digital twin model framework. A digital twin instance containing a unique identifier, three-dimensional geometric model, material properties, installation location, operating parameters, and real-time status is created for each lighting facility. Spatial topological relationships and power supply network connections between lighting facilities are also established to form a complete digital twin model of the lighting system.
3. The flying robot control method for an intelligent lighting system according to claim 2, characterized in that, Based on the spatial distribution of lighting facilities in the digital twin model, the quantum annealing algorithm is used to optimize the inspection path of the flying robot, generating an adaptive inspection trajectory that covers all lighting facilities and has the lowest energy consumption. The quantum annealing algorithm includes the following steps for joint optimization, where flight distance, obstacle avoidance, and lighting facility priority are used as constraints: The spatial coordinate information of all lighting facilities is extracted from the digital twin model. The center position of each lighting facility is used as a task node. The straight-line flight distance between any two task nodes is calculated as the edge weight. At the same time, priority weights are assigned to each task node according to the operating status and historical failure frequency of the lighting facilities recorded in the digital twin model. A directed inspection task graph containing node priority and edge weights is constructed. Extract the three-dimensional spatial location and boundary information of obstacles in the digital twin model, set a safety buffer for each obstacle, divide the three-dimensional space into a flyable area and a no-fly zone, detect whether each edge in the inspection task map crosses the no-fly zone, generate detour constraints for path edges that cross the no-fly zone, and ensure that candidate trajectories meet spatial safety requirements. The inspection path optimization problem is transformed into an extended form of the traveling salesman problem. A qubit variable is defined to represent the order and path selection of the flying robot's access to each lighting facility. The flight distance minimization target, obstacle avoidance constraints, and priority access requirements of priority lighting facilities are encoded as energy terms of quantum Hamiltonians. The low-energy state corresponds to the superior trajectory scheme with short flight distance, obstacle avoidance, and priority access to high-priority facilities. The initial temperature parameters and annealing rate of the quantum annealing algorithm are set. By simulating the quantum tunneling effect, the system gradually evolves from a high-energy state to a low-energy state. During the annealing process, the system is allowed to accept a state transition with a certain probability of temporarily increasing energy in order to escape the local optimum. After multiple rounds of annealing iterations, the system converges to the stable state with the lowest energy. The quantum bit configuration corresponding to this state represents the optimized inspection path access sequence. The discrete task node access sequence output by the quantum annealing algorithm is converted into a continuous three-dimensional space track. The track is smoothed by Bézier curve or spline interpolation algorithm to meet the dynamic constraints of the flying robot. The generated track is verified to meet the physical limitations such as maximum flight speed, maximum acceleration and minimum turning radius. Track segments that do not meet the constraints are locally adjusted and optimized, and finally an adaptive inspection track is generated.
4. The flying robot control method for an intelligent lighting system according to claim 3, characterized in that, The steps of controlling the flying robot to fly along the adaptive inspection trajectory, performing non-contact inspection of each lighting facility using onboard thermal imaging sensors and a spectral analyzer, and real-time acquiring multi-dimensional detection data including thermal distribution images, spectral characteristic curves, and electromagnetic radiation intensity, and then encrypting and uploading the multi-dimensional detection data to cloud storage via a blockchain network include: The instructions corresponding to the adaptive inspection trajectory are loaded into the flight control system of the flying robot, and the autonomous navigation mode is activated to make the flying robot fly along the predetermined trajectory. When it approaches the target lighting facility, the flight attitude and hovering position of the flying robot are automatically adjusted according to the spatial pose information of the lighting facility stored in the digital twin model, so that the detection optical axis of the thermal imaging sensor and the spectral analyzer are aligned with the key detection part of the lighting facility, and the best observation angle and detection distance are maintained. After the flying robot reaches the preset stable detection posture, the thermal imaging sensor, the spectrum analyzer and the electromagnetic radiation detector are triggered simultaneously to collect data. The thermal imaging sensor captures a two-dimensional temperature distribution image of the surface of the lighting facility to identify abnormal hot spots. The spectrum analyzer measures the spectral characteristic curve emitted by the lighting facility to evaluate the performance of the light source and the color temperature shift. The electromagnetic radiation detector detects the electromagnetic field strength around the lighting facility to determine the electrical safety status. Each set of detection data is automatically labeled with the collection timestamp, facility identifier and spatial location information. The system performs real-time preprocessing on the acquired raw multidimensional detection data, including noise reduction filtering of thermal distribution images, baseline correction of spectral curves, and interference suppression of electromagnetic radiation signals. It judges whether the data quality meets the analysis requirements by evaluating image sharpness, calculating spectral signal-to-noise ratio, and checking signal integrity. It triggers a re-acquisition command for data that does not meet the quality requirements, ensuring that the acquired detection data has the reliability and accuracy required for subsequent intelligent analysis. The quality-verified multidimensional detection data, along with metadata information, is encapsulated into a standardized data packet. An asymmetric encryption algorithm is used to encrypt the data packet to generate ciphertext data. The hash digest value of the data packet is calculated as a digital fingerprint. The ciphertext data, hash digest, and timestamp information are packaged to construct a blockchain transaction request. The integrity of the data is verified through a smart contract, and the transaction record is written into a distributed ledger to ensure the immutability and traceability of the detection data. The encrypted detection data is uploaded to the cloud storage system. A distributed storage strategy is adopted to distribute data copies across multiple physical nodes to improve data reliability. At the same time, a retrieval structure indexed by lighting facility identifier, collection time and data type is established in the cloud database. A mapping relationship between blockchain transaction hash and cloud storage address is generated to realize the rapid location of detection data and version management of historical data, providing efficient data access services for subsequent data analysis and decision support.
5. The flying robot control method for an intelligent lighting system according to claim 4, characterized in that, The process of fusing and analyzing the multidimensional detection data using a deep convolutional neural network to identify abnormal patterns in lighting facilities, including abnormal light decay, overheating faults, and flicker defects, and autonomously generating intelligent decision-making instructions that include fault location, repair priority, and suggested maintenance plans based on the severity of the abnormal patterns, includes: Multidimensional detection data of the target lighting facility is extracted from the cloud storage system. The thermal distribution image is converted into a temperature feature matrix, the spectral feature curve is converted into a spectral intensity vector, and the electromagnetic radiation intensity is converted into a field strength feature sequence. These heterogeneous data are organized according to the time dimension, spatial dimension, and feature dimension to construct a four-dimensional feature tensor. At the same time, the historical detection data and normal operation benchmark data of the target lighting facility are loaded as a comparison reference to provide a standardized input data structure for neural network analysis. The constructed four-dimensional feature tensor is input into a pre-trained deep convolutional neural network model. The spatial pattern features of thermal distribution, the frequency domain features of spectral curves, and the temporal features of electromagnetic radiation are extracted through multiple convolutional and pooling layers. An attention mechanism is used in the middle layer of the network to weight and fuse different modal features, so that the network adaptively focuses on the feature dimension that contributes the most to anomaly recognition. The fused features are mapped to the anomaly pattern classification space through a fully connected layer, and the output includes the classification probability distribution of light decay anomaly, overheating fault, stroboscopic defect and normal state. Based on the classification probability distribution output by the deep convolutional neural network, the category with the highest probability is selected as the current operating status determination result of the lighting facility. When an abnormal state is identified, the activation response map of the intermediate layer of the network is extracted to locate the precise spatial location of the abnormal area on the lighting facility. The confidence index of the abnormal determination is calculated to evaluate the reliability of the diagnosis result. For ambiguous cases with low confidence, a manual review process is triggered or a flying robot is arranged to conduct a second fine inspection to ensure the accuracy and reliability of the abnormal identification. For each identified anomaly pattern, a comprehensive assessment is conducted based on the degree of deviation of the anomaly characteristics, duration, and scope of impact. The severity of the anomaly is divided into four levels: emergency, important, general, and minor. Emergency anomalies indicate that the lighting facilities are at risk of immediate failure or safety hazards. Important anomalies indicate that the facility performance has significantly degraded and requires immediate attention. General anomalies indicate that the facility has potential problems and can be maintained in a planned manner. Minor anomalies indicate that the facility is operating basically normally but requires continuous monitoring. Each anomaly is assigned a quantitative risk score. Based on the anomaly pattern type, severity level, and location information of lighting facilities, the system retrieves matching maintenance plan templates from a pre-built maintenance knowledge base. It automatically generates a detailed maintenance plan that includes a precise fault location description, recommended repair measures, a list of required spare parts, and estimated repair time. According to the risk score and the anomaly distribution of all lighting facilities in the area, a multi-objective optimization algorithm is used to calculate the maintenance priority ranking of each facility. The system generates intelligent decision instructions that include a priority list, repair route suggestions, and resource scheduling plans, and pushes them to operation and maintenance management personnel and on-site maintenance teams through the IoT control platform.
6. The flying robot control method for an intelligent lighting system according to claim 5, characterized in that, The steps of synchronizing the intelligent decision-making commands to the IoT control platform of the intelligent lighting system to trigger remote dimming, zone control, or fault isolation operations of the lighting facilities, and dynamically adjusting the next inspection cycle and key monitoring areas of the flying robot based on changes in ambient light and user feedback, to achieve closed-loop optimization management of the lighting system, include: The intelligent decision-making instructions are synchronously transmitted to the IoT control platform of the intelligent lighting system through a secure communication protocol. The platform's decision analysis module extracts fault location information, maintenance priorities, and suggested maintenance plans. It automatically matches the corresponding control strategy according to the type of abnormal mode. Specifically, it generates dimming compensation instructions for light decay abnormalities to maintain the target illuminance, generates power reduction operation or temporary shutdown instructions for overheating faults to prevent equipment damage, generates drive circuit parameter adjustment instructions for flicker defects, and generates fault isolation instructions for emergency faults to cut off the power supply to the facility and activate backup lighting. The IoT control platform sends the parsed control commands to the smart controllers of each lighting facility through a wireless communication network. After receiving the commands, the smart controllers execute corresponding dimming operations, zone switching control, or fault isolation actions. During the execution process, they monitor the operating parameters of the lighting facilities in real time, such as voltage, current, power, and light output. The execution results and equipment status change information are sent back to the IoT control platform through the feedback channel. The platform verifies whether the control commands have been successfully executed and records the operation log. For commands that fail to be executed, an alarm is triggered and manual intervention is arranged. By continuously collecting real-time environmental data such as ambient light intensity, weather conditions, and human activity density through illuminance sensors, human presence sensors, and environmental monitoring equipment deployed in the lighting area, and collecting user demand data such as lighting effect satisfaction evaluations, illuminance adjustment requests, and fault repair information from the user feedback system, these multi-source data are aggregated into the data analysis module of the IoT control platform to construct a comprehensive dataset that includes environmental change trends and user behavior patterns. Based on the collected environmental perception data and user demand data, combined with the current health assessment results of lighting facilities and historical fault statistics, machine learning algorithms are used to analyze the spatiotemporal patterns and environmental correlations of lighting facility faults, identify time periods, areas and environmental conditions prone to faults, dynamically adjust the next round of inspection cycle of the flying robot, appropriately extend the inspection interval for areas with good health and high user satisfaction, shorten the inspection cycle and increase the inspection frequency for key monitoring areas with frequent faults or many user complaints, and update the inspection task priority list and the list of facilities to be monitored. By integrating the results of control command execution, changes in the operating status of lighting facilities, environmental data, and user feedback data to form a closed-loop feedback information flow, the lighting facility status parameters and operating history in the digital twin model are updated. This accumulated data is used to continuously train and optimize the deep convolutional neural network model to improve the accuracy of anomaly identification. At the same time, the parameter configuration of the quantum annealing algorithm is optimized to improve the trajectory planning effect. Through a data-driven continuous learning mechanism, the detection accuracy, control effect, and resource allocation efficiency of the entire intelligent lighting system are continuously improved over time, realizing the evolution from responsive management to predictive management and adaptive optimization management.
7. A flight robot control system for an intelligent lighting system, configured to execute the flight robot control method for an intelligent lighting system as described in any one of claims 1 to 6, characterized in that, include: Flying robots and servers; The flying robot is configured to: use a multispectral camera and lidar mounted on the flying robot to perform spatial scanning of the lighting facilities in the target area, and acquire three-dimensional point cloud data including the position, attitude and spectral characteristics of the lighting facilities; The server is configured as follows: A digital twin model of the lighting facilities is constructed based on the three-dimensional point cloud data. The digital twin model includes the spatial coordinates, physical parameters, and real-time operating status of each lighting facility. Based on the spatial distribution of lighting facilities in the digital twin model, the quantum annealing algorithm is used to optimize the inspection path of the flying robot, generating an adaptive inspection trajectory that covers all lighting facilities and has the lowest energy consumption. The quantum annealing algorithm uses flight distance, obstacle avoidance and lighting facility priority as constraints for joint optimization. The flight robot is controlled to fly along the adaptive inspection trajectory. It performs non-contact inspection of each lighting facility using the thermal imaging sensor and spectral analyzer it is equipped with. It collects multi-dimensional detection data in real time, including thermal distribution images, spectral characteristic curves and electromagnetic radiation intensity, and uploads the multi-dimensional detection data to the cloud for storage via a blockchain network. The multidimensional detection data is fused and analyzed using a deep convolutional neural network to identify abnormal patterns of lighting facilities, including abnormal light decay, overheating faults, and flicker defects. Based on the severity of the abnormal patterns, intelligent decision-making instructions are autonomously generated, including fault location, repair priority, and suggested maintenance solutions. The intelligent decision-making instructions are synchronized to the IoT control platform of the intelligent lighting system, triggering remote dimming, zone control, or fault isolation operations of the lighting facilities. At the same time, based on changes in ambient light and user feedback, the next round of inspection cycle and key monitoring areas of the flying robot are dynamically adjusted to achieve closed-loop optimization management of the lighting system.
Citation Information
Patent Citations
Unmanned aerial vehicle flight data secure storage system based on block chain
CN120408663A
Industrial robot multi-target path planning method based on quantum annealing algorithm
CN120685101A
Offshore wind turbine inspection monitoring system based on digital twinning
CN121143167A