Multi-algorithm fusion intelligent lighting system target detection method and system
The integration of video and millimeter wave radar data with multiple algorithms improves smart lighting systems' detection accuracy and adaptability, addressing precision and adaptability issues in complex environments.
Patent Information
- Application Number
- CN202510358050.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-25
- Publication Date
- 2025-07-15
AI Technical Summary
The existing intelligent lighting systems have insufficient detection accuracy in environments with dense people, complex scenes or frequent light changes, and have a single algorithm and poor adaptability, which affects the accuracy of lighting control and energy saving effect.
By combining video images and millimeter-wave radar data, a variety of algorithms are used for fusion detection, including data acquisition, preprocessing, association matching, fusion recognition and joint string frame analysis, identifying target location, trajectory, speed and action, and using deep learning, spatiotemporal feature fusion and adaptive multi-level weight allocation algorithms, high-precision target recognition is achieved.
It improves the accuracy and adaptability of target detection, can accurately identify target information in complex environments, provides high-precision control basis for intelligent lighting systems, and improves the accuracy of lighting control and energy saving effect.
Smart Images
Figure CN120318901A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent lighting systems, and particularly to a target detection method and system for an intelligent lighting system integrating multiple algorithms. Background Art
[0002] With the rapid development of smart homes and intelligent buildings, intelligent lighting systems have gradually become an important technical means to improve the quality of life and energy utilization efficiency. Traditional intelligent lighting systems mostly rely on simple sensors or preset control strategies, and cannot accurately perform dynamic adjustment according to the position of people, activity status, and ambient light conditions.
[0003] In recent years, intelligent lighting systems based on target detection have gradually emerged. By using visual sensors such as cameras, the real-time monitoring of the position of people is realized, and lighting devices are adjusted accordingly. However, most of the existing systems have problems such as insufficient detection accuracy, single algorithm, and poor adaptability to complex environments, making it difficult to meet the increasingly diverse intelligent lighting needs. Especially in environments with dense people, complex scenes, or frequent light changes, the detection effect is often unsatisfactory, thus affecting the accuracy of lighting control and the energy saving effect. Summary of the Invention
[0004] The present invention aims to provide a target detection method and system for an intelligent lighting system integrating multiple algorithms. By collecting video image and millimeter-wave radar data, and combining multiple algorithms to detect and identify targets, accurate identification of the position, trajectory, speed, and actions of targets is achieved, thereby providing a high-precision basis for the control of intelligent lighting systems.
[0005] The technical solution adopted by the present invention to solve its technical problems is as follows:
[0006] A target detection method for an intelligent lighting system integrating multiple algorithms, comprising the following steps:
[0007] Step S1: Data collection, collecting video image data and millimeter-wave radar data of targets in the environment through a video image acquisition device and a millimeter-wave radar.
[0008] Step S2: Data preprocessing, preprocessing the collected video image data and millimeter-wave radar data. Among them, enhancing the video image, grouping and caching the radar point cloud data, and filtering the radar point cloud data using the Gaussian filtering formula to remove the noise of the radar point cloud data, and performing basic feature recognition and extraction on moving targets.
[0009] Among them, the image enhancement uses the following formula:
[0010]
[0011] Among them, h out(k) represents the cumulative distribution function value of the output image at gray level k, and n i is the number of pixels of the input image at gray level i, and L is the total number of gray levels.
[0012] The optical flow method is used to identify and extract the basic features in the video image, and the suspected targets in the image are identified:
[0013]
[0014] Among them, Ix, Iy, and It are the spatial gradient and temporal gradient of the image in the x and y directions respectively, and u and v are the moving speeds of the pixel points in the x and y directions.
[0015] Step S3: An association matching algorithm is adopted to perform primary fusion matching on the basic features of each group of video data and the radar point cloud data, and the fusion algorithm type for target recognition of each group of data is output.
[0016] Among them, primary fusion matching is to fuse and match the basic features of the video image with the radar point cloud data to quickly determine what kind of algorithm should be used for fusion calculation of the current video data and radar data.
[0017] An association matching function is constructed to measure the similarity of the target basic features in the video data and the radar point cloud data. The formula is:
[0018]
[0019] Among them, Sim pre (i, jj) represents the similarity between the basic features of the i-th video target and the basic features of the j-th radar target, and F video_base (i) and F radar_base (j) are the basic feature vectors of the video target and the radar target respectively. Primary feature fusion matching is performed according to the similarity. When Simpre(i, j) is greater than the set threshold, it is considered that the two belong to the same target.
[0020] The fusion algorithm type selection model calculates the applicability score of each fusion algorithm type according to factors such as the proportion of the number of successful target pairs in the primary matching, the complexity of the basic features of the video image (such as the irregularity of the target shape, the diversity of colors, etc.), and the stability of the basic features of the radar point cloud (such as the change rate of the target speed, the fluctuation degree of the position, etc.). The formula of the fusion algorithm selection model is:
[0021] S alg = w1·N match + w2·C video + w3·S radar
[0022] Among them, S algis the applicability score of a certain type of fusion algorithm, w1, w2, and w3 are the weight coefficients of the three factors: the proportion of the number of successfully matched target pairs, the complexity of the basic features of the video image, and the stability of the basic features of the radar point cloud. match is the ratio of the number of video and radar target pairs that are successfully matched at the first level, C video It is the complexity index of the basic features of the video image, which can be calculated by analyzing the irregularity of the target shape, the diversity of colors and other features. rradar It is a stability index of the basic features of radar point cloud, which can be obtained by calculating the rate of change of target speed, degree of position fluctuation and other features.
[0023] After calculating the suitability scores of all fusion algorithm types, the fusion algorithm type with the highest score is selected as the final fusion algorithm for the current data.
[0024] Fusion algorithm types include deep learning fusion algorithm, spatiotemporal feature fusion algorithm, adaptive multi-level weight allocation fusion algorithm, and attention feature map fusion algorithm.
[0025] Step S4: For the video data and point cloud data of each area, the fusion algorithm determined in step S3 is used to perform target recognition after secondary fusion of the data, including the type, position, speed, movement, trajectory and expected trajectory of the target.
[0026] Among them, for the identified scenes or areas with high dependence on historical data, such as the movement patterns of pedestrians or other targets showing certain regularity in different time periods, a deep learning fusion algorithm is used to improve the YOLOv5s algorithm in a deep learning way to quickly identify the target data; for the identified scenes or areas that pay more attention to the target behavior, a spatiotemporal feature fusion algorithm is used to quickly identify the target data by analyzing the spatiotemporal features such as the movement trajectory and residence time of people or vehicles; for the identified scenes or areas with large changes in environmental conditions such as light intensity, interfering light sources, and cluttered radar point cloud features, an adaptive multi-level weight allocation fusion algorithm is used to adaptively adjust the weight coefficients of the loss values of each layer in the YOLOv5s algorithm according to the lighting conditions in the lighting space, the position of the interfering light source, the time of occurrence, etc., to quickly identify the target data; for the identified scenes or areas with important target features, according to the trained attention feature map, the YOLOv5s algorithm is focused on the key feature area of the target to be identified, the feature identification of the feature area is enhanced, and the target data is quickly identified.
[0027] Step S5: Using a joint frame algorithm, the target data of each area is jointly analyzed in chronological order to form a data sequence of each target in the entire area. After formatting the target data sequence, it is sent to the lighting control module of the intelligent lighting system to control the space lighting fixtures.
[0028] Specifically, the target data at different times and in different regions are sorted and integrated according to timestamps to construct a target data sequence. The formula is as follows:
[0029] S = {G1, G2,..., G n}
[0030] where S is the target data sequence, and G i represents the target data set at the i-th time point, including information such as the target positions, speeds, types, and trajectory vectors in each region. Through the joint frame concatenation algorithm, a comprehensive understanding of the dynamic changes of targets in the entire region is achieved, providing data support from a global perspective for formulating subsequent lighting control strategies.
[0031] Preferably, for the filtering algorithm in step S2, one of median filtering, Kalman filtering, and low-order Butterworth filtering can also be selected.
[0032] Preferably, for the deep learning algorithm in step S4, one or several of Faster R-CNN, Mask R-CNN, and SSD can be introduced according to different scenarios.
[0033] The present invention also provides a system for implementing the above-mentioned multi-algorithm fusion intelligent lighting system target detection method, including:
[0034] Data acquisition module: used to acquire video image data and millimeter-wave radar data of the target through a video image acquisition device and a millimeter-wave radar.
[0035] Among them, the video image acquisition device selects a high-definition camera with adjustable focal length and aperture to adapt to the shooting requirements of different scenarios; the millimeter-wave radar selects a suitable working frequency band and detection distance according to the actual application scenario, and is equipped with a corresponding signal processing circuit to ensure the accuracy and reliability of the data.
[0036] Data preprocessing module: preprocess the acquired video image data and millimeter-wave radar data. Among them, enhance the video image, frame and cache the radar point cloud data, and filter it using the Gaussian filtering formula to remove the noise of the radar point cloud data, and identify and extract the basic features of moving targets.
[0037] Primary correlation matching module: perform primary fusion matching on the basic features of each group of video data and the radar point cloud data, and output the fusion algorithm type for target recognition of each group of data.
[0038] Secondary fusion recognition module: For the video data and point cloud data of each region, using the fusion algorithm determined in the primary correlation matching module, perform target recognition after secondary fusion of the data, including the type, position, speed, action, trajectory, and predicted trajectory of the target.
[0039] Joint frame series module: Analyze the target data of each region jointly in chronological order to form a sequence of target data in the entire region.
[0040] Result output module: After formatting the target data sequence, send it to the lighting control module of the intelligent lighting system to control the space lighting fixtures.
[0041] Preferably, the filtering algorithm in the data preprocessing module can also be one of median filtering, Kalman filtering, and low-order Butterworth filtering.
[0042] Advantages of the present invention compared with the prior art:
[0043] 1. Improve detection accuracy: By combining video images and millimeter-wave radar data and using multiple algorithms for fusion detection calculation, it is possible to more accurately detect and identify information such as the position, trajectory, speed, and action of the target, thereby improving the detection accuracy.
[0044] 2. Strong adaptability: According to factors such as the scene, the relative position of people and vehicles, and the position of the lighting fixtures, select multiple algorithms for fusion calculation to adapt to the complex environment within the system monitoring area and at different times. After partitioning and identifying the environment, package the unified recognition result data. Brief Description of the Drawings
[0045] Figure 1 It is a schematic structural diagram of the multi-algorithm fusion target detection system of the present invention.
[0046] Figure 2 It is a schematic diagram of the multi-algorithm fusion target detection method of the present invention. Detailed Embodiments
[0047] To make the purpose, technical solutions, and advantages of the present invention more clear, the following will describe the technical solutions in the embodiments of the present invention in more detail with reference to the accompanying drawings in the embodiments of the present invention. In the drawings, the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions from beginning to end. The described embodiments are some embodiments of the present invention, rather than all embodiments. The embodiments described below by referring to the drawings are exemplary and are intended to explain the present invention, and should not be construed as a limitation to the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention. The following further specifically describes the technical solutions of the present invention with reference to the accompanying drawings, and gives an embodiment of the present invention.
[0048] In the intelligent lighting system of a certain urban road, video image data and millimeter-wave radar data of vehicles and pedestrians are collected through video image acquisition devices and millimeter-wave radars installed on street lights. The camera captures images with a resolution of 1080p at a rate of 25 frames per second, covering an area of about 50 meters on both sides of the intersection and the road; the millimeter-wave radar operates in the 77GHz frequency band, has a detection range of 200 meters, and has high-precision speed and distance measurement capabilities.
[0049] After the intelligent lighting system runs, the camera collects images with a resolution of 1080p at a rate of 25 frames per second, and the coverage area is about 50 meters on both sides of the intersection and the road. The millimeter-wave radar operates in the 77GHz frequency band, with a detection range of 200 meters, and can measure the speed and distance of the target with high precision.
[0050] The histogram equalization technology is used to enhance the image contrast, so that pedestrians and vehicles can maintain good clarity under different lighting conditions (such as at night, on rainy and cloudy days).
[0051] The radar point cloud data is denoised by Gaussian filtering, and the processed point cloud data is clustered. The DBSCAN clustering algorithm is used to divide the point cloud into multiple target clusters, and eigenvalue such as the center coordinates, speed, and radial distance of each cluster are extracted.
[0052] Set a similarity threshold, generally taking a value between 0.6 and 0.8, and fuse and match the basic features of the video image with the radar point cloud data to quickly determine the type of fusion algorithm for the current data.
[0053] Set a primary matching weight value, and calculate the applicable scores of the fusion algorithm types in different video and radar monitoring areas. Generally, the weight value should be set according to the system scenario. In the road lighting system, the weight of the proportion of the number of pairs of successfully matched video and radar targets is 0.4, the weight of the complexity index of the basic features of the video image is 0.3, and the weight of the stability index of the basic features of the radar point cloud is 0.3.
[0054] On a straight lane with dense traffic, a deep learning fusion algorithm is used. The Faster R-CNN model pre-trained on the COCO dataset is adopted. Through transfer learning, it is transferred to the urban road scenario and fine-tuned on the local dataset (1000 urban road scenario images, annotated with targets such as pedestrians, cars, and bicycles). When fine-tuning, the SGD optimizer is used, the learning rate is 0.001, and it is trained for 10 epochs.
[0055] At an intersection with mixed traffic of people and vehicles, a spatio-temporal feature fusion algorithm is used. The dense optical flow method is used to calculate the motion vectors of pixels between consecutive frames. Combining the position and speed information of the targets, the motion trajectories of vehicles and pedestrians are accurately predicted to accurately track the trajectories of pedestrians and non-motor vehicles.
[0056] At a small intersection with a large number of pedestrians and non-motor vehicles, the YOLOv5s fusion algorithm with CBAM attention feature maps is used to focus on the recognition of small targets and enhance the detection ability of pedestrians and non-motor vehicles.
[0057] The target data of the straight lanes and each intersection in the system monitoring area are combined in series according to the time sequence to form a time series of regional target data. The time series is input into the intelligent lighting system. The lighting system further determines common targets such as pedestrians, cars, and bicycles based on the time series, and determines whether the pedestrian is walking, whether it is likely to fall, whether it is avoiding vehicles, whether it is carrying large items, etc., and automatically adjusts the brightness and angle of the street lights to keep the illuminance in the area around the pedestrian between 300 - 500 lux. When a vehicle approaches, the brightness of the street lights is appropriately reduced to avoid glare for the driver, ensuring the safe passage of pedestrians and vehicles in key areas.
[0058] The parts not elaborated in detail in the present invention are common knowledge in the art.
[0059] Through the above embodiments, the purpose of the present invention is fully and effectively achieved. Those skilled in the art can understand that the present invention includes but is not limited to the content described in the drawings and the above specific embodiments. Although the present invention has been described with respect to the currently considered most practical and preferred embodiments, it should be understood that the present invention is not limited to the disclosed embodiments, and any modification that does not deviate from the functional and structural principles of the present invention will be included in the scope of the claims.
Claims
1. An object detection method for an intelligent lighting system with multi-algorithm fusion, characterized in that, It includes the following steps: Step S1: Data acquisition, where video image data and millimeter-wave radar data of targets in the environment are acquired through video image acquisition devices and millimeter-wave radars; Step S2: Data preprocessing, where the acquired video image data and millimeter-wave radar data are preprocessed. Among them, the video images are enhanced, the radar point cloud data is framed and cached, and Gaussian filtering formula is used for filtering to remove the noise of the radar point cloud data, and the basic features of moving targets are identified and extracted; Step S3: Primary fusion matching, where an association matching algorithm is used to perform primary fusion matching on the basic features of each group of video data and radar point cloud data, and the fusion algorithm type for target recognition of each group of data is output; Step S4: Secondary fusion and target recognition, where for the video data and point cloud data in each area, using the fusion algorithm determined in the primary fusion matching, the data is secondarily fused and then target recognition is performed, including the type, position, speed, action, trajectory and predicted trajectory of the target; Step S5: Joint serial frame analysis, where a joint serial frame algorithm is used to perform joint analysis on the target data in each area in chronological order to form a data sequence of each target in the entire area. After formatting the target data sequence, it is sent to the lighting control module of the intelligent lighting system to control the space lighting fixtures.
2. The target detection method of the intelligent lighting system with multi-algorithm fusion according to claim 1, characterized in that, In the data preprocessing step, histogram equalization technology is used for image enhancement, and its formula is: where h out (k) represents the cumulative distribution function value of the output image at gray level k, and n i is the number of pixels of the input image at gray level i, and L is the total number of gray levels.
3. The object detection method of the intelligent lighting system with multi-algorithm fusion according to claim 1, characterized in that, In the data preprocessing step, an optical flow method is used to identify and extract the basic features in the video image, and its formula is: where Ix and Iy are the spatial gradients of the image in the x and y directions respectively, It is the temporal gradient, and u and v are the moving speeds of the pixel points in the x and y directions.
4. The object detection method of the intelligent lighting system with multi-algorithm fusion according to claim 1, characterized in that, In the primary fusion matching step, an association matching function is constructed to measure the similarity of the basic features of the targets in the video data and radar point cloud data, and its formula is: Among them, Sim pre (i, j) represents the similarity between the i-th video target basic feature and the j-th radar target basic feature, and F video_base (i) and F radar_base (j) are the basic feature vectors of the video target and the radar target respectively.
5. The object detection method for the intelligent lighting system with multi-algorithm fusion according to claim 1, characterized in that In the primary fusion matching step, the formula for the fusion algorithm type selection model is: S alg = w1·N match + w2·C video + w3·S radar Among them, S alg is the applicability score of a certain type of fusion algorithm. w1, w2, and w3 are the weight coefficients of three factors: the number of successfully matched target pairs, the complexity of the basic features of the video image, and the stability of the basic features of the radar point cloud. N match is the number of video and radar target pairs with successful first-level matching. C video is the complexity index of the basic features of the video image. S radar is the stability index of the basic features of the radar point cloud.
6. The object detection method of the intelligent lighting system with multi-algorithm fusion according to claim 1, characterized in that In the joint serial frame analysis step, the formula for constructing the target data sequence is: S = {G1, G2,..., G n} Among them, S is the target data sequence, and G i represents the target data set at the i-th time point, including information such as the target positions, speeds, types, trajectory vectors, etc. of each region.
7. A system adopting the target detection method of the multi-algorithm fusion intelligent lighting system according to any one of claims 1-6, characterized in that, It includes the following modules: Data acquisition module: Acquire video image data and millimeter-wave radar data of the target through video image acquisition devices and millimeter-wave radars; Data preprocessing module: Preprocess the acquired video image data and millimeter-wave radar data. Among them, the video images are enhanced, the radar point cloud data is framed and cached, and Gaussian filtering formula is used for filtering to remove the noise of the radar point cloud data, and the basic features of moving targets are identified and extracted; Primary association matching module: Perform primary fusion matching on the basic features of each group of video data and radar point cloud data, and output the fusion algorithm type for target recognition of each group of data;; Secondary fusion recognition module: For the video data and point cloud data in each area, using the fusion algorithm determined in the primary association matching module, perform secondary fusion on the data and then perform target recognition, including the type, position, speed, action, trajectory and predicted trajectory of the target; Joint serial frame module: Jointly analyze the target data in each area in chronological order to form a data sequence of each target in the entire area; Result output module: After processing the format of the target data sequence, it is sent to the lighting control module of the intelligent lighting system to control the space lighting fixtures.