Ray-vision integrated intrusion early warning device and method thereof

Abstract

The invention belongs to the technical field of road early warning, and particularly relates to a thunder-vision integrated intrusion early warning device and a method thereof. The device mainly comprises a thunder-vision integrated detection box, a photosensitive sensor, a red-blue stroboscopic LED lamp panel and a loudspeaker. Wherein the two red and blue stroboscopic LED lamp panels and the two loudspeakers are respectively installed at two sides of the thunder-vision integrated detection box, the thunder-vision integrated detection box is utilized to design an intrusion early warning method, and the method can complete high-precision detection of an intruded object in different scenes of daytime and night. The thunder and vision integrated detection box can detect an intruding object in front of the equipment through a radar and a camera, sounds are made through a red and blue stroboscopic LED lamp panel and a tweeter, warning is given out through stroboscopic LED light, normal development of construction tasks in an operation area is guaranteed, and meanwhile safety of intruding passersby is prevented.

Description

Technical Field

[0001] The present invention belongs to the technical field of highway warning, and particularly relates to a radar-vision integrated intrusion warning device and method. Background Art

[0002] When a highway is being rebuilt or maintained, the construction unit usually sets obvious construction signs and safety warning signs at both ends of the construction section, such as road construction warning lights or warning signs. If vehicle detours are required, the construction unit should set guiding signs at the detour intersections; if the construction section cannot be detoured, a temporary road needs to be built to ensure the normal passage of vehicles and pedestrians. However, the existing warning devices have limited effects at night or in bad weather, posing safety hazards. For example, when you see the yellow warning light on the construction sign, it indicates that there may be construction, road surface damage, etc. ahead. At this time, you should slow down moderately and stay vigilant. When you see the red light on, you need to stop and wait for the construction team or other vehicles to pass.

[0003] Scientific research data shows that observing traffic rules and paying attention to safety matters in the construction section can greatly reduce the risk of traffic accidents. According to data from the US Traffic Safety Institute, approximately 90% of accidents in construction areas are caused by drivers not observing traffic lights and signs. Especially when vehicles or pedestrians pass through the construction section at night, they need to be extra cautious. At this time, warning signs and warning facilities in the construction area are one of the important measures to effectively reduce highway accidents. If vehicle drivers or pedestrians see the warning lights, warning sounds, and warning signs in the construction area, they can stop and check the situation ahead and take corresponding countermeasures. At night, due to limited visibility, construction signs and safety warning devices can effectively remind passing vehicles and pedestrians of the existence of the construction section and enable them to take corresponding safety measures. The existing highway construction area warning devices mainly include construction signs and safety warning signs. They have good warning effects during the day, but due to insufficient light intensity at night, the warning effects of construction signs and safety warning signs are still insufficient, resulting in certain safety hazards in the construction area. Further research on construction signs and safety warning devices has great practical application value and significance. Summary of the Invention

[0004] The present invention provides a radar-vision integrated intrusion warning device and method to solve the technical problem of insufficient warning effects and safety hazards of warning devices in the prior art at night or in bad weather.

[0005] To achieve the above object, the present invention adopts the following technical solutions: A combined radar and vision intrusion warning device, comprising a combined radar and vision detection box, on which a combined radar and vision detection device is installed for radar scanning and video monitoring; the combined radar and vision detection box is connected to an acoustic-optical warning device, and an embedded controller is arranged inside the acoustic-optical warning device. A photosensitive sensor is fixed above the combined radar and vision detection box, and the photosensitive sensor is connected to the input end of the embedded controller through a connection line, and the output end of the embedded controller is connected to the acoustic-optical warning device.

[0006] The combined radar and vision detection device is embedded in the combined radar and vision detection box. The combined radar and vision detection device includes a camera and a radar. Supplementary lights are evenly and symmetrically installed outside the camera. The camera and the radar are connected to the data input end of the embedded controller through a connection circuit.

[0007] The acoustic-optical alarm device is installed on an installation shell. The combined radar and vision detection box is fixed in the middle position on the front of the installation shell. The acoustic-optical alarm device includes a high-pitched horn and a red-blue strobe light. The high-pitched horn is installed at both ends of the installation shell, and the red-blue strobe light is located between the high-pitched horns at both ends of the installation shell and is evenly distributed.

[0008] The photosensitive sensor is installed on a solar photovoltaic panel. The bottom of the solar photovoltaic panel is connected to a fixed rod, and the lower end of the fixed rod is fixed inside the installation shell. The fixed rod is of a hollow structure.

[0009] A solar photovoltaic cell panel is installed on the solar photovoltaic panel. The photosensitive sensor is fixed above the solar photovoltaic cell panel. The connection lines at the output ends of the solar cell panel and the photosensitive sensor are connected to the embedded controller through the hollow part of the fixed rod.

[0010] A combined radar and vision intrusion warning method, and the warning method is as follows: Step 1: Divide the light intensity within the warning range in a day into different light intensity levels, and divide the warning range into regions according to the visible range of the combined radar and vision intrusion warning device; Step 2: The embedded controller obtains light intensity data, radar detection data, and camera detection data through the photosensitive sensor and the combined radar and vision detection device; Step 3: The embedded controller performs corresponding data fusion on the radar detection data and the camera detection data according to the light intensity level to obtain target data; Step 4: The embedded controller judges the warning area of intrusion according to the target data and generates a warning control instruction. The embedded controller sends the warning control instruction to the warning device, and the warning device performs corresponding acoustic-optical warning processing according to the warning control instruction.

[0011] Before the radar detection data is fused with the camera detection data, the radar detection data and the camera detection data need to be preprocessed. The DBSCAN algorithm is used to cluster the radar detection data to generate point set data frames. Then, the joint probabilistic data association algorithm is used to associate the point set data frame of the current scan with the historical data frame. Finally, the unscented Kalman filter algorithm is applied to filter the historical data frame and the current data frame to obtain the final estimated value. The camera detection data needs to be converted into frame pictures, and then the YOLO-v3 object detection algorithm is used for preliminary object detection processing. Then, the DEEPSORT tracking algorithm is used to reprocess the objects in the video pictures. The preprocessed radar detection data is fused with the camera detection data through coordinate transformation.

[0012] The real-time light intensity in the warning area within 24 hours a day is divided into intensity levels from 1 to 10 from weak to strong. When the light intensity level is greater than or equal to 6 and less than or equal to 10, when fusing the detection data, the detection data of the camera is used as the standard. When associating the detection target data of the radar with the detection target data of the camera to form a data frame, the detection target data of the camera is used as the main associated data. When the light intensity level is greater than 6 and less than 3, when associating the detection target data of the radar with the detection target data of the camera to form a data frame, the common detection target data of the radar and the camera are both used as the main associated data. When the light intensity level is less than or equal to 3, when associating the detection target data of the radar with the detection target data of the camera to form a data frame, the detection target data of the radar is used as the main associated data.

[0013] When the light intensity level is greater than or equal to 6 and less than or equal to 10, the association is performed according to the distance d between the radar detection data and the camera detection data. When the distance d 5, , represents a camera detection data, represents a radar detection data, represents the association relationship between two detection target data; when the distance d 5, , M is the number of detection target data of radar 17 that satisfies d 5; when the light intensity level is greater than 6 and less than 3, if the distance d 5, , if the distance d 5, ; when the light intensity level is less than or equal to 3, if the distance d 5, , if the distance d 5, .

[0014] Within the visible range of the camera, the visible range is divided into fan-shaped areas with different radii. The radii of division decrease in order as L0, L1, L2, L3, L4. The fan-shaped rings between different radii and the fan-shaped area with a radius of L2 are respectively defined as area Z0, area Z1, area Z2, area Z3, area Z4, and the warning levels gradually increase. When the intruding object is determined to be located in area Z0 - Z1, the STM32 controller controls the warning device to activate the first audible and visual warning mechanism. When the intruding object enters area Z2 - Z3, the STM32 controller controls the warning device to activate the second audible and visual warning mechanism. When the intruding object enters area Z4, the STM32 controller controls the warning device to activate the third audible and visual warning mechanism. The operating frequencies and powers of the first audible and visual warning mechanism, the second audible and visual warning mechanism, and the third audible and visual warning mechanism increase in turn.

[0015] Compared with the prior art, the present invention has the following beneficial effects: A radar-vision integrated intrusion warning device and method proposed by the present invention, through the cooperation of a camera and a radar, the device can provide a method for intrusion warning in a highway construction area using radar-vision integration, which can realize the fusion detection of dual-channel sensors of radar and camera. This method is not only applicable to the accurate detection of intruders during the day, but also can effectively detect intruders at night. A solar photovoltaic panel is installed on the upper part of the device, which can provide battery life power for the device when the device is powered off. Through the cooperation of the radar-vision integrated detection box and the red and blue strobe lights and the high-pitched horn, it can timely detect the pedestrians and intruding objects that have strayed in, and warn the intruders in the form of high-frequency strobe lights and voices, thereby improving the effect of safety warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 : Top view of the installation housing of a radar-vision integrated intrusion warning device provided by the invention; Figure 2 : Front view of a radar-vision integrated intrusion warning device; Figure 3 : Figure 2 Front view of the radar-vision integrated detection box of the intrusion warning device shown; Figure 4 : Figure 2 Side view of the housing of the radar-vision integrated detection box of the intrusion warning device shown; Figure 5 : Detection range diagram of the radar-vision integrated intrusion warning device; Figure 6 : Flowchart of the detection method of the radar-vision integrated warning device.

[0017] Label description: 1. Installation housing; 2. Thunder and vision integrated detection box housing; 3. Solar photovoltaic panel; 4. Fixed rod; 5. Thunder and vision integrated detection box; 6. Solar photovoltaic cell panel; 7. Photosensitive sensor; 8. High - pitched horn; 9. Red - blue strobe light; 10. Fastening bracket; 11. Manual switch for controlling the thunder and vision integrated detection box; 12. Manual switch for red - blue strobe light; 13. Manual switch for high - pitched horn; 14. Main switch of the device; 15. Supplementary light; 16. Camera; 17. Radar; 18. Rain - proof and light - shielding edge. Detailed implementation mode

[0018] To further understand the content of the present invention, the following describes the present invention in detail with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments are only for explaining the present invention rather than limiting it.

[0019] The following details the embodiments of the present invention with reference to the accompanying drawings.

[0020] This embodiment proposes a thunder and vision integrated intrusion warning device, and its specific implementation method is as follows: See Figures 1 to 4, a combined radar and camera warning device, comprising a combined radar and camera detection box 5. An embedded combined radar and camera detection device is installed inside the combined radar and camera detection box 5. The combined radar and camera detection device includes a camera 16 and a radar 17. Fill lights 15 are installed on both sides of the camera 16, and the radar 17 is installed below the camera 16. The combined radar and camera detection box 5 is embedded and installed inside the housing 2 of the combined radar and camera detection box. The top of the housing 2 of the combined radar and camera detection box is provided with a rainproof and light-shielding edge 18 extending outwards. The back of the housing 2 of the combined radar and camera detection box is fixed to the middle of the front of an installation housing 1. Alarm devices are arranged on both sides of the housing 2 of the combined radar and camera detection box on the front of the installation housing 1. The alarm devices include a red and blue strobe light 9 and a high-pitched horn 8. The red and blue strobe lights 9 are evenly arranged on the parts of the front of the installation housing 1 on both sides of the housing 2 of the combined radar and camera detection box. The high-pitched horn 8 is arranged outside the red and blue strobe lights 9. A manual switch 11 for the combined radar and camera detection box, a manual switch 12 for the red and blue strobe lights, a manual switch 13 for the high-pitched horn, and a main switch 14 of the device are sequentially arranged at the bottom of the installation housing 1. A fastening device 10 is fixed in the middle of the lower end of the installation housing 1. The fastening device 10 is used to install the combined radar and camera warning device at a warning detection point. A fixing rod 4 is fixed to the upper end of the installation housing 1. The lower end of the fixing rod 4 is installed inside the installation housing 1 through a bearing and a bearing fixing seat. The fixing rod 4 is of a hollow structure. A solar photovoltaic panel 3 is fixedly installed at the top of the fixing rod 4. A photovoltaic panel 6 is installed on the solar photovoltaic panel 3. Three photosensitive sensors 7 are linearly and evenly installed at the upper end of the solar photovoltaic panel 6. The fastening bracket 10 is of a square structure. Screw fixing holes are formed in the four outer side surfaces of the fastening bracket 10 and are distributed in a square array. One ends of the screw fixing holes are all communicated with the inner wall of the fastening bracket 10. The staff will Figure 1 The square fastening bracket 10 at the lower end of the combined radar and camera intrusion warning device shown in the figure is fixedly installed on the installation column at the intrusion detection position in the construction area, and the combined radar and camera intrusion warning device is fixed in the construction area with the screw fixing holes and hand-tightening bolts to provide an intrusion warning function for the construction area.

[0021] The fixed rod 4 on the radar and vision integrated intrusion warning device is connected to the installation housing 1 through a bearing and a bearing fixing seat. The lower end of the fixed rod 4 is installed inside the installation housing 1, and the fixed rod 4 can rotate at different angular positions on the installation housing 1 through the bearing. The upper end of the fixed rod 4 is fixed with a solar photovoltaic panel 3 by a fixed clamp and screws. After the radar and vision integrated intrusion warning device is installed, the angle of the fixed rod 4 is adjusted so that the solar photovoltaic panel 3 can fully receive sunlight. The solar photovoltaic panel 3 is directly installed with a solar photovoltaic cell panel 6 through a buckle. The solar photovoltaic cell panel 6 can convert solar energy into electrical energy and store it in the lithium battery pack of the device and provide the device with the endurance ability in the case of power failure of the device. The fixed rod 4 is of a hollow structure. Three photosensitive sensors 7 are linearly and evenly installed at the upper end of the solar photovoltaic cell panel 6. The connecting wires of the solar photovoltaic cell panel 6 and the photosensitive sensors 7 are connected to the embedded controller inside the installation housing 1 through the hollow part of the fixed rod 4. The solar photovoltaic cell panel 6 can convert solar energy into electrical energy and store it in the lithium battery pack of the device and provide the device with the ability to continue working in the case of power failure of the device. A filter is arranged inside the photosensitive sensor 7. After filtering the light intensity signal of sunlight, the received light intensity signal is transmitted to the embedded controller inside the installation housing 1 through a connecting line.

[0022] According to the Figure 6 flowchart of the radar and vision integrated warning detection method described above, a specific implementation method of a radar and vision integrated intrusion warning device of the present invention is as follows: Perform interruption initialization settings. Measure and set the height of the installation position as the initial installation height H0, set the levelness of the device during installation as the initial levelness L0, and set the light intensity monitored by the photosensitive sensor 7 during installation as the initial light intensity R0. At the same time, the warning area is also divided. As Figure 5 shown in, within the visible range of the camera 16 inside the radar and vision integrated detection box 5, the visible range is divided into fan-shaped areas with different radii, and the divided radii are successively divided into L0, L1, L2, L3, and L4 from large to small. Among them, the fan-shaped ring between the radii L0 and L1 is defined as area Z0, the fan-shaped ring between the radii L1 and L2 is defined as area Z1, the fan-shaped ring between the radii L2 and L3 is defined as area Z2, the fan-shaped ring between the radii L3 and L4 is defined as area Z3, and the fan-shaped area with the radius L2 is defined as area Z4. The warning levels of areas Z0, Z1, Z2, Z3, and Z4 gradually increase. The specific setting of the warning area is as follows: (1), Initial parameter setting: Initial installation height: H0; Initial levelness: L0; Initial light intensity: R 0。

[0023] (2), Warning area division: The visible range of the camera 16 inside the integrated radar and vision detection box 5 is a fan-shaped area, and the division radii are L0, L1, L2, L3, L4 in descending order. Define r as the radial distance from the detection point to the camera 16, which can be specifically defined as: Area Z0: L0 ≤ r < L1; Area Z1: L1 ≤ r < L2; Area Z2: L2 ≤ r < L3; Area Z3: L3 ≤ r < L4; Area Z4: r = L2.

[0024] (3) Definition of warning levels: Let the warning level be W, then the warning levels of each area satisfy: W(Z0) < W(Z1) < W(Z2) < W(Z3) < W(Z4), that is, from area Z0 to Z4, the warning level increases step by step.

[0025] The embedded controller in the present invention uses an STM32 controller, and an MPU6050 chip is used inside the STM32 controller. The chip is a sensor module integrating a three-axis gyroscope and a three-axis accelerometer, which can accurately detect the attitude change of the device. The inclination angle of the intrusion warning device in the three-dimensional space is accurately measured by the MPU6050 six-axis sensor on the STM32 controller, and then the initial level L0 of the attitude of the intrusion warning device and the subsequent real-time level L1 are measured. The horizontal offset value B of the intrusion warning device is calculated through the difference between the initial level L0 and the real-time level L1 of the intrusion warning device, that is: . The STM32 controller is set with a horizontal offset threshold. During the long-term operation of the device, due to the influence of external environmental factors (such as earthquakes, strong winds, equipment vibrations, etc.), the attitude of the device may change. By calculating the horizontal offset value B in real time, the slight changes in the attitude of the device can be detected in time. Once the horizontal offset value B exceeds the set horizontal offset threshold, the STM32 controller sends control instructions to the high-pitched horn 8 and the red-blue stroboscopic light 9 for corresponding acoustic and optical warnings. The specific steps are as follows: 1. Attitude detection and measurement: Let θ0 be the initial level L0, and it is measured by the MPU6050 six-axis sensor: L0 = θ0; Let θ1 be the real-time level L1, and it is measured in real time by the MPU6050 six-axis sensor: L1 = θ1.

[0026] 2. Calculation of the horizontal offset value: The horizontal offset value B is defined as the difference between the real-time level and the initial level: B = L1 - L0 = θ1 - θ0 3. Warning mechanism: Set the horizontal offset threshold as Bthreshold. When the horizontal offset value B meets the following conditions, an alarm is triggered: ∣B∣>Bthreshold After the alarm is triggered, the STM32 controller sends control commands to the high - pitched horn 8 and the red - blue strobe light 9 to start the acoustic - optical alarm.

[0027] Through the above steps, the functions of attitude detection, horizontal offset calculation and alarm of the embedded controller can be completed.

[0028] Sensor detection input and signal processing.

[0029] Three photosensitive sensors 7 arranged linearly on the solar photovoltaic panel 6 detect the real - time light intensity R at the location of the intrusion warning device in real - time. The photosensitive sensor 7 transmits the detected real - time light intensity R to the STM32 controller through the connection circuit. Inside the photosensitive sensor 7 is a photodiode, which is very sensitive to the change of sunlight but not affected by temperature. Therefore, the photosensitive sensor 7 detects the change of the light intensity of sunlight in real - time through the photodiode. Since there is natural light and artificial light in the construction area, the real - time light intensity R within the range where the device is located within 24 hours of a day is divided into 1 - 10, a total of 10 intensity levels from weak to strong. The radar 17 in the radar - vision integrated detection box 15 inputs the radar detection data of the intruding target into the STM32 controller, and the camera 16 in the radar - vision integrated detection box 15 inputs the camera detection data of the monitoring into the STM32 controller. The STM32 controller fuses the radar detection data and the camera detection data according to different light levels. The fusion method of the radar detection data and the camera detection data is as follows: Use the DBSCAN (Density - Based Spatial Clustering of Applications with Noise) algorithm to perform trajectory clustering on a large number of data points captured by radar scanning to generate point - set data frames. DBSCAN identifies clusters through the principle of density reachability and can effectively process noise data. The specific method is as follows: For each data point , calculate the number of points in its neighborhood:

[0030] where, is the data set, is the data point and is the distance between . If is a core point. The radar 17 inside the radar-vision integrated detection box 5 scans multiple moving targets in front of the construction area, including vehicles and pedestrians. The DBSCAN algorithm calculates the number of points within the neighborhood of each data point to identify core points, form clusters of density-reachable points, effectively remove noise data, and improve the accuracy and usability of the data.

[0031] Use the Joint Probabilistic Data Association (JPDA) algorithm to associate the current scanned point set data frame with the historical data frame, considering the association probability between multiple targets. Calculate the association probability between each observation value and each target to improve the association accuracy. The specific method is as follows: Let be the observation value of the current frame be the state prediction value of the previous frame, and calculate the association probability between the observation value and the target :

[0032] where is the likelihood function of the observation value in the state of the target .

[0033] Update the target state estimate:

[0034] where is the Kalman gain, is the observation matrix. is the optimal estimate of the target t state at time k, is the predicted value of the target t state based on the information at time k-1 at time k, is the target distance and speed information within the warning area scanned by the radar 17 at time k.

[0035] Apply the UKF (Unscented Kalman Filter) algorithm to filter the historical data frame and the current data frame to obtain the final estimated value. The UKF algorithm processes the nonlinear system through unscented transformation, avoiding the calculation requirement of the Jacobian matrix and improving the filtering accuracy. The specific method is as follows: Select Sigma points: , where is the state dimension, is the scaling parameter, is the state covariance matrix.

[0036] Propagate Sigma points: , where is the state transition function.

[0037] Calculate the predicted state and covariance: ;

[0038] where represents the predicted covariance of the state estimate based on the information at time k - 1 at time k, which is used to measure the uncertainty of the state estimate; and represent the weight coefficients of the i-th Sigma point, and different Sigma points correspond to different weights; represents the predicted value of the i-th Sigma point at time k based on the state at time k - 1; represents the predicted value of the state at time k based on the state at time k - 1; represents the process noise covariance matrix at time k.

[0039] Update the state and covariance:

[0040]

[0041]

[0042] After updating the state and covariance through the unscented Kalman filter algorithm, a more accurate and reliable target state estimation result can be obtained, providing a solid foundation for subsequent intrusion warning decisions.

[0043] The processing method for the video data detected by camera 16 is as follows: Convert the video into the original frame pictures, and use the YOLO-v3 object detection algorithm to perform preliminary object detection processing on the video acquisition data. This method has the advantages of high detection accuracy and strong real-time performance; then use the DeepSORT (Deep learning based SORT, a visual object tracking algorithm based on deep learning) tracking algorithm to reprocess the objects in the video pictures, especially to perform re-tracking processing on the occluded targets. The specific processing method for the video data of camera 16 is as follows: 1. Video frame extraction: Let the video data be V, and convert it into a frame sequence {F1, F2…, F n}, where Fi represents the i-th frame image: V → {F1, F2,…, F n} 2. Object Detection: The YOLO-v3 object detection algorithm is used to process each frame Fi to obtain the object detection result Di: Di = YOLO-v3(Fi) Among them, Di contains the bounding box Bi and class probability Pi of the detected object.

[0044] 3. Object Tracking: The DeepSORT algorithm is used to track the object detection result Di to generate the tracking result Ti: Ti = DeepSORT(Di) DeepSORT realizes continuous tracking of occluded objects by combining the appearance features and motion information of the objects.

[0045] 4. Data Processing Flow: The overall data processing flow can be expressed as V → {F1, F2…, Fn} → {D1, D2,…, Dn} → {T1, T2,…, Tn} Through the above steps, frame extraction, object detection, and object tracking of video data processing can be achieved.

[0046] Coordinate Transformation. In order to achieve synchronization in the spatial dimension between Radar 17 and Camera 16, combined with the device attitude detection data, coordinate transformation is carried out using the coordinate systems where the radar and the camera are located, so as to transform the data in the coordinate system where the radar is located to the pixel coordinate system. During the coordinate transformation process, the origin of the camera coordinate system and the origin of the radar coordinate system are made to coincide. The transformation process is as follows: The two-dimensional radar coordinate system is transformed into a three-dimensional world coordinate system, then into a three-dimensional camera coordinate system, then into a two-dimensional image coordinate system, and finally into a two-dimensional pixel coordinate system. After coordinate transformation, the detection data of Radar 17 and Camera 16 are in the same two-dimensional plane. In the two-dimensional pixel coordinate system, the target coordinates detected by Radar 17 and the target coordinates detected by Camera 16 are measured, and the distance d between the two target coordinates is calculated.

[0047] Data Fusion. In practical applications, the objects detected by the radar-vision integrated detection box 5 are divided into three major categories: the objects detected by Radar 17, the objects detected by Camera 16, and the objects jointly detected by Radar 17 and Camera 16. The present invention uses the criterion of virtual warning area setting and minimum statistical distance to associate the detection data of Radar 17 and Camera 16, and then uses the light intensity as the main target decision condition to make a fusion decision on the warning objects in different scenarios for the final object determination result. The specific process of processing the data fusion of the radar and the camera according to different light levels is as follows: Three sunlight sensors 7 are linearly installed on the upper part of the solar photovoltaic panel 6 to detect the light intensity R at the location of the real-time detection device. Different calculation methods are selected according to the difference between the current detection value and the previous detection value of the light intensity R, and then the final calculated value S of the light intensity is calculated. The STM32 controller is set with a light intensity difference threshold. If the difference between the current detection value and the previous detection value of the light intensity R is greater than or equal to the light intensity difference threshold, the previous detection value of the light intensity R is the final calculated value S of the light intensity; if the difference between the current detection value and the previous detection value of the light intensity R is less than the light intensity difference threshold, the weighted average method is used to update the detection value. The calculated values of the light intensity corresponding to the three photosensitive sensors 7 The calculation method is as follows:

[0048] In the formula, represents the weighting coefficient, which can be adjusted according to the reliability of the sensor, and the default value is 0.2. represents a small positive constant, k = 1, 2, 3. represents the current detection value of the sunlight sensor, represents the previous detection value of the sunlight sensor, represents the detection calculation value of the k-th sunlight sensor, represents the previous detection calculation value of the k-th sunlight sensor, The final calculated value S of the sunlight intensity of the device

[0049] In the formula, represents the weight of each sensor, satisfying .

[0050] When the real-time light intensity is 6, the light is good at this time. When fusing the detection data, the detection data of the camera 16 is used as the standard. When correlating the detection target data of the radar 17 with the detection target data of the camera 16 in the data frame, the detection target data of the camera 16 is used as the main correlation data. At the same time, correlation is performed according to the distance d between the detection target data of the radar 17 and the detection target data of the camera 16. When the distance d is 5, one detection target data of the camera is associated with one detection target data of the radar, that is , represents one detection target data of the camera 16, represents one detection target data of the radar 17, represents the association relationship between the two detection target data; when the distance d is 5, , multiple detection target data of the radar 17 The detection target data associated with a camera 16 after averaging , where M is the number of detection target data of the radar 17 that satisfies d 5.

[0051] When 6 > S > 3, considering normal lighting, when associating the detection target data of the radar 17 and the detection target data of the camera 16 with the data frame, the common detection target data of the radar 17 and the camera 16 are used as the main associated data. In addition, according to the distance d between the detection target data of the radar 17 and the detection target data of the camera 16, when the distance d 5, the detection target data of one camera 16 is associated with the detection target data of one radar 17 , that is ; when the distance d 5, the detection target data of multiple radars 17 and cameras 16 are averaged and then data association is performed, that is: .

[0052] When S 3, considering poor lighting, the fused detection data is based on the radar detection data. When associating the detection target data of the radar 17 and the detection target data of the camera 16 with the data frame, the radar detection target data is used as the main associated data. In addition, according to the distance between the radar detection target data and the camera detection target data, when the distance d < 5, the detection target data of one camera 16 is associated with the detection target data of one radar 17, that is: ; when the distance d ≥ 5, the detection target data of multiple cameras 16 are averaged and then associated with the detection target data of one radar 17, that is: .

[0053] For each frame of data N , data association is performed according to the above rules, and the state of the target data is updated and then the associated target data is output. Through the above algorithm, data fusion of the radar and the camera can be realized for different lighting levels in the actual system.

[0054] Intrusion object detection. According to as Figure 5The five warning areas divided as shown are, in sequence, area Z0, area Z1, area Z2, area Z3, and area Z4. According to the division scope of the warning areas, first determine the intrusion area. For the object that intrudes into the warning range, the target position information obtained after data fusion of the detection target data of radar 17 and the detection target data of camera 16 is compared with the division scope of each warning area to determine the warning area where the object that intrudes into the warning range is located. Determine the moving object. By analyzing the position information of the target in multiple consecutive time frames, if the position of the target changes significantly in different time frames, it is determined as a moving object; if the position remains basically unchanged, it may be a stationary object. Determine the characteristic attributes of the object. Combining the detection data of radar 17 and camera 16, through the analysis and processing of the detection data, determine the characteristic attributes of the intruding object. Start the intrusion timing. The embedded controller presets a warning area residence time threshold. If the residence time of the intruding object is greater than the warning area residence time threshold, it is determined as an intruding object, and corresponding warning processing is performed according to the warning area where it is located, and continue to detect and judge the next moving object.

[0055] Warning processing. The corresponding warning processing for the five warning areas is as follows: When the intruding object is determined to be located in areas Z0 - Z1, the system activates the low-frequency sound and light warning mechanism. After the STM32 controller inside the device receives the signal that the intruding object is in the Z0 - Z1 area, according to the preset program, it calls the low-frequency warning voice file stored in the memory, amplifies the voice signal through the audio amplification circuit, and drives the high-pitched horn 8 to emit sound. At the same time, the STM32 controller controls the light driving circuit to start the red and blue strobe lights 9 at a lower frequency. When the intruding object enters areas Z2 - Z3, it means that the risk level has increased, and the system switches to medium-frequency sound and light warning. The STM32 controller adjusts the audio output parameters of the high-pitched horn 8, increases the volume and frequency of the warning voice, so that the sound emitted by the high-pitched horn 8 is louder and clearer and can be heard at a farther distance; at the same time, the STM32 controller adjusts the parameters of the light driving circuit to increase the flashing frequency of the red and blue strobe lights 9. When the intruding object enters area Z4, which is the highest-level warning area, the system immediately activates the high-frequency sound and light warning. The STM32 controller turns the volume of the high-pitched horn 8 to the maximum and continuously broadcasts the emergency warning voice in a high-frequency manner. At the same time, the STM32 controller increases the flashing frequency of the red and blue strobe lights 9 to the highest, emitting strong and fast-flashing red and blue warning lights.

[0056] Since the warning levels of areas Z0, Z1, Z2, Z3, and Z4 gradually increase, therefore, the warning content, the volume of the warning horn, and the red and blue strobe lights for warning are set to different frequencies and levels to conduct targeted persuasion and evacuation processing for the intruding personnel.

[0057] If the radar-vision integrated detection and warning device is in special weather conditions such as rain, snow, wind, thunderstorm, and sandstorm, the detection target data of the radar 17 and the detection target data of the camera 16 will both show abnormal data monitoring. The abnormal data is processed separately according to different weather conditions, and the non-associated processing reduces the impact of weather conditions on the detection results, ensuring that the radar-vision integrated detection and warning device can still perform normal detection and warning in harsh weather. The specific steps of non-associated processing are as follows: I. Abnormal data monitoring Under special weather conditions, the detection data of the radar and the camera may have noise, loss, or deviation. First, it is necessary to detect the abnormal data. Define the following variables: : The set of target data detected by the radar 17, ; : The set of target data detected by the camera 16, ; : The noise threshold of the radar target detection data; : The noise threshold of the camera target detection data.

[0058] For the target data detected by the radar 17 , if its deviation from the historical data detected by the radar 17 exceeds , it is determined as abnormal data, that is: ， Among them, is the mean value of the historical data detected by the radar 17.

[0059] For the target data detected by the camera 16 , if its deviation from the historical data detected by the camera 16 exceeds , it is determined as abnormal data, that is: , among which, is the mean value of the historical data detected by the camera 16.

[0060] II. Abnormal data correction (1) Correction of the detection data of the radar 17: If the target data detected by the radar 17 is determined to be abnormal, it is corrected by the method of weighted average of its adjacent data:

[0061] Among them, is 's adjacent data, is the weight, and this value is usually inversely proportional to the distance. If it cannot be corrected, the data is excluded.

[0062] (2) Correction of the detection data of camera 16 If the target data detected by camera 16 is determined to be abnormal, it is corrected by the method of the weighted average of its adjacent data:

[0063] Among them, is adjacent data of, is the weight, and this value is usually inversely proportional to the distance. If it cannot be corrected, the data is excluded.

[0064] III. Fusion of non-associated data For the target data detected by radar 17 that cannot be matched and the target data detected by camera 16 , they are processed independently: If the target data detected by radar 17 cannot be matched, it is predicted according to historical data and environmental conditions:

[0065] Among them, is the weight coefficient, usually .

[0066] If the target data detected by camera 16 cannot be matched, it is predicted according to historical data and environmental conditions:

[0067] Among them, is the weight coefficient, usually .

[0068] IV. Final target determination After the abnormal data correction and non-associated data processing, the corrected radar data and camera data are obtained. The final target determination result is generated by the following rules: (1) If the detection data of radar 17 and camera 16 match successfully, the fused radar detection data and camera detection data are used as the final target determination result.

[0069] (2) If the data of the radar 17 and the camera 16 cannot be matched, the main data source is selected according to the light intensity R: according to the different light intensity R, the calculated value S of the light intensity of the photosensor 7 is calculated, and the main data source is selected according to the calculated value S of the light intensity. When it is less than 6, the corrected data of the camera 16 is taken as the main; when it is greater than or equal to 6, the corrected data of the radar 17 is taken as the main.

[0070] Through the above method, non-associated data processing can effectively correct the abnormal data of the radar 17 and the camera 16 under special weather conditions, and generate reliable target determination results through data fusion and prediction methods. This method improves the robustness and accuracy of the radar-vision integrated detection device in complex environments.

[0071] In addition, it should be understood that although this specification is described according to the embodiments, not every embodiment only contains an independent technical solution. The narrative way of this specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art. The above content is only to illustrate the technical idea of the present invention, and the protection scope of the present invention cannot be limited by this. Any modification made on the basis of the technical solution according to the technical idea proposed by the present invention falls within the protection scope of the claims of the present invention.

Claims

1. A radar-visual integrated intrusion warning device, characterized in that: The invention comprises a radar and vision integrated detection box (5), on which a radar and vision integrated detection device is installed, and the radar and vision integrated detection device is used for radar scanning and video monitoring; the radar and vision integrated detection box (5) is connected to a sound and light warning device, and an embedded controller is arranged in the sound and light warning device; a photosensitive sensor (7) is fixed above the radar and vision integrated detection box (5), and the photosensitive sensor (7) is connected to the input end of the embedded controller through a connecting line, and the output end of the embedded controller is connected to the sound and light warning device.

2. The integrated radar and visual intrusion warning device according to claim 1, characterized in that: The integrated thunder and vision detection device is embedded in the integrated thunder and vision detection box (5), and comprises a camera (16) and a radar (17). Filling lights (15) are evenly and symmetrically installed outside the camera (16). The camera (16) and the radar (17) are connected to the data input terminal of the embedded controller via a connecting circuit.

3. The integrated radar and visual intrusion warning device according to claim 2, characterized in that: The sound and light alarm device is fixedly mounted on the mounting shell (1), the integrated thunder and vision detection box (5) is fixed at the middle position of the front face of the mounting shell (1), the sound and light alarm device comprises a tweeter (8) and a red and blue strobe light (9), the tweeter (8) is mounted at both ends of the mounting shell (1), and the red and blue strobe light (9) is located between the tweeters (8) at both ends of the mounting shell (1) and is evenly distributed.

4. The integrated radar and visual intrusion warning device according to claim 3, characterized in that: The light-sensitive sensor (7) is mounted on the solar photovoltaic panel (3); the bottom of the solar photovoltaic panel (3) is connected via a fixing rod (4); the lower end of the fixing rod (4) is fixed inside the mounting housing (1); and the fixing rod (4) is a hollow structure.

5. The integrated radar and visual intrusion warning device according to claim 4, characterized in that: A solar photovoltaic cell panel (6) is mounted on the solar photovoltaic panel (3); a light sensor (7) is fixed above the solar photovoltaic cell panel (6); and a connection line between the output end of the solar panel (6) and the light sensor (7) is connected to an embedded controller via a hollow portion of the fixing rod (4).

6. A method for early warning of an intrusion with integrated lightning and vision, based on a device for early warning of an intrusion with integrated lightning and vision as claimed in any one of claims 1 to 5, characterized in that: The warning methods are as follows: Step 1: Divide the light intensity within the warning range within one day into different light intensity levels, and divide the warning range into regions according to the visual range of the integrated radar and visual intrusion warning device; Step 2: The embedded controller obtains light intensity data, radar detection data and camera detection data through the light-sensitive sensor (7) and the integrated radar and visual detection device; Step 3: The embedded controller performs corresponding data fusion on the radar detection data and the camera detection data according to the light intensity levels of different light intensities to obtain the target data; Step 4: The embedded controller determines the warning area that the target has entered based on the target data and generates a warning control instruction. The embedded controller sends the warning control instruction to the warning device, and the warning device performs corresponding sound and light warning processing according to the warning control instruction.

7. The method for early warning of intrusion with integrated radar and vision according to claim 6, characterized in that: Before the radar detection data is fused with the camera detection data, the radar detection data and the camera detection data need to be preprocessed; the DBSCAN algorithm is used to cluster the radar detection data to generate a point set data frame, and then the joint probability data association algorithm is used to associate the current scanned point set data frame with the historical data frame, and finally the unscented Kalman filter algorithm is used to filter the historical data frame and the current data frame to obtain the final estimated value; the camera detection data needs to be converted into a frame image, and then the YOLO-v3 target detection algorithm is used for preliminary target detection processing, and then the DEEPSORT tracking algorithm is used to reprocess the objects in the video image, and the preprocessed radar detection data is fused with the camera detection data through coordinate transformation.

8. The method for early warning of intrusion with integrated radar and vision according to claim 7, characterized in that: The real-time light intensity of the warning area within 24 hours of a day is divided into intensity levels of 1 to 10 from weak to strong; when the light intensity level is greater than or equal to 6 and less than and equal to 10, the fusion detection data is based on the detection data of the camera (16), and when the detection target data of the radar (17) is associated with the detection target data of the camera (16) in the data frame, the detection target data of the camera (16) is used as the main associated data; when the light intensity level is greater than 6 and less than 3, when the detection target data of the radar (17) is associated with the detection target data of the camera (16) in the data frame, the common detection target data of the radar (17) and the camera (16) are used as the main associated data; when the light intensity level is less than or equal to 3, when the detection target data of the radar (17) is associated with the detection target data of the camera (16) in the data frame, the radar detection target data is used as the main associated data.

9. The method for early warning of intrusion with integrated radar and vision according to claim 8, characterized in that: When the light intensity level is greater than or equal to 6 and less than or equal to 10, the radar detection data and the camera detection data are associated according to the distance d. 5 o'clock, , Represents a camera detection data, Represents a radar detection data, Indicates the correlation between two detection target data; when the distance d 5 o'clock, , M is to satisfy d 5, the number of target detection data of the radar 17; when the light intensity level is greater than 6 and less than 3, if the distance d 5. , if the spacing d 5. ; When the light intensity level is less than or equal to 3, if the spacing d 5. , if the spacing d 5. .

10. The method for early warning of intrusion with integrated radar and vision according to claim 6, characterized in that: Within the visible range of the camera (16), the visible range is divided into fan-shaped areas with different radii, and the radii are L0, L1, L2, L3, and L4 from large to small. The fan-shaped rings between different radii and the fan with a radius of L2 are defined as area Z0, area Z1, area Z2, area Z3, and area Z4, respectively, and the warning level increases gradually; when the intruder is determined to be located in area Z0-Z1, the STM32 controller controls the warning device to start the first sound and light warning mechanism; when the intruder enters area Z2-Z3, the STM32 controller controls the warning device to start the second sound and light warning mechanism; when the intruder enters area Z4, the STM32 controller controls the warning device to start the third sound and light warning mechanism; the frequency and power of the first sound and light warning mechanism, the second sound and light warning mechanism, and the third sound and light warning mechanism increase in sequence.

Images