Intelligent lamp pole and curve-approaching vehicle early warning method based on intelligent lamp pole

Abstract

This invention discloses a smart light pole and a method for warning of oncoming vehicles on curves based on the smart light pole, belonging to the field of traffic safety technology. The method uses an infrared camera to acquire video images, which are then preprocessed and input into an improved YOLOv8 target detection algorithm model to identify vehicle positions. If no vehicle is detected, a fog-free image is generated using an infrared image defogging module, and the YOLOv11 algorithm is used for a second vehicle position detection. Simultaneously, radar is used to acquire vehicle position information, which is then fused with the vehicle position information obtained from the video to complete vehicle motion estimation. Different levels of warnings are triggered based on the distance between the vehicle and the light pole, thereby improving the vehicle detection rate and warning response delay in low visibility conditions. This method is suitable for curved road sections in mountainous and suburban areas, reducing the occurrence of traffic accidents.

Description

Technical Field

[0001] This invention relates to the field of traffic safety, specifically to a smart light pole and a method for warning of oncoming vehicles on curves based on the smart light pole. Background Technology

[0002] Streetlights are lighting facilities used on roads. Most existing streetlights are non-smart, with limited functionality, especially in mountainous, winding roads. Mountainous areas have numerous curves and extremely complex terrain, and are frequently shrouded in fog. Traditional lighting methods are insufficient to guarantee reliable warnings of oncoming vehicles on curves. Existing methods either rely on a single sensor, perform poorly in fog, or have high computational costs. Due to obstructed visibility, drivers cannot anticipate oncoming vehicles at curves, significantly increasing the probability of collisions. However, a smart streetlight method for warning of oncoming vehicles on curves, based on infrared image defogging and radar assistance, can effectively coordinate infrared image defogging and accurate warnings of oncoming vehicles on curves, improving road traffic safety. Summary of the Invention

[0003] To achieve the above objectives, the present invention provides a smart light pole and a method for warning of oncoming vehicles on curves based on the smart light pole.

[0004] The present invention adopts the following technical solution:

[0005] A method for warning of oncoming vehicles on curves based on smart light poles includes the following steps:

[0006] S1. Acquire multiple consecutive video frames and generate infrared preprocessed images;

[0007] S2. First, the infrared preprocessed image is transmitted to the improved YOLOv8 target detection algorithm to identify the vehicle location information, and then the process jumps to step S4. If the vehicle cannot be identified, the image is transmitted to the infrared image dehazing module to perform dehazing processing and obtain the reconstructed fog-free image data.

[0008] S3. The reconstructed fog-free image data is transmitted to the YOLOv11-based target detection algorithm to identify vehicle location information;

[0009] S4. The radar detects the vehicle's position information and performs coordinate system matching with the vehicle's position information identified in step S2 or step S3.

[0010] S5. Fuse the vehicle detection results obtained from the two sensors to estimate the vehicle's motion;

[0011] S6. If vehicles are detected on both sides of the smart light pole, calculate the distance between the predicted positions of the vehicles on both sides and the smart light pole;

[0012] S7. If the distance between the predicted position of a vehicle on only one side and the smart light pole is less than the warning distance, turn on the LED warning light;

[0013] S8. Continue to calculate the distance between the predicted positions of vehicles on both sides and the smart light pole. If the distance between the predicted positions of vehicles on both sides and the smart light pole is less than the warning distance, turn on the loudspeaker and LED warning lights to remind the driver to drive carefully.

[0014] Furthermore, taking advantage of the fact that infrared detection is unaffected by light fog, this invention employs a two-stage target detection strategy: the first stage uses the improved YOLOv8 algorithm to perform rapid detection on the original infrared image to identify vehicles in clear and light fog scenes; if no target is detected, the defogging module is triggered to reconstruct the image; the second stage uses the YOLOv11 algorithm, which has a deeper structure and stronger feature extraction capabilities, to perform fine detection on the defogging image to ensure recognition accuracy in low visibility scenes.

[0015] Furthermore, the multiple consecutive video frames are acquired by an infrared camera installed on the device; the warning distance is 60 meters.

[0016] Furthermore, step S1 specifically involves acquiring multiple consecutive infrared video images of the curved area using an infrared camera, and performing preprocessing operations on the acquired raw infrared images, including image denoising and contrast enhancement, to generate a preprocessed infrared image. The specific operations are as follows:

[0017] The image denoising method adopts a nonlocal mean filtering method, which eliminates noise by using the texture and structural information of similar blocks in the image through grayscale conversion, block processing and similar block matching.

[0018] The contrast enhancement employs adaptive histogram equalization, which divides the image into multiple local blocks and performs histogram equalization on each local block. This stretches the image's histogram to the entire range of local pixel values, thereby increasing the difference between pixel values ​​and improving the visibility of the infrared image.

[0019] Furthermore, step S2 uses an improved YOLOv8 target detection algorithm, including optimizations to the backbone network, connection network, and detection network. The specific improvements are as follows:

[0020] The optimization of the backbone network involves inserting an SCConv module after the C2f module of the original YOLOv8 backbone network to form a C2f_SCConv module, in order to suppress feature redundancy.

[0021] The optimization of the connection network is based on the Slimneck idea, using GSConv to replace ordinary convolution and VoVGSCSP module to replace C2f module, in order to reduce parameter redundancy and improve information transmission efficiency.

[0022] The detection network is optimized and lightweighted by merging the original three-channel detection network into a single-channel detection network, which adopts a combination of SCConv+Conv structure.

[0023] Furthermore, in step S2, if the vehicle cannot be identified, the infrared preprocessed image is first cropped, and the cropped infrared preprocessed image is transmitted to the infrared image dehazing module to generate a superpixel image and estimate the global atmospheric light value. and transmittance Derive the fog-free image The specific formula is as follows:

[0024]

[0025] in, It is a degraded image;

[0026] The infrared image dehazing module includes the following steps:

[0027] A hierarchical subdivision SLIC algorithm based on image texture and region merging is used, with an initial number of superpixels. The image is coarsely segmented to form large superpixel blocks, introducing the image's entropy value. :

[0028]

[0029] in, It is the number of grayscale intervals; It is the probability of each gray-level interval obtained through histogram statistics, when the superpixel block entropy value When it is greater than the threshold Q, for Perform segmentation and set the number of sub-superpixels. Further segmentation After two-level segmentation, a series of finer sub-superpixels are obtained. A region adjacency map is constructed, and the average brightness of adjacent superpixels is calculated. With texture entropy To reasonably represent region similarity :

[0030]

[0031]

[0032]

[0033] in, and It is a superpixel block of adjacent nodes; g and h are two weight factors, and the sum of g and h is 1; when When it is greater than the threshold of 0.8, and Highly similar, merge sub-superpixels to obtain the final superpixel segmentation map, traverse each superpixel block, calculate the average brightness value of each superpixel block, and mark the superpixel blocks with the top 5% brightness values ​​as glare blocks based on the statistical sorting of the average brightness values ​​of all superpixel blocks.

[0034] The initial segmentation generates coarse superpixels, and each superpixel is labeled as... :

[0035]

[0036] in It is superpixel The total number of pixels in the image All are labeled with the corresponding superpixels. :

[0037]

[0038] in Representing pixels Next, the image is divided into four equal-sized parts, and the labels appearing in the part with the largest average value are counted as φ. A new sub-block is constructed using the superpixels corresponding to the labels in φ. :

[0039]

[0040] in It is the new sub-block formed in the nth iteration. Finally, the sub-block... Perform a new round of quadtree subdivision, repeating this process until every pixel within the largest average quarter of the subtree belongs to the same label, resulting in the final block. The average brightness value is atmospheric light. The accurate estimate:

[0041]

[0042] in This represents the brightness value of a pixel. It is a region The total number of pixels in;

[0043] Introducing mean square error contrast Represents the variance of pixel values, for each reconstructed superpixel. It can be represented as:

[0044]

[0045] Where N is the number of superpixels. It is the average brightness value of the superpixel. It is the transmittance of each superpixel, as... Decrease The value will continue to increase, but for 16-bit infrared images, the pixel value cannot exceed the valid range of data bits. Input values ​​in the range [0, α] and [β, 65535] will be truncated to 0 or 65535. For simplicity, the sum of squares of the truncated values ​​is used to represent the information loss cost, and its expression is:

[0046]

[0047] Where α represents the lower cutoff boundary and β represents the upper cutoff boundary. This represents the brightness value of each pixel in the reconstructed image, and the formula must satisfy the following constraints:

[0048]

[0049] Based on the constraints and the formula derived from fog-free infrared images, we can conclude that:

[0050]

[0051] The value varies Increase and decrease, choose full foot The minimum value of this formula is taken as the optimal transmittance. The transmittance of each superpixel can be calculated using the following formula:

[0052]

[0053] in It is a constant used to control the degree of defogging;

[0054] After deriving the preliminary fog-free infrared image based on the formula for deriving fog-free infrared images, the previously marked glare blocks are processed:

[0055]

[0056] Where, α = 0.7, This represents the pixel value at position x in the final fog-free infrared image. This represents the pixel value at position x in the preliminary fog-free infrared image;

[0057] The final fog-free image is obtained.

[0058] Furthermore, in step S4, the infrared camera and radar are installed on the same smart light pole with the same coordinate origin, and coordinate system matching is achieved through coordinate rotation to realize the coordinate system unification of the data.

[0059] Furthermore, step S5, which involves estimating the motion of the vehicle, is as follows: based on the state vector of each target vehicle, which includes the position coordinates of the target vehicle and its lateral and longitudinal velocities in the global coordinate system, the average prediction and estimated state uncertainty covariance matrix of each target vehicle's position is obtained using kinematic methods based on a constant velocity model.

[0060] The predicted state of the target vehicle's position and speed is projected onto the sensor measurement space to obtain the projection mean and covariance matrix. The difference between the predicted state and the measured value is reconciled, and the state and covariance of the target vehicle are corrected by fusing the infrared camera and radar measurements.

[0061] After multiple iterations, the distance between the vehicle and the smart light pole is calculated using the corrected position coordinates of the target vehicle.

[0062] Specifically, in the prediction phase, the vehicle detection results obtained from the two sensors are fused to estimate the vehicle's motion, and the state vector of each vehicle is calculated:

[0063]

[0064] and It is the center position of the target vehicle, and and These represent the lateral and longitudinal velocities in the global coordinate system, respectively, and are the estimated values ​​for each target vehicle at the current step size. Based on a constant velocity model, the state of each target vehicle at the current moment is estimated using kinematic methods, specifically:

[0065]

[0066]

[0067] , This represents the average predicted position of the target vehicle between the previous and current moments. , This represents the covariance matrix of the estimated state uncertainty between the previous time step and the current time step. It is a state transition matrix. This is the system noise matrix, and T denotes the transpose operation. The state vector of the sensor measurements is given by the following equation:

[0068]

[0069]

[0070] and These represent the state vectors from the infrared camera and radar measurements, respectively. In the state vectors, , The vehicle's position was measured by an infrared camera, and , , , The vehicle position and velocity are measured by millimeter-wave radar. The predicted target vehicle is projected onto the sensor measurement space using the following equation:

[0071]

[0072]

[0073] and This indicates that the estimated projection mean and covariance matrix are used in the prediction phase. This is the sensor measurement transformation matrix, where index j represents two different sensors: an infrared camera and a millimeter-wave radar. The value of j can be c or r, where c represents the infrared camera and r represents the millimeter-wave radar. The Kalman filter assumes that the predicted state and the two measurements follow a Gaussian distribution. To reconcile the difference between the predicted state and the measurements from the two sensors, the optimal estimate is obtained by multiplying the probability density function of the predicted state by the two measurements. First, the infrared camera measurements are multiplied, then the radar measurements are multiplied. After multiplying the infrared camera measurements, the target vehicle's state and covariance are:

[0074]

[0075]

[0076]

[0077] in, This indicates the state of the target vehicle after multiplying the infrared camera measurements. This represents the covariance of the target vehicle after multiplying the measurements from the infrared cameras. , T represents the transpose operation. It is a matrix called the Kalman filter gain, which minimizes the covariance, and This is the measurement noise from the infrared camera. Using the same multiplication method, the radar measurements are further used to correct the target vehicle's state and covariance. After radar correction, the target vehicle's state and covariance can be written as:

[0078]

[0079]

[0080]

[0081] in, This indicates the state of the target vehicle after multiplying the millimeter-wave radar measurements. This represents the covariance of the target vehicle after multiplying the millimeter-wave radar measurements. , T represents the transpose operation. It is a matrix called the Kalman filter gain. It is the radar measurement noise:

[0082] After multiple iterations, use the corrected and The distance between the vehicle and the smart light pole is calculated as follows:

[0083]

[0084] in, This indicates the predicted distance between the vehicle and the smart light pole.

[0085] Furthermore, in step 3, the YOLOv11 target detection algorithm model performs multi-scale feature fusion and attention mechanism processing on the dehazed image and radar data, and the data processing adopts semi-supervised learning and data augmentation strategies. The specific scheme is as follows:

[0086] The multi-scale fusion module constructs a feature pyramid by extracting multi-scale features through convolutional kernels of different sizes (5×5, 3×3, 1×1), which correspond to global fog distribution, local target structure, and detailed texture, respectively. It also uses cross-layer connections to fuse features of different scales and uses an adaptive weighting mechanism to suppress background noise and enhance the response of the target region.

[0087] The attention mechanism is a combination of spatial attention and channel attention. Spatial attention generates a spatial weight map through the interaction of the position dimension and the channel dimension, focusing on the target area (such as vehicles and pedestrians) and suppressing uniform fog interference. Channel attention is based on adaptive threshold segmentation, adjusting the RGB channel weights, enhancing the luminance channel features, and alleviating the problem of decreased image contrast in foggy weather. RGB represents the colors of the three channels: red (R), green (G), and blue (B).

[0088] The semi-supervised learning method trains the model using a small amount of labeled data and a large amount of unlabeled data. For labeled data, supervised cross-entropy loss and mean squared error loss functions are used to guide the model learning. For unlabeled data, consistency regularization based on continuity and clustering assumptions is used to make the model make similar predictions for all data points within the potential increase radius. Adding perturbations does not affect the output, making full use of the potential information in the unlabeled data.

[0089] The data augmentation strategy involves using AutoAugmentation to perform random cropping, translation, rotation, automatic contrast adjustment, negativeing, histogram equalization, threshold inversion, bit reduction, contrast control, color balance, brightness adjustment, blurring, random erasure, and sample overlay on images. This allows the model to learn object features under different foggy environmental conditions, improving the model's generalization ability in foggy road curve scenarios. AutoAugmentation refers to an automatic data augmentation technique.

[0090] The present invention also provides a smart light pole, including an infrared camera, a millimeter-wave radar, an LED warning light, a speaker, a processor, a communication module, and a power supply module for powering the above components, all mounted on the light pole.

[0091] The infrared camera and millimeter-wave radar are installed at the same height on the light pole facing the curve area, covering the vehicle trajectory on both sides of the curve area, and the detection direction of the millimeter-wave radar overlaps with the detection range of the infrared camera.

[0092] LED warning lights and speakers are installed on the light poles so that they are within the driver's field of vision in both directions;

[0093] The processor, communication module, and power module are all housed in an equipment box at the top of the light pole.

[0094] Furthermore, the specific requirements for this smart light pole are as follows:

[0095] The infrared camera uses a 16-bit depth infrared imaging sensor with a resolution of no less than 1280×1024, a frame rate of 30fps, and an effective detection distance of ≥200 meters.

[0096] The LED warning light has a single-lamp brightness of ≥5000cd / m² and a response delay of ≤50ms;

[0097] The millimeter-wave radar operates in the 77GHz frequency band, with a detection radius of ≥200 meters and a positioning accuracy of ±0.1 meters.

[0098] The processor uses an embedded AI computing unit, and its GPU is an NVIDIA Jetson Xavier.

[0099] It has 8GB of RAM and 256GB of storage space.

[0100] The speaker 4 has a sound pressure level ≥90dB, supports custom voice broadcast content, and has a voice playback delay ≤50ms;

[0101] The communication module supports 5G / Ethernet dual-mode communication with a data upload latency of ≤10ms.

[0102] The power module 5 has a maximum power consumption of ≤100W under full load and a standby power consumption of ≤15W.

[0103] The housing and mounting bracket are made of aluminum alloy with an IP67 protection rating.

[0104] This invention can support continuous operation in an environment of -30℃ to 70℃ and all weather conditions. Its mean time between failures is ≥50,000 hours in an environment of 25℃, and the system can still maintain 80% detection accuracy when a single sensor fails, with a false alarm rate of <0.1%.

[0105] Beneficial effects:

[0106] This invention discloses a smart light pole and a method for warning of oncoming vehicles on curves based on the smart light pole. The smart light pole integrates technologies such as infrared image target detection, defogging, multi-sensor fusion, and vehicle prediction. By acquiring video images, infrared target detection is first performed. If vehicle information cannot be identified, defogging is performed to determine the vehicle's position. This information is then fused with vehicle information acquired by radar to improve vehicle position detection accuracy. An optimized algorithm is used to accurately predict the vehicle's future position, and the distance between the vehicle and the smart light pole device is calculated. This distance is compared with the warning distance to determine whether to issue an alarm response, thereby achieving a warning of oncoming vehicles on curves, reminding drivers to drive cautiously, and reducing the incidence of traffic accidents. Attached Figure Description

[0107] Figure 1 This is a schematic flowchart of the method of the present invention;

[0108] Figure 2 Here is a flowchart of the superpixel segmentation process;

[0109] Figure 3 Here is a flowchart of the infrared defogging process;

[0110] Figure 4 Flowchart for obtaining vehicle information;

[0111] Figure 5 This is a schematic diagram of the smart light pole device of the invention. Detailed Implementation

[0112] The present invention will be described in detail below with reference to the embodiments and accompanying drawings, but this is not intended to limit the scope of protection of this application.

[0113] Example 1

[0114] The smart light pole and the curve approach warning method based on the smart light pole disclosed in this invention are applicable to winding roads in mountainous areas, especially for scenarios where heavy fog is frequent, traditional lighting facilities have limited effectiveness, and drivers' visibility is obstructed, leading to a high risk of collisions on curves. The core objective is to achieve accurate warnings of oncoming vehicles on curves by integrating infrared image target detection, defogging processing, multi-sensor data fusion, and vehicle motion estimation technologies, thereby reducing the traffic accident rate. It can support continuous operation in all weather conditions from -30℃ to 70℃, maintain 80% detection accuracy even when a single sensor fails, and has a false alarm rate of <0.1%.

[0115] refer to Figure 1 This invention discloses a smart street light vehicle approach warning method based on infrared target detection and defogging and radar assistance, comprising the following steps;

[0116] S1. An infrared camera captures multiple consecutive video images of the curved area and preprocesses them to generate an infrared preprocessed image;

[0117] S2. The infrared preprocessed image is transmitted to the improved YOLOv8 target detection algorithm model to identify vehicle location information, and then the process jumps to step S4. If the vehicle cannot be identified, the image is transmitted to the infrared image dehazing module to perform dehazing processing and obtain the reconstructed fog-free image data.

[0118] S3. Transmit the reconstructed fog-free image data to the YOLOv11-based target detection algorithm model to identify vehicle location information;

[0119] S4. The radar detects the vehicle's position information and performs coordinate system matching with the vehicle's position information identified in step S2 or step S3.

[0120] S5. Fuse the vehicle detection results after coordinate system matching and estimate the vehicle motion;

[0121] S6. If vehicles are detected on both sides of the smart light pole within the curve area, calculate the distance between the predicted positions of the vehicles on both sides and the smart light pole in real time;

[0122] S7. When the calculated distance between the predicted position of the vehicle on only one side and the smart light pole is less than the preset warning distance, the LED warning light will be turned on.

[0123] When the calculated predicted distances between the vehicles on both sides and the smart light pole are both less than the preset warning distance, the loudspeaker and LED warning lights will be activated.

[0124] The specific steps are as follows:

[0125] S1: Infrared preprocessing image acquisition and preprocessing;

[0126] The infrared camera is activated to capture multiple consecutive frames of infrared video images of the curved area in real time, with the acquisition frequency consistent with the camera's frame rate (30 frames per second). The video is then segmented to obtain the original infrared image. A non-local mean filtering method is used to first convert the original infrared image to grayscale, converting the color image (if any) to grayscale. Next, block processing is performed to divide the image into several sub-blocks of the same size. Finally, similar block matching is used to eliminate image noise and retain vehicle target features by utilizing the texture and structural information of similar sub-blocks in the image. Then, adaptive histogram equalization technology is applied to divide the preprocessed image into multiple local blocks. Histogram equalization is performed on each local block separately, stretching the histogram of the local block to the entire range of local pixel values, increasing the difference between pixel values, improving the distinction between the vehicle and the background in the infrared image, and generating an infrared preprocessed image.

[0127] S2: Vehicle target detection and defogging based on the improved YOLOv8 target detection algorithm model;

[0128] First, the preprocessed infrared image is fed into the improved YOLOv8 vehicle target detection algorithm. This algorithm inserts an SCConv module after the C2f module of the original YOLOv8 backbone network to form a C2f_SCConv module. The feature filtering capability of the SCConv module is used to suppress redundant features and focus on vehicle target features. Based on the Slimneck idea, GSConv is used to replace ordinary convolution, reducing the number of convolution operation parameters. The VoVGSCSP module is used to replace the C2f module, improving feature transfer efficiency and reducing the computational load of the model. Lightweight processing is also performed, merging the original three-channel detection network into a single-channel detection network. The single-channel network adopts an "SCConv+Conv" structure combination, which improves the operation speed while ensuring detection accuracy. If the algorithm successfully identifies the vehicle location information, it directly proceeds to step S4. If the vehicle cannot be identified, dehazing is performed.

[0129] The defogging process specifically involves the following steps: if a vehicle cannot be identified, the infrared preprocessed image is first cropped, then the cropped infrared preprocessed image is transmitted to the infrared image defogging module to generate a superpixel image and estimate the global atmospheric light value. and transmittance Derivation of fog-free infrared images The specific formula is as follows:

[0130]

[0131] in, It is a degraded image, referring to a pre-processed infrared image that has been cut and contains fog.

[0132] Superpixel images are region partitioning maps obtained by segmenting the pre-processed infrared image using an improved SLIC algorithm during the dehazing process of infrared images. Their function is to more accurately recover the haze-free infrared image by using the brightness and texture information within the superpixel blocks, thereby improving the estimation accuracy of atmospheric physical model parameters and obtaining a clearer haze-free infrared image.

[0133] refer to Figure 2 , Figure 3 Infrared preprocessing for image dehazing first utilizes the SLIC algorithm, a layered subdivision algorithm based on image texture and region merging, to coarsely segment the image with a preset initial number of superpixels, forming large-size superpixel blocks and introducing the image's entropy value. :

[0134]

[0135] in, It is the number of grayscale intervals; It is the probability of each gray-level interval obtained through histogram statistics.

[0136] When the i-th superpixel block entropy value When it is greater than the threshold Q, for Perform segmentation and set the number of sub-superpixels. Further segmentation After two-level segmentation, a series of finer sub-superpixels are obtained. A region adjacency map is constructed, and the average brightness of adjacent superpixel blocks is calculated. , With texture entropy , Similarity is used to reasonably represent region similarity. :

[0137]

[0138]

[0139]

[0140] in, and It is a superpixel block of adjacent nodes; g and h are two weight factors, and the sum of g and h is 1; when When it is greater than the threshold of 0.8, and Highly similar, merge sub-superpixels to obtain the final superpixel segmentation map, traverse each superpixel block, calculate the average brightness value of each superpixel block, and mark the superpixel blocks with the top 5% brightness values ​​as glare blocks based on the statistical sorting of the average brightness values ​​of all superpixel blocks.

[0141] The initial segmentation generates coarse superpixels, and each superpixel is labeled as... :

[0142]

[0143] Among them, This represents the index of the superpixel, with a value ranging from 1 to s. It is the total number of superpixels, each pixel in the image. All are labeled with the corresponding superpixels. :

[0144]

[0145] in Representing pixels Next, the image is divided into four equal parts, and the labels appearing in the part with the largest average value are counted. , used with The superpixel corresponding to the label in the code constructs a new sub-block. :

[0146]

[0147] in It is the new sub-block formed in the nth iteration. Finally, the sub-block... Perform a new round of quadtree subdivision, repeating this process until every pixel within the largest average quarter of the subtree belongs to the same label, resulting in the final block. The average brightness value is the global atmospheric light value. The accurate estimate:

[0148]

[0149] in This represents the brightness value of a pixel. It is a region The total number of pixels in;

[0150] Introducing mean square error contrast The variance of pixel values, the mean squared error of each reconstructed superpixel, and the contrast ratio. It can be represented as:

[0151]

[0152] Where N is the number of superpixels. It is the average brightness value of the superpixel. It is the transmittance of each superpixel, as... Decrease The value will continue to increase, but for 16-bit infrared images, the pixel value cannot exceed the valid range of data bits. Input values ​​in the range [0, α] and [β, 65535] will be truncated to 0 or 65535. For simplicity, the sum of squares of the truncated values ​​is used to represent the information loss cost, and its expression is:

[0153]

[0154] Where α represents the lower cutoff boundary and β represents the upper cutoff boundary. This represents the brightness value of each pixel in the reconstructed image, min[·,·]. 2 Max[·,·] represents the sum of squares of pixel values ​​below 0. 2 This represents the sum of squares of the pixel values ​​exceeding 65535; and the formula must satisfy the following constraints:

[0155]

[0156] Based on the constraints and the formula derived from fog-free infrared images, we can conclude that:

[0157]

[0158] The value varies The transmittance increases while decreasing; the minimum value satisfying this formula is selected as the optimal transmittance. The transmittance of each superpixel can be calculated using the following formula:

[0159]

[0160] in It is a constant used to control the degree of defogging;

[0161] After deriving the preliminary fog-free infrared image based on the formula for deriving fog-free infrared images, the previously marked glare blocks are processed:

[0162]

[0163] Where, α = 0.7, This represents the pixel value at position x in the final haze-free image. This represents the pixel value at position x in the initial haze-free image;

[0164] The final fog-free image data is obtained.

[0165] S3: Vehicle detection based on the YOLOv11 object detection algorithm model;

[0166] The fog-free image data reconstructed in step S2 is input into the target detection algorithm model based on YOLOv11. The YOLOv11 target detection algorithm model is used to identify the vehicle's position information by recognizing the high detection accuracy of small and fast-moving targets in clear images and outputting the vehicle's coordinate parameters in the image coordinate system.

[0167] S4: Coordinate system matching;

[0168] When the infrared camera and millimeter-wave radar are installed on the same smart light pole, their coordinate origins are consistent, eliminating the need for translation calibration. It is only necessary to rotate the vehicle position information coordinates identified by the infrared camera and the vehicle three-dimensional coordinates detected by the millimeter-wave radar to the Northeast (YX) coordinate system: through a preset rotation matrix (determined based on the smart light pole installation azimuth angle), the vehicle position information coordinates identified by the infrared camera and the vehicle three-dimensional coordinates detected by the millimeter-wave radar (both sharing the same rotation matrix) are converted into planar coordinates in the Northeast coordinate system, thus unifying the coordinate systems of the two sensor data.

[0169] S5: Reference Figure 4 The steps for estimating the motion of a vehicle are as follows:

[0170] Based on the state vector of each target vehicle, which includes the position coordinates of the target vehicle and its lateral and longitudinal velocities in the global coordinate system, the average prediction and covariance matrix of the state uncertainty of each target vehicle are obtained using kinematic methods based on the constant velocity model.

[0171] The predicted state of the target vehicle's position and speed is projected onto the sensor measurement space to obtain the projection mean and covariance matrix. The difference between the predicted state and the measured value is reconciled, and the state and covariance of the target vehicle are corrected by fusing the infrared camera and radar measurements.

[0172] After multiple iterations, the distance between the vehicle and the smart light pole is calculated using the corrected position coordinates of the target vehicle.

[0173] The specific calculation steps are as follows:

[0174] The state vector of each vehicle is defined as:

[0175]

[0176] and It is the center position of the target vehicle, and and These represent the lateral and longitudinal velocities in the global coordinate system, respectively, and are the estimated values ​​for each target vehicle at the current step size. Based on a constant velocity model, the state of each target vehicle at the current moment is estimated using kinematic methods, specifically:

[0177]

[0178] , This represents the average predicted position of the target vehicle between the previous and current moments. , This represents the covariance matrix of the estimated state uncertainty between the previous time step and the current time step. It is a state transition matrix. This is the system noise matrix, and T denotes the transpose operation. The state vector of the sensor measurements is given by the following equation:

[0179] and These represent the state vectors from the infrared camera and radar measurements, respectively. In the state vectors, , The vehicle's position was measured by an infrared camera, and , , , The vehicle's position and velocity are measured by millimeter-wave radar. The predicted position and velocity of the target vehicle are projected onto the sensor measurement space using the following equation:

[0180]

[0181] and This indicates that the estimated projection mean and covariance matrix are used in the prediction phase. This is the sensor measurement transformation matrix, where index j represents two different sensors: an infrared camera and a millimeter-wave radar. The value of j can be c or r, where c represents the infrared camera and r represents the millimeter-wave radar. The Kalman filter assumes that the predicted state and the two measurements follow a Gaussian distribution. To reconcile the difference between the predicted state and the measurements from the two sensors, the optimal estimate is obtained by multiplying the probability density function of the predicted state by the two measurements. First, the infrared camera measurements are multiplied, then the radar measurements are multiplied. After multiplying the infrared camera measurements, the target vehicle's state and covariance are:

[0182]

[0183]

[0184]

[0185] in, This indicates the state of the target vehicle after multiplying the infrared camera measurements. This represents the covariance of the target vehicle after multiplying the measurements from the infrared cameras. , T represents the transpose operation. It is the matrix of the Kalman filter gain, which minimizes the covariance, and This is the measurement noise from the infrared camera. Using the same multiplication method, the radar measurements are further used to correct the target vehicle's state and covariance. After radar correction, the target vehicle's state and covariance can be written as:

[0186]

[0187]

[0188] in, This indicates the state of the target vehicle after multiplying the millimeter-wave radar measurements. This represents the covariance of the target vehicle after multiplying the millimeter-wave radar measurements. , T represents the transpose operation. It is the matrix of Kalman filter gain. It is the radar measurement noise:

[0189] After multiple iterations, use the corrected and The distance between the vehicle and the smart light pole is calculated as follows:

[0190]

[0191] in, This indicates the predicted distance between the vehicle and the smart light pole.

[0192] Multi-scale feature fusion and attention mechanism processing are applied to the dehazed images and radar data. The data processing employs semi-supervised learning and data augmentation strategies. The specific scheme is as follows:

[0193] The multi-scale fusion module constructs a feature pyramid by extracting multi-scale features through convolutional kernels of different sizes (5×5, 3×3, 1×1), which correspond to global fog distribution, local target structure, and detailed texture, respectively. It also uses cross-layer connections to fuse features of different scales and uses an adaptive weighting mechanism to suppress background noise and enhance the response of the target region.

[0194] The attention mechanism is a combination of spatial attention and channel attention. Spatial attention generates a spatial weight map through the interaction of the position dimension and the channel dimension, focusing on the target area (such as vehicles and pedestrians) and suppressing uniform fog interference. Channel attention is based on adaptive threshold segmentation, adjusting the RGB channel weights, enhancing the luminance channel features, and alleviating the problem of decreased image contrast in foggy weather. RGB represents the colors of the red, green, and blue channels.

[0195] Semi-supervised learning trains a model using a small amount of labeled data and a large amount of unlabeled data. For labeled data, supervised cross-entropy loss and mean squared error loss functions are used to guide the model learning. For unlabeled data, consistency regularization based on continuity and clustering assumptions is used to make the model make similar predictions for all data points within the potential increase radius. Adding perturbations does not affect the output, making full use of the potential information in the unlabeled data.

[0196] The data augmentation strategy involves using AutoAugmentation to randomly crop, translate, rotate, automatically adjust contrast, apply negatives, histogram equalization, threshold inversion, reduce bit depth, control contrast, balance color, adjust brightness, blur, randomly erase, and overlay samples to enable the model to learn object features under different foggy environmental conditions, thereby improving the model's generalization ability in foggy road curve scenarios; AutoAugmentation refers to an automatic data augmentation technique.

[0197] S6-S7: Early warning judgment and execution;

[0198] If the distance d between the predicted position of a vehicle on one side of the curve and the smart light pole is less than 60 meters, the processor sends a control signal to the LED warning light to activate it, alerting the driver on that side to oncoming traffic. The processor continuously monitors the distance between the predicted positions of vehicles on both sides of the curve and the smart light pole. If the distance d between the predicted positions of vehicles on both sides of the curve and the smart light pole is less than 60 meters, the processor simultaneously sends signals to the LED warning light and the speaker. Activating the LED warning light triggers the speaker to play a preset warning message, providing a two-way reminder to the driver: "There is oncoming traffic on the curve ahead, please drive carefully," thus preventing a collision.

[0199] Example 2

[0200] refer to Figure 5 The smart light pole device of the present invention includes: an infrared camera 1, an LED warning light 2, a millimeter-wave radar 3, a processor, a speaker 4, a communication module, and a power supply module for supplying power to the above components. The device requirements are as follows:

[0201] The infrared camera 1 uses a 16-bit depth infrared imaging sensor with a resolution of no less than 1280×1024, a frame rate of 30fps, and an effective detection distance of ≥200 meters.

[0202] The LED warning light 2 has a single-lamp brightness of ≥5000cd / m² and a response delay of ≤50ms.

[0203] The millimeter-wave radar 3 operates in the 77GHz frequency band, with a detection radius of ≥200 meters and a positioning accuracy of ±0.1 meters.

[0204] The processor uses an embedded AI computing unit, and its GPU is an NVIDIA Jetson Xavier.

[0205] It has 8GB of RAM and 256GB of storage space.

[0206] The speaker 4 has a sound pressure level ≥90dB, supports custom voice broadcast content, and has a voice playback delay ≤50ms.

[0207] The communication module supports 5G / Ethernet dual-mode communication with a data upload latency of ≤10ms; the infrared camera 1, millimeter-wave radar 3, and processor communicate through the communication module.

[0208] The power module 5 has a maximum power consumption of ≤100W under full load and a standby power consumption of ≤15W.

[0209] All hardware components of this invention are integrated into the mounting bracket of the smart light pole. The mounting bracket is made of aluminum alloy and has an IP67 protection rating.

[0210] Infrared camera 1 and millimeter-wave radar 3 are installed at the same height on the light pole facing the curve area, ensuring that their coordinate origins are consistent and covering the vehicle trajectories on both sides of the curve. LED warning lights 2 and speakers 4 are installed in conspicuous positions on the light pole, ensuring that both drivers can clearly see the lights and hear the voice commands within their field of vision. The processor, power module, and communication module are housed in an equipment box at the top of the light pole's mounting bracket. The equipment box is made of aluminum alloy and meets IP67 protection requirements. During installation, the infrared camera angle must be calibrated to ensure that its shooting range completely covers the curve's blind spot. The millimeter-wave radar needs to be adjusted to overlap with the infrared camera's detection range to avoid detection blind spots. After powering on, the stability of the power supply to each hardware component is tested, and the data transmission delay is verified to meet the requirement of ≤10ms through the communication module.

[0211] Infrared camera 1 captures multiple consecutive video images of the curved area and generates a pre-processed infrared image;

[0212] The infrared preprocessed image is transmitted to the processor, and the improved YOLOv8 target detection algorithm is used to identify the vehicle location information. If the vehicle cannot be identified, it is transmitted to the infrared image dehazing module to dehaze the image and obtain the reconstructed fog-free image data.

[0213] The reconstructed fog-free image data is then fed into a YOLOv11-based target detection algorithm to identify vehicle location information.

[0214] Simultaneously, the millimeter-wave radar 3 is used to detect vehicle position information, and coordinate system matching is performed with the vehicle position information identified by the infrared camera 1.

[0215] The vehicle detection results obtained by infrared camera 1 and millimeter-wave radar 3 are fused to estimate the vehicle's motion.

[0216] If vehicles are detected on both sides of the smart light pole, calculate the distance between the predicted positions of the vehicles on both sides and the smart light pole;

[0217] If only one side of the vehicle's predicted position is less than the warning distance from the smart light pole, turn on LED warning light 2;

[0218] The system continues to calculate the distance between the predicted positions of vehicles on both sides and the smart light pole. If the distance between the predicted positions of vehicles on both sides and the smart light pole is less than the warning distance, the speaker 4 and LED warning light 2 are turned on to remind the driver to drive carefully, thus realizing the warning of vehicles approaching on the curve and reducing the traffic accident rate.

Claims

1. A method for warning of oncoming vehicles on curves based on smart light poles, characterized in that, Includes the following steps: S1. An infrared camera captures multiple consecutive video images of the curved area and preprocesses them to generate an infrared preprocessed image; S2. The infrared preprocessed image is transmitted to the improved YOLOv8 target detection algorithm model to identify vehicle location information, and then the process jumps to step S4. If the vehicle cannot be identified, the image is transmitted to the infrared image dehazing module to perform dehazing processing and obtain the reconstructed fog-free image data. S3. Transmit the reconstructed fog-free image data to the YOLOv11-based target detection algorithm model to identify vehicle location information; S4. The radar detects the vehicle's position information and performs coordinate system matching with the vehicle's position information identified in step S2 or step S3. S5. Fuse the vehicle detection results after coordinate system matching and estimate the vehicle motion; S6. If vehicles are detected on both sides of the smart light pole within the curve area, calculate the distance between the predicted positions of the vehicles on both sides and the smart light pole in real time; S7. When the calculated distance between the predicted position of the vehicle on only one side and the smart light pole is less than the preset warning distance, the LED warning light will be turned on. When the calculated predicted distances between the vehicles on both sides and the smart light pole are both less than the preset warning distance, the loudspeaker and LED warning lights will be activated.

2. The method for warning of oncoming vehicles on curves based on smart light poles according to claim 1, characterized in that, In step S1, the preprocessing of the acquired video images includes image denoising and contrast enhancement, and the specific steps are as follows: The image denoising method employs nonlocal mean filtering, which utilizes the texture and structural information of similar blocks in the image to eliminate noise through grayscale conversion, block processing, and similar block matching. The contrast enhancement employs adaptive histogram equalization, which divides the image into multiple local blocks and performs histogram equalization on each local block, stretching the image's histogram to the entire range of local pixel values.

3. The method for warning of oncoming vehicles on curves based on smart light poles according to claim 1, characterized in that, In step S2, the improved YOLOv8 target detection algorithm includes optimizations to the backbone network, connection network, and detection network.

4. The method for warning of oncoming vehicles on curves based on smart light poles according to claim 3, characterized in that, The optimization of the backbone network involves inserting an SCConv module after the C2f module of the original YOLOv8 backbone network to form a C2f_SCConv module, in order to suppress feature redundancy. The optimization of the connection network is based on the Slimneck idea, which uses the GSConv module to replace the ordinary convolution and the VoVGSCSP module to replace the C2f module, in order to reduce parameter redundancy and improve information transmission efficiency. The detection network is optimized by lightweighting the network, merging the original three-channel detection network into a single-channel detection network.

5. The method for warning of oncoming vehicles on curves based on smart light poles according to claim 1, characterized in that, In step S2, if the vehicle cannot be identified, the infrared preprocessed image is first cut, and then the cut infrared preprocessed image is transmitted to the infrared image dehazing module to generate a superpixel image. The global atmospheric light value and transmittance are estimated to derive a fog-free image.

6. The method for warning of oncoming vehicles on curves based on smart light poles according to claim 1, characterized in that, In step S4, the infrared camera and radar are installed on the same smart light pole with the same coordinate origin. Coordinate system matching is achieved by rotating the coordinate system to realize the data coordinate system unity.

7. The method for warning of oncoming vehicles on curves based on smart light poles according to claim 1, characterized in that, In step S5, the motion estimation step for the vehicle is as follows: Based on the state vector of each target vehicle, which includes the position coordinates of the target vehicle and its lateral and longitudinal velocities in the global coordinate system, the average prediction and covariance matrix of the state uncertainty of each target vehicle are obtained using kinematic methods based on the constant velocity model. The predicted state of the target vehicle's position and speed is projected onto the sensor measurement space to obtain the projection mean and covariance matrix. The difference between the predicted state and the measured value is reconciled, and the state and covariance of the target vehicle are corrected by fusing the infrared camera and radar measurements. After multiple iterations, the distance between the vehicle and the smart light pole is calculated using the corrected position coordinates of the target vehicle.

8. A method for warning of oncoming vehicles on curves based on smart light poles according to claim 1 or 5, characterized in that, In step 3, the YOLOv11 target detection algorithm model performs multi-scale feature fusion and attention mechanism processing on the defogging-free image data and radar vehicle position information, and adopts semi-supervised learning and data augmentation strategies for data processing.

9. The method for warning of oncoming vehicles on curves based on smart light poles according to claim 8, characterized in that, The multi-scale feature fusion specifically involves constructing a feature pyramid, extracting multi-scale features through convolutional kernels of different sizes, corresponding to global fog distribution, local target structure, and detailed texture, respectively, and fusing features of different scales through cross-layer connections. An adaptive weighting mechanism is used to suppress background noise and enhance the response of the target region. The attention mechanism is a combination of spatial attention and channel attention. Spatial attention generates a spatial weight map through the interaction of the position dimension and the channel dimension, focuses on the target area, and suppresses uniform haze interference. Channel attention is based on adaptive threshold segmentation, adjusts the weights of the red, green and blue color channels, and enhances the brightness channel features. The semi-supervised learning method trains the YOLOv11 object detection algorithm model using labeled and unlabeled data. For labeled data, supervised cross-entropy loss and mean squared error loss functions are used to guide model learning. For unlabeled data, consistency regularization based on continuity and clustering assumptions is used to enable the model to make predictions for all data points within the potential increase radius, making full use of the potential information in the unlabeled data. The data augmentation strategy uses automatic data augmentation techniques to perform random cropping, translation, rotation, automatic contrast adjustment, negativeing, histogram equalization, threshold inversion, bit reduction, contrast control, color balance, brightness adjustment, blurring, random erasure, and sample overlay on images, enabling the model to learn object features under different foggy environmental conditions.

10. A smart light pole, characterized in that, This includes an infrared camera, millimeter-wave radar, LED warning lights, speakers, processor, communication module, and power supply module that power the above components, all mounted on the light pole. The infrared camera and millimeter-wave radar are installed at the same height on the light pole facing the curve area, covering the vehicle trajectory on both sides of the curve area, and the detection direction of the millimeter-wave radar overlaps with the detection range of the infrared camera. LED warning lights and speakers are installed on the light poles so that they are within the driver's field of vision in both directions; The processor, communication module, and power module are all housed in an equipment box at the top of the light pole.

Images