A vehicle trajectory tracking and channel safety warning method based on multi-source fusion

Abstract

This invention discloses a vehicle trajectory tracking and channel safety early warning method based on multi-source fusion, relating to the field of power system transmission line safety monitoring technology. The method includes: acquiring video images and lidar point cloud data of the transmission channel, and preprocessing them separately; extracting image feature points and key points of the point cloud, constructing 2D-3D matching point pairs and solving the transformation matrix to establish a scene matching relationship model; detecting construction vehicles through image recognition, extracting contours and classifying them, and retrieving the actual three-dimensional dimensions of the vehicles; tracking the trajectory of the same vehicle based on continuous time-series images, determining the motion state and trend by combining size changes and displacement changes, and triggering graded safety early warnings by fusing multi-dimensional information such as vehicle type, speed, and trajectory. This invention achieves accurate quantification of the three-dimensional dimensions of construction vehicles, continuous trajectory tracking, and advanced graded early warning, improving the intelligent prevention and control capabilities of external damage hazards in transmission channels.

Description

Technical Field

[0001] This invention relates to the field of power system transmission line safety monitoring technology, and in particular to a vehicle trajectory tracking and channel safety early warning method based on multi-source fusion. Background Technology

[0002] Monitoring the external damage hazards of construction vehicles within overhead transmission line corridors is a crucial aspect of ensuring the safe operation of the power grid. Accidents involving line tripping and equipment damage caused by construction vehicles illegally entering corridors are frequent, necessitating precise perception and trend prediction of vehicle location, size, and movement status. Existing automated monitoring technologies primarily rely on two-dimensional visual recognition, using image contour extraction and deep learning target detection to locate vehicles. However, two-dimensional images lack depth dimension information, making it impossible to quantify the vehicle's physical dimensions; they can only qualitatively identify hazard types, not quantitatively assess risk levels. Some technologies incorporate lidar or binocular vision to acquire depth data, but these generally suffer from insufficient data fusion, low feature matching accuracy, and poor robustness in complex scenarios. While three-dimensional point cloud technology can accurately acquire target depth and three-dimensional coordinates, compensating for the deficiencies of two-dimensional images, pure point cloud monitoring faces shortcomings such as missing texture features, low target contour recognition, and large data processing volumes. Therefore, achieving efficient fusion of two-dimensional image texture features and three-dimensional point cloud depth information to construct accurate 2D-3D matching relationships has become a core technological bottleneck in the intelligent monitoring of construction vehicles.

[0003] Existing technology establishes a unified three-dimensional spatial coordinate system by combining two-dimensional pixel data collected by on-site cameras with three-dimensional laser point cloud data pre-set in the power transmission channel, and performs 2D-3D coordinate mapping to achieve preliminary data correlation. Then, a pre-trained convolutional neural network is used to extract the two-dimensional detection box of the construction vehicle, solve for the minimum spatial distance between the vehicle and the conductor, and thereby assess the risk of external damage. This solution initially achieves data alignment between vision and point cloud.

[0004] However, the aforementioned existing technologies still have the following shortcomings: First, they lack the ability to quantify vehicle dimensions. They rely solely on a two-dimensional detection frame combined with ground point cloud computing for single-point depth, failing to extract the feature point set of the vehicle's complete outline and thus unable to deduce the vehicle's true three-dimensional dimensions (length, width, and height). This leads to false alarms for small vehicles and missed alarms for large vehicles. Second, the warning rules are one-sided. They use only the distance between the vehicle and the conductor as the sole basis for warnings, without considering the vehicle's motion state (speed, direction) and continuous trajectory characteristics for tiered warnings, easily resulting in false alarms or delayed alarms. Third, they lack continuous motion trajectory tracking capabilities. Existing technologies only achieve target recognition and positioning distance measurement at a single moment, failing to utilize feature point matching information between multiple frames of images. This makes it impossible to continuously track the motion trajectory of construction vehicles, hindering the prediction of vehicle intrusion trends and failing to meet the practical engineering needs of forward-looking safety management and full-process traceability for power transmission channels. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, this invention aims to provide a vehicle trajectory tracking and channel safety early warning method based on multi-source fusion, which enables continuous and accurate tracking of the movement trajectory of construction vehicles, real-time monitoring of vehicle operation status, and achieves the goal of early prevention and proactive risk avoidance in power transmission channel monitoring, ultimately meeting the dynamic monitoring needs of power transmission channel construction scenarios.

[0006] A method for vehicle trajectory tracking and lane safety early warning based on multi-source fusion includes the following steps: S1. Obtain on-site channel video through power transmission channel visualization equipment, acquire channel images by frame extraction, and complete the full-domain point cloud data acquisition of the channel using LiDAR. The channel images and the full-domain point cloud data of the channel form multi-source data. S2. Preprocess the multi-source data collected in step S1, perform noise reduction, enhancement, and distortion correction on the channel images, and perform noise reduction, registration, ground filtering, and key point extraction on the point cloud, removing redundant noise data. S3. Extract image feature points and point cloud key points, construct 2D-3D matching point pairs, solve the coordinate transformation matrix, and build a power transmission channel scene matching relationship model. S4. Use image recognition technology to detect construction vehicles, extract their outlines, classify vehicle types, calculate the two-dimensional pixel size of the vehicles, and invert the actual spatial size of the vehicles through a scene matching relationship model. S5. Extract vehicles of the same type from continuous historical frame images, determine the motion state by size changes, determine the motion trend by displacement changes, and trigger graded safety warnings by combining trajectory, vehicle type, vehicle size, and warning rules.

[0007] Further, step S3 includes: S3.1. The SIFT algorithm is used to extract image feature points, including scale space construction, scale space extremum detection, feature point selection and localization, and feature point descriptor generation. S3.2. Using the ISS algorithm, key points in the point cloud are extracted, including radius filtering to remove isolated outliers, constructing a local neighborhood of the point cloud, calculating the local covariance matrix, and performing eigenvalue decomposition to obtain three eigenvalues. ; Corresponding to the main extension direction, Corresponding to the secondary extension direction, Two thresholds are set in the normal direction. and If satisfied and If so, then this point is a key feature point; S3.3. The FLANN approximate nearest neighbor matcher is used to perform bidirectional matching between image feature points and point cloud key points to initially obtain candidate matching point pairs; then, the RANSAC algorithm is used to remove mismatched point pairs and retain the set of matching points. ,in Let i be the coordinates of the i-th image feature point. These are the corresponding key points in the point cloud; S3.4. Based on the matching point set M, the EPnP algorithm is used to solve the camera extrinsic parameters, and a 2D-3D forward mapping model and a 3D-2D reverse mapping model are constructed. S3.5. The Levenburg-Marquardt algorithm is used to iteratively optimize the camera extrinsic parameters and minimize the reprojection error.

[0008] Furthermore, in step S3.2, for isolated outliers, a radius filtering algorithm is used to remove noise. The filtering determination formula is as follows: ; in, Let n be the search radius, and n be the number of neighboring points within the statistical radius. For neighboring point clouds, For point clouds to be determined, This is an indicator function that returns 1 if the condition within the parentheses is true, and 0 otherwise. The minimum neighborhood number threshold, if Then determine Noise point; This is the result of the judgment.

[0009] Furthermore, in step S3.4, the projection transformation formulas for constructing the 2D-3D forward mapping model and the 3D-2D backward mapping model are as follows: ; in, As a scale factor, For the camera intrinsic parameter matrix, , For camera Directional focal length, Image principal pixel coordinates, It is a 3×3 rotation matrix. It is a 3×1 translation matrix. The coordinates of a point in 3D space in the world coordinate system. These are the corresponding image pixel coordinates.

[0010] Furthermore, in step S3.5, the reprojection error The calculation formula is: ; in: Let i be the coordinates of the i-th image feature point. represents the pixel coordinates of the 3D point reprojection, and n represents the total number of image feature points.

[0011] Further, in step S4, an improved YOLOv8n model is used to detect construction vehicles. Canny edge detection is used to extract the complete outline of the vehicle and fit the minimum bounding rectangle to obtain pixel-level length and width. Based on the scene matching relationship model, the bottom center point of the vehicle detection box is extracted to invert its three-dimensional spatial coordinates, and ground markers are selected to solve the ratio coefficient between pixels and actual size. The actual length and width of the vehicle are calculated based on the ratio coefficient, and the actual height is obtained by the difference between the spatial height inverted from the top pixel of the vehicle and the ground elevation.

[0012] Furthermore, based on the scene matching relationship model, the pixel coordinates of the bottom center point of the vehicle detection box are extracted. Inversion to obtain the 3D spatial coordinates of the point Select ground markers in the point cloud baseline map and obtain their actual lengths. With corresponding pixel length Solve for the actual size ratio coefficient corresponding to a single pixel. Based on the scaling factor Calculate the actual length of the vehicle Actual width and actual height : ; in, The pixel length of the vehicle detection box or contour in the image. The width is in pixels. To determine the spatial height of the pixels on the top of the vehicle; This refers to the ground elevation at the bottom of the vehicle.

[0013] Furthermore, in step S5, historical time-series images are retrieved, and a multi-target tracking algorithm is used to bind the vehicle's unique ID to generate a spatiotemporal trajectory sequence; the motion state is determined by comparing the actual length changes of the vehicle in consecutive frames; displacement and velocity are calculated, and the future position is predicted by linear extrapolation of the direction; and graded early warning rules are set according to vehicle type, motion state, speed, and trajectory.

[0014] Furthermore, calculate the spatial displacement of vehicles in adjacent frames. Speed ​​of movement Predicting the future position of the vehicle by combining the direction of displacement :

[0015] in, The vehicle's direction angle. and These are the three-dimensional spatial coordinates of the vehicle in two adjacent image frames. The spatial coordinates of the vehicle in the current frame. The time interval between adjacent frames. This is the predicted time interval.

[0016] The beneficial effects of this invention are: I. Achieving precise quantification of vehicle 3D dimensions. By using a 2D-3D scene matching model and combining it with pixel ratio coefficient inversion, the actual 3D dimensions of a vehicle (length, width, and height) can be accurately calculated. This not only enables precise identification of high-risk vehicle types but also quantitative assessment of the risk level of external damage to vehicles, providing data support for hazard assessment.

[0017] Second, it possesses continuous trajectory tracking capabilities, enabling dynamic control of the entire process. Existing technologies only support static recognition of single-frame images and single-point ranging, lacking continuous trajectory analysis functions and failing to grasp the entire process of vehicle movement. This invention achieves unique vehicle ID binding based on time-series images, constructs a spatiotemporal trajectory sequence, and can track vehicle displacement, speed, and direction in real time, recording the entire vehicle's trajectory and enabling full-process control of construction vehicles from their appearance, movement, approach, to departure.

[0018] Third, the early warning mechanism is multi-dimensional and comprehensive, enabling advanced prediction and tiered response. Existing technologies rely solely on the distance between the vehicle and the guide wire as the sole basis for early warning, resulting in one-sided and delayed warning rules that are prone to false alarms and missed alarms. This invention innovatively integrates four dimensions—vehicle type, motion state, speed, and trajectory direction—to construct a four-level tiered early warning system. Combined with a linear extrapolation algorithm, it predicts the future position of vehicles in advance, achieving advanced early warning of potential hazards. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0020] Figure 1 A flowchart of a construction vehicle trajectory tracking and access safety early warning method based on multi-source data fusion; Figure 2 Visualize the image of the channel; Figure 3 Point cloud data; Figure 4 This is the result of vehicle identification. Detailed Implementation

[0021] The invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0022] To acquire live video of the channel from the visualization device, a fixed-time frame-sampling method is used to obtain channel photos, such as... Figure 2The channel visualization image shown is based on point cloud data collection of the channel in advance, such as... Figure 3 The point cloud data images shown are preprocessed for these two types of data. By constructing 2D-3D matching point pairs, a matching relationship model for power transmission channel scenarios is built.

[0023] Image recognition technology is used to identify the outline and type of construction vehicles, calculate the two-dimensional dimensions of the construction vehicles, and calculate the actual size of the vehicles through scene matching models; Based on continuous historical visualization channel images, construction vehicles in the historical images are identified. For construction vehicles of the same type, the vehicle's movement status is judged based on changes in vehicle size, and the vehicle's movement trend is judged based on changes in vehicle displacement. Channel safety warnings are issued by combining the movement trajectory and preset warning rules.

[0024] Data acquisition employs a combination of fixed visualization and UAV point cloud acquisition: Visualization equipment (high-definition bullet camera) is fixedly installed on the transmission tower, with the lens aimed at the entire transmission channel, capturing real-time video of the channel. Frame-by-frame extraction is used to periodically acquire on-site photos of the channel, with a resolution of at least 2592×1944. UAVs are used to perform a full-area scan of the target transmission channel beforehand, acquiring point cloud data with a density ≥150 points / m². 2 Point cloud data contains three-dimensional coordinate information of all elements within the channel, including towers, conductors, ground, vegetation, and structures.

[0025] like Figure 1 As shown, the vehicle trajectory tracking and lane safety early warning method based on multi-source fusion of the present invention includes the following steps: S1, Multi-source data acquisition The video stream of the power transmission channel is acquired using visualization equipment, and the channel photos are generated by frame extraction at fixed intervals. At the same time, the entire channel area is scanned in advance using lidar to complete the high-density point cloud data acquisition.

[0026] S2, Data Preprocessing The system performs noise reduction, enhancement, and distortion correction on images, and denoising, registration, ground filtering, and key point extraction on point clouds, while removing redundant noise data.

[0027] S2.1 Image Data Preprocessing Distortion correction, noise reduction, and enhancement are performed sequentially on the channel image. Based on the radial and tangential distortion coefficients obtained from camera calibration, the original pixel coordinates are corrected using the following formula:

[0028] in, These are the original pixel coordinates. The corrected coordinates, The radial distortion coefficient is... The tangential distortion coefficient is... The radial distance from a pixel to the principal point of the image. Gaussian noise is suppressed by using a Gaussian convolution kernel. The kernel function follows a two-dimensional Gaussian distribution and is adjusted according to the noise intensity of the construction scene to preserve the vehicle outline and the edge of the guide wire while eliminating noise.

[0029] The expression for the convolution kernel of a Gaussian filter is: ; in, The standard deviation is Gaussian, ranging from 1.0 to 1.5, and is adjusted according to the noise intensity of the construction scene to ensure that the vehicle outline details are preserved while eliminating noise. Relative coordinates within the convolution kernel.

[0030] The image is divided into blocks for histogram equalization, with the block size set to 8x8 and the contrast limit threshold set to 2.0, in order to improve the recognizability of vehicle outlines and textures.

[0031] S2.2 Point Cloud Data Preprocessing Point cloud preprocessing mainly includes noise removal and downsampling. First, a statistical filtering algorithm is used to remove isolated noise points: for each point, its k nearest neighbors are searched, where k ranges from 10 to 20, and the average distance between the neighbors is calculated. If the average distance between a point and its neighbors exceeds a set threshold (1.5 to 2 times the global average distance), the point is marked as noise and removed. After denoising, voxel lattice filtering is used for downsampling, dividing the 3D space into cubic lattices with side length v, where v ranges from 0.05 to 0.1 meters depending on the point cloud density. Within each voxel lattice, a representative point, such as the voxel center, is retained, and the rest are discarded, thus simplifying the point cloud scale and improving the real-time performance of subsequent processing.

[0032] S3, 2D-3D Matching and Scene Modeling Extract image feature points and point cloud key points, construct accurate matching point pairs, solve the coordinate transformation matrix, and build a matching relationship model for power transmission channel scenarios.

[0033] S3.1 Two-dimensional feature point extraction Based on the obtained construction vehicle outline, the SIFT algorithm is used to extract stable feature points from the image: (1) Scale space construction The scale space is constructed using the difference-of-Gaussian pyramid, and the difference-of-Gaussian image is obtained by subtracting the Gaussian blurred images of adjacent scales.

[0034] Gaussian difference image The calculation formula is: ;in, The image is a Gaussian blurred image, where k is a scale factor with a value of 1.2.

[0035] (2) Scale-space extreme value detection In the difference of Gaussian image, local extrema are detected, that is, points where the current point is at a maximum or minimum in its own 8-neighborhood and at the corresponding positions of the adjacent scales above and below, and these points are used as candidate feature points.

[0036] (3) Feature point selection and localization By fitting a three-dimensional quadratic function, candidate points are precisely located at the sub-pixel level, and points with low contrast and strong edge response are eliminated, while stable key points are retained.

[0037] (4) Feature point descriptor generation Centered on the feature point, a 16×16 neighborhood window is selected and divided into 4×4 sub-windows. Gradient histograms in 8 directions are calculated for each sub-window to generate a 128-dimensional feature descriptor for subsequent feature matching.

[0038] S3.2, 3D Feature Point Extraction Key feature points were extracted from the point cloud of construction vehicles using the ISS algorithm. To eliminate outlier noise, radius filtering is used: for any point P, the number of neighboring points within the search radius is counted; if the number of points is less than the preset minimum neighbor number... If P is identified as noise, it will be deleted.

[0039] Filtering determination formula: ; in, Let n be the search radius, and n be the number of neighboring points within the statistical radius. For neighboring point clouds, For point clouds to be determined, This is an indicator function that returns 1 if the condition within the parentheses is true, and 0 otherwise. The minimum neighborhood number threshold, if Then determine Noise point; This is the result of the judgment.

[0040] For the retained points, construct a local neighborhood, centered on the target point P, with a fixed search radius. The preferred setting is 0.5 meters, to obtain k-nearest neighbor set Construct a local neighborhood space, calculate the local covariance matrix based on the neighborhood point set, and construct the target point. The three-dimensional covariance matrix To characterize the spatial distribution features of a local neighborhood, the covariance matrix The formula is: ;in, The target point's three-dimensional coordinate vector. Let be the three-dimensional coordinate vector of the neighborhood points, and k be the number of neighborhood points. For the covariance matrix... Eigenvalue decomposition yields three eigenvalues. The eigenvalues ​​reflect the distribution characteristics of the point cloud in different directions. Corresponding to the main extension direction, Corresponding to the secondary extension direction, The direction should be normal. Preferably, The value ranges from 0.6 to 0.8. The value ranges from 0.05 to 0.1, if it satisfies... and If so, then that point is a key feature point.

[0041] S3.3, 2D-3D Feature Matching The SIFT features of the image are associated with the FPFH features of the point cloud. A FLANN approximate nearest neighbor searcher is used to perform bidirectional matching to obtain preliminary candidate matching point pairs.

[0042] Then, the RANSAC random sampling consensus algorithm is used to remove erroneous matches, ultimately forming a set of exact matching points. ,in For image pixels, For the corresponding point cloud spatial points, the number of matching points Ensure the accuracy of the model solution.

[0043] S3.4 Scene Matching Model Construction Based on the precise matching of point pairs, the EPnP algorithm is used to solve the camera's external parameters, preferably a 3x3 rotation matrix R and a 3x1 translation vector T, thereby establishing a forward mapping from two-dimensional pixels to three-dimensional space and a reverse mapping from three-dimensional space to two-dimensional pixels.

[0044] Core projection transformation formula: ;in This is the scale factor (depth value in camera coordinate system). This is the camera intrinsic parameter matrix (obtained through calibration). , For camera Directional focal length, Image principal pixel coordinates, It is a 3×3 rotation matrix. It is a 3×1 translation matrix. The coordinates of a point in 3D space in the world coordinate system. These are the corresponding image pixel coordinates.

[0045] S3.5 Model Optimization and Error Calibration The Levenburg-Marquardt algorithm is used to iteratively optimize the rotation matrix R and translation vector T, with the goal of minimizing the reprojection error.

[0046] Formula for calculating reprojection error: ; in: These are the actual pixel coordinates. The reprojection pixel coordinates of 3D points are used. After optimization, the reprojection error is ≤0.5 pixels, and the overall scene mapping error is ≤0.1m.

[0047] S4. Vehicle Recognition and Size Calculation: Image recognition technology is used to detect construction vehicles, extract their contours, and classify vehicle types. The system then fits the bounding rectangle of the vehicle pixels and, based on an optimized scene mapping model, inverts the contour pixels to a 3D spatial coordinate system to directly calculate the actual 3D spatial dimensions of the vehicle. Figure 4 The image shows the vehicle recognition results.

[0048] S4.1 Vehicle Detection and Contour Extraction An improved YOLOv8n target detection model was used to detect construction vehicles in preprocessed images.

[0049] It should be noted that the improved YOLOv8n target detection model is an ultra-lightweight model in the YOLOv8 series, emphasizing speed and small parameter count. However, the original model suffers from insufficient accuracy and false positives / missed detections in complex construction scenarios, irregularly shaped construction vehicles, and situations where small / large targets coexist. Therefore, targeted improvements have been made specifically for construction vehicle scenarios: Head improvement—adapting to five types of anchor frames specific to construction vehicles. The model training set covers five types of high-risk vehicles, including cranes, excavators, pump trucks, dump trucks, and loaders, with approximately 100,000 labeled samples; Detection output: Vehicle category Confidence level Pixel detection box.

[0050] The confidence threshold is set to 0.85, and detection results below the threshold are directly rejected.

[0051] The complete outline of the vehicle is extracted by Canny edge detection and fitted with the minimum bounding rectangle to obtain the pixel coordinates of the four corner points of the vehicle outline bounding rectangle, providing a pixel input reference for subsequent 3D spatial dimension calculation.

[0052] It should be noted that, in order to reduce errors, the selection of ground representations follows these principles: Same-plane principle: Select markers that are at the same elevation and horizontal plane as the ground where construction vehicles are parked.

[0053] Same line-of-sight depth principle: Select the straight-line distance and depth level from the camera optical center to the marker, and the pole base, ground calibration plate, road square marking, site standard components, etc. that are similar to the depth of the target vehicle detection frame.

[0054] S4.2 Calculation of the actual three-dimensional dimensions of the vehicle Based on the scene transformation matrix and 3D scene mapping model optimized by the Levenberg-Marquardt algorithm of S3.5, the pixel coordinates of the four corner points of the minimum bounding rectangle of the vehicle outline are inverted point by point to obtain the real 3D spatial coordinates of each corner point.

[0055] In a three-dimensional coordinate system, the spatial distances between the diagonals and adjacent sides of the rectangle are calculated using the Euclidean distance formula, and the actual length and width of the vehicle are directly obtained.

[0056] For the vehicle height dimension, the top feature pixels and bottom ground pixels of the vehicle detection box are selected, and the corresponding three-dimensional spatial elevation coordinates are obtained by inversion. The actual height of the vehicle is calculated by the difference between the upper and lower elevations.

[0057] S5, Tracking and Early Warning Extract vehicles of the same type from continuous historical frames, determine their motion state by size changes and motion trend by displacement changes, and trigger graded safety warnings by combining trajectory, vehicle type, vehicle size, and warning rules.

[0058] S5.1, Continuous Frame Vehicle Multi-Target Tracking Retrieve historical time-series images (time intervals) This employs a multi-target tracking algorithm, uniquely binding each vehicle with an ID and eliminating interfering targets based on vehicle category and size; it then generates a continuous spatiotemporal trajectory sequence for a single vehicle type. ; Where: ID is the unique identifier for the vehicle; For collection timestamps; For vehicle spatial coordinates.

[0059] S5.2 Vehicle Motion Status Determination Compare the actual length changes of the vehicle in two consecutive frames. Combined with size error threshold ( (Take a distance of 0.05m) to determine the vehicle's stationary / moving state. .

[0060] S5.3 Vehicle Motion Trend Prediction Calculate the spatial displacement of vehicles in adjacent frames Speed ​​of movement The future position of the vehicle is predicted by combining the displacement direction, where: Displacement calculation formula: Instantaneous velocity calculation formula: Predicting the future position of a vehicle through linear extrapolation. ; in, The vehicle's direction angle. For the predicted time interval, preferably .

[0061] S5.4 Multi-dimensional hierarchical early warning rules Based on vehicle type, motion status, speed, and trajectory obtained through target detection and tracking, the system constructs a four-level early warning and response mechanism to address the risks posed by construction vehicles near power transmission corridors in a differentiated manner. The specific level classification and response strategies are as follows: Green alerts are automatic response levels and are applicable to scenarios where non-high-risk vehicles are stationary or far from the road. The system automatically filters out such targets and does not generate alarms or records.

[0062] A blue alert is considered a "watch level" alert, indicating that non-high-risk vehicles are approaching the passage at low speeds, not exceeding 0.5 meters per second. In this situation, the system only records the event information on the platform; no on-site personnel intervention is required, but the information can be used for retrospective analysis.

[0063] An orange alert is a warning level alert triggered when high-risk vehicles, such as cranes and pump trucks, approach the passage at low speeds, not exceeding 0.5 meters per second. The system will automatically send SMS reminders to relevant maintenance personnel, prompting them to pay attention to the on-site situation and prepare for intervention in advance.

[0064] A red alert is an emergency level alert, triggered when a high-risk vehicle approaches a passage at high speed, exceeding 1.5 meters per second. The system will immediately activate on-site audible and visual alarms, and simultaneously send multi-level SMS and voice notifications to responsible personnel, demanding immediate cessation of unauthorized construction work until the danger has passed.

[0065] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for vehicle trajectory tracking and lane safety early warning based on multi-source fusion, characterized in that, Includes the following steps: S1. Obtain on-site channel video through power transmission channel visualization equipment, acquire channel images by frame extraction, and complete the full-domain point cloud data acquisition of the channel using LiDAR. The channel images and the full-domain point cloud data of the channel form multi-source data. S2. Preprocess the multi-source data collected in step S1, perform noise reduction, enhancement, and distortion correction on the channel images, and perform noise reduction, registration, ground filtering, and key point extraction on the point cloud, removing redundant noise data. S3. Extract image feature points and point cloud key points, construct 2D-3D matching point pairs, solve the coordinate transformation matrix, and build a power transmission channel scene matching relationship model. S4. Use image recognition technology to detect construction vehicles, extract their outlines, classify vehicle types, calculate the two-dimensional pixel size of the vehicles, and invert the actual spatial size of the vehicles through a scene matching relationship model. S5. Extract vehicles of the same type from continuous historical frame images, determine the motion state by size changes, determine the motion trend by displacement changes, and trigger graded safety warnings by combining trajectory, vehicle type, vehicle size, and warning rules.

2. The method for vehicle trajectory tracking and lane safety early warning based on multi-source fusion according to claim 1, characterized in that, Step S3 includes: S3.

1. The SIFT algorithm is used to extract image feature points, including scale space construction, scale space extremum detection, feature point selection and localization, and feature point descriptor generation. S3.

2. Using the ISS algorithm, key points in the point cloud are extracted, including radius filtering to remove isolated outliers, constructing a local neighborhood of the point cloud, calculating the local covariance matrix, and performing eigenvalue decomposition to obtain three eigenvalues. ; Corresponding to the main extension direction, Corresponding to the secondary extension direction, Two thresholds are set in the normal direction. and If satisfied and If so, then this point is a key feature point; S3.

3. The FLANN approximate nearest neighbor matcher is used to perform bidirectional matching between image feature points and point cloud key points to initially obtain candidate matching point pairs; then, the RANSAC algorithm is used to remove mismatched point pairs and retain the matching point set. ,in Let i be the coordinates of the i-th image feature point. These are the corresponding key points in the point cloud; S3.

4. Based on the matching point set M, the EPnP algorithm is used to solve the camera extrinsic parameters, and a 2D-3D forward mapping model and a 3D-2D reverse mapping model are constructed. S3.

5. The Levenburg-Marquardt algorithm is used to iteratively optimize the camera extrinsic parameters and minimize the reprojection error.

3. The method for vehicle trajectory tracking and lane safety early warning based on multi-source fusion according to claim 2, characterized in that, In step S3.2, for isolated outliers, a radius filtering algorithm is used to remove noise. The filtering determination formula is as follows: ; in, Let n be the search radius, and n be the number of neighboring points within the statistical radius. For neighboring point clouds, For point clouds to be determined, This is an indicator function that returns 1 if the condition within the parentheses is true, and 0 otherwise. The minimum neighborhood number threshold, if Then determine Noise point; This is the result of the judgment.

4. The method for vehicle trajectory tracking and channel safety early warning based on multi-source fusion according to claim 2, characterized in that, In step S3.4, the projection transformation formulas for constructing the 2D-3D forward mapping model and the 3D-2D backward mapping model are as follows: ; in, As a scale factor, For the camera intrinsic parameter matrix, , For camera Directional focal length, Image principal pixel coordinates, For rotation matrix, It is a translation matrix. The coordinates of a point in 3D space in the world coordinate system. These are the corresponding image pixel coordinates.

5. The method for vehicle trajectory tracking and lane safety early warning based on multi-source fusion according to claim 2, characterized in that, In step S3.5, the reprojection error The calculation formula is: ； in: Let i be the coordinates of the i-th image feature point. represents the pixel coordinates of the 3D point reprojection, and n represents the total number of image feature points.

6. The method for vehicle trajectory tracking and lane safety early warning based on multi-source fusion according to claim 1, characterized in that, Step S4 uses an improved YOLOv8n model to detect construction vehicles, and uses Canny edge detection to extract the complete outline of the vehicle and fit the minimum bounding rectangle to obtain pixel-level length and width. Based on the scene matching relationship model, the bottom center point of the vehicle detection box is extracted and its three-dimensional spatial coordinates are inverted. The ratio coefficient between the pixel and the actual size is calculated by selecting ground markers. The actual length and width of the vehicle are calculated based on the ratio coefficient. The actual height is then obtained by the difference between the spatial height inverted by the top pixel of the vehicle and the ground elevation.

7. The method for vehicle trajectory tracking and lane safety early warning based on multi-source fusion according to claim 6, characterized in that, Based on the scene matching relationship model, the pixel coordinates of the bottom center point of the vehicle detection box are extracted. Inversion to obtain the 3D spatial coordinates of the point ; Select ground markers in the point cloud baseline map and obtain their actual lengths. With corresponding pixel length Solve for the actual size ratio coefficient corresponding to a single pixel. Based on the scaling factor Calculate the actual length of the vehicle Actual width and actual height : ; in, The pixel length of the vehicle detection box or contour in the image. The width is in pixels. To determine the spatial height of the pixels on the top of the vehicle; This refers to the ground elevation at the bottom of the vehicle.

8. The method for vehicle trajectory tracking and lane safety early warning based on multi-source fusion according to claim 1, characterized in that, In step S5, historical time-series images are retrieved, and a multi-target tracking algorithm is used to bind the unique vehicle ID to generate a spatiotemporal trajectory sequence. The motion state is determined by comparing the actual length changes of the vehicle in consecutive frames. The displacement and velocity are calculated, and the future position is predicted by linear extrapolation of the direction. A graded warning rule is set according to the vehicle type, motion state, speed, and trajectory.

9. A method for vehicle trajectory tracking and lane safety early warning based on multi-source fusion according to claim 8, characterized in that, Calculate the spatial displacement of vehicles in adjacent frames Speed ​​of movement Predicting the future position of the vehicle by combining the direction of displacement : in, The vehicle's direction angle. and These are the three-dimensional spatial coordinates of the vehicle in two adjacent image frames. The spatial coordinates of the vehicle in the current frame. The time interval between adjacent frames. This is the predicted time interval.

Images