An intersection collision warning method combining low-altitude aerial photography and double-branch deep behavior prediction
By using large-model-driven UAV swarm image acquisition and an improved S_MBLBP operator, perspective transformation model, and bi-branch trajectory prediction model, the problems of blind spots, image distortion, and vehicle detection deviation in intersection collision warning were solved, achieving accurate collision risk assessment and improving the reliability and practicality of the warning.
Patent Information
- Application Number
- CN202610292978.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-11
- Publication Date
- 2026-07-07
AI Technical Summary
Existing intersection collision warning technologies suffer from problems such as blind spots, insufficient image coordinate transformation accuracy, vehicle detection deviation, limited trajectory prediction accuracy, and inaccurate collision risk assessment, which affect the reliability and practicality of the warnings.
A large-model-driven UAV swarm image acquisition and classification architecture is adopted, combined with an improved S_MBLBP operator and perspective transformation model for image correction, a directed bounding box vehicle detection algorithm for localization and tracking, a bi-branch cooperative trajectory inference model for prediction, and risk assessment based on bounding box intersection area and dynamic threshold.
It achieves accurate intersection collision warning throughout the entire process, solving problems such as blind spots, image distortion, vehicle detection deviation, and inaccurate trajectory prediction, thus improving the accuracy and reliability of the warning.
Smart Images
Figure CN122347857A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent transportation and computer vision interdisciplinary technology, specifically for traffic collision early warning at urban intersections. The core technologies revolve around key technologies such as drone aerial image acquisition, image correction, vehicle detection and tracking, trajectory prediction, and collision risk assessment, to achieve accurate and real-time early warning of vehicle collision risks at intersections. Background Technology
[0002] Urban intersections are core areas where traffic flows converge and are also high-risk locations for collisions. Existing intersection collision warning technologies have several limitations: First, traditional fixed monitoring equipment is easily obstructed by buildings and large vehicles, creating blind spots and failing to fully cover key areas of the intersection; second, images acquired by drones suffer from perspective distortion and noise interference, resulting in insufficient accuracy in converting pixel coordinates to real-world geographic coordinates, affecting vehicle positioning accuracy; third, target detection models often output non-directional, axis-aligned bounding boxes, making it difficult to adapt to complex driving scenarios such as vehicle turning and lane changes at intersections, easily leading to detection bias; fourth, single trajectory prediction models struggle to balance physical plausibility and semantic adaptability, resulting in limited prediction accuracy; fifth, fixed collision risk thresholds fail to consider dynamic factors such as vehicle type, speed, and distance, leading to insufficient warning accuracy. These problems severely restrict the reliability and practicality of intersection collision warning systems, necessitating a comprehensive, high-precision, and adaptive warning technology solution. Summary of the Invention
[0003] This invention proposes an intersection collision warning method that combines low-altitude aerial photography with dual-branch depth behavior prediction. It achieves accurate warning throughout the entire process through five key steps, addressing the pain points of existing technologies:
[0004] R1 adopts a large model-driven drone swarm image acquisition and classification architecture, dynamically optimizing drone layout and flight trajectory to achieve blind-spot coverage of intersections and intelligent image classification and storage.
[0005] R2 is based on the improved S_MBLBP operator and perspective transformation model to achieve high-precision image correction and establish a precise mapping between pixel coordinates and geographic coordinates.
[0006] R3 is based on a directed bounding box vehicle detection algorithm to achieve directional vehicle localization, angle perception, motion parameter calculation and continuous tracking.
[0007] R4 employs a dual-branch collaborative trajectory extrapolation model to collaboratively generate the best vehicle trajectory prediction results that are physically reasonable and semantically adapted.
[0008] R5 uses bounding box intersection area and dynamic threshold to achieve accurate classification and determination of collision risk.
[0009] Specifically, step R1 is as follows:
[0010] R1-1: Large Model-Driven Target Acquisition and Scene Analysis:
[0011] The basic parameters of the crossroads (defining the intersection as a "grid" shaped structure) are set. ,Width Drone flight altitude Camera field of view Historical image features (coordinates of high-frequency occlusion areas, vehicle density distribution during peak hours) are input into a pre-trained traffic scene model. The model identifies key areas prone to blind spots (such as non-motorized vehicle lanes obscured by large vehicles turning, and pedestrian crossing corners) through semantic understanding, and outputs scene adaptation weights (defining the weights of blind spot areas). ordinary area This provides a basis for subsequent layout optimization.
[0012] R1-2: Dynamic calculation of UAV spatial layout parameters:
[0013] The large model calculates the side length of the camera area of a single drone based on scene adaptation weight differentiation: the ordinary area is calculated according to the basic formula. Calculations are performed to ensure regular coverage; the side lengths of blind spots are corrected. (By narrowing the area to increase coverage), the accuracy of flight altitude adjustment is improved simultaneously.
[0014] The number of deployments is dynamically calculated based on scenario adaptation weights: the horizontal number is... ( (Adapt weights for horizontal scenarios); vertical quantity ( (Adapting weights for vertical scenarios); Total number The system ensures that drones are distributed according to the principle of "blind spot encryption and general adaptation," with no duplicate or blank coverage.
[0015] R1-3: Large-scale collaborative flight trajectory planning:
[0016] The large model generates a trajectory strategy based on real-time traffic conditions (vehicle density and speed initially collected by drones): during peak hours (vehicle density...). (Vehicles / km) adopts a "short-distance, high-frequency" strategy, lateral movement distance Longitudinal movement distance Duration of stay During off-peak hours, a "long-distance, low-frequency" strategy is adopted, with the moving distance remaining the same but the dwell time extended.
[0017] The drone moves in the order of "first covering blind areas laterally, then extending to ordinary areas vertically" to avoid the lag in blind area data collection caused by the conventional order. GPS location information is recorded synchronously with each movement to ensure that the trajectory is traceable.
[0018] R1-4: End-to-end image acquisition and intelligent classification:
[0019] The acquired images are transmitted to the large model module embedded at the edge. The model intelligently classifies and stores the image information according to the core elements: it is stored under three categories of tags: "collision warning core image" (including dynamic vehicle targets), "scene baseline image" (including static facilities), and "redundant filtered image" (without key targets).
[0020] Specifically, step R2 is as follows:
[0021] R2-1: Feature point selection:
[0022] To address the issues of image distortion caused by perspective, altitude, and lens distortion in wide-angle lens imaging of drones, as well as the inconsistency between image coordinates and real-world coordinates, this study combines improved LBP features to enhance feature matching stability in complex environments. This achieves accurate conversion between pixel coordinates and geographic coordinates, providing a precise "image-real-world" mapping for vehicle positioning. The study focuses on extracting feature points with significant and stable textures at intersections, including the intersection center point, lane line intersections, stop line edges, pedestrian crossing corners, and turning point of guide lines, ensuring uniform coverage of key areas and providing accurate references for deviation identification.
[0023] R2-2: Improved S_MLBP feature extraction:
[0024] Traditional LBP (Local Binary Pattern) operators are robust to changes in illumination, but are susceptible to noise and dynamic background interference. An improved S_MLBP (Stable Multi-Scale Block Local Binary Pattern) operator is adopted, which uses block averaging and neighborhood compensation strategies to suppress the influence of dynamic background such as abrupt changes in illumination and improve the stability of feature extraction.
[0025] The image is divided into blocks of a preset size, and each block is further divided into smaller regions. The mean gray value of all pixels in each small region is calculated and used to replace the gray value of a single pixel in the small region, thus avoiding interference from isolated noise pixels in feature extraction.
[0026] A neighborhood compensation strategy is introduced. If the grayscale mean of a small area changes abruptly due to the dynamic background, the stability of the grayscale mean is further improved by weighting the grayscale mean of the small area and 2-3 surrounding small areas (the weight increases with the distance from the center area).
[0027] Using key feature points in the image (intersection center point, lane line intersection point, stop line edge point, etc.) as the center, a circular neighborhood with a preset radius is selected (adapting to texture structures in different directions to reduce the impact of directional deviation on feature description). The grayscale mean of the central region is compared with that of each region in the neighborhood. If the mean of the neighborhood region is greater than or equal to the mean of the central region, it is recorded as "1", otherwise it is recorded as "0". Based on this, a unique binary code for each feature point is generated.
[0028] A feature histogram is constructed based on the binary code of all feature points, forming an LBP feature descriptor with illumination robustness. This descriptor is then matched against a feature point sample library with known real geographic coordinates to determine the pixel coordinates corresponding to each feature point in the current image. Then, the pixel coordinates are compared with the real-world 3D coordinates of the feature points measured beforehand. Each image is compared individually. If the deviation between the two exceeds a preset threshold, the area is marked as an image geometric deviation area, providing a precise deviation reference for subsequent image correction.
[0029] R2-3: Perspective Transformation Model Construction and Deviation Correction:
[0030] To address the perspective distortion characteristics of drone aerial images, a perspective transformation model is used to construct a mapping relationship between pixel coordinates and real-world geographic coordinates, thereby achieving geometric deviation correction.
[0031] The core principle of perspective transformation is to transform the image plane using homogeneous coordinates. Mapped to the world plane Its mathematical expression is:
[0032]
[0033] in, Image pixel coordinates Using real-world planar coordinates, These are the parameters of the perspective transformation matrix.
[0034] Parameter solving process: Select at least 4 non-collinear feature points extracted from R2-1, and obtain the pixel coordinates of each feature point. and their corresponding real-world coordinates Minimize the error function using the least squares method The parameters of the perspective transformation matrix are obtained by solving.
[0035] Specifically, step R3 is as follows:
[0036] R3-1: Deployment of Directed Object Detection Models:
[0037] Compared to traditional object detection models that can only output undirected axis-aligned bounding boxes, the YOLOv11-OBB model can output angled rotated bounding boxes, which can more accurately fit the actual contours and driving directions of vehicles, avoiding detection deviations caused by vehicle tilting. It is especially suitable for complex driving scenarios such as vehicle turning and lane changing at intersections.
[0038] Here, the YOLOv11-OBB model is used to detect vehicles on the corrected 2D intersection plan. This model directly outputs oriented bounding boxes, whose parameters are a six-tuple: .in, These are the coordinates of the center point of the directed bounding box. , For the width and length of the bounding box, The bounding box major axis (front direction) and two-dimensional plan view The included angle of the axis, For vehicle codes.
[0039] R3-2: Vehicle angle and position information acquisition:
[0040] Center point coordinate transformation: Using the perspective transformation model in (II), the center point pixel coordinates are transformed. Convert to real-world coordinates .
[0041] Driving direction angle reading: Vehicle steering angle Obtained directly from the output of the YOLOv11-OBB (You Only Look Once version 11 - Oriented Bounding Box) model.
[0042] R3-3: Motion parameter calculation and tracking optimization:
[0043] Exporting continuous frame images from a time series, the center point coordinates and angles of the same vehicle in different frames are obtained to complete the basic displacement data acquisition; the displacement is decomposed based on the vehicle's driving direction to obtain the displacement in the actual driving direction, and then the instantaneous velocity is obtained. If the vehicle is turning, the displacement decomposition result is corrected by averaging the angle; combining the velocity difference between adjacent moments with the time interval... Obtain acceleration To form complete vehicle motion parameters .
[0044] Based on the vehicle's unique code The system achieves continuous vehicle tracking through "location association + feature matching": the search area is defined in consecutive frames, vehicle features are matched, and the parameters of the same vehicle are continuously updated; if tracking is interrupted, the vehicle position during the occlusion period can be predicted based on historical trajectory, or tracking can be resumed through feature rematching to ensure tracking continuity.
[0045] Specifically, step R4 is as follows:
[0046] R4-1: Mamba-VSF hybrid trajectory prediction model:
[0047] Mamba-VSF (Mamba-Vision-Semantic Fusion) adopts a two-branch collaborative architecture. The Mamba branch is responsible for generating physically reasonable candidate trajectories based on historical trajectory data, while the VSF branch uses the cognitive ability of the visual language model to evaluate the semantic reasonableness of the candidate trajectories. Finally, the optimal predicted trajectory is output through an adaptive fusion mechanism.
[0048] R4-2: Mamba trajectory generation branch:
[0049] Input: Historical trajectory sequence of the vehicle, including past... The speed of each time step acceleration Direction angle Real-world coordinates .
[0050] The temporal trajectory mapping relationship is constructed based on the state-space model (SSM). The specific process is as follows:
[0051] State initialization: Initial time is set through the embedding layer. The motion parameters are converted into a high-dimensional state vector: .in, It is a fully connected embedding layer. ( (where the state vector dimension is used) to achieve low-dimensional to high-dimensional mapping of motion parameters.
[0052] State transition is calculated based on capturing the temporal dependence of the trajectory using linear state transition equations: .in, This is the state transition matrix, which describes the influence of the previous state on the current state. The input matrix is weighted to account for the contribution of the motion parameters at the current time step. This is the motion parameter vector for the current time step; The vector is a Gaussian noise vector, simulating small disturbances in real traffic scenarios.
[0053] Predictive output mapping: Mapping a high-dimensional state vector to a predicted future location, using the following formula: .in, For the output matrix, The bias matrix, For the first Predicted location of the time step.
[0054] Output: Generate A physically plausible candidate trajectory Each candidate trajectory contains future... Predicted location of time step Direction angle and generate confidence scores .
[0055] R4-3: VSF semantic evaluation branch:
[0056] Trajectory visualization: The candidate trajectories generated by the Mamba branch are overlaid on the corrected 2D intersection plan to generate a trajectory-scene fusion image.
[0057] VLM Semantic Scoring: The semantic reasonableness of each trajectory-scene fusion image is evaluated using a pre-trained visual language model, taking into account factors such as whether the trajectory complies with traffic rules, the safety of the trajectory's interaction with other traffic participants, and the feasibility of the trajectory in a real traffic environment.
[0058] Output: VLM semantic score for each candidate trajectory .
[0059] R4-4: Adaptive Fusion and Trajectory Selection:
[0060] Design an adaptive weight fusion mechanism to dynamically adjust the contributions of the two branches based on the complexity of the scenario: .in, For adaptive weight parameters It can be dynamically adjusted based on the complexity of the scenario.
[0061] Select the candidate trajectory with the highest final score as the prediction result and output the future trajectory. Predicted vehicle location at time and direction angle .
[0062] Specifically, step R5 is as follows:
[0063] R5-1: Predicted location overlay and intersection area calculation:
[0064] The real-time locations of all vehicles are overlaid with their future locations predicted by the Mamba-VSF hybrid model on the same spatiotemporal map. For any two vehicles... and Obtain the directed bounding boxes of each pair of bounding boxes, and sort the vertices of the intersecting polygons of the two bounding boxes in clockwise order. Calculate the cross area accordingly. : .
[0065] R5-2: Risk Level Determination:
[0066] The collision risk level is determined based on the ratio of the intersection area of the two vehicles' directed bounding boxes to the minimum area of the two vehicles. The core parameter for this determination is the collision threshold. and collision hazard assessment threshold ,and , Must meet The constraints, among which This is the minimum difference between the threshold values.
[0067] , Base threshold The corresponding value ranges are divided according to vehicle scenarios, with large vehicles being... Small vehicles are Mixed scenes are There is an implicit relationship between the minimum basic threshold values in each scenario, i.e. .
[0068] , The base threshold needs to be combined with the speed correction coefficient. and distance correction factor Correction, among which Based on the relative speed of the two vehicles Divide the interval and take values. (High-speed threshold) , (Medium speed range) take , (Low speed threshold) And satisfy ; Based on the real-time distance between the two vehicles Divide the interval and take values. (Near distance threshold) , (Medium distance interval) take , (Long distance threshold) And satisfy .
[0069] The final judgment threshold is calculated by combining the base threshold and the correction coefficient, using the following formula:
[0070]
[0071]
[0072] Using the final threshold as the standard, combined with the cross area The area of the two vehicles' directional enclosure (vehicle )and (vehicle The risk level is determined according to the following rules: when When, it is determined that a collision is imminent; when When a collision risk is determined, When ), it is considered safe. Attached Figure Description
[0073] Figure 1 This is a flowchart of the steps of the present invention, used to clarify the execution order and data flow of each module in specific implementation.
[0074] Figure 2 This is a schematic diagram of a vehicle collision warning simulation, which is a visualization of the results of the R5 step. It is used to intuitively show the bounding box intersection status in the spatiotemporal graph. Detailed Implementation
[0075] R1: Large-Model-Driven UAV Swarm Image Acquisition and Intelligent Classification
[0076] This step uses a large model to dynamically optimize the drone's layout and trajectory, solving the problems of "blind spot coverage at intersections" and "image data filtering." The specific implementation is as follows:
[0077] R1-1 Input Parameter Definition
[0078] Basic parameters of the intersection: Assume the lateral length of the "grid" shaped structure of the cross intersection is... Vertical width is The reference altitude for drone flight is The camera's field of view is .
[0079] Historical image features: Let the set of coordinates of the occluded high-frequency regions be denoted as . Vehicle density during peak hours Vehicle density during off-peak hours is .
[0080] Large model configuration: Select a pre-trained large traffic scene model. (backbone) for The size of the pre-trained dataset is After inputting the above parameters, the model outputs scene adaptation weights: blind zone area weights. ordinary region weight ( (Ensure priority coverage of blind spots).
[0081] R1-2 UAV Spatial Layout Calculation
[0082] Side length of the camera area for a single drone: for ordinary areas, use the basic formula. Calculations are performed to ensure regular coverage; the side length of the blind zone is corrected to... , To increase the coverage density by narrowing the area (in order to improve the coverage density in blind spots), the flight altitude adjustment accuracy will be simultaneously increased from... Upgraded to .
[0083] Number of drones deployed: Number of drones deployed laterally (along the coast) direction) ,in Adapting weights for horizontal scenes ( Blind spot area ordinary areas take ); Number of vertical deployments (along) direction) ,in Weights are adapted for vertical scenarios (the same rules apply). The total number of deployments is Ensure there are no duplicates or blank covers.
[0084] R1-3 Flight Trajectory Planning and Execution
[0085] Trajectory strategy adaptation: peak hours ( , Using a "short-distance, high-frequency" strategy (with a density threshold), the lateral movement distance... Longitudinal movement distance Duration of stay Off-peak hours ( Adopting a "long-distance, low-frequency" strategy, the movement distance is maintained. , Unchanged, but the stay time is extended to ( ).
[0086] Trajectory execution and recording: The drone moves in the order of "blind spot first, then normal area," and records its location information via GPS module after each movement. Generate trajectory log ( (Time stamp).
[0087] R1-4 Image Acquisition and Classification Storage
[0088] Acquisition parameters: Camera resolution set to... (pixels), frame rate Matching the dwell time, the image format is .
[0089] Classification rule: Edge node embedding model The acquired images are categorized and stored. The storage path for core images (including dynamic vehicle targets) is: The storage path for the reference image (including static facilities) is: ( (Date); the storage path for redundant images (without critical targets) is (reserve (Automatic cleanup after 1 day).
[0090] R2: High-precision image correction based on S_MBLBP and perspective transformation
[0091] This step addresses the issues of "image distortion" and "pixel-geographic coordinate mismatch" through feature point extraction, improved LBP feature calculation, and perspective transformation. The specific implementation is as follows:
[0092] R2-1 Feature Point Selection and Labeling
[0093] Feature point set: Select static, high-texture feature points within the intersection, denoted as... Including: the center point of the intersection Lane intersection ; Stop line edge point Pedestrian crossing corner Turning point of the guide wire .
[0094] Coordinate annotation: Using the annotation tool labelimg_0BB, the pixel coordinates of each feature point are annotated on the baseline image. At the same time, its real-world coordinates are measured. ( (Coordinates in the geographic coordinate system).
[0095] R2-2 Improved S_MBLBP Feature Extraction
[0096] Image segmentation and mean calculation: dividing the image into blocks... Divide into large blocks, and each large block is further divided into... Small blocks; calculate the average gray value of the small blocks. ( For large block indexes, For small block index), use Replace pixel grayscale values within a small block to suppress isolated noise.
[0097] Neighborhood compensation correction: if ( The grayscale difference threshold. If the average value is the value of adjacent small blocks, then a weighted adjustment is applied: ( , (Increases with distance).
[0098] Binary codes and feature histograms: Choose a radius centered on A circular neighborhood is compared between the center and the neighborhood mean: , Conversely), generate Bit binary code Construct feature histograms (dimension is) ), statistics of all Frequency of occurrence.
[0099] Feature matching and bias labeling: Compared with standard feature sample library Perform cosine similarity matching to determine and The correspondence; if ( (Assuming a coordinate deviation threshold), this area is marked as a geometric deviation area. .
[0100] R2-3 Perspective Transformation Model Construction and Correction
[0101] Model parameter solution: Based on the perspective transformation model, select n ≥ 4 non-collinear feature points. Minimize the error function using the least squares method ( To predict pixel coordinates, solve for the matrix parameters.
[0102]
[0103] in For pixel coordinates, Geographic coordinates These are the parameters of the transformation matrix.
[0104] Correction execution: Input the image to be corrected into the perspective transformation function. Substitute the solution parameters and output the corrected image. Verify correction accuracy: If If so, the correction is successful.
[0105] R3: Vehicle Directed Localization, Angle Awareness, and Number Tracking Based on Directed Bounding Box Vehicle Detection Algorithm
[0106] This step achieves full-dimensional state perception of the vehicle through a directed detection model and tracking logic, and the specific implementation is as follows:
[0107] R3-1 YOLOv11-OBB Model Deployment and Parameter Configuration
[0108] R3-1-1 Model Training and Optimization
[0109] Build includes Dataset of vehicle images at an intersection Mark the directed bounding box parameters of the vehicle. Set batch size Learning rate Training rounds Training parameters, loss function adopted in, To lose weight, For bounding box loss, The loss is classified; the validation criterion is the vehicle detection accuracy on the test set. And angle prediction error .
[0110] R3-1-2 Model Inference and Output Parsing
[0111] With computing power Edge computing nodes The acceleration framework is adopted in China Increase reasoning speed to Input the corrected image The model outputs a set of vehicle parameters. ,in ( The coordinates of the pixel center point The width and length of the bounding box (Direction angle).
[0112] R3-2 Calculation of vehicle coordinates, angles and motion parameters.
[0113] R3-2-1 Real-world coordinate transformation
[0114] Through perspective transformation function The inverse transform will change the pixel center point coordinates Convert to geographic coordinates ,Right now .
[0115] R3-2-2 Calculation of Direction Angle and Motion Parameters
[0116] The direction angle output by the model The unified scope is (like ,but Based on two consecutive frames of images , The same vehicle coordinates , Calculate displacement and instantaneous speed Speed based on three consecutive frames , , Calculate acceleration .
[0117] R3-3 Vehicle Continuous Tracking and Optimization
[0118] For each vehicle in the first frame Assign a unique code And store the state In subsequent frames In the middle, for Define the search area , ( , Matching feature similarity Goals and updates If the tracking is interrupted, a function fitting the historical trajectory will be used. Predicted location or based on feature library Rematch Recovery .
[0119] R4: Vehicle trajectory prediction based on the Mamba-VSF hybrid model
[0120] This step generates the optimal trajectory using a dual-branch collaborative architecture, as detailed below:
[0121] R4-1 Manba-VSF Model Architecture and Parameter Configuration
[0122] R4-1-1 Model Structure
[0123] The model comprises two branches, Mamba and VSF, and a fusion layer: the Mamba branch is based on the State-Space Model (SSM), which takes a vehicle's historical trajectory sequence as input and outputs physically plausible candidate trajectories; the VSF branch is based on the Visual Language Model (VLM), which takes a trajectory-scene fusion image as input and outputs a semantic score; the fusion layer uses dynamic weights... By combining the outputs of the two branches, the optimal trajectory is output. .
[0124] R4-1-2 Input Data Preprocessing
[0125] With a time step of Internal collection past Motion parameters at each time step are used to form the input trajectory sequence. ; For sequences The parameters are normalized to make .
[0126] R4-2 Mamba Branch: Generation of Physically Reasonable Candidate Trajectories
[0127] R4-2-1 State Vectors and Transitions
[0128] Through the embedding layer Transform the initial parameters into a high-dimensional state vector. , (for the state dimension); then through Complete the state transition, where Here is the state transition matrix. For the input matrix, For the input vector, Gaussian noise ( (Noise standard deviation).
[0129] R4-2-2 Candidate Trajectory Generation
[0130] pass Implement prediction mapping, where For the output matrix, The bias matrix, Adjust Gaussian noise Generate K candidate trajectories Each trajectory It contains the position and angle information of the next T time steps. And calculate the generation confidence level. ,in .
[0131] R4-3 VSF Branch: Semantic Reasonableness Assessment
[0132] Candidate trajectory Superimposed on the corrected image In the middle, draw the trajectory line With prediction point Generate fused image ;enter The evaluation criteria (determining whether the trajectory conforms to traffic rules) are fed into the VLM to output a semantic score. ,in
[0133] R4-4 Adaptive Fusion and Optimal Trajectory Selection
[0134] R4-4-1 Dynamic Weight Calculation
[0135] Calculate the complexity of the scenario ,in For saturated density, For the proportion of vehicles turning, , As weights; and through Determine the dynamic weights, where Based on weights, To adjust the coefficient, The larger, The smaller.
[0136] R4-4-2 Determining the Optimal Trajectory
[0137] Calculate the final score for each candidate trajectory. Select The largest As Output .
[0138] R5: Collision risk assessment based on bounding box intersection area
[0139] This step uses dynamic thresholds and cross-area calculations to determine risk classification. The specific implementation is as follows:
[0140] R5-1 Vehicle Directed Boundary Overlay and Intersection Area Calculation
[0141] Establishing a spatiotemporal map of a geographic coordinate system The real-time vehicle location is overlaid on the image. With the optimal trajectory Predicted location For any two cars , At time t, its directed bounding box is obtained. , Calculate the set of vertices of intersecting polygons. ; and through Calculate the area of the bounding box intersection .
[0142] R5-2 Dynamic Risk Threshold Setting
[0143] R5-2-1 Quantification of Influencing Factors
[0144] Vehicle types are categorized into three types: large, small, and mixed-use, with corresponding basic thresholds set for each type. For large vehicles Small vehicles Mixed Scenes ,satisfy The implicit relationship.
[0145] Based on relative velocity Divide the speed range into high-speed, medium-speed, and low-speed ranges, and match the corresponding speed correction coefficients. Satisfying Relationships .when (High-speed threshold) (High-speed correction factor); (Medium speed range) (Medium speed correction factor); 1 (low speed threshold): (Low-speed correction factor).
[0146] Based on the real-time distance between the two vehicles Divide the distance into short, medium, and long ranges, and match the corresponding distance correction coefficients. ),satisfy .when (Near distance threshold) (Close-range correction factor); (Mid-distance range) (Mid-range correction factor); (Long-distance threshold) (Long-distance correction factor).
[0147] R5-2-2 Threshold Calculation
[0148] The collision determination threshold is calculated based on the quantified influencing factors. , Collision hazard assessment threshold , The constraints are satisfied. ( (Minimum difference of the threshold).
[0149] R5-3 Risk Classification and Early Warning Output
[0150] R5-3-1 Decision Logic
[0151] Calculate the area of the bounding box according to the decision logic. Take the minimum of the two. Then, based on the relationship between the cross area and the threshold, the classification is performed: It was determined to be a collision at that time. It was determined to be a collision hazard at that time. It was determined to be safe at that time.
[0152] R5-3-2 Early Warning Output
[0153] Based on the classification results, corresponding warnings are output; a collision triggers a red warning. (Audible and visual alarm); Yellow alert triggered by collision hazard. (Light alarm); simultaneously send warning information ( (To predict collision time) the data is transmitted to the traffic signal system. With cloud platform .
Claims
1. A method for intersection collision warning combining low-altitude aerial photography and dual-branch depth behavior prediction, characterized in that, Includes the following steps: R1. To address the blind spot problem at intersections, a "large model-driven UAV swarm end-to-end image acquisition and intelligent classification" architecture is adopted. The intersection structure parameters, UAV equipment parameters, and historical image features are input into a pre-trained large traffic scene model, which dynamically outputs a UAV spatial layout scheme to achieve seamless coverage of the intersection road surface, lane lines, stop lines, and blind spots. Simultaneously, edge nodes are embedded in the large model to process the acquired images end-to-end, classify and store them, providing a structured image data source for subsequent processing. R2. Based on Local Binary Feature (LBP) and perspective transformation models, this paper addresses the distortion problem in images acquired by UAVs by combining the noise resistance and dynamic background suppression advantages of the improved S_MBLBP (Stable Multi-Scale Block LBP) operator. It performs high-precision image correction, achieves accurate conversion between pixel coordinates and geographic coordinates, and obtains corrected images that can directly correspond to real-world distances, providing an accurate benchmark for subsequent vehicle positioning and motion parameter calculation. R3. Based on the directed bounding box vehicle detection algorithm, vehicle detection is performed on the intersection 2D plan view corrected by R2. Through the entire process of "directed detection - information conversion - parameter calculation - continuous tracking", the directed localization, angle perception, motion parameter acquisition and number tracking of vehicles are realized. R4. Based on the vehicle's historical trajectory sequence, a dual-branch collaborative trajectory estimation model is used to predict the trajectory. The Mamba branch generates physically reasonable candidate trajectories, the VSF branch evaluates the semantic reasonableness of the candidate trajectories, and the optimal predicted trajectory is output through an adaptive fusion mechanism to obtain the vehicle's future position and orientation angle. R5. Based on the real-time location and predicted trajectory of the vehicle, the directional bounding boxes of the vehicle are superimposed onto a unified spatiotemporal map. The intersection area of any two vehicle bounding boxes is calculated. The risk threshold is dynamically set in combination with vehicle type, speed and distance. The collision risk is determined according to the proportion of the intersection area to the minimum area of the two vehicle bounding boxes. The collision risk is then determined by three categories: collision, collision hazard and safety.
2. The intersection collision warning method combining low-altitude aerial photography and dual-branch depth behavior prediction according to claim 1, characterized in that, Step R1 specifically involves: R1-1: Large Model-Driven Target Acquisition and Scene Analysis: The basic parameters of the crossroads (defining the intersection as a "grid" shaped structure) are... ,Width Drone flight altitude Camera field of view Historical image features (coordinates of high-frequency occlusion areas, vehicle density distribution during peak hours) are input into a pre-trained traffic scene model. The model identifies key areas prone to blind spots (such as non-motorized vehicle lanes obscured by large vehicles turning, and pedestrian crossing corners) through semantic understanding, and outputs scene adaptation weights (defining the weights of blind spot areas). ordinary area This provides a basis for subsequent layout optimization. R1-2: Dynamic calculation of UAV spatial layout parameters: The large model calculates the side length of the camera area of a single drone based on scene adaptation weight differentiation: the ordinary area is calculated according to the basic formula. Calculations are performed to ensure regular coverage; The side length of the blind zone will be corrected to (By narrowing the area to increase coverage), the accuracy of flight altitude adjustment is improved simultaneously. The number of deployments is dynamically calculated based on scenario adaptation weights: the horizontal number is... ( (Adapt weights for horizontal scenarios); vertical quantity ( (Adapting weights for vertical scenarios); Total number The system ensures that drones are distributed according to the principle of "blind spot encryption and general adaptation," with no duplicate or blank coverage. R1-3: Large-scale collaborative flight trajectory planning: The large model generates a trajectory strategy based on real-time traffic conditions (vehicle density and speed initially collected by drones): during peak hours (vehicle density...). (Vehicles / km) adopts a "short distance, high frequency" strategy, lateral movement distance Longitudinal movement distance Duration of stay During off-peak hours, a "long-distance, low-frequency" strategy is adopted, with the moving distance remaining the same but the dwell time extended. The drone moves in the order of "first covering blind areas laterally, then extending to ordinary areas vertically" to avoid the lag in blind area data collection caused by the conventional order. GPS location information is recorded synchronously with each movement to ensure that the trajectory is traceable. R1-4: End-to-end image acquisition and intelligent classification: The acquired images are transmitted to the large model module embedded at the edge. The model intelligently classifies and stores the image information according to the core elements: it is stored under three categories of tags: "collision warning core image" (including dynamic vehicle targets), "scene reference image" (including static facilities), and "redundant filtered image" (without key targets).
3. The intersection collision warning method combining low-altitude aerial photography and dual-branch depth behavior prediction according to claim 1, characterized in that, Step R2 specifically involves: R2-1: Feature point selection: To address the issues of image distortion caused by perspective, altitude, and lens distortion in wide-angle lens imaging of drones, as well as the inconsistency between image coordinates and real-world coordinates, this paper combines improved LBP features to enhance feature matching stability in complex environments. This achieves accurate conversion between pixel coordinates and geographic coordinates, providing a precise "image-real-world" mapping for vehicle positioning. The paper focuses on extracting feature points with significant and stable textures at intersections, including the intersection center point, lane line intersections, stop line edges, pedestrian crossing corners, and turning point of guide lines, ensuring uniform coverage of key areas and providing accurate references for deviation identification. R2-2: Improved S_MBLBP feature extraction: Traditional LBP (Local Binary Pattern) operators are robust to changes in illumination, but are susceptible to noise and dynamic background interference. An improved S_MBLBP (Stable Multi-Scale Block Local Binary Pattern) operator is adopted, which uses block averaging and neighborhood compensation strategies to suppress the influence of dynamic background such as abrupt changes in illumination and improve the stability of feature extraction. The image is divided into blocks of a preset size, and each block is further divided into smaller regions. The mean gray value of all pixels in each small region is calculated and used to replace the gray value of a single pixel in the small region, thus avoiding interference from isolated noise pixels in feature extraction. A neighborhood compensation strategy is introduced. If the grayscale mean of a small area changes abruptly due to the dynamic background, the stability of the grayscale mean is further improved by weighting the grayscale mean of the small area and 2-3 surrounding small areas (the weight increases with the distance from the center area). Using key feature points in the image (intersection center point, lane line intersection point, stop line edge point, etc.) as the center, select a circular neighborhood with a preset radius (adapting to texture structures in different directions and reducing the impact of directional deviation on feature description). Compare the grayscale mean of the central region with that of each region in the neighborhood. If the mean of the neighborhood region is greater than or equal to the mean of the central region, it is recorded as "1", otherwise it is recorded as "0". Based on this, a unique binary code for each feature point is generated. A feature histogram is constructed based on the binary code of all feature points, forming an LBP feature descriptor with illumination robustness. This descriptor is then matched against a feature point sample library with known real geographic coordinates to determine the pixel coordinates corresponding to each feature point in the current image. Then, the pixel coordinates are compared with the real-world 3D coordinates of the feature points measured beforehand. Each image is compared individually. If the deviation between the two exceeds a preset threshold, the area is marked as an image geometric deviation area, providing a precise deviation reference for subsequent image correction. R2-3: Perspective Transformation Model Construction and Deviation Correction: To address the perspective distortion characteristics of drone aerial images, a perspective transformation model is used to construct a mapping relationship between pixel coordinates and real-world geographic coordinates, thereby achieving geometric deviation correction. The core principle of perspective transformation is to transform the image plane using homogeneous coordinates. Mapped to the world plane Its mathematical expression is: in, Image pixel coordinates Using real-world planar coordinates, These are the parameters of the perspective transformation matrix. Parameter solution process: Select at least 4 non-collinear feature points extracted from R2-1, and obtain the pixel coordinates of each feature point. and their corresponding real-world coordinates Minimize the error function using the least squares method The parameters of the perspective transformation matrix are obtained by solving.
4. The intersection collision warning method combining low-altitude aerial photography and dual-branch depth behavior prediction according to claim 1, characterized in that, Step R3 specifically involves: R3-1: Deployment of Directed Object Detection Models: Compared to traditional object detection models that can only output undirected axis-aligned bounding boxes, the YOLOv11-OBB model can output angled rotated bounding boxes, which can more accurately fit the actual contours and driving directions of vehicles, avoiding detection deviations caused by vehicle tilting. It is especially suitable for complex driving scenarios such as vehicle turning and lane changing at intersections. Here, the YOLOv11-OBB model is used to detect vehicles on the corrected 2D intersection plan. This model directly outputs oriented bounding boxes, whose parameters are a six-tuple: . in, These are the coordinates of the center point of the directed bounding box. , For the width and length of the bounding box, The bounding box major axis (front direction) and two-dimensional plan view The included angle of the axis, For vehicle codes. R3-2: Vehicle angle and position information acquisition: Center point coordinate transformation: Using the perspective transformation model in (II), the center point pixel coordinates are transformed. Convert to real-world coordinates . Driving direction angle reading: Vehicle steering angle Obtained directly from the output of the YOLOv11-OBB (You Only Look Once version 11 - Oriented Bounding Box) model. R3-3: Motion parameter calculation and tracking optimization: Exporting continuous frame images from a time series, the center point coordinates and angles of the same vehicle in different frames are obtained to complete the basic displacement data acquisition; the displacement is decomposed based on the vehicle's driving direction to obtain the displacement in the actual driving direction, and then the instantaneous velocity is obtained. If the vehicle turns, the displacement decomposition result is corrected by the average angle; this is combined with the speed difference and time interval between adjacent moments. Obtain acceleration To form complete vehicle motion parameters . Based on the vehicle's unique code The system achieves continuous vehicle tracking through "location association + feature matching": the search area is defined in consecutive frames, vehicle features are matched, and the parameters of the same vehicle are continuously updated; if tracking is interrupted, the vehicle position during the occlusion period can be predicted based on historical trajectory, or tracking can be resumed through feature rematching to ensure tracking continuity.
5. The intersection collision warning method combining low-altitude aerial photography and dual-branch depth behavior prediction according to claim 1, characterized in that, Step R4 specifically involves: R4-1: Mamba-VSF hybrid trajectory prediction model: Mamba-VSF (Mamba-Vision-Semantic Fusion) adopts a two-branch collaborative architecture. The Mamba branch is responsible for generating physically reasonable candidate trajectories based on historical trajectory data, while the VSF branch uses the cognitive ability of the visual language model to evaluate the semantic reasonableness of the candidate trajectories. Finally, the optimal predicted trajectory is output through an adaptive fusion mechanism. R4-2: Mamba trajectory generation branch: Input the vehicle's historical trajectory sequence, including past... The speed of each time step acceleration Direction angle Real-world coordinates . The temporal trajectory mapping relationship is constructed based on the state-space model (SSM). The specific process is as follows: State initialization: Initial time is set through the embedding layer. The motion parameters are converted into a high-dimensional state vector: in, It is a fully connected embedding layer. ( (where the state vector dimension is used) to achieve low-dimensional to high-dimensional mapping of motion parameters. State transition is calculated based on capturing the temporal dependence of the trajectory using linear state transition equations: in, This is the state transition matrix, which describes the influence of the previous state on the current state. The input matrix is weighted to account for the contribution of the motion parameters at the current time step. This is the motion parameter vector for the current time step; The vector is a Gaussian noise vector, simulating small disturbances in real traffic scenarios. The prediction output mapping maps the high-dimensional state vector to predicted future positions, as shown in the formula: in, For the output matrix, The bias matrix, Let t be the predicted position at time step t. Output generation A physically plausible candidate trajectory Each candidate trajectory contains future... Predicted location of time step Direction angle and generate confidence scores . R4-3: VSF semantic evaluation branch: Trajectory visualization involves overlaying candidate trajectories generated by the Mamba branch onto the corrected 2D intersection plan view to generate a trajectory-scene fusion image. VLM semantic scoring uses a pre-trained visual language model to evaluate the semantic reasonableness of each trajectory-scene fusion image, taking into account factors such as: whether the trajectory complies with traffic rules, the safety of the trajectory's interaction with other traffic participants, and the feasibility of the trajectory in a real traffic environment. Output the VLM semantic score for each candidate trajectory. . R4-4: Adaptive Fusion and Trajectory Selection: Design an adaptive weight fusion mechanism to dynamically adjust the contributions of the two branches based on the complexity of the scenario: in, For adaptive weight parameters It can be dynamically adjusted based on the complexity of the scenario. Select the candidate trajectory with the highest final score as the prediction result and output the future trajectory. Predicted vehicle location at time and direction angle .
6. The intersection collision warning method combining low-altitude aerial photography and dual-branch depth behavior prediction according to claim 1, characterized in that, Step R5 specifically involves: R5-1: Predicted location overlay and intersection area calculation: The real-time locations of all vehicles are overlaid with their future locations predicted by the Mamba-VSF hybrid model on the same spatiotemporal map. For any two vehicles... and Obtain the directed bounding boxes of each pair of bounding boxes, and sort the vertices of the intersecting polygons of the two bounding boxes in clockwise order. Calculate the cross area accordingly. : R5-2: Risk Level Determination: The collision risk level is determined based on the ratio of the intersection area of the two vehicles' directed bounding boxes to the minimum area of the two vehicles. The core parameter for this determination is the collision threshold. and collision hazard assessment threshold ,and , Must meet The constraints, among which This is the minimum difference between the threshold values. , Base threshold The corresponding value ranges are divided according to vehicle scenarios, with large vehicles being... Small vehicles are Mixed scenes are There is an implicit relationship between the minimum basic threshold values in each scenario, i.e. . , The base threshold needs to be combined with the speed correction coefficient. and distance correction factor Correction, among which Based on the relative speed of the two vehicles Divide the interval and take values. (High-speed threshold) , (Medium speed range) take , (Low speed threshold) And satisfy ; Based on the real-time distance between the two vehicles Divide the interval and take values. (Near distance threshold) , (Medium distance interval) take , (Long distance threshold) And satisfy . The final judgment threshold is calculated by combining the base threshold and the correction coefficient, using the following formula: Using the final threshold as the standard, combined with the cross area The area of the two vehicles' directional enclosure (vehicle )and (vehicle The risk level is determined according to the following rules: when When, it is determined that a collision is imminent; when When, it is determined that there is a risk of collision; when When ), it is considered safe.