85th vehicle speed survey method based on deep learning

Abstract

The invention relates to a 85th vehicle speed survey method based on deep learning, and belongs to the field of vehicle tracking. The method comprises the following steps: realizing multi-target vehicle detection by using a deep learning algorithm, identifying and classifying vehicles, and continuously updating the position and state information of the vehicles; mapping a camera picture to geographic coordinates, calculating coordinates of four angular points in a shooting range according to parameters of the unmanned aerial vehicle, and updating a corresponding relation between the coordinates; determining the position of the vehicle in a camera coordinate system through a deep learning model, and calculating the geographic coordinates of the vehicle through pixel coordinate normalization; calculating the geographical distance of driving in one second by utilizing a vehicle tracking result, connecting speed values at all moments to obtain a speed-time curve, calculating a distance-time curve by combining a road model created by track information, and calculating the 85th vehicle speed; the current position of the unmanned aerial vehicle can be corrected by using continuous images of continuous frames. Accurate and reliable data support is provided for applications such as vehicle tracking.

Description

Technical Field

[0001] The present invention belongs to the field of vehicle tracking, and in particular relates to a vehicle speed investigation method based on deep learning. Background Art

[0002] The mapping model between camera images and geographic coordinates provides a method for converting camera coordinates and geographic coordinates. Vincenty calculates the distance s between the geographic coordinates of vehicles between adjacent fixed frames on the WGS84 ellipsoid, providing accurate and reliable data support for applications such as vehicle tracking. It also creates a comprehensive model of road speeds at the 85th position, providing real-world data information for traffic flow analysis, road facility assessment, and traffic safety.

[0003] Previous vehicle speed survey methods mainly include manual observation, radar speed measurement, and ground-sensing coil speed measurement. Unlike the vehicle speed survey method described in the present invention, the manual method is time-consuming and labor-intensive, and is subject to significant subjective factors. Although the radar and coil methods can achieve automatic speed measurement, their installation and maintenance costs are high, and they can only be measured at specific locations, failing to achieve comprehensive coverage of the entire road network. In addition, these methods can often only measure the instantaneous speed of a vehicle, failing to obtain the vehicle's driving trajectory and speed changes, making it difficult to meet the refined requirements of traffic planning and management for vehicle speed information in a planar environment. Summary of the Invention

[0004] The purpose of the present invention is to provide an 85th vehicle speed survey method based on deep learning, which uses various parameters such as the position of the drone, its altitude, and the camera's field of view to complete the calculation and derivation of the geographic coordinates of the four corner points of the vehicle within the drone's shooting range. Subsequently, an association relationship is established between each pixel in the video image and the corresponding actual geographic coordinates, thereby achieving an effective mapping from the video image to the actual geographic coordinates. Next, the trajectory information of the vehicle between all adjacent fixed F frames of the deep learning detection target is merged, and when a comprehensive road model is created in real time, a direction-based weighted average method is used to fuse the path, and the direction vector of the path is taken into account. In this way, the directional characteristics of the path are better preserved, and the comprehensive road model can be reflected more accurately. Finally, the 85th vehicle speed calculation is achieved, and the drone position correction is completed.

[0005] To achieve the above objectives, the technical solution of the present invention is: a method for investigating the 85th vehicle speed based on deep learning, comprising:

[0006] Step 1: Multi-target vehicle tracking and identification: Use deep learning algorithms to track and identify multiple vehicles, identify and classify vehicles, and update vehicle location and status information;

[0007] Step 2: Mapping the camera image to geographic coordinates: Calculate the geographic coordinates of the four corner points of the drone's shooting range and update the correspondence between the video image and the geographic coordinates;

[0008] Step 3: Calculate the vehicle's geographic coordinates: Determine the position of the tracked vehicle in the camera coordinate system and calculate the tracked vehicle's geographic coordinates.

[0009] Step 4: Vehicle speed calculation: Calculate the geographic distance traveled by the vehicle in 1 second, and calculate the vehicle speed based on the geographic distance and time.

[0010] Step 5: Speed-time creation: Connect the speed values at each moment to obtain the real-time speed-time curve;

[0011] Step 6: Create a comprehensive road model: Use the dynamic time warping (DTW) method and the Gaussian mixture model (GMM) to perform trajectory fusion, merge vehicle trajectory information, create a comprehensive road model, and use the comprehensive road model to build a distance-time curve.

[0012] Step 7: Calculate the 85th vehicle speed: Calculate the 85th vehicle speed based on the speed-time curve and the distance-time curve;

[0013] Step 8. UAV position correction: Use the visual odometry (VO) and visual inertial odometry (VIO) to estimate the UAV position in real time, and perform bundle adjustment (BA) to optimize the trajectory regularly.

[0014] Furthermore, in step one, the Kalman filter is used to predict the trajectory of the high-speed moving vehicle, and the target information is updated in real time. At the same time, the cascade matching and Hungarian algorithm are used to match the predicted target trajectory and the detected target trajectory.

[0015] Furthermore, in step 2, based on the parameters of the drone including the position, altitude and field of view of the camera, the geographic coordinates of the four corner points of the drone's shooting range are calculated, and then each pixel point in the video screen is associated with the actual geographic coordinates to achieve the mapping of the video screen to the actual geographic coordinates.

[0016] Furthermore, the geographic coordinates of the four corner points of the drone's shooting range are calculated as follows: considering the horizontal field of view angle θ h and vertical field angle θ v The rotation angles of the drone, including pitch angle α, roll angle β and yaw angle γ, as well as the altitude h of the drone, are accurately calculated in real time using the Vincenty formula on the earth reference model WGS84 ellipsoid to calculate the geographic coordinates of the four corner points of the camera's field of view, namely: the latitude and longitude of the top left corner, the top right corner, the bottom left corner, and the bottom right corner.

[0017] Furthermore, the specific method for calculating the geographic coordinates of the four corner points of the drone's shooting range is as follows:

[0018] Assume that the drone is located at a point (λ0, φ0) on the Earth's surface, where λ0 is longitude and φ0 is latitude. The conversion between the drone's coordinate system and the Earth's coordinate system requires a rotation matrix. First, a standard rotation matrix is defined, and then the drone's coordinate system is rotated from its original position to the specified angle. The parameters of the Earth reference model WGS84 ellipsoid are: semi-major axis a = 6378137m, flattening e = 1 / 298.257223563, semi-minor axis b = a(1-1 / e), and focal length or zoom factor f.

[0019] (1) Position of corner points in the local coordinate system

[0020] Determine the positions of the four corner points of the drone's field of view in the local coordinate system v top-left 、v top-right 、v bottom-left 、v bottom-right :

[0021]

[0022] Among them, d h =2htan(θ h / 2), d v =2htan(θ v / 2);

[0023] (2) Rotation Matrix

[0024] Apply a rotation matrix to transform the four corner points from the local coordinate system to the global coordinate system:

[0025] For a given three Euler angles, the rotation matrix R is expressed as:

[0026]

[0027] v' corner =R·v corner

[0028] Where: v corner represents the corner point coordinate vector in the local coordinate system; v′ corner Represents the corner point coordinate vector in the global coordinate system;

[0029] (3) Converting 3D coordinates to geographic coordinates

[0030] The positions v′ of the four corner points in the global coordinate system corner Convert to geographic coordinates as follows:

[0031] 1) Geographic coordinates to geodetic coordinates

[0032] Convert the current position of the drone (λ0, φ0, h) to the Earth Center-Earth Fixed Coordinate System ECEF coordinates:

[0033] a. Convert geographic coordinates to geodetic coordinates:

[0034]

[0035] in is the square of the first flattening;

[0036] x0=(h+N)cos(λ0)cos(φ0)

[0037] y0=(h+N)sin(λ0)cos(φ0)

[0038] z0=[N(e 2 +1)+h]sin(φ0)

[0039] b. Convert the three-dimensional coordinates of the corner point to the Earth Center-Earth Fixed Coordinate System ECEF:

[0040]

[0041] in: Represents the coordinate vector of the corner point under ECEF; Represents the coordinate vector of the reference point in the local coordinate system under ECEF;

[0042] 2) Geodetic coordinates to geographic coordinates

[0043] Iterate and solve to find λ and φ that satisfy the ellipsoid equation, and convert the points under ECEF back to geographic coordinates:

[0044] a. Let x, y, z be points in ECEF;

[0045] b. Initial estimated longitude λ′ and latitude φ′;

[0046] c. Iterate until convergence:

[0047] Calculate the normal gravity radius N(φ′);

[0048] Update λ′ and φ′:

[0049] λ'=arctan2(u,x)

[0050]

[0051] in is the square of the second flattening;

[0052] Calculate λ′ and φ′ until λ′ and φ′ converge;

[0053] The λ′ and φ′ finally calculated are the geographic coordinates to be solved.

[0054] Furthermore, in step 3, in the calculation of the vehicle geographic coordinates, the Vincenty formula is used to accurately calculate the distance s between the geographic coordinates (λ1, φ1) and (λ2, φ2) corresponding to the vehicle between adjacent fixed frames on the earth reference model WGS84 ellipsoid;

[0055] 1) Calculate auxiliary variables: Based on the given latitude and longitude, auxiliary latitudes u1 and u2 are used to calculate the longitude difference λ;

[0056]

[0057] in: is the latitude of the first point, the latitude of the second point (radians); is the longitude of the first point, the longitude of the second point (in radians);

[0058] 2) Iterative solution: Use the iterative method to solve the central angle σ between two points;

[0059]

[0060] cosσ=sin(u1)sin(u2)+cos(u1)cos(u2)cos(λ)

[0061]

[0062] cos 2 α=1-sin 2 α

[0063]

[0064] λ=L+(1-C)fsinα(σ+Csinσ(cos(2σ m )+Ccosσ(-1+2cos 2 (2σ m ))))

[0065] Where: C is an auxiliary constant; L is the initial longitude difference approximation; σ m represents an intermediate angle introduced in the iterative process to calculate the high-order correction term related to the ellipsoid flattening (parameter f); specifically, it is expressed by the formula definition;

[0066] Termination condition: When the change of λ is less than the preset threshold, the iteration stops;

[0067] 3) Calculate distance: calculate the distance between two points;

[0068] u 2 =cos 2 α(a 2 -b 2 ) / b 2

[0069]

[0070] Where: A and B are auxiliary coefficients designed to more accurately calculate the distance between two points under the Earth ellipsoid model. By expanding the polynomial of u^2, the influence of factors such as the Earth's flattening on the distance is quantified, and the accuracy of the distance calculation is optimized;

[0071]

[0072] s=bA(σ-Δσ).

[0073] Furthermore, in step 4, the vehicle speed is calculated as follows:

[0074] Assuming the frame rate of the video is F, and the distance between the geographic coordinates of the vehicle between adjacent fixed F frames is s, the vehicle's speed v is calculated using the following formula:

[0075]

[0076] Here, 1 in the numerator represents 1 second.

[0077] Furthermore, step six is specifically implemented as follows:

[0078] 1) Use dynamic time warping (DTW) to construct a reference path curve

[0079] Each path T i Represented as a series of points:

[0080]

[0081] where n i represents the number of points on the i-th path;

[0082] Construct the cumulative distance matrix D:

[0083] The element D[i,j] of the cumulative distance matrix D represents the path T i The first i points and path T j The cumulative distance between the first j points of ; the formula for calculating D is as follows:

[0084]

[0085] Among them, d(Ti [i],T j [j]) is the path T i The i-th point and path T j The distance between the jth points is calculated accurately on the WGS84 ellipsoid using the Vincenty formula;

[0086] Backtracking the optimal path: Find the last element D[n i ,n j ], and then backtrack to find the optimal alignment path, the optimal alignment path is from (n i ,n j ) to (1,1) so that the cumulative distance is minimized;

[0087] Path=argmin (i,j) D[i,j]

[0088] Path represents the optimal alignment path determined by the search process;

[0089] 2) Construct Gaussian mixture model GMM translation trajectory

[0090] a. Use K Gaussian components to model the distribution of trajectories;

[0091] Initializing GMM parameters includes:

[0092] The weight ω of each Gaussian component k ;

[0093] The mean μ of each Gaussian component k ;

[0094] The covariance matrix Σ of each Gaussian component k ;

[0095] b. Apply EM algorithm to optimize GMM parameters

[0096] Use the EM algorithm to iteratively optimize the GMM parameters until convergence;

[0097] The E step is the expected step:

[0098] Calculate the responsibility of each trajectory point to belong to each Gaussian component k

[0099]

[0100] where ω l Represents the weight of the lth Gaussian component in the Gaussian mixture model (GMM);

[0101] in is the probability density function of the Gaussian distribution:

[0102]

[0103] Where D is the dimension of the data;

[0104] The M step is the maximization step:

[0105] Update parameter ω k ,μ k ,Σ k

[0106]

[0107]

[0108] Repeat the E and M steps until the parameters converge or the predetermined maximum number of iterations is reached;

[0109] 3) Projection to the most likely curve

[0110] For each trajectory T i , calculate the probability of it falling into each Gaussian component k, and project the trajectory point onto the reference path corresponding to the component with the highest probability;

[0111] For each trajectory point T i [j](T i represents the i-th trajectory, and T i [j] represents the jth point in this trajectory), and calculate the probability that it belongs to each Gaussian component k:

[0112]

[0113] Find the most likely Gaussian component k:

[0114] k * =argmax k p(T i [j]|μ k ,Σ k )

[0115] Then, T i [j] is projected to the corresponding Gaussian component k * On, if Represents the center point of the corresponding component, then the projection is to move the point to the nearest

[0116]

[0117] 4) Fusion calculation of comprehensive road model

[0118] GMM is used to describe the distribution of all trajectories, and a comprehensive road model is constructed based on the road model calculated by the weighted average of the means of all Gaussian components calculated by GMM:

[0119]

[0120] Furthermore, in step seven, based on the real-time speed v-time t curve and the distance S-time t curve, a distance S and a corresponding set of speeds V are obtained per second, i.e., per frame F, to calculate the 85th vehicle speed, as follows:

[0121] Assume that V has n velocity values, recorded as V=[v1,v2,…,v n ], sort the speed values from small to large to get a new ordered array V sorted ;

[0122] For each velocity value v i Calculate the cumulative frequency or cumulative probability. The cumulative frequency is calculated by calculating whether it is less than or equal to v i The ratio of the number of speed values to the total number is obtained;

[0123] The cumulative distribution function CDF is expressed as:

[0124]

[0125] Find the CDF(v 85 )≥v 85 The minimum speed value v 85 :

[0126] If n×0.85 is an integer k, then the 85th vehicle speed v 85 That is V sorted [k](round);

[0127] If n×0.85 is not an integer, assume it is equal to k+f, then v 85 The calculation is as follows:

[0128] v 85 =V sorted [k]+(V sorted [k+1]-V sorted [k])×f.

[0129] The present invention also provides a computer-readable storage medium on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, any of the method steps described above can be implemented.

[0130] Compared with the prior art, the present invention has the following beneficial effects: the present invention uses various parameters such as the position of the drone, its altitude, and the camera's field of view to complete the calculation and derivation of the geographic coordinates of the four corner points of the vehicle within the drone's shooting range. Subsequently, an association is established between each pixel point in the video screen and the corresponding actual geographic coordinates, thereby achieving effective mapping from the video screen to the actual geographic coordinates. Next, the trajectory information of the vehicle between all adjacent fixed F frames of the deep learning detection target is merged to create a comprehensive road model in real time. The direction-based weighted average method is used to fuse the path and take into account the direction vector of the path. In this way, the directional characteristics of the path are better preserved, and the comprehensive road model can be reflected more accurately. Finally, the 85th vehicle speed calculation is achieved, and the drone position correction is completed. The present invention can provide accurate and reliable data support for applications such as vehicle tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0131] Figure 1 The 85th vehicle speed survey method process based on deep learning;

[0132] Figure 2 Visualization of multi-target vehicle tracking and recognition process using deep learning algorithms;

[0133] Figure 3 Schematic diagram of drone locations and parameters.

[0134] Figure 4 Road alignment - 85th speed road model rendering. DETAILED DESCRIPTION

[0135] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0136] like Figure 1 As shown, the present invention provides an 85th vehicle speed investigation method based on deep learning, comprising:

[0137] Step 1: Multi-target vehicle tracking and identification: Use deep learning algorithms to track and identify multiple vehicles, identify and classify vehicles, and update vehicle location and status information;

[0138] Step 2: Mapping the camera image to geographic coordinates: Calculate the geographic coordinates of the four corner points of the drone's shooting range and update the correspondence between the video image and the geographic coordinates;

[0139] Step 3: Calculate the vehicle's geographic coordinates: Determine the position of the tracked vehicle in the camera coordinate system and calculate the tracked vehicle's geographic coordinates.

[0140] Step 4: Vehicle speed calculation: Calculate the geographic distance traveled by the vehicle in 1 second, and calculate the vehicle speed based on the geographic distance and time.

[0141] Step 5: Speed-time creation: Connect the speed values at each moment to obtain the real-time speed-time curve;

[0142] Step 6: Create a comprehensive road model: Use the dynamic time warping (DTW) method and the Gaussian mixture model (GMM) to perform trajectory fusion, merge vehicle trajectory information, create a comprehensive road model, and use the comprehensive road model to build a distance-time curve.

[0143] Step 7: Calculate the 85th vehicle speed: Calculate the 85th vehicle speed based on the speed-time curve and the distance-time curve;

[0144] Step 8. UAV position correction: Use the visual odometry (VO) and visual inertial odometry (VIO) to estimate the UAV position in real time, and perform bundle adjustment (BA) to optimize the trajectory regularly.

[0145] This invention incorporates a deep learning cascade matching mechanism into traditional vehicle detection methods. It performs a rigorous preliminary match of specific vehicle features under relatively strict conditions. For unmatched detection results and trajectories, an IOU matching attempt is then performed. This significantly improves matching accuracy and efficiency, and reduces the probability of false matches.

[0146] Given the complex and ever-changing road environment and the different driving characteristics of different vehicles, tracking loss and matching errors are prone to occur when detecting high-speed moving targets and various occlusion situations. Kalman filtering is used to predict the trajectory of high-speed moving targets, updating target information in real time. At the same time, cascade matching and Hungarian algorithm are used to match the predicted target trajectory with the detected target trajectory, enhancing tracking performance and reducing matching error rates. Figure 2 shown.

[0147] For the calculation of geographic coordinates of four corner points, such as Figure 3 As shown in the figure, based on the parameters such as the position and height of the drone and the field of view of the camera, the geographic coordinates of the four corner points of the vehicle in the drone's shooting range are calculated, and then each pixel point in the video image is associated with the actual geographic coordinates, thereby realizing the mapping of the video image to the actual geographic coordinates.

[0148] Based on the above method of the present invention, the present invention constructs the following system:

[0149] 1. Multi-target detection module

[0150] Combined with step 1, use deep learning algorithms to track and identify multiple vehicles. Identify and classify vehicles, and update vehicle location and status information:

[0151] 2. Geographic coordinate calculation module for detection targets

[0152] The coordinates of the drone video obtained in step 1 are converted into geographic coordinates through the rotation matrix.

[0153] Assume that the drone is located at a point (λ0, φ0) on the Earth's surface, where λ0 is longitude and φ0 is latitude. The conversion between the drone's coordinate system and the Earth's coordinate system requires a rotation matrix. First, a standard rotation matrix is defined, and then the drone's coordinate system is rotated from its original position to the specified angle. The parameters of the Earth reference model WGS84 ellipsoid are: semi-major axis a = 6378137m, flattening e = 1 / 298.257223563, semi-minor axis b = a(1-1 / e), and focal length or zoom factor f.

[0154] (1) First determine the position v of the UAV's field of view corner in the local coordinate system top-left 、v top-right 、v bottom-left 、v bottom-right :

[0155]

[0156] Among them, d h =2htan(θ h / 2), d v =2htan(θ v / 2);

[0157] (2) Apply the rotation matrix to transform the four corner points from the local coordinate system to the global coordinate system:

[0158] For a given three Euler angles, the rotation matrix R is expressed as:

[0159]

[0160] v′ corner =R·v corner

[0161] Where: v corner represents the corner point coordinate vector in the local coordinate system; v′ corner Represents the corner point coordinate vector in the global coordinate system; (3) Convert three-dimensional coordinates to geographic coordinates

[0162] First, the current position of the UAV (λ0, φ0, h) is converted into the Earth-Centered Earth-Fixed (ECEF) coordinate system.

[0163] 1) Geographic coordinates to geodetic coordinates (NED, North East coordinate system):

[0164]

[0165] in is the square of the first flattening;

[0166] x0=(h+N)cos(λ0)cos(φ0)

[0167] y0=(h+N)sin(λ0)cos(φ0)

[0168] z0=[N(e 2 +1)+h]sin(φ0)

[0169] 2) Convert the three-dimensional coordinates of the corner point to the Earth Center-Earth Fixed Coordinate System ECEF:

[0170]

[0171] in: Represents the coordinate vector of the corner point under ECEF; Represents the coordinate vector of the reference point in the local coordinate system under ECEF;

[0172] Next, convert the ECEF coordinates back to geographic coordinates.

[0173] Let x, y, z be points in ECEF;

[0174] Initial estimate of longitude λ′ and latitude φ′ (the initial estimate can be the position of the drone);

[0175] c. Iterate until convergence:

[0176] Calculate the normal gravity radius N(φ′);

[0177] Update λ′ and φ′:

[0178] λ′=arctan 2(y,x)

[0179]

[0180] in is the square of the second flattening;

[0181] Calculate λ′ and φ′ until λ′ and φ′ converge;

[0182] The λ′ and φ′ finally calculated are the geographic coordinates to be solved.

[0183] The deep learning model determines the position of the tracked vehicle in the camera coordinate system and calculates the geographic coordinates of the tracked vehicle. The pixel coordinates (u, v) are normalized to the position (x′, y′, 1) in the camera coordinate system. Thus, the position of the point in the camera coordinate system can be expressed as (x′, y′, 1), where: (cx ,c y ) is the center point of the image. The geographic coordinates of the tracked vehicle are calculated through step 2.

[0184] Using the tracking results, calculate the geographical distance the vehicle travels in 1 second.

[0185] (1) Use the Vincenty formula to accurately calculate the distance s between the geographic coordinates (λ1, φ1) and (λ2, φ2) corresponding to the vehicle between adjacent fixed frames on the WGS84 ellipsoid.

[0186] 1) Calculate auxiliary variables: Based on the given latitude and longitude, auxiliary latitudes u1 and u2 are used to calculate the longitude difference λ;

[0187]

[0188] in: is the latitude of the first point, the latitude of the second point (radians); is the longitude of the first point, the longitude of the second point (in radians);

[0189] 2) Iterative solution: Use the iterative method to solve the central angle σ between two points;

[0190]

[0191] cosσ=sin(u1)sin(u2)+cos(u1)cos(u2)cos(λ)

[0192]

[0193] cos 2 α=1-sin 2 α

[0194]

[0195]

[0196] λ=L+(1-C)fsinα(σ+Csinσ(cos(2σ m )+Ccosσ(-1+2cos 2 (2σ m ))))

[0197] Where: C is an auxiliary constant; L is the initial longitude difference approximation; σ m represents an intermediate angle introduced in the iterative process to calculate the high-order correction term related to the ellipsoid flattening (parameter f); specifically, it is expressed by the formula definition;

[0198] Termination condition: When the change of λ is less than the preset threshold, the iteration stops;

[0199] 3) Calculate distance: calculate the distance between two points;

[0200] u 2 =cos 2 α(a 2 -b 2 ) / b 2

[0201]

[0202] Where: A and B are auxiliary coefficients designed to more accurately calculate the distance between two points under the Earth ellipsoid model. By expanding the polynomial of u^2, the influence of factors such as the Earth's flattening on the distance is quantified, and the accuracy of the distance calculation is optimized;

[0203]

[0204] s=bA(σ-Δσ).

[0205] (2) Calculate the vehicle's speed. Assuming the video frame rate is F fps / s and the distance between the vehicle's address coordinates between adjacent fixed F frames is sm, the vehicle's speed v can be calculated using the following formula:

[0206] Assuming the video frame rate is F fps / s and the distance between the vehicle's geographic coordinates between adjacent fixed F frames is sm, the vehicle's speed v is calculated using the following formula:

[0207]

[0208] Here, 1 in the numerator represents 1 second.

[0209] 3. Comprehensive Road Model Creation Module

[0210] The speed v values at each moment (1s interval, i.e., adjacent fixed F frames) are connected to obtain the real-time speed v-time t curve of all deep learning detection targets.

[0211] The trajectory information of all vehicles between adjacent fixed F frames of deep learning detection targets is merged to create a comprehensive road model in real time for calculating the distance S-time t curve.

[0212] The paths are fused using a direction-based weighted average method, taking into account the direction vector of the path:

[0213] (1) Use the dynamic time warping (DTW) method to construct a reference path curve

[0214] Each path T iRepresented as a series of points:

[0215]

[0216] where n i represents the number of points on the i-th path;

[0217] 1) Construct the cumulative distance matrix D:

[0218] The element D[i,j] of the cumulative distance matrix D represents the path T i The first i points and path T j The cumulative distance between the first j points of ; the formula for calculating D is as follows:

[0219]

[0220] Among them, d(T i [i],T j [j]) is the path T i The i-th point and path T j The distance between the jth points is calculated accurately on the WGS84 ellipsoid using the Vincenty formula;

[0221] 2) Backtrack the optimal path: find the last element D[n i ,n j ], and then backtrack to find the optimal alignment path, the optimal alignment path is from (n i ,n j ) to (1,1) so that the cumulative distance is minimized;

[0222] Path=argmin (i,j) D[i,j]

[0223] Path represents the optimal alignment path determined by the search process;

[0224] (2) Constructing Gaussian mixture model GMM translation trajectory

[0225] 1) Use K Gaussian components to model the distribution of trajectories;

[0226] Initializing GMM parameters includes:

[0227] The weight ω of each Gaussian component k ;

[0228] The mean μ of each Gaussian component k ;

[0229] The covariance matrix Σ of each Gaussian component k ;

[0230] 2) Apply EM algorithm to optimize GMM parameters

[0231] Use the EM algorithm to iteratively optimize the GMM parameters until convergence;

[0232] E-step (expected step):

[0233] Calculate the responsibility of each trajectory point to belong to each Gaussian component k

[0234]

[0235] where ω l Represents the weight of the lth Gaussian component in the Gaussian mixture model (GMM);

[0236] in is the probability density function of the Gaussian distribution:

[0237]

[0238] where D is the dimension of the data (typically 2 in this case, since each point is a pair of latitude and longitude);

[0239] M step (maximization step):

[0240] Update parameter ω k ,μ k ,Σ k

[0241]

[0242] Repeat the E and M steps until the parameters converge or the predetermined maximum number of iterations is reached;

[0243] (3) Projection to the most likely curve

[0244] For each trajectory T i , calculate the probability of it falling into each Gaussian component k, and project the trajectory point onto the reference path corresponding to the component with the highest probability;

[0245] For each trajectory point T i [j](T i represents the i-th trajectory, and T i [j] represents the jth point in this trajectory), and calculate the probability that it belongs to each Gaussian component k:

[0246]

[0247] Find the most likely Gaussian component k:

[0248] k * =argmax k p(Ti [j]|μ k ,Σ k )

[0249] Then, T i [j] is projected to the corresponding Gaussian component k * On, if Represents the center point of the corresponding component, then the projection is to move the point to the nearest

[0250]

[0251] (4) Fusion calculation of comprehensive road model

[0252] GMM is used to describe the distribution of all trajectories, and a comprehensive road model is constructed based on the road model calculated by the weighted average of the means of all Gaussian components calculated by GMM:

[0253]

[0254] (5) According to the real-time speed v-time t curve and the distance S-time t curve, a distance S and a corresponding set of speeds V are obtained every second (per frame F), and the 85th vehicle speed can be calculated as follows:

[0255] Assume that V has n velocity values, recorded as V=[v1,v2,…,v n ], sort the speed values from small to large to get a new ordered array V sorted ;

[0256] For each velocity value v i Calculate the cumulative frequency or cumulative probability. The cumulative frequency is calculated by calculating whether it is less than or equal to v i The ratio of the number of speed values to the total number is obtained;

[0257] The cumulative distribution function CDF is expressed as:

[0258]

[0259] Find the CDF(v 85 )≥v 85 The minimum speed value v 85 :

[0260] If n×0.85 is an integer k, then the 85th vehicle speed v 85 That is V sorted [k](round);

[0261] If n×0.85 is not an integer, assume it is equal to k+f, then v 85 The calculation is as follows:

[0262] v 85 =V sorted [k]+(V sorted [k+1]-V sorted [k])×f.

[0263] 4. UAV Correction Module

[0264] Continuous images using consecutive frames can be used to correct the current position of the drone (λ0, φ0, h).

[0265] (1) Real-time estimation of visual odometry (VO) and visual inertial odometry (VIO):

[0266] When each frame arrives, VO is used to estimate the relative motion of the camera.

[0267] Combined with IMU data, VIO is used for fusion to obtain more accurate pose estimation.

[0268] (2) Bundle Adjustment (BA) Periodic Optimization:

[0269] BA is performed periodically (for example, every certain number of frames) to optimize the entire trajectory and reduce the accumulated error.

[0270] The present invention also provides a computer-readable storage medium on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, any of the method steps described above can be implemented.

[0271] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions and effects do not exceed the scope of the technical solution of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A method for investigating vehicle speed at the 85th place based on deep learning, characterized in that: include: Step 1: Multi-target vehicle tracking and identification: Use deep learning algorithms to track and identify multiple vehicles, identify and classify vehicles, and update vehicle location and status information; Step 2: Mapping the camera image to geographic coordinates: Calculate the geographic coordinates of the four corner points of the drone's shooting range and update the correspondence between the video image and the geographic coordinates; Step 3: Calculate the vehicle's geographic coordinates: Determine the position of the tracked vehicle in the camera coordinate system and calculate the tracked vehicle's geographic coordinates. Step 4: Vehicle speed calculation: Calculate the geographic distance traveled by the vehicle in 1 second, and calculate the vehicle speed based on the geographic distance and time. Step 5: Speed-time creation: Connect the speed values at each moment to obtain the real-time speed-time curve; Step 6: Create a comprehensive road model: Use the dynamic time warping (DTW) method and the Gaussian mixture model (GMM) to perform trajectory fusion, merge vehicle trajectory information, create a comprehensive road model, and use the comprehensive road model to build a distance-time curve. Step 7: Calculate the 85th vehicle speed: Calculate the 85th vehicle speed based on the speed-time curve and the distance-time curve; Step 8. UAV position correction: Use the visual odometry (VO) and visual inertial odometry (VIO) to estimate the UAV position in real time, and perform bundle adjustment (BA) to optimize the trajectory regularly.

2. The 85th vehicle speed investigation method based on deep learning according to claim 1 is characterized in that: In step 1, Kalman filtering is used to predict the trajectory of high-speed moving vehicles, and the target information is updated in real time. At the same time, cascade matching and Hungarian algorithm are used to match the predicted target trajectory with the detected target trajectory.

3. The 85th vehicle speed investigation method based on deep learning according to claim 1 is characterized in that: In step 2, based on the parameters of the drone including position, altitude and field of view of the camera, the geographic coordinates of the four corner points of the drone's shooting range are calculated, and then each pixel point in the video screen is associated with the actual geographic coordinates to realize the mapping of the video screen to the actual geographic coordinates.

4. The 85th vehicle speed investigation method based on deep learning according to claim 3 is characterized in that: The way to calculate the geographic coordinates of the four corner points of the drone's shooting range is: considering the horizontal field of view angle θ h and vertical field angle θ v The rotation angles of the drone, including pitch angle α, roll angle β and yaw angle γ, as well as the altitude h of the drone, are accurately calculated in real time using the Vincenty formula on the earth reference model WGS84 ellipsoid to calculate the geographic coordinates of the four corner points of the camera's field of view, namely: the latitude and longitude of the top left corner, the top right corner, the bottom left corner, and the bottom right corner.

5. The 85th vehicle speed investigation method based on deep learning according to claim 4 is characterized in that: The specific method for calculating the geographic coordinates of the four corner points of the drone's shooting range is as follows: Assume that the drone is located at a point (λ0, φ0) on the Earth's surface, where λ0 is longitude and φ0 is latitude. The conversion between the drone's coordinate system and the Earth's coordinate system requires a rotation matrix. First, a standard rotation matrix is defined, and then the drone's coordinate system is rotated from its original position to the specified angle. The parameters of the Earth reference model WGS84 ellipsoid are: semi-major axis a = 6378137m, flattening e = 1 / 298.257223563, semi-minor axis b = a(1-1 / e), and focal length or zoom factor f. (1) Position of corner points in the local coordinate system Determine the positions of the four corner points of the drone's field of view in the local coordinate system v top-left 、v top-right 、v bottom-left 、v bottom-right : among them,d h =2htan(θ h / 2),d v =2htan(θ v / 2); (2) Rotation Matrix Apply a rotation matrix to transform the four corner points from the local coordinate system to the global coordinate system: For a given three Euler angles, the rotation matrix R is expressed as: v′ corner =R·v corner Where: v corner represents the corner point coordinate vector in the local coordinate system; v′ corner Represents the corner point coordinate vector in the global coordinate system; (3) Converting 3D coordinates to geographic coordinates The positions v′ of the four corner points in the global coordinate system corner Convert to geographic coordinates as follows: 1) Geographic coordinates to geodetic coordinates Convert the current position of the drone (λ0, φ0, h) to the Earth Center-Earth Fixed Coordinate System ECEF coordinates: a. Convert geographic coordinates to geodetic coordinates: in is the square of the first flattening; x0=(h+N)cos(λ0)cos(φ0) y0=(h+N)sin(λ0)cos(φ0) z0=[N(e 2 +1)+h]sin(φ0) b. Convert the three-dimensional coordinates of the corner point to the Earth Center-Earth Fixed Coordinate System ECEF: in: Represents the coordinate vector of the corner point under ECEF; Represents the coordinate vector of the reference point in the local coordinate system under ECEF; 2) Geodetic coordinates to geographic coordinates Iterate and solve to find λ and φ that satisfy the ellipsoid equation, and convert the points under ECEF back to geographic coordinates: a. Let x, y, z be points in ECEF; b. Initial estimated longitude λ′ and latitude φ′; c. Iterate until convergence: Calculate the normal gravity radius N(φ′); Update λ′ and φ′: λ′=arctan2(y,x) in is the square of the second flattening; Calculate λ′ and φ′ until λ′ and φ′ converge; The λ′ and φ′ finally calculated are the geographic coordinates to be solved.

6. The 85th vehicle speed investigation method based on deep learning according to claim 5 is characterized in that: In step 3, in the calculation of the vehicle's geographic coordinates, the Vincenty formula is used to accurately calculate the distance s between the vehicle's corresponding geographic coordinates (λ1, φ1) and (λ2, φ2) between adjacent fixed frames on the earth reference model WGS84 ellipsoid; 1) Calculate auxiliary variables: Based on the given latitude and longitude, auxiliary latitudes u1 and u2 are used to calculate the longitude difference λ; in: is the latitude of the first point and the latitude of the second point; is the longitude of the first point, the longitude of the second point; 2) Iterative solution: Use the iterative method to solve the central angle σ between two points; cosσ=sin(u1)sin(u2)+cos(u1)cos(u2)cos(λ) cos 2 α=1-sin 2 a λ=L+(1-C)fsinα(σ+Csinσ(cos(2σ m )+Ccosσ(-1+2cos 2 (2p m )))) Where: C is an auxiliary constant; L is the initial longitude difference approximation; σ m represents an intermediate angle introduced in the iterative process to calculate the high-order correction term related to the ellipsoid flattening f; Termination condition: When the change of λ is less than the preset threshold, the iteration stops; 3) Calculate distance: calculate the distance between two points; you 2 =cos 2 a(a 2 -b 2 ) / b 2 Among them: A and B are the auxiliary coefficients of the design; s=bA(σ-Δσ).

7. The 85th vehicle speed investigation method based on deep learning according to claim 6 is characterized in that: In step 4, the vehicle speed is calculated as follows: Assuming the frame rate of the video is F, and the distance between the geographic coordinates of the vehicle between adjacent fixed F frames is s, the vehicle's speed v is calculated using the following formula: Here, 1 in the numerator represents 1 second.

8. The 85th vehicle speed investigation method based on deep learning according to claim 7 is characterized in that: Step 6: The specific implementation is as follows: 1) Use dynamic time warping (DTW) to construct a reference path curve Each path T i Represented as a series of points: where n i represents the number of points on the i-th path; Construct the cumulative distance matrix D: The element D[i,j] of the cumulative distance matrix D represents the path T i The first i points and path T j The cumulative distance between the first j points of ; the formula for calculating D is as follows: Among them, d(T i [i],T j [j]) is the path T i The i-th point and path T j The distance between the jth points is calculated accurately on the WGS84 ellipsoid using the Vincenty formula; Backtracking the optimal path: Find the last element D[n i ,n j ], and then backtrack to find the optimal alignment path, the optimal alignment path is from (n i ,n j ) to (1,1) so that the cumulative distance is minimized; Path=argmin (i,j) D[i,j] Path represents the optimal alignment path determined by the search process; 2) Constructing Gaussian mixture model GMM translation trajectory a. Use K Gaussian components to model the distribution of trajectories; Initializing GMM parameters includes: The weight ω of each Gaussian component k ; The mean μ of each Gaussian component k ; The covariance matrix Σ of each Gaussian component k ; b. Apply EM algorithm to optimize GMM parameters Use the EM algorithm to iteratively optimize the GMM parameters until convergence; The E step is the expected step: Calculate the responsibility of each trajectory point to belong to each Gaussian component k where ω l Represents the weight of the lth Gaussian component in GMM; in is the probability density function of the Gaussian distribution: Where D is the dimension of the data; The M step is the maximization step: Update parameter ω k ,μ k ,Σ k Repeat the E and M steps until the parameters converge or the predetermined maximum number of iterations is reached; 3) Projection to the most likely curve For each trajectory T i , calculate the probability of it falling into each Gaussian component k, and project the trajectory point onto the reference path corresponding to the component with the highest probability; For each trajectory T i The jth trajectory point T in i [j], calculate the probability that it belongs to each Gaussian component k: Find the most likely Gaussian component k: k * =argmax k p(T i [j]|m k ,S k ) Then, T i [j] is projected to the corresponding Gaussian component k * On, if Represents the center point of the corresponding component, then the projection is to move the point to the nearest 4) Fusion calculation of comprehensive road model GMM is used to describe the distribution of all trajectories, and a comprehensive road model is constructed based on the road model calculated by the weighted average of the means of all Gaussian components calculated by GMM:

9. The 85th vehicle speed investigation method based on deep learning according to claim 8 is characterized in that: Step 7: Based on the real-time speed v-time t curve and the distance S-time t curve, a distance S and a corresponding set of speeds V are obtained per second, i.e., per frame F, to calculate the 85th vehicle speed, as follows: Assume that V has n velocity values, recorded as V=[v1,v2,…,v n ], sort the speed values from small to large to get a new ordered array V sorted ; For each velocity value v i Calculate the cumulative frequency or cumulative probability. The cumulative frequency is calculated by calculating whether it is less than or equal to v i The ratio of the number of speed values to the total number is obtained; The cumulative distribution function CDF is expressed as: Find the CDF(v 85 )≥v 85 The minimum speed value v 85 : If n×0.85 is an integer k, then the 85th vehicle speed v 85 That is V sorted [k], [k] means rounding k); If n×0.85 is not an integer, assume it is equal to k+f, then v 85 The calculation is as follows: v 85 =V sorted [k]+(V sorted [k+1]-V sorted [k])×f。 10. A computer-readable storage medium storing computer program instructions that can be executed by a processor, wherein when the processor executes the computer program instructions, the method steps according to any one of claims 1 to 9 can be implemented.

Images