A method for long-range vehicle speed recognition and tracking based on target detection

By combining deep learning and time series modeling with YOLOv12 and Kalman filtering, vehicle speed estimation is achieved directly in the image pixel space. This solves the problems of vehicle occlusion and lighting changes in video speed measurement technology, improving the accuracy and adaptability of speed estimation, and is suitable for video surveillance and autonomous driving.

CN121482728BActive Publication Date: 2026-11-17HANGZHOU KUANGXING TECHNOLOGY CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202511759489.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-11-27
Publication Date
2026-11-17
Estimated Expiration
2045-11-27

AI Technical Summary

Technical Problem

Existing video speed measurement technologies cannot correctly correlate target trajectories when vehicles are obscured or lighting conditions change, resulting in discontinuous speed calculations. They also require precise camera parameter calibration, leading to high deployment complexity, and especially low speed detection accuracy at low frame rates.

Method used

We employ a deep learning-based object detection method combined with time series modeling. By using a pre-trained YOLOv12 model and Kalman filtering, we can directly map vehicle position changes to real speed in the image pixel space, reducing the dependence on camera parameter calibration. We also use FC-LSTM-CorrNet ​​for dynamic correction to improve speed estimation accuracy.

Benefits of technology

It significantly improves the accuracy and adaptability of speed estimation, reduces the dependence on camera calibration, and is suitable for long-distance vehicle tracking and speed estimation in video surveillance and autonomous driving scenarios. It is also more flexible in deployment and adaptable to complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121482728B_ABST
    Figure CN121482728B_ABST
Patent Text Reader

Abstract

This invention discloses a method for long-distance vehicle speed recognition and tracking based on target detection. The method includes: acquiring real-time video of vehicle movement from a fixed camera position and angle; identifying vehicles in each frame of the video using a pre-trained YOLOv12 target detection model; performing preliminary speed estimation of the vehicle using inter-frame difference, then constraining the rate of speed change and applying Kalman filtering for temporal correction to obtain a corrected speed estimate; calculating the displacement and deflection angle in the horizontal and vertical directions of adjacent frames, and integrating these with the corrected speed estimate to form a time-series input feature; and modeling the historical trajectory using a trained time-series deep learning model to achieve dynamic speed correction. This invention significantly reduces the dependence on physical space calibration and improves the model's ability to perceive motion trends, making it suitable for vehicle tracking and speed measurement tasks in long-distance, low-resolution scenarios.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to a method for long-distance vehicle speed recognition and tracking based on target detection. Background Technology

[0002] Vehicle speed measurement systems are widely used in road monitoring and autonomous driving. Existing vehicle speed measurement methods are mainly divided into two categories: one relies on hardware for speed measurement, such as radar speed measurement; the other analyzes captured video footage for speed measurement, such as the DeepSORT algorithm and the TrackRNN algorithm.

[0003] Compared to visual speed measurement solutions, speed measurement methods based on dedicated hardware require additional sensor modules, resulting in higher system integration complexity and increased equipment and maintenance costs. Therefore, video speed measurement is the future trend. However, video speed measurement technology also has a series of problems. When a vehicle is obscured for several frames, such as by a large vehicle or sudden changes in lighting, the system may fail to correctly associate the target trajectory, leading to ID switching or detection interruption. The speed calculation may instantly drop to zero or jump, resulting in speed abrupt changes that do not conform to physical laws, affecting data continuity and reliability. Video speed measurement usually requires on-site camera calibration, including precise measurements of installation height, pitch angle, focal length, etc. If the camera moves or the viewing angle is adjusted, recalibration is required, which increases deployment complexity. Speed ​​calculation is based on inter-frame displacement. When the frame rate is low, the target moves a large distance in adjacent frames, resulting in low speed detection accuracy. Moreover, due to the reduced sampling frequency, it is difficult to capture speed change trends.

[0004] Therefore, it is essential to invent an anti-interference video speed measurement algorithm, which has significant application value. Summary of the Invention

[0005] The purpose of this invention is to address the limitations of existing vehicle speed measurement technologies by providing a long-distance vehicle speed recognition and tracking method based on target detection. This invention utilizes deep learning algorithms and introduces time-series modeling, effectively avoiding the need for precise camera parameter calibration in traditional methods. It directly maps pixel displacement to actual physical velocity in the image pixel space, thereby significantly improving the accuracy and adaptability of speed estimation. This method is suitable for long-distance continuous vehicle tracking and speed estimation in video surveillance and autonomous driving scenarios.

[0006] The objective of this invention is achieved through the following technical solution: a method for long-distance vehicle speed recognition and tracking based on target detection, comprising the following steps: Step 1: Real-time acquisition of vehicle driving video using a fixed camera position and angle; identification of vehicles in each frame of the driving video using a pre-trained target detection model; extraction of the center coordinates of the detection box as the geometric center of the vehicle image; obtaining information on the positional changes of the vehicle in consecutive frames. Step 2: Combining the vehicle's historical location information, the inter-frame difference method is used to perform a preliminary speed estimation of the car to obtain a preliminary speed estimate. Then, the speed change rate is constrained to obtain a constrained speed estimate. Kalman filtering is used to perform time-series correction to obtain a corrected speed estimate. Step 3: Based on the vehicle's position change information in consecutive frames, calculate the displacement in the horizontal and vertical directions and the motion deflection angle between the current frame and the previous frame, and integrate them with the corrected speed estimate to form the time series input features. Step 4: Input the time series input features into the trained time series deep learning model, and dynamically correct the corrected velocity estimate by learning historical motion features to obtain the final velocity estimate.

[0007] Furthermore, the pre-trained target detection model is obtained through the following method: First, images of cars in traffic environments under different weather conditions, lighting conditions, vehicle speeds, and angles are acquired. Then, the images are enhanced using various data augmentation methods. Finally, bounding box annotations are added to the vehicle targets in the enhanced images using open-source annotation tools to construct a single-class car target detection dataset with high annotation accuracy. Then YOLOv12 was selected as the object detection model, with the image as input and the car category probability, confidence score and detection box as output. Secondly, YOLOv12 was trained using a single-class car target detection dataset. During the training process, the model parameters of YOLOv12 were adjusted with the goal of minimizing the total loss function in order to obtain a well-trained YOLOv12. Finally, a ByTrack attention mechanism is added after the output of the trained YOLOv12 detection head to serve as a pre-trained object detection model.

[0008] Furthermore, the data augmentation method includes brightness perturbation, blur interference, and noise perturbation.

[0009] Furthermore, the step of training YOLOv12 using a single-class vehicle target detection dataset specifically includes: Images from a single-class car target detection dataset are input into YOLOv12 to obtain car category probabilities, confidence scores, and bounding boxes. Classification loss is calculated based on the car category probability and the corresponding ground truth class of the image. Bounding box regression loss is obtained by calculating the intersection-union ratio (IU / U) based on the bounding box labels and the corresponding bounding box labels in the single-class car target detection dataset. The total loss function of the target detection model is obtained based on the classification loss and the bounding box regression loss. With minimizing the total loss function of the target detection model as the optimization objective, the AdamW optimizer and cosine annealing learning rate scheduler are used to adjust the YOLOv12 model parameters through backpropagation until a preset number of training epochs are reached, resulting in a trained YOLOv12.

[0010] Furthermore, by combining the vehicle's historical location information and employing the inter-frame difference method to perform a preliminary speed estimation, a preliminary speed estimate is obtained, specifically including: Based on the vehicle's position change information in consecutive frames obtained in step 1, i.e., the vehicle position sequence The inter-frame difference method is used to calculate the pixel displacement of the vehicle between adjacent frames; the pixel displacement is then converted into a preliminary speed estimate based on the conversion factor between the pixel and the actual distance. , represented as: In the formula, This represents the initial velocity estimate at time t; This represents the conversion factor at time t, calculated by comparing the average height H of the actual vehicle with the height of the target detection box. Obtaining the ratio between them; This represents the pixel displacement between adjacent frames at time t. , The x and y coordinates represent the vehicle's position at time t in video frame t; This indicates the time interval corresponding to the video frame rate.

[0011] Furthermore, the process of obtaining a constrained velocity estimate by constraining the rate of change of velocity, and then performing time-series correction using Kalman filtering to obtain a corrected velocity estimate, specifically includes: First, the rate of change of velocity is constrained. Specifically, the preliminary velocity estimate is automatically cropped according to the following formula to ensure that the rate of change of velocity is less than or equal to a preset maximum acceleration threshold: In the formula, This represents the constrained speed estimate. Represents the rate of change of velocity. , This indicates the preset maximum acceleration threshold; Then, Kalman filtering is used to perform time-series correction on the constrained velocity estimates, and the constrained velocity estimates are... As a new velocity observation input for each frame The Kalman filter is updated in the following two stages: Prediction phase: ; Correction phase: ; In the formula, Represents the state vector. and Let these represent the vehicle's velocity and acceleration at time t, respectively. Indicates the Kalman state transition moments, Represents the observation moments; The Kalman gain is expressed as a function of the system noise covariance. and observation noise covariance Calculated; The first component of the final Kalman filtered state vector This is the corrected speed estimate. .

[0012] Further, in step 3, the time series input features are represented as follows: ,in This represents the time series input features at time t. and Let represent the displacement increments in the x and y directions of adjacent frames at time t, respectively. This represents the pixel displacement between adjacent frames at time t. and Let represent the sine and cosine values ​​of the deflection angle at time t, respectively. This represents the corrected speed estimate.

[0013] Furthermore, the time series deep learning model includes a feature transformation layer, a temporal modeling layer, and a nonlinear correction layer. The feature transformation layer uses a fully connected network to map the time series input features into the time series deep learning model to a high-dimensional representation space. The temporal modeling layer uses a single-layer long short-term memory unit to perform time-dependent modeling on the feature sequence output by the feature transformation layer. The nonlinear correction layer is used to convert the output of the temporal modeling layer into a final velocity estimate through nonlinear mapping, so as to achieve dynamic correction of the corrected velocity estimate.

[0014] Furthermore, the trained time-series deep learning model is obtained through the following method: A training dataset is constructed based on the collected videos of vehicles driving and the corresponding actual observed speeds; Based on the videos in the training dataset, the corresponding final velocity estimates are obtained through steps 1-3 above. The SmoothL1 loss function is used to supervise the training of the final velocity estimate and its corresponding real observed velocity. At the same time, an acceleration regularization term is introduced to obtain the total loss function of the time series deep learning model. With minimizing the total loss function of the time series deep learning model as the optimization objective, the AdamW optimizer and cosine annealing learning rate scheduler are used to adjust the parameters of the time series deep learning model until the preset number of training rounds is reached, thus obtaining a well-trained time series deep learning model.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention breaks through the limitations of camera calibration and is more flexible in deployment. The “FC-LSTM-CorrNet” network structure proposed in this invention combines the feature transformation capability of fully connected networks with the time series modeling advantages of LSTM to establish a mapping relationship between pixel displacement and velocity, which significantly reduces the dependence on physical space calibration and has stronger practical deployment adaptability.

[0016] (2) This invention integrates deep detection and time series learning, which can accurately model dynamic trajectories. This invention is the first to organically integrate YOLOv12 and LSTM architecture, introduce features such as motion direction, deflection angle, and preliminary speed estimation, and form multi-dimensional time series input, which improves the model's ability to perceive motion trends. It is particularly suitable for vehicle tracking and speed measurement tasks in long-distance, low-resolution scenarios, and significantly improves the accuracy and adaptability of speed estimation. Attached Figure Description

[0017] Figure 1 This is a flowchart of the vehicle long-distance speed recognition and tracking method based on target detection according to the present invention; Figure 2 This is a schematic diagram of the network structure of the A2C2f module of the present invention; Figure 3 This is a training result diagram of YOLOv12 of the present invention; Figure 4 This is a comparison chart of the speed prediction, preliminary predicted speed, and actual observed speed of the time series deep learning model of this invention. Detailed Implementation

[0018] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not intended to limit this application.

[0019] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0020] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to determination," or "includes." Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process or method. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0021] The present invention will now be described in detail with reference to the accompanying drawings. Unless otherwise specified, the features of the following embodiments and implementations can be combined with each other.

[0022] This invention provides a method for long-range vehicle speed recognition and tracking based on target detection, applicable to fixed-view video surveillance or vehicle-mounted camera scenarios, such as... Figure 1 As shown, the specific steps include: Step 1: Real-time acquisition of vehicle driving video from a fixed position and angle; identification of vehicles in each frame of the driving video using a pre-trained target detection model; extraction of the center coordinates of the detection box as the geometric center of the vehicle image; and obtaining information on the positional changes of the vehicle in consecutive frames.

[0023] In this embodiment, the pre-trained target detection model is obtained through the following steps: Step 1.1: First, select typical urban roads and highway scenes, and use cameras to capture images of cars in traffic environments under different weather conditions, lighting conditions, vehicle speeds, and angles (such as front, side, and turning angles). Then, enhance the images using various enhancement methods and use open-source annotation tools such as LabelImg to annotate the bounding boxes of vehicle targets in the enhanced images to construct a single-class car target detection dataset with high annotation accuracy.

[0024] Data augmentation methods include brightness perturbation, blur interference, and noise perturbation.

[0025] It should be noted that this single-class vehicle target detection dataset is used for customized training of target detection models. It not only covers different climates, lighting, vehicle speeds and angles during the data collection process, but also introduces enhancement methods such as brightness perturbation, blur interference and noise perturbation in the image quality dimension to ensure the stable operation of the target detection model. This can improve the adaptability of the target detection model to real complex environments, expand the original dataset, and increase the training samples for subsequent target detection models.

[0026] Step 1.2: To improve the accuracy and real-time performance of the model in obtaining the geometric center coordinates of the car, the existing YOLOv12 is selected as the object detection model. Its input is an image, and its output is the car category probability, confidence score, and detection box. YOLOv12 is a publicly available object detection model; its network structure will not be detailed here, but it includes an A2C2f module. The structure of the A2C2f module is as follows... Figure 2 As shown, it specifically includes convolutional layers, an Ablock module, and a stitching layer. The Ablock module contains region attention A2 and convolutional layers. From Figure 2 As can be seen, the A2C2f module introduces a residual shortcut connection from input to output, which ensures the gradient stability of YOLOv12, enabling the object detection model to converge effectively even in complex scenarios. At the same time, YOLOv12 uses only one convolutional layer to compress the input channels to a uniform dimension at once, eliminating redundant splitting operations. This results in a simultaneous decrease in the number of parameters, memory usage, and floating-point operations (FLOPs), thereby effectively reducing inference latency.

[0027] Step 1.3: Train YOLOv12 using a single-class car target detection dataset. During training, minimize the total loss function as the optimization objective and adjust the YOLOv12 model parameters to obtain a well-trained YOLOv12.

[0028] Specifically, images from a single-class car target detection dataset are input into YOLOv12 to obtain car category probabilities, confidence scores, and detection boxes. Classification loss is calculated based on the car category probability and the corresponding ground truth class of the image. Boundary box regression loss is obtained by calculating the intersection-union ratio (IU / R) based on the detection boxes and the corresponding bounding box labels in the single-class car target detection dataset. The total loss function of the target detection model is obtained based on the classification loss and the bounding box regression loss. With minimizing the total loss function of the target detection model as the optimization objective, the AdamW optimizer and a cosine annealing learning rate scheduler (initial learning rate can be set to 0.01, number of iterations can be set to 500, batch size can be set to 32) are used to adjust the YOLOv12 model parameters through backpropagation until the preset number of training epochs is reached, thus obtaining a trained YOLOv12.

[0029] It should be understood that using the AdamW optimizer and cosine annealing learning rate scheduler can accelerate the convergence speed of the YOLOv12 model, and the training results are as follows: Figure 3 As shown, when the confidence level reaches 0.973, the model accuracy reaches 1.0, which meets the requirements for efficient deployment of vehicle recognition in monitoring and intelligent driving scenarios.

[0030] Step 1.4: After the output of the trained YOLOv12 detection head, add the ByTrack attention mechanism as a pre-trained object detection model for subsequent inference. The addition of the ByTrack attention mechanism enhances the consistency of vehicle identity judgment between consecutive frames, enabling long-term vehicle tracking and preventing the vehicle from moving too fast and losing the target.

[0031] Furthermore, during the inference phase, the pre-trained target detection model is applied to driving videos captured from fixed camera positions, and real speed data recorded by speed sensors is used for comparison. A ByTrack mechanism is integrated during detection to continuously lock onto targets and prevent target loss due to occlusion or passing. The pre-trained target detection model is used to analyze the captured fixed-camera videos frame by frame, identifying vehicles in each frame to extract and locate the bounding boxes of cars in each frame, and accurately recording their geometric center coordinate sequences. In this process, the geometric center coordinates of the detection box are extracted frame by frame to construct a vehicle position sequence, which provides information on the vehicle's position changes in consecutive frames. This provides stable and consistent target position information input for the subsequent speed estimation stage, achieving accurate single-target detection of the vehicle. This represents the sequence of geometric center coordinates of the detection box, and also the sequence of vehicle positions. The x and y coordinates represent the vehicle's position at time t in video frame t, where T represents the total number of video frames.

[0032] Step 2: Combining the historical location information of the vehicle obtained in Step 1, the vehicle's speed is initially estimated using the inter-frame difference method (i.e., Euclidean distance divided by the time interval). Then, the speed estimate is obtained by constraining the rate of change of speed. Finally, the constrained speed estimate is corrected by using Kalman filtering to obtain the corrected speed estimate, making the obtained speed smoother.

[0033] Furthermore, combining the vehicle's historical location information, the inter-frame difference method is used to perform a preliminary speed estimation of the vehicle, obtaining a preliminary speed estimate. Specifically, this includes: based on the vehicle position sequence obtained in step 1... The inter-frame difference method is used to calculate the pixel displacement of the vehicle between adjacent frames; the pixel displacement is then converted into a preliminary speed estimate based on the conversion factor between the pixel and the actual distance. , represented as: In the formula, This represents the initial velocity estimate at time t; The conversion factor at time t is calculated by comparing the average height H of the actual vehicle (H = 1.6m in this embodiment) with the height of the target detection box. The ratio between them is used to construct the conversion coefficient between pixels and actual distances. ; This represents the pixel displacement between adjacent frames at time t. , The x and y coordinates represent the vehicle's position at time t in video frame t; This represents the time interval corresponding to the video frame rate (in this embodiment, the frame rate is 60fps, then...). ).

[0034] Furthermore, by constraining the rate of change of velocity to obtain a constrained velocity estimate, a Kalman filter is used to perform time-series correction to obtain a corrected velocity estimate. Specifically, to prevent abnormal velocity transitions caused by factors such as occlusion and false detection, the system further constrains the rate of change of velocity. Specifically, the initial velocity estimate is automatically trimmed according to the following formula to ensure that the rate of change of velocity is less than or equal to a preset maximum acceleration threshold. : In the formula, This represents the constrained speed estimate. To represent the rate of change of velocity, it is required that... , The preset maximum acceleration threshold is used. Subsequently, to improve the stability and temporal continuity of the velocity estimation, a Kalman filter is introduced. The Kalman filter is used to perform temporal correction on the constrained velocity estimates, thus modifying the constrained velocity estimates. As a new velocity observation input for each frame ,Right now The Kalman filter is updated in the following two steps: ①Prediction phase: ; ② Correction stage: ; In the formula, Represents the state vector. and Let these represent the vehicle's velocity and acceleration at time t, respectively. Indicates the Kalman state transition moments, Represents the observation moments; The Kalman gain is expressed as a function of the system noise covariance. and observation noise covariance Calculated; This is the intermediate state vector, which has no practical meaning. The first component of the state vector after Kalman filtering. This is the corrected speed estimate. ,Right now This corrected velocity estimate serves as one of the input features for subsequent deep learning models. The first component of the final Kalman-filtered state vector... This is the corrected speed estimate. .

[0035] Step 3: Based on the vehicle position change information in consecutive frames obtained in Step 1, calculate the displacement in the horizontal and vertical directions and the motion deflection angle between the current frame and the previous frame, and integrate them with the corrected speed estimate obtained in Step 2 to form the time series input features.

[0036] Specifically, based on the vehicle's position change information in consecutive frames obtained in step 1, i.e., the vehicle position sequence The formulas for calculating the displacement increments in the horizontal and vertical directions and the sine and cosine values ​​of the motion deflection angles of adjacent frames are as follows: In the formula, This represents the displacement increment in the x-direction between adjacent frames at time t. This represents the displacement increment in the y-direction between adjacent frames at time t. and Let x and y represent the x and y coordinates of the vehicle's position at time t in video frame t, respectively. and Let represent the sine and cosine values ​​of the deflection angle at time t, respectively. To prevent the perturbation term introduced by division by zero, the displacement increments of adjacent frames in the horizontal and vertical directions obtained above are... and and the sine of the deflection angle. Sum of cosine values Combined with time and the pixel displacement obtained in step 2 and the corrected speed estimate Finally, time series input features are constructed. This provides a stable and semantically rich input representation for subsequent velocity estimation.

[0037] Step 4: Input the time series input features obtained in Step 3 into the trained time series deep learning (Fully-Connected LSTM Correction Network, FC-LSTM-CorrNet) model. By learning historical motion features, the corrected velocity estimate is dynamically corrected to obtain a more accurate and precise final velocity estimate.

[0038] In this embodiment, the FC-LSTM-CorrNet ​​model is a lightweight, high-precision temporal learning network that integrates fully connected feature maps and the sequence modeling capabilities of Long Short-Term Memory (LSTM) units. Leveraging its ability to model temporal patterns of motion trajectories, the aim is to use the trained FC-LSTM-CorrNet ​​model to refine the initial velocity estimation results (i.e., the corrected velocity estimate obtained in step 2). Dynamic correction is performed to achieve high-precision speed identification and output a more accurate final speed estimate.

[0039] Specifically, the FC-LSTM-CorrNet ​​model comprises three layers: a feature transformation layer, a temporal modeling layer, and a nonlinear correction layer. The feature transformation layer uses a fully connected network to map the time-series input features to the FC-LSTM-CorrNet ​​model to a high-dimensional representation space, as shown below: In the formula, This represents the output of the feature transformation layer. , Representing the time series input features, , This represents the weight matrix of the feature transformation layer. , This represents the bias vector of the feature transformation layer. This represents the ReLU activation function. The temporal modeling layer uses a single-layer Long Short-Term Memory (LSTM) unit to model the temporal dependencies of the feature sequence output from the feature transformation layer, capturing the temporal dependency patterns of velocity changes, as shown below: In the formula, and Let these represent the hidden state and the memory state at time t, respectively, as output by the temporal modeling layer. This represents a Long Short-Term Memory (LSTM) unit. The nonlinear correction layer is used to convert the output of the timing modeling layer into a final velocity estimate through a nonlinear mapping, enabling dynamic correction of the corrected velocity estimate. This is represented as: In the formula, This represents the final velocity estimate after dynamic correction. This represents the weight matrix of the nonlinear correction layer. , This represents the bias vector of the nonlinear correction layer.

[0040] Furthermore, the trained FC-LSTM-CorrNet ​​model is obtained through the following steps: Step 4.1: Construct a training dataset based on the collected car driving videos and the corresponding actual observed speeds.

[0041] Step 4.2: Based on the videos in the training dataset, obtain the corresponding final velocity estimates through steps 1-3 above. .

[0042] Step 4.3: Apply the SmoothL1 loss function to the final velocity estimate. Its corresponding actual observation speed Supervised training is performed, and a lightweight acceleration regularization term is introduced to obtain the total loss function of the FC-LSTM-CorrNet ​​model, which is expressed as: In the formula, This represents the total loss function of the FC-LSTM-CorrNet ​​model. Represents the SmoothL1 loss function. This represents the acceleration regularization term. This represents the error factor, used to balance the effects of velocity error and acceleration error. In this embodiment, Introducing the SmoothL1 loss function and acceleration regularization term can improve the speed smoothness, physical plausibility, and continuity of the FC-LSTM-CorrNet ​​model across consecutive frames.

[0043] Step 4.4: With minimizing the total loss function of the FC-LSTM-CorrNet ​​model as the optimization objective, the AdamW optimizer and cosine annealing learning rate scheduler are used to adjust the parameters of the FC-LSTM-CorrNet ​​model until the preset number of training epochs is reached, such as 500 training epochs, to obtain the trained FC-LSTM-CorrNet ​​model.

[0044] Furthermore, the sequence feature tensor captured in real time The input is fed into the FC-LSTM-CorrNet ​​model to obtain the final velocity estimate after dynamic correction. Dynamically corrected velocity (i.e., final velocity estimate) ) and the initial predicted speed (i.e., the corrected speed estimate) A comparison chart of actual observed speeds is shown below. Figure 4 As shown.

[0045] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target detection-based long-distance vehicle speed recognition and tracking method, characterized in that, Includes the following steps: Step 1: Real-time acquisition of vehicle driving video using a fixed camera position and angle; identification of vehicles in each frame of the driving video using a pre-trained target detection model; extraction of the center coordinates of the detection box as the geometric center of the vehicle image; obtaining information on the positional changes of the vehicle in consecutive frames. Step 2: Combining the vehicle's historical location information, the inter-frame difference method is used to perform a preliminary speed estimation of the car to obtain a preliminary speed estimate. Then, the speed change rate is constrained to obtain a constrained speed estimate. Kalman filtering is used to perform time-series correction to obtain a corrected speed estimate. Step 3: Based on the vehicle's position change information in consecutive frames, calculate the displacement in the horizontal and vertical directions and the motion deflection angle between the current frame and the previous frame, and integrate them with the corrected speed estimate to form the time series input features. Step 4: Input the time series input features into the trained time series deep learning model, and dynamically correct the corrected velocity estimate by learning historical motion features to obtain the final velocity estimate; The time series deep learning model includes a feature transformation layer, a temporal modeling layer, and a nonlinear correction layer. The feature transformation layer uses a fully connected network to map the time series input features into the time series deep learning model to a high-dimensional representation space. The temporal modeling layer uses a single-layer long short-term memory unit to perform time-dependent modeling on the feature sequence output by the feature transformation layer. The nonlinear correction layer is used to convert the output of the time-series modeling layer into the final velocity estimate through nonlinear mapping, so as to achieve dynamic correction of the corrected velocity estimate. 2.The target detection based car long distance speed recognition and tracking method according to claim 1, characterized in that, The pre-trained target detection model is obtained through the following method: First, images of cars in traffic environments under different weather conditions, lighting conditions, vehicle speeds, and angles are acquired. Then, the images are enhanced using various data augmentation methods. Finally, bounding box annotations are added to the vehicle targets in the enhanced images using open-source annotation tools to construct a single-class car target detection dataset with high annotation accuracy. Then YOLOv12 was selected as the object detection model, with the image as input and the car category probability, confidence score and detection box as output. Secondly, YOLOv12 was trained using a single-class car target detection dataset. During the training process, the model parameters of YOLOv12 were adjusted with the goal of minimizing the total loss function in order to obtain a well-trained YOLOv12. Finally, a ByteTrack attention mechanism is added after the output of the trained YOLOv12 detection head to serve as a pre-trained object detection model. 3.The target detection based car long distance speed recognition and tracking method according to claim 2, characterized in that, The data augmentation methods include brightness perturbation, blur interference, and noise perturbation.

4. The method of claim 3, wherein the target detection-based vehicle long-distance speed recognition and tracking method is characterized by, The training of YOLOv12 using a single-class car target detection dataset specifically includes: Images from a single-class car target detection dataset are input into YOLOv12 to obtain car category probabilities, confidence scores, and bounding boxes. Classification loss is calculated based on the car category probability and the corresponding ground truth class of the image. Bounding box regression loss is obtained by calculating the intersection-union ratio (IU / U) based on the bounding box labels and the corresponding bounding box labels in the single-class car target detection dataset. The total loss function of the target detection model is obtained based on the classification loss and the bounding box regression loss. With minimizing the total loss function of the target detection model as the optimization objective, the AdamW optimizer and cosine annealing learning rate scheduler are used to adjust the YOLOv12 model parameters through backpropagation until a preset number of training epochs are reached, resulting in a trained YOLOv12.

5. The method for long-distance vehicle speed recognition and tracking based on target detection according to claim 1, characterized in that, The process involves combining the vehicle's historical location information with an inter-frame difference method to perform a preliminary speed estimate, yielding a preliminary speed estimate, specifically including: Based on the vehicle's position change information in consecutive frames obtained in step 1, i.e., the vehicle position sequence The inter-frame difference method is used to calculate the pixel displacement of the vehicle between adjacent frames; the pixel displacement is then converted into a preliminary speed estimate based on the conversion factor between the pixel and the actual distance. , represented as: In the formula, This represents the initial velocity estimate at time t; This represents the conversion factor at time t, calculated by comparing the average height H of the actual vehicle with the height of the target detection box. Obtaining the ratio between them; This represents the pixel displacement between adjacent frames at time t. , The x and y coordinates represent the vehicle's position at time t in video frame t; This indicates the time interval corresponding to the video frame rate.

6. The method for long-distance vehicle speed recognition and tracking based on target detection according to claim 1, characterized in that, The process of obtaining a constrained velocity estimate by imposing constraints on the rate of change of velocity, and then applying a Kalman filter to perform time-series correction on the estimate to obtain a corrected velocity estimate, specifically includes: First, the rate of change of velocity is constrained. Specifically, the preliminary velocity estimate is automatically cropped according to the following formula to ensure that the rate of change of velocity is less than or equal to a preset maximum acceleration threshold: In the formula, This represents the constrained speed estimate. Represents the rate of change of velocity. , This indicates the preset maximum acceleration threshold; Then, Kalman filtering is used to perform time-series correction on the constrained velocity estimates, and the constrained velocity estimates are... As a new velocity observation input for each frame The Kalman filter is updated in the following two stages: Prediction phase: ; Correction phase: ; In the formula, Represents the state vector. and Let these represent the vehicle's velocity and acceleration at time t, respectively. Indicates the Kalman state transition moments, Represents the observation moments; The Kalman gain is expressed as a function of the system noise covariance. and observation noise covariance Calculated; The first component of the final Kalman filtered state vector This is the corrected speed estimate. .

7. The method for long-distance vehicle speed recognition and tracking based on target detection according to claim 1, characterized in that, In step 3, the time series input features are represented as follows: ,in This represents the time series input features at time t. and Let represent the displacement increments in the x and y directions of adjacent frames at time t, respectively. This represents the pixel displacement between adjacent frames at time t. and Let represent the sine and cosine values ​​of the deflection angle at time t, respectively. This represents the corrected speed estimate.

8. The method for long-distance vehicle speed recognition and tracking based on target detection according to claim 1, characterized in that, The trained time-series deep learning model is obtained through the following method: A training dataset is constructed based on the collected videos of vehicles driving and the corresponding actual observed speeds; Based on the videos in the training dataset, the corresponding final velocity estimates are obtained through steps 1-3 above. The SmoothL1 loss function is used to supervise the training of the final velocity estimate and its corresponding real observed velocity. At the same time, an acceleration regularization term is introduced to obtain the total loss function of the time series deep learning model. With minimizing the total loss function of the time series deep learning model as the optimization objective, the AdamW optimizer and cosine annealing learning rate scheduler are used to adjust the parameters of the time series deep learning model until the preset number of training rounds is reached, thus obtaining a well-trained time series deep learning model.

Citation Information

Patent Citations

  • Vehicle detection and tracking method for highway traffic monitoring scene

    CN116453012A

  • Multi-target tracking method and system based on unscented Kalman filtering and long and short term memory fusion

    CN117611627A