A vehicle speed measurement method and system based on monocular vision

Abstract

The application discloses a monocular vision-based vehicle speed measurement method and system, which comprises the following steps: acquiring a plurality of image frames in a vehicle video; detecting target vehicles in the plurality of image frames through a target detection network, tracking the detected target vehicles through a multi-target tracking network, and obtaining detection boxes corresponding to the same target vehicle in adjacent two image frames; converting pixel coordinate values corresponding to the midpoints of the lower edges of the two detection boxes into corresponding image physical coordinate values; converting the two groups of image physical coordinate values into first world coordinate values through a preset coordinate mapping relationship obtained in a monocular vision camera calibration process; and calculating the speed of the target vehicle according to the two groups of first world coordinate values and the sampling frequency of the vehicle video. The method improves the accuracy of vehicle speed measurement and is suitable for vehicle speed measurement in various vehicle scenes.

Description

Technical Field

[0001] This application relates to the fields of computer vision and intelligent transportation technology, specifically to a vehicle speed measurement method and system based on monocular vision. Background Technology

[0002] In recent years, vehicle speed measurement technologies based on monocular vision cameras have included radar speed measurement, laser speed measurement, methods based on virtual line segments or virtual regions, and three-frame difference methods. Among these, radar and laser speed measurement methods are susceptible to adverse weather conditions such as rain and fog, making accuracy difficult to guarantee, and the equipment costs are high. Methods based on virtual line segments or virtual regions pre-set two virtual line segments or fixed virtual regions on the road and measure the actual distance within these regions. However, this method cannot handle scenarios involving lane changes, turns, acceleration, and deceleration, resulting in low accuracy in speed measurement. The three-frame difference method uses three consecutive frames of images, employing two difference operations and one logical AND operation to accurately detect the target vehicle. Then, by tracking the target vehicle's centroid, its second pixel displacement in the image is calculated. Finally, combined with camera calibration parameters, the second pixel displacement is converted into real-world physical speed. This method requires the vehicle to move in a preset direction, leading to a high false detection rate for the target vehicle in occluded scenarios, resulting in inaccurate speed measurement. Summary of the Invention

[0003] In view of this, this application provides a monocular vision-based vehicle speed measurement method, the method comprising:

[0004] This process involves acquiring multiple image frames from vehicle videos captured by a monocular vision camera; detecting target vehicles within these frames using an object detection network; and tracking the detected target vehicles using a multi-object tracking network to obtain bounding boxes for the same target vehicle in adjacent image frames. Specifically, the object detection network is constructed by replacing PaFPN with AFPN in the YOLOv7 network structure and integrating the LSK-Net spatial attention mechanism into ELAN; the multi-object tracking network is built by introducing an appearance feature update strategy based on EMA into the DeepSORT network structure and employing NSA. The Kalman filter replaces the original Kalman filter; the pixel coordinates of the midpoints of the lower edges of the two detection boxes are obtained, and the two sets of pixel coordinates are converted into their respective image physical coordinates; the two sets of image physical coordinates are converted into first world coordinates in the world coordinate system through the preset coordinate mapping relationship obtained during the monocular vision camera calibration; the XOY plane of the world coordinate system is located on the ground directly below the monocular vision camera, and the Z-axis is vertically upward; the preset coordinate mapping relationship is the correlation between the image physical coordinates, world coordinates, and focal length of the ground feature points constructed during the monocular vision camera calibration process; the speed of the target vehicle is calculated based on the two sets of first world coordinates and the sampling frequency of the vehicle video.

[0005] Optionally, target vehicles are detected in multiple image frames using an object detection network, and the detected target vehicles are tracked using a multi-target tracking network to obtain detection boxes corresponding to the same target vehicle in two adjacent image frames. This includes: inputting multiple image frames into the CSPDarknet backbone network for feature extraction to obtain initial feature maps of multiple scales for each image frame; fusing the initial feature maps of adjacent scales corresponding to each image frame using the progressive fusion strategy of AFPN to obtain a preliminary fusion result, and then fusing the preliminary fusion result with initial feature maps of other scales to obtain a first feature map corresponding to each image frame; using ELAN with LSK-Net spatial attention mechanism to extract features from the first feature map in parallel using multiple convolutional kernels of different sizes, and adaptively assigning weights to different spatial locations in the first feature map to generate an enhanced second feature map; generating target detection results for the corresponding image frames based on the second feature map; the target detection results include the target vehicle's detection box data, detection confidence, and appearance features; and using an appearance update strategy based on EMA and NSA. The Kalman filter performs state estimation and tracking association on the detected target based on appearance features, detection box data, and detection confidence, and obtains the detection boxes corresponding to the same target vehicle in two adjacent image frames.

[0006] Optionally, when the monocular vision camera is a roadside camera, the speed of the target vehicle is calculated based on two sets of first world coordinate values ​​and the sampling frequency of the vehicle video, including: calculating the Euclidean distance between the two sets of first world coordinate values ​​and using the Euclidean distance as the first displacement of the target vehicle during the acquisition of two adjacent image frames by the monocular vision camera; and calculating the vehicle speed of the target vehicle based on the first displacement and the sampling frequency of the vehicle video.

[0007] Optionally, when the monocular vision camera is an in-vehicle camera, the speed of the target vehicle is determined based on two sets of first world coordinate values ​​and the sampling frequency of the vehicle video, including: acquiring the second displacement of the vehicle during the acquisition of two adjacent image frames by the in-vehicle camera; the in-vehicle vehicle is a vehicle equipped with a monocular vision camera and located behind the target vehicle; and calculating the speed of the target vehicle based on the two sets of first world coordinate values, the second displacement, and the sampling frequency.

[0008] Alternatively, the vehicle-mounted monocular vision forward perception speed measurement model can be represented as:

[0009] ;

[0010] in, Indicates the speed of the target vehicle. This indicates the second displacement of the target vehicle. , These are the x and y coordinates of the world coordinates corresponding to the previous frame in adjacent image frames, respectively. , These are the x and y coordinates of the world coordinates corresponding to the next frame in an adjacent image frame. This indicates the sampling frequency of the vehicle video.

[0011] Optionally, the vehicle speed measurement method based on a monocular vision camera further includes: acquiring data from at least one auxiliary sensor that is spatiotemporally synchronized with the monocular vision camera; the data from the auxiliary sensor includes the position coordinates of the target vehicle;

[0012] The position coordinates are converted to second-world coordinates in the world coordinate system. If the Euclidean distance between the second-world coordinates and the first-world coordinates is greater than or equal to the coordinate threshold, the first-world coordinates are corrected to obtain the third-world coordinates.

[0013] The speed of the target vehicle is calculated based on the third-world coordinates and the sampling frequency of the vehicle video.

[0014] This application provides a vehicle speed measurement system based on monocular vision, comprising:

[0015] The image frame acquisition module is used to acquire multiple image frames from vehicle videos captured by a monocular vision camera.

[0016] The target detection-tracking module is used to detect target vehicles in multiple image frames through a target detection network and track the detected target vehicles using a multi-target tracking network, obtaining the detection boxes corresponding to the same target vehicle in two adjacent image frames. The target detection network is constructed by replacing PaFPN with AFPN in the YOLOv7 network structure and integrating the LSK-Net spatial attention mechanism in ELAN. The multi-target tracking network is constructed by introducing an appearance feature update strategy based on EMA in the DeepSORT network structure and replacing the original Kalman filter with an NSA Kalman filter.

[0017] The physical coordinate acquisition module is used to acquire the pixel coordinate values ​​corresponding to the midpoints of the lower edges of the two detection boxes, and convert the two sets of pixel coordinate values ​​into their respective image physical coordinate values.

[0018] The positioning module is used to convert two sets of image physical coordinate values ​​into first world coordinate values ​​in the world coordinate system by using a preset coordinate mapping relationship obtained during the calibration of the monocular vision camera. The XOY plane of the world coordinate system is located on the ground directly below the monocular vision camera, and the Z-axis is vertically upward. The preset coordinate mapping relationship is the correlation between the image physical coordinate values, world coordinate values ​​and focal length of the ground feature points constructed during the calibration of the monocular vision camera.

[0019] The vehicle speed calculation module is used to calculate the speed of the target vehicle based on two sets of first-world coordinate values ​​and the sampling frequency of the vehicle video.

[0020] According to the scheme provided in the embodiments of this application, multiple image frames from vehicle videos captured by a monocular vision camera are acquired; target vehicles are detected in the multiple image frames using a target detection network, and the detected target vehicles are tracked using a multi-target tracking network to obtain detection boxes corresponding to the same target vehicle in two adjacent image frames; pixel coordinate values ​​corresponding to the midpoints of the lower edges of the two detection boxes are acquired, and the two sets of pixel coordinate values ​​are converted into their respective image physical coordinate values; the two sets of image physical coordinate values ​​are converted into first world coordinate values ​​in the world coordinate system using a preset coordinate mapping relationship obtained during the monocular vision camera calibration process; the XOY plane of the world coordinate system is located on the ground directly below the monocular vision camera, and the Z-axis is vertically upward; the preset coordinate mapping relationship is the correlation between the image physical coordinate values, world coordinate values, and focal length of the ground feature points constructed during the monocular vision camera calibration process; the speed of the target vehicle is calculated based on the two sets of first world coordinate values ​​and the sampling frequency of the vehicle video. In this process, an improved YOLOv7 network based on AFPN and LSK-Net attention mechanisms can reduce false detections and false negatives, and improve the detection capability of small targets in complex scenes, thus obtaining more accurate target detection results. An improved DeepSORT network based on the EMA appearance update strategy and NSA Kalman filter tracks the target detection results, reducing the impact of detection noise and ID switching, improving the continuity of trajectory tracking, and making the detection boxes of the same target vehicle in adjacent image frames more accurate. Based on this, through the single preset coordinate mapping relationship established during the monocular vision camera calibration process, the image physical coordinate values ​​of the midpoint of the lower edge of the detection box are converted into corresponding world coordinate values. Based on the world coordinate values ​​and the sampling frequency of the vehicle video, the speed of the target vehicle can be determined simply and quickly, improving the real-time performance and accuracy of vehicle speed measurement. Furthermore, the method provided in this application does not require radar, laser, or other sensors, has low cost, and is applicable to vehicle speed measurement in various vehicle scenarios. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0022] Figure 1 A flowchart illustrating a vehicle speed measurement method based on monocular vision provided in an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of a vehicle speed measurement system based on monocular vision, provided as an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0026] It should be noted that the terms "first" and "second" used in the embodiments of the present invention are only used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0027] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of this application pertain. It should also be understood that terms such as those defined in general dictionaries should be understood to have a meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0028] Figure 1 A flowchart illustrating a monocular vision-based vehicle speed measurement method provided in this application embodiment is shown below. Figure 1 As shown, the monocular vision vehicle speed measurement method provided in this application includes:

[0029] S101. Acquire multiple image frames from the vehicle video captured by the monocular vision camera.

[0030] In some embodiments, a monocular vision camera can capture real-time video of vehicles on the road. The video can include objects such as vehicles, pedestrians, green belts, roadside curbs, and road signs, and can record the movement or motion information of various vehicles in real time. The monocular vision camera can be fixedly installed on the roadside to capture images of vehicles in the surrounding environment; the monocular vision camera can also be installed on a vehicle (the vehicle itself) to capture images of other vehicles in the environment in front of the vehicle.

[0031] In some embodiments, after obtaining vehicle video captured by a monocular vision camera, the vehicle video can be segmented into frames to obtain multiple consecutive video image frames.

[0032] S102. Detect target vehicles in multiple image frames using a target detection network, track the detected target vehicles using a multi-target tracking network, and obtain the detection boxes corresponding to the same target vehicle in two adjacent image frames.

[0033] The object detection network is constructed by replacing the path aggregation feature pyramid network (PaFPN) with an asymptotic feature pyramid network (AFPN) in the YOLOv7 network structure, and integrating a large selective kernel network (LSK-Net) spatial attention mechanism in the efficient layer aggregation network (ELAN). The multi-object tracking network is constructed by introducing an appearance feature update strategy based on exponential moving average (EMA) in the DeepSORT network structure, and replacing the original Kalman filter with an NSA Kalman filter.

[0034] In some embodiments, the improved YOLOv7 network and the improved DeepSORT network can be integrated into an integrated target detection-tracking model. The target detection results output by the improved YOLOv7 network can be directly used as the input of the improved DeepSORT network to reduce information redundancy in the target detection and target tracking process and improve the real-time performance of vehicle speed measurement.

[0035] In some embodiments, the improved YOLOv7 network includes AFPN and ELAN with LSK-Net attention mechanism. AFPN can generate weights corresponding to feature maps of different scales and perform weighted fusion based on the weights and corresponding feature maps, so that the fused features have richer information, thereby alleviating the gap between high-level feature semantics and low-level feature semantics, and thus improving multi-scale object detection capability. ELAN with LSK-Net attention mechanism can perform feature filtering and feature enhancement on the feature map fused by AFPN, reducing false negatives and false positives.

[0036] In some embodiments, the EMA-based appearance update strategy allows the model to dynamically determine the update intensity of appearance features, thereby ensuring the stability and reliability of the appearance feature library and reducing ID switching. The NSA Kalman filter adaptively adjusts the noise covariance based on the detection confidence, dynamically correlates the observation noise with the detection confidence, thereby improving the robustness of motion state prediction and reducing the impact of detection noise. This ultimately leads to more accurate trajectory matching results based on the updated appearance features and motion state data, improving the accuracy of vehicle speed measurement.

[0037] In some embodiments, by improving the YOLOv7 network and the DeepSORT network, the trajectory of a target vehicle in multiple consecutive image frames can be obtained. A trajectory may include multiple consecutive detection boxes of the same target vehicle. In practice, for the speed determination of the same target vehicle, the detection boxes of the target vehicle in two adjacent image frames can be selected for calculation.

[0038] S103. Obtain the pixel coordinate values ​​corresponding to the midpoints of the lower edges of the two detection boxes, and convert the two sets of pixel coordinate values ​​into their respective image physical coordinate values.

[0039] In some embodiments, the origin of the image pixel coordinates is located at the upper left corner of the image, and the origin of the image physical coordinates is located at the center of the image. The pixel coordinate values ​​of the midpoints of the lower edges of the two detection boxes can be converted into the corresponding image physical coordinate values ​​in turn according to the conversion relationship between the image pixel coordinates and the image physical coordinates (which can be calculated by combining the number of pixels in the horizontal direction and the number of pixels in the vertical direction of the image frame).

[0040] S104. Using the preset coordinate mapping relationship obtained during the monocular vision camera calibration process, convert the two sets of image physical coordinate values ​​into the first world coordinate value in the world coordinate system.

[0041] It should be noted that the XOY plane of the world coordinate system is located on the ground directly below the monocular vision camera, and the Z-axis is vertically upward; the preset coordinate mapping relationship obtained during the monocular vision camera calibration process is the correlation between the image physical coordinate values, world coordinate values, and focal length of the ground feature points established during the monocular vision camera calibration process.

[0042] In some embodiments, the two sets of image physical coordinates can be substituted into the variable focal length model to calculate the corresponding focal length. Then, based on the calculated focal length, the corresponding image physical coordinate values, and a preset coordinate mapping relationship, the corresponding first world coordinate values ​​can be obtained. The variable focal length model is a cubic polynomial function relating the image physical coordinates of ground feature points and the focal length, constructed during the calibration of a monocular vision camera.

[0043] S105. Calculate the speed of the target vehicle based on the two sets of first-world coordinate values ​​and the sampling frequency of the vehicle video.

[0044] In some embodiments, the displacement of the target vehicle in two adjacent video image frames can be calculated based on two sets of first world coordinate values, and the speed of the target vehicle can be further calculated based on the displacement and the sampling frequency.

[0045] In this embodiment, multiple image frames from vehicle videos captured by a monocular vision camera are acquired; target vehicles are detected in the multiple image frames using a target detection network, and the detected target vehicles are tracked using a multi-target tracking network to obtain detection boxes corresponding to the same target vehicle in two adjacent image frames; pixel coordinate values ​​corresponding to the midpoints of the lower edges of the two detection boxes are acquired, and the two sets of pixel coordinate values ​​are converted into their respective image physical coordinate values; the two sets of image physical coordinate values ​​are converted into first world coordinate values ​​in the world coordinate system using a preset coordinate mapping relationship obtained during the monocular vision camera calibration process; the XOY plane of the world coordinate system is located on the ground directly below the monocular vision camera, and the Z-axis is vertically upward; the preset coordinate mapping relationship is the correlation between the image physical coordinate values, world coordinate values, and focal length of the ground feature points constructed during the monocular vision camera calibration process; the speed of the target vehicle is calculated based on the two sets of first world coordinate values ​​and the sampling frequency of the vehicle video. In this process, an improved YOLOv7 network based on AFPN and LSK-Net attention mechanisms can reduce false detections and false negatives, and improve the detection capability of small targets in complex scenes, thus obtaining more accurate target detection results. An improved DeepSORT network based on the EMA appearance update strategy and NSA Kalman filter tracks the target detection results, reducing the impact of detection noise and ID switching, improving the continuity of trajectory tracking, and making the detection boxes of the same target vehicle in adjacent image frames more accurate. Based on this, the image physical coordinates of the midpoint of the lower edge of the detection box are converted into corresponding world coordinates through the preset coordinate mapping relationship constructed during the monocular vision camera calibration process. Based on the world coordinates and the sampling frequency of the vehicle video, the speed of the target vehicle can be determined simply and quickly, improving the real-time performance and accuracy of vehicle speed measurement. Furthermore, the method provided in this application does not require radar, laser, or other sensors, has low cost, and is applicable to vehicle speed measurement in various vehicle scenarios.

[0046] In some embodiments of this application, the detection of target vehicles in multiple image frames by a target detection network in step S102, the tracking of the detected target vehicles by a multi-target tracking network, and the acquisition of the detection boxes corresponding to the same target vehicle in two adjacent image frames can be achieved by the following steps S1021 to S1024. Each step is described below.

[0047] S1021. Input multiple image frames into the CSPDarknet backbone network to extract features respectively, and obtain initial feature maps of multiple scales corresponding to each image frame.

[0048] In some embodiments, the CSPDarknet (Cross Stage Partial Network) backbone network can be the backbone network of the YOLOv7 model. Multiple image frames can be sequentially input into the CSPDarknet backbone network. The CSPDarknet backbone network can extract features of different levels of semantics for each image frame, thereby obtaining feature maps of multiple scales. The number and richness of features (such as texture features and semantic features) contained in the feature maps of each scale are different.

[0049] For example, the feature maps corresponding to each image frame at multiple scales may include high-resolution detail feature maps, mid-level semantic feature maps, deep semantic feature maps, and ultra-deep feature maps. The spatial size (scale) of each feature map may be 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the video image frame to be processed, respectively. For example, for a video image frame with a spatial size of 640x640, the spatial sizes of the corresponding multiple feature maps may be 160x160, 80x80, 40x40, and 20x20, respectively.

[0050] S1022. The progressive fusion strategy of AFPN is used to fuse the initial feature maps of adjacent scales corresponding to each image frame to obtain a preliminary fusion result. Then, the initial fusion result is fused with the initial feature maps of other scales to obtain the first feature map corresponding to each image frame.

[0051] In some embodiments, AFPN can fuse feature maps of multiple scales corresponding to each image frame using a progressive fusion strategy. By first fusing feature maps of adjacent scales and then fusing feature maps of other scales, resolution alignment can be performed during the fusion of initial feature maps of different scales, thereby obtaining the first feature map.

[0052] S1023. Using ELAN with LSK-Net spatial attention mechanism, multiple convolutional kernels of different sizes are used in parallel to extract features from the first feature map, and weights are adaptively assigned to different spatial locations in the first feature map to generate an enhanced second feature map; based on the second feature map, the target detection result corresponding to each image frame is generated.

[0053] In some embodiments, ELAN, which includes the LSK-Net spatial attention mechanism, can sequentially perform feature extraction, adaptive filtering, and feature enhancement on the first feature map corresponding to each input image frame. Adaptive filtering can be seen as filtering features that are important and significant to the target and can be used to distinguish between the target and non-target. Feature enhancement can be seen as transforming and combining these features to generate new features that are more conducive to machine model understanding and learning.

[0054] In some embodiments, the spatial attention mechanism of LSK-Net can use multiple convolutional kernels of different sizes (representing receptive fields of different sizes) in parallel to scan the first feature map. Then, through a selection mechanism, the LSK-Net network learns on its own which region of the current scene should use a large or small receptive field, thereby giving feature extraction dynamic selection capabilities, thus enabling more accurate localization and recognition of small targets.

[0055] In some embodiments, adaptive filtering and feature enhancement can be achieved by the model learning to determine the weights corresponding to each feature in the first feature map, and based on these weights, the corresponding features can be filtered or enhanced, thereby improving the quality and richness of the features in the first feature map.

[0056] In some embodiments, the target detection result includes the target vehicle's bounding box data, detection confidence, and appearance features. After processing the image frames using AFPN and ELAN containing the LSK-Net spatial attention mechanism to obtain the second feature maps corresponding to each image frame, the features in each second feature map can be analyzed to determine the target vehicle's bounding box data, detection confidence, and appearance features.

[0057] S1024. Using an EMA-based appearance update strategy and an NSA Kalman filter, state estimation and tracking association are performed on the detected target based on appearance features, detection box data, and detection confidence, to obtain the detection boxes corresponding to the same target vehicle in two adjacent image frames.

[0058] In some embodiments, the Exponential Moving Average (EMA) strategy can smooth appearance features. Instead of directly replacing old appearance features with new ones, it gradually updates the appearance features in the appearance feature library according to a smoothing coefficient. The smoothing coefficient can adjust the contribution of old and new appearance features to the current appearance feature library, and it can be adaptively adjusted so that the update intensity can be dynamically determined according to the quality of the new features or the drastic change in the appearance of the detected target.

[0059] It should be noted that the observation noise of a traditional Kalman filter is fixed, but in practice, the reliability of the detection box varies. High-confidence detection boxes are more reliable, while low-confidence detection boxes may be inaccurate or background noise. Therefore, the NSA Kalman filter can calculate the observation noise based on the detection confidence of the detection box, thus achieving a dynamic correlation between the observation noise and the detection confidence. This can improve the robustness of the motion state estimation of the detected target and increase the accuracy of the predicted trajectory state data.

[0060] In some embodiments, the NSA Kalman filter can calculate the corresponding adaptive observation noise based on the detection confidence, and combine the adaptive observation noise and the detection box data to estimate the motion state of the detected target, thereby obtaining the predicted trajectory state data of the detected target. The predicted trajectory state data may include the motion speed, position, etc. of the detected target.

[0061] In some embodiments, the trajectory matching of the detected target can be performed based on the updated appearance features and predicted trajectory state data, and a cascaded matching strategy can be adopted, such as performing IoU (Intersection over Union) matching first and then appearance matching, so as to obtain the target tracking result. The target tracking result may include the trajectory shape of the detected target, the position of the detected target, the movement speed, etc.

[0062] In some embodiments, the Intersection over Union (IoU) between each detection box and the predicted box (obtained from the predicted trajectory state data) of each video image frame can be calculated. The Hungarian algorithm (also known as the KM algorithm) is used to solve the allocation problem. The matching principle is to minimize the total IoU distance; that is, if the calculated IoU distance is greater than a certain threshold, the match fails. For unmatched trajectories and detections after IoU matching, appearance features are used for matching. Detected appearance features are extracted and cosine distances are calculated with the appearance feature library of the trajectory. The Hungarian algorithm is used for matching, while setting an appearance distance threshold. Only when the distance is less than this threshold is the match considered successful. Successfully matched detection boxes can be assigned the correct ID, and the output is the final tracking trajectory. Unmatched detection boxes can be initialized as new trajectories, and unmatched trajectories can be attempted to be retrieved in subsequent frames.

[0063] Understandably, the progressive fusion strategy of AFPN fuses the initial feature maps of adjacent scales corresponding to each image frame to obtain a preliminary fusion result. This initial fusion result is then fused with initial feature maps of other scales, mitigating the gap between high-level and low-level feature semantics and thus improving multi-scale object detection capabilities. The spatial attention mechanism of LSK-Net in ELAN uses multiple convolutional kernels of different sizes in parallel to extract features from the first feature map and adaptively assigns weights to different spatial locations within the first feature map. This allows the model to automatically learn the receptive field size of each first feature map at different spatial locations, dynamically selecting different convolutional kernels for feature extraction, reducing false positives and false negatives, and enhancing the model's adaptability and discrimination ability in complex scenes. The appearance feature update strategy based on EMA allows the model to dynamically determine the update intensity of appearance features based on the quality of new appearance features or the drastic change in the appearance of the detected target, ensuring the stability and reliability of the appearance feature library, reducing ID switching, and improving the continuity of trajectory tracking. The NSA... The Kalman filter estimates the motion state of the detected target based on the detection box data and detection confidence, dynamically correlates the observation noise with the detection confidence, thereby improving the robustness of motion state prediction, reducing the impact of detection noise, and ultimately obtaining a more accurate matching trajectory. This improves the accuracy of the detection boxes of the same detected target in adjacent image frames, thus enhancing the accuracy of vehicle speed measurement.

[0064] In some embodiments of this application, the conversion of two sets of physical coordinate values ​​of images into first world coordinate values ​​in the world coordinate system by the preset coordinate mapping relationship obtained during the monocular vision camera calibration process in step S103 can be achieved by the following steps S1031 to S1034, and each step will be described below.

[0065] S1031. Substitute the physical coordinates of the reference image into the variable focal length model for calculation to obtain the reference focal length of the monocular vision camera.

[0066] It should be noted that the physical coordinates of the reference image are any one of the two sets of physical coordinates; the variable focal length model is a cubic polynomial function between the ground feature points constructed during the calibration of the monocular vision camera and the focal length of the monocular vision camera.

[0067] In some embodiments, during the calibration of a monocular vision camera, a variable focal length model can be established based on the nonlinear mapping relationship between the image physical coordinates of ground feature points in the image frames acquired by the monocular vision camera and the equivalent focal length corresponding to the ground feature points. The equivalent focal length can be a quantity that changes with the image physical coordinates of the ground feature points, and can be used to compensate for distortion, perspective effects, etc., of the monocular vision camera. The variable focal length model can be used to handle variable focal length and distortion in the calibration of monocular vision cameras. For example, in wide-angle lenses or fisheye lenses, the focal length may change with the image position, and this change can be approximated by the variable focal length model.

[0068] In some embodiments, the variable focal length model can be represented by the following formula (1):

[0069] (1);

[0070] in, The value of the image is ( The equivalent focal length corresponding to the ground feature points. These are the fitting coefficients (which can be solved using the least squares algorithm). All are integers greater than or equal to 0 and less than 4, i.e. The values ​​of are all 0, 1, 2, 3. The right side of the equation (1) is a double cubic polynomial, which, after expansion, includes 16 fitting coefficients. This variable focal length model can fit extremely complex nonlinear surfaces.

[0071] In some embodiments, after determining the physical coordinates of the reference image (the physical coordinates of the midpoint of the lower edge of any detection box in two adjacent image frames), the physical coordinates of the reference image can be directly substituted into formula (1) to calculate the corresponding reference focal length.

[0072] S1032. Based on the reference focal length, the physical coordinates of the reference image, and the preset coordinate mapping relationship, obtain the first reference world coordinates corresponding to the reference pixel coordinates.

[0073] In some embodiments, the preset coordinate mapping relationship can be the correlation between the image physical coordinates, world coordinates, and focal length of the monocular vision camera, created during the monocular vision camera calibration stage based on the camera imaging model, pinhole imaging principle, etc. This correlation can be represented by the model shown in formula (2) below:

[0074] (2);

[0075] in, , These are the x-coordinate and y-coordinate values ​​in the first reference world coordinate system, respectively. , These represent the x-coordinate and y-coordinate values ​​in the physical coordinates of the reference image, respectively. This represents the reference focal length calculated using the variable focal length model and the physical coordinates of the reference image. This represents the angle value of the tilt angle of a monocular vision camera. This indicates the angle between a specific straight line and the Y-axis of the world coordinate system; the specific straight line is the line connecting the ground feature point and the origin O of the world coordinate system.

[0076] Here, the tilt angle of the monocular vision camera can be the predicted focal length calculated using a variable focal length model and image physical coordinates during the monocular vision camera calibration process, and the tilt angle of the monocular vision camera calculated using the image physical coordinates. A nonlinear function can be established based on the geometric relationship of camera imaging, relating the image physical coordinates, predicted focal length, tilt angle of the monocular vision camera, and a specific distance; the tilt angle can be obtained by iteratively solving the nonlinear function using Newton's method and Brent's method. This nonlinear function can be expressed by formula (3):

[0077] (3);

[0078] in, It represents a specific distance (the distance from a ground feature point to a reference point, where the reference point is the intersection of the optical center of the monocular vision camera perpendicularly downwards and the plane containing the ground; the distance from the ground feature point to the reference point can be the ground Euclidean distance). Indicates the height of the monocular vision camera above the ground. This represents the angle value of the tilt angle of a monocular vision camera. This indicates that the physical coordinates of the image are ( The predicted focal length is calculated from the ground feature points.

[0079] In the above formula (3), the left side of the equal sign is the known ground distance, and the right side is the equivalent focal length calculated based on the physical coordinates of the ground feature points in the image and the variable focal length model, as well as the height of the monocular vision camera and the angle value of the downtilt angle to be solved. The distance was calculated theoretically based on the pinhole model and coordinate system rotation.

[0080] In some embodiments, after the reference focal length corresponding to the physical coordinates of the reference image is calculated by formula (1), the angle value of the downtilt angle, the reference focal length and the physical coordinates of the reference image obtained in the monocular vision calibration stage can be substituted into formula (2) for calculation, and the first reference world coordinate value corresponding to the reference pixel coordinate value can be obtained.

[0081] S1033. Calculate the positioning error compensation value based on the physical coordinates of the reference image and the positioning error compensation model.

[0082] It should be noted that the positioning error compensation model is a fifth-order polynomial function relating the positioning error between the physical pixel coordinates of the ground feature points constructed during the calibration of the monocular vision camera and the world coordinates. The positioning error compensation model can be expressed by the following formulas (4) and (5):

[0083] (4);

[0084] (5);

[0085] in, These represent the positioning errors corresponding to the x and y coordinates, respectively. The predicted physical coordinates of ground feature points in the image. All of these represent the fitting coefficients associated with the physical coordinates of the image.

[0086] In some embodiments, the predicted physical coordinates of multiple ground feature points can be calculated based on the nonlinear function shown in formula (3) and the angle value of the downtilt angle of the calibrated monocular vision camera, and the predicted physical coordinates of the images can be converted into the corresponding predicted world coordinates. Based on the real world coordinates of each ground feature point and the corresponding predicted world coordinates, the positioning error of each ground feature point is calculated. The positioning error and the corresponding predicted physical coordinates of the images are fitted to create a fifth-order polynomial function between the positioning error and the predicted physical coordinates of the images. The fifth-order polynomial function is then solved using the least squares algorithm to obtain the fitting coefficients associated with the physical coordinates of the images. Thus, the solved positioning error compensation model can be obtained. The solved positioning error compensation model can calculate the positioning error compensation value in real time during the monocular vision camera operation stage (the actual use stage after calibration).

[0087] In some embodiments, the physical coordinates of the reference image can be directly substituted into the positioning error compensation model for calculation, thereby obtaining the corresponding positioning error compensation value.

[0088] S1034. Determine the first world coordinate value based on the positioning error compensation value and the first reference world coordinate value.

[0089] In some embodiments, the positioning error compensation value can be directly added to the corresponding first reference world coordinate value to obtain the first world coordinate value. The first reference world coordinate value includes two values: the horizontal coordinate value and the vertical coordinate value. Correspondingly, the positioning error compensation value also includes two values: the horizontal coordinate compensation value and the vertical coordinate compensation value of the world coordinate value. The horizontal coordinate value of the first reference world coordinate value and the horizontal coordinate compensation value of the world coordinate value can be added together to obtain the horizontal coordinate value of the first world coordinate value; the vertical coordinate value of the first reference world coordinate value and the vertical coordinate value of the world coordinate value can be added together to obtain the horizontal coordinate value of the first world coordinate value.

[0090] Understandably, by substituting the physical coordinates of the reference image into the variable focal length model to calculate the reference focal length of the monocular vision camera, and through the mapping relationship between the reference focal length, the physical coordinates of the reference image, and the preset coordinates, the physical coordinates of the midpoint of the lower edge of the target vehicle's detection box can be quickly converted into the corresponding world coordinates. Furthermore, by using a positioning error compensation model to compensate for the positioning error of the world coordinates, errors in camera calibration and 2D positioning caused by factors such as camera calibration errors and imaging distortion can be reduced, making the positioning of the midpoint of the lower edge of the target vehicle's detection box more accurate and improving the accuracy of vehicle speed measurement.

[0091] In some embodiments of this application, when the monocular vision camera is a roadside camera, the calculation of the target vehicle speed in step S105 based on the two sets of first world coordinate values ​​and the sampling frequency of the vehicle video can be achieved by the following steps S1051A to S1052A, and each step will be described below.

[0092] S1051A Calculate the Euclidean distance between the two sets of first world coordinate values, and use the Euclidean distance as the first displacement of the target vehicle during the acquisition of two adjacent image frames by the monocular vision camera.

[0093] In some embodiments, by calculating the Euclidean distance between two sets of first world coordinate values ​​(the world coordinate values ​​of the midpoint of the lower edge of the detection box of the target vehicle in two adjacent image frames), the displacement of the target vehicle in these two adjacent image frames acquired by the monocular vision camera can be obtained, namely the first displacement.

[0094] S1052A. Calculate the speed of the target vehicle based on the first displacement and the sampling frequency of the vehicle video.

[0095] In some embodiments, the first digit can be removed as the reciprocal of the sampling frequency of the vehicle video to obtain the speed of the target vehicle.

[0096] Understandably, when the monocular vision camera is a roadside camera, the vehicle speed can be directly obtained by calculating the Euclidean distance between the world coordinates of the midpoint of the lower edge of the detection box of the target vehicle in adjacent image frames (two sets of first world coordinates), and then using this Euclidean distance and the sampling frequency of the vehicle video. This method is computationally simple and has low complexity, which can improve the real-time performance and accuracy of vehicle speed measurement in roadside scenarios.

[0097] In some embodiments of this application, when the monocular vision camera is an in-vehicle camera, the determination of the target vehicle speed in step S105 based on two sets of first world coordinate values ​​and the sampling frequency of the vehicle video can also be achieved through the following steps S1051B to S1052B. Each step will be described separately below.

[0098] S1051B: Acquire the second displacement of the vehicle during the process of the vehicle-mounted camera acquiring two adjacent image frames.

[0099] It should be noted that the "self-vehicle" refers to a vehicle equipped with a monocular vision camera and positioned behind the target vehicle. The monocular vision camera can be mounted on the front of the self-vehicle to capture images of the target vehicle in front of it. The self-vehicle can be stationary or in motion, and the target vehicle can include one or more vehicles.

[0100] In some embodiments, the second displacement of the vehicle can be calculated by on-board equipment or devices (such as a vehicle control unit) on the vehicle. When the vehicle is stationary, the second displacement is zero. The second displacement can be understood as the distance the vehicle moves at the instant when the monocular vision camera on the vehicle captures two adjacent image frames.

[0101] S1052B: Calculate the speed of the target vehicle based on two sets of first world coordinate values, second displacement, and sampling frequency.

[0102] In some embodiments, the vehicle-mounted monocular vision forward perception speed measurement model can be a target vehicle speed calculation model established based on the relative motion trend of the vehicle and the target vehicle. The vehicle-mounted monocular vision forward perception speed measurement model can be expressed by the following formula (6):

[0103] (6);

[0104] in, Indicates the speed of the target vehicle. This indicates the second displacement of the target vehicle. , These are the x and y coordinates of the world coordinates corresponding to the previous frame in adjacent image frames, respectively. , These are the x and y coordinates of the world coordinates corresponding to the next frame in an adjacent image frame. This indicates the sampling frequency of the vehicle video.

[0105] Understandably, when the monocular vision camera is an in-vehicle camera, the vehicle speed can be quickly obtained based on the second displacement of the vehicle during the acquisition of two adjacent image frames by the in-vehicle camera, the first world coordinate value of the midpoint of the lower edge of the target vehicle detection box in the adjacent image frames, and the sampling frequency of the vehicle video. This method is simple to calculate and can improve the real-time performance and accuracy of speed measurement of forward vehicles in in-vehicle camera scenarios.

[0106] It should be noted that in calculating the target vehicle speed in both scenarios where the monocular vision camera is a roadside camera or an in-vehicle camera, a higher sampling frequency of the vehicle video results in a higher frequency of image frames input to the target detection-tracking model, leading to smoother instantaneous speed changes and a more accurate reflection of the target vehicle's actual speed. However, an excessively high sampling frequency increases computational load. Therefore, in practical scenarios, an appropriate time interval can be selected based on the sampling frequency of the vehicle video and the performance of the target detection-tracking algorithm. For example, a longer time interval can be chosen for scenarios involving acceleration, deceleration, and turning; while for scenarios involving stable, constant-speed driving, a relatively shorter time interval can be chosen if sufficient computing power is available. This improves the overall efficiency of the monocular vision-based vehicle speed measurement method provided in this application while ensuring accurate speed measurement.

[0107] In some embodiments of this application, the above-described vehicle speed measurement method based on monocular vision may further include the following steps:

[0108] S201. Acquire data from at least one auxiliary sensor that is spatiotemporally synchronized with the monocular vision camera.

[0109] The data from the auxiliary sensors includes the position coordinates of the target vehicle. These auxiliary sensors may include millimeter-wave radar, lidar, roadside magnetic induction coils, piezoelectric sensors, GNSS / RTK differential reference stations, or onboard V2X roadside units, etc.

[0110] In some embodiments, the auxiliary sensor may be located in the same vehicle driving scene as the monocular vision camera, and data of the target vehicle may be collected while the monocular vision camera is acquiring vehicle video.

[0111] S202. Convert the position coordinates to second world coordinates in the world coordinate system, and if the Euclidean distance between the second world coordinates and the first world coordinates is greater than or equal to the coordinate threshold, correct the first world coordinates to obtain the third world coordinates.

[0112] In some embodiments, the position coordinates of the target vehicle acquired by the auxiliary sensor may not be in the same coordinate system as the first world coordinates of the target vehicle determined by the vehicle video acquired by the monocular vision camera and the preset coordinate mapping relationship. Therefore, coordinate unification can be performed first to convert the position coordinates into second world coordinates in the world coordinate system. Then, the Euclidean distance between the first world coordinates and the second world coordinates can be calculated. If the Euclidean distance is greater than or equal to the coordinate threshold, it indicates that the error of the first world coordinates determined by the vehicle video acquired by the monocular vision camera and the preset coordinate mapping relationship is large. In this case, by correcting the first world coordinates so that the Euclidean distance between the corrected third world coordinates and the second world coordinates is less than the coordinate threshold, a third world coordinate value with higher positioning accuracy can be obtained.

[0113] In some embodiments, the first world coordinate value can be corrected using Kalman fusion or factor graph optimization methods. The first world coordinate value can be used as the first observation, and the second world coordinate value after coordinate transformation of the position coordinate value measured by the auxiliary sensor can be used as the second observation. An extended Kalman filter or sliding window factor graph containing a uniform motion model can be established. The corrected third world coordinate value is obtained by iteratively solving the Mahalanobis distance between the two observations. When the Mahalanobis distance in a single iteration is greater than a preset threshold, the second observation (second world coordinate value) is discarded, and only the first observation (first world coordinate value) is retained.

[0114] S203. Calculate the speed of the target vehicle based on the third-world coordinates and the sampling frequency of the vehicle video.

[0115] In some embodiments, after converting the first-world coordinates to obtain third-world coordinates, the speed of the target vehicle can be directly calculated based on the third-world coordinates and the sampling frequency of the vehicle video. Calculating the speed of the target vehicle based on the third-world coordinates and the sampling frequency of the vehicle video is similar to the method described in steps S1051A to S1052A and S1051B to S1052B, and will not be repeated here.

[0116] It is understandable that by acquiring the position coordinates of the target vehicle collected by at least one auxiliary sensor that is spatiotemporally synchronized with a monocular vision camera, converting these position coordinates into second-world coordinates in the same coordinate system as the first-world coordinates, and correcting the first-world coordinates when the Euclidean distance between the first-world and second-world coordinates is greater than or equal to a preset threshold, a third-world coordinate with higher positioning accuracy can be obtained. This allows for more accurate vehicle speed measurement results based on the third-world coordinates and the sampling frequency of the vehicle video.

[0117] In this embodiment, multiple image frames from vehicle videos captured by a monocular vision camera are acquired; target vehicles are detected in the multiple image frames using a target detection network, and the detected target vehicles are tracked using a multi-target tracking network to obtain detection boxes corresponding to the same target vehicle in two adjacent image frames; pixel coordinate values ​​corresponding to the midpoints of the lower edges of the two detection boxes are acquired, and the two sets of pixel coordinate values ​​are converted into their respective image physical coordinate values; the two sets of image physical coordinate values ​​are converted into first world coordinate values ​​in the world coordinate system using a preset coordinate mapping relationship obtained during the monocular vision camera calibration process; the XOY plane of the world coordinate system is located on the ground directly below the monocular vision camera, and the Z-axis is vertically upward; the preset coordinate mapping relationship is the correlation between the image physical coordinate values, world coordinate values, and focal length of the ground feature points constructed during the monocular vision camera calibration process; the speed of the target vehicle is calculated based on the two sets of first world coordinate values ​​and the sampling frequency of the vehicle video. In this process, an improved YOLOv7 network based on AFPN and LSK-Net attention mechanisms can reduce false detections and false negatives, and improve the detection capability of small targets in complex scenes, thus obtaining more accurate target detection results. An improved DeepSORT network based on the EMA appearance update strategy and NSA Kalman filter tracks the target detection results, reducing the impact of detection noise and ID switching, improving the continuity of trajectory tracking, and making the detection boxes of the same target vehicle in adjacent image frames more accurate. Based on this, the image physical coordinates of the midpoint of the lower edge of the detection box are converted into corresponding world coordinates through the preset coordinate mapping relationship constructed during the monocular vision camera calibration process. Based on the world coordinates and the sampling frequency of the vehicle video, the speed of the target vehicle can be determined simply and quickly, improving the real-time performance and accuracy of vehicle speed measurement. Furthermore, the method provided in this application does not require radar, laser, or other sensors, has low cost, and is applicable to vehicle speed measurement in various vehicle scenarios.

[0118] In real-world road environments, the vehicle speed measurement method based on monocular vision in this application is compared with existing speed measurement methods based on YOLOv7+DeepSORT. The comparison results include the minimum accuracy, average accuracy, multi-object tracking accuracy (MOTA), multi-object tracking localization accuracy (MOTP), identity recognition F1 score (IDF1), and average frame rate (aFPS) for each method. The comparison results are shown in Table 1 below:

[0119] Table 1 Comparison of the accuracy of speed measurement using different methods

[0120]

[0121] As shown in Table 1, the vehicle speed measurement method based on monocular vision provided in this application significantly improves the accuracy of speed measurement compared to existing vehicle speed measurement methods based on YOLOv7+DeepSORT, effectively solving the common problem in monocular vision speed measurement where changes in motion state significantly affect the accuracy. For multi-target detection and tracking, an improved YOLOv7+improved DeepSORT approach is adopted, resulting in a stronger network structure that significantly improves tracking and speed measurement accuracy while ensuring fast speed measurement. Although unavoidable issues such as changes in vehicle posture, occlusion of the target object, and inaccurate recognition can lead to abrupt changes in the preselection box size, affecting the localization and thus the speed measurement, this application uses the midpoint of the lower border of the preselection box as the localization point. Therefore, the impact on the coordinates of the midpoint of the lower edge of the predicted box is relatively small when the preselection box changes abruptly, effectively reducing the impact on speed measurement accuracy and ensuring the stability of the vehicle speed measurement method based on monocular vision proposed in this application.

[0122] This application provides a vehicle speed measurement system based on monocular vision, such as... Figure 2 As shown, the vehicle speed measurement system 300 based on monocular vision includes:

[0123] The image frame acquisition module 301 is used to acquire multiple image frames from the vehicle video captured by the monocular vision camera.

[0124] The target detection-tracking module 302 is used to detect target vehicles in multiple image frames through a target detection network and track the detected target vehicles using a multi-target tracking network to obtain the detection boxes corresponding to the same target vehicle in two adjacent image frames. The target detection network is constructed by replacing PaFPN with AFPN in the YOLOv7 network structure and integrating the LSK-Net spatial attention mechanism in ELAN. The multi-target tracking network is constructed by introducing an appearance feature update strategy based on EMA in the DeepSORT network structure and replacing the original Kalman filter with an NSA Kalman filter.

[0125] The physical coordinate acquisition module 303 is used to acquire the pixel coordinate values ​​corresponding to the midpoints of the lower edges of the two detection boxes, and convert the two sets of pixel coordinate values ​​into their respective image physical coordinate values.

[0126] The positioning module 304 is used to convert two sets of image physical coordinate values ​​into first world coordinate values ​​in the world coordinate system by using a preset coordinate mapping relationship obtained during the monocular vision camera calibration process. The XOY plane of the world coordinate system is located on the ground directly below the monocular vision camera, and the Z-axis is vertically upward. The preset coordinate mapping relationship is the correlation between the image physical coordinate values, world coordinate values ​​and focal length of the ground feature points constructed during the monocular vision camera calibration process.

[0127] The vehicle speed calculation module 305 is used to calculate the speed of the target vehicle based on two sets of first-world coordinate values ​​and the sampling frequency of the vehicle video.

[0128] The description of the above system embodiments is similar to that of the above method embodiments, and has the same beneficial effects as the method embodiments. For technical details not disclosed in the system embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0129] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0130] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0131] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0132] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A vehicle speed measurement method based on monocular vision, characterized in that, include: Acquire multiple image frames from vehicle videos captured by a monocular vision camera; The target vehicle is detected in the multiple image frames by a target detection network, and the detected target vehicle is tracked by a multi-target tracking network to obtain the detection box corresponding to the same target vehicle in two adjacent image frames. The target detection network is constructed by replacing the path aggregation feature pyramid network PaFPN with a progressive feature pyramid network AFPN in the YOLOv7 network structure, and integrating the large convolutional kernel network LSK-Net spatial attention mechanism in the efficient layer aggregation network ELAN. The multi-target tracking network is constructed by introducing an appearance feature update strategy based on exponential moving average EMA in the DeepSORT network structure, and replacing the original Kalman filter with an adaptive Kalman NSA Kalman filter. Obtain the pixel coordinates of the midpoints of the lower edges of the two detection boxes, and convert the two sets of pixel coordinates into their respective physical image coordinates. Using the preset coordinate mapping relationship obtained during the monocular vision camera calibration process, the two sets of image physical coordinate values ​​are respectively converted into the first world coordinate value in the world coordinate system; the XOY plane of the world coordinate system is located on the ground directly below the monocular vision camera, and the Z axis is vertically upward; the preset coordinate mapping relationship model is the correlation between the image physical coordinate value, world coordinate value and focal length of the ground feature point constructed during the calibration of the monocular vision camera; The speed of the target vehicle is calculated based on two sets of first-world coordinate values ​​and the sampling frequency of the vehicle video. The step of detecting target vehicles in the multiple image frames using a target detection network, tracking the detected target vehicles using a multi-target tracking network, and obtaining detection boxes corresponding to the same target vehicle in two adjacent image frames includes: The multiple image frames are input into the CSPDarknet backbone network for feature extraction, resulting in initial feature maps of multiple scales for each image frame. The progressive fusion strategy of AFPN will fuse the initial feature maps of adjacent scales corresponding to each image frame to obtain a preliminary fusion result. Then, the preliminary fusion result will be fused with the initial feature maps of other scales to obtain the first feature map corresponding to each image frame. The first feature map is extracted in parallel using multiple convolutional kernels of different sizes through ELAN, which incorporates the LSK-Net spatial attention mechanism. Weights are adaptively assigned to different spatial locations in the first feature map to generate an enhanced second feature map. The target detection result of the corresponding image frame is generated based on the second feature map. The target detection result includes the detection box data, detection confidence, and appearance features of the target vehicle. Using an EMA-based appearance update strategy and an NSA Kalman filter, state estimation and tracking association are performed on the detected target based on the appearance features, detection box data, and detection confidence, to obtain the detection boxes corresponding to the same target vehicle in two adjacent image frames.

2. The method according to claim 1, characterized in that, When the monocular vision camera is a roadside camera, calculating the speed of the target vehicle based on two sets of first world coordinate values ​​and the sampling frequency of the vehicle video includes: Calculate the Euclidean distance between the two sets of first world coordinate values, and use the Euclidean distance as the first displacement of the target vehicle during the process of the monocular vision camera acquiring the two adjacent image frames; The speed of the target vehicle is calculated based on the first displacement and the sampling frequency of the vehicle video.

3. The method according to claim 1, characterized in that, When the monocular vision camera is a vehicle-mounted camera, determining the speed of the target vehicle based on two sets of first world coordinate values ​​and the sampling frequency of the vehicle video includes: The second displacement of the vehicle is obtained during the process of the vehicle-mounted camera acquiring two adjacent image frames; the vehicle is a vehicle equipped with the monocular vision camera and located behind the target vehicle. The speed of the target vehicle is calculated based on the two sets of first world coordinate values, the second displacement, and the sampling frequency.

4. The method according to claim 3, characterized in that, The speed of the target vehicle is calculated based on the two sets of first world coordinate values, the second displacement, and the sampling frequency using the following formula: ； in, Indicates the speed of the target vehicle. This indicates the second displacement of the vehicle. , These are the x and y coordinates of the world coordinates corresponding to the previous frame in adjacent image frames, respectively. , These are the x and y coordinates of the world coordinates corresponding to the next frame in an adjacent image frame. This indicates the sampling frequency of the vehicle video.

5. The method according to claim 1, characterized in that, Also includes: Data from at least one auxiliary sensor that is spatiotemporally synchronized with the monocular vision camera is acquired; the data from the auxiliary sensor includes the position coordinates of the target vehicle. The position coordinates are converted into second world coordinates in the world coordinate system. If the Euclidean distance between the second world coordinates and the first world coordinates is greater than or equal to a coordinate threshold, the first world coordinates are corrected to obtain third world coordinates. The speed of the target vehicle is calculated based on the third-world coordinates and the sampling frequency of the vehicle video.

6. A vehicle speed measurement system based on monocular vision, characterized in that, include: The image frame acquisition module is used to acquire multiple image frames from vehicle videos captured by a monocular vision camera. The target detection-tracking module is used to detect target vehicles in the multiple image frames through a target detection network, and to track the detected target vehicles using a multi-target tracking network to obtain the detection boxes corresponding to the same target vehicle in two adjacent image frames. The target detection network is constructed by replacing PaFPN with AFPN in the YOLOv7 network structure and integrating the LSK-Net spatial attention mechanism in ELAN. The multi-target tracking network is constructed by introducing an appearance feature update strategy based on EMA in the DeepSORT network structure and replacing the original Kalman filter with an NSA Kalman filter. The physical coordinate acquisition module is used to acquire the pixel coordinate values ​​corresponding to the midpoints of the lower edges of the two detection boxes, and convert the two sets of pixel coordinate values ​​into their respective image physical coordinate values. The positioning module is used to convert two sets of image physical coordinate values ​​into first world coordinate values ​​in the world coordinate system based on a preset coordinate mapping relationship obtained during the calibration of the monocular vision camera. The XOY plane of the world coordinate system is located on the ground directly below the monocular vision camera, with the Z-axis pointing vertically upwards. The preset coordinate mapping relationship is the correlation between the image physical coordinate values, world coordinate values, and focal length of the monocular vision camera of the ground feature points constructed during the calibration of the monocular vision camera. The vehicle speed calculation module is used to calculate the speed of the target vehicle based on two sets of first world coordinate values ​​and the sampling frequency of the vehicle video. The target detection-tracking module is further configured to input the multiple image frames into the CSPDarknet backbone network for feature extraction, obtaining initial feature maps of multiple scales corresponding to each image frame; using the progressive fusion strategy of AFPN, the initial feature maps of adjacent scales corresponding to each image frame are fused to obtain a preliminary fusion result, and then the preliminary fusion result is fused with the initial feature maps of other scales to obtain a first feature map corresponding to each image frame; using ELAN, which includes the LSK-Net spatial attention mechanism, multiple convolutional kernels of different sizes are used in parallel to extract features from the first feature map, and weights are adaptively assigned to different spatial locations in the first feature map to generate an enhanced second feature map; based on the second feature map, target detection results for the corresponding image frames are generated; the target detection results include the detection box data, detection confidence, and appearance features of the target vehicle; using an appearance update strategy based on EMA and an NSA Kalman filter, state estimation and tracking association are performed on the detected target based on the appearance features, detection box data, and detection confidence to obtain the detection boxes corresponding to the same target vehicle in two adjacent image frames.

Images