Campus scene speed measurement method and system based on binocular vision

Abstract

The invention provides a campus scene speed measurement method and system based on binocular vision, and the method comprises the following steps: S1, training a YOLOv8 model to recognize a vehicle, employing DeepSORT for tracking, and calculating the loss and performance indexes of the YOLOv8; s2, a binocular camera is used for shooting, parallax is calculated through an SGBM algorithm, and the depth is estimated through the triangulation principle; s3, preprocessing the depth map, extracting a mean value by using GMM fitting, and selecting an optimal model to determine the depth; and S4, maintaining the instance by using Kalman filtering, and managing the life cycle of the filter by combining GMM observation update.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and more particularly to a method and system for measuring speed in a campus scene based on binocular vision. Background Technology

[0002] Currently, the main methods for measuring road vehicle speed both domestically and internationally include laser speed measurement, radar speed measurement, and video speed measurement. Each method has the following shortcomings:

[0003] Radar speed measurement is the most widely used method, but it has a small measurement range, requires the installation of dedicated radar equipment, and has high manpower and material costs. The radar only outputs "speed", and the license plate depends on the checkpoint camera. The two sets of equipment have out-of-sync clocks and incompatible coordinate systems. There are safety hazards related to human radiation and interference issues with vehicle-mounted radar. The radar point cloud is sparse, and it is impossible to distinguish the correspondence between targets when multiple vehicles are parallel or non-motorized vehicles are mixed. It requires manual review of video to verify, which is inefficient and prone to missing penalties.

[0004] In laser speed measurement mode, severe weather conditions such as heavy rain, dense fog, and heavy snow can significantly attenuate the laser signal, leading to a shortened measurement distance or even failure. Road dust and direct sunlight can interfere with signal reception, causing data deviations. A 15° direct angle to the target is required; if the angle between the vehicle's direction of travel and the speedometer is too large, a "speed measurement blind zone" exists, resulting in inaccurate data. Dark or rough surfaces of the target will absorb the laser signal, affecting measurement accuracy. Only one target can be locked at a time, and the monitoring efficiency in high-traffic areas is far lower than that of radar speed measurement.

[0005] In video speed measurement, the development of the virtual coil method requires knowledge of the exact location of the coil in the video image, which limits its application scenarios. In low-light conditions at night, under oncoming high beams, or in fog and haze, the recall rate of monocular visual detection decreases. Summary of the Invention

[0006] This invention aims to at least solve the technical problems existing in the prior art, and in particular, it innovatively proposes a method and system for speed measurement in campus scenes based on binocular vision.

[0007] To achieve the above-mentioned objectives of this invention, this invention provides a speed measurement method for campus scenes based on binocular vision, comprising:

[0008] S1: Train the YOLOv8 model to recognize vehicles, track them using DeepSORT, and calculate the YOLOv8 loss and performance metrics.

[0009] S2: Use a binocular camera to take pictures, calculate the parallax using the SGBM algorithm, and estimate the depth using the triangulation principle;

[0010] S3: Preprocess the depth map, use GMM to fit and extract the mean, and select the optimal model to determine the depth;

[0011] S4: Use Kalman filtering to maintain instances, combined with GMM observation updates, to manage the filter lifecycle.

[0012] In a preferred embodiment of the above technical solution, S1 includes:

[0013] Calculation of YOLOv8 network structure feature extraction formula

[0014]

[0015] Where the initial feature F0 = X is extracted input Subsequently, PANet is used for top-down and bottom-up fusion to output the final multi-scale features P for detection.

[0016]

[0017] Where Ci represents intermediate feature maps from different stages of the backbone network; the core role of Upsample here is to transmit the semantic information of high-level features and enhance low-level features; ⊕ indicates that the channel dimensions are concatenated and then reduced by 1 × 1 convolution.

[0018] Overall loss function ;

[0019] in , , These are the weights of the bounding box regression loss, the target confidence loss, and the classification loss, respectively. box L obj L cls These are the corresponding loss terms;

[0020] Bounding box regression loss (CIoU Loss):

[0021] ,

[0022] Shape loss weighting coefficient:

[0023] ,

[0024] Shape difference measurement:

[0025]

[0026] Where IoU is the intersection-union ratio, b is the center of the predicted bounding box, and b gt ρ is the center of the true bounding box. 2 (·) represents the squared Euclidean distance, c is the length of the diagonal of the smallest rectangle enclosing the predicted bounding box and the ground truth bounding box, and w and h are the width and height of the predicted bounding box, respectively. gt h gtThe actual bounding box width and height are represented by v; v is the aspect ratio consistency loss term. It is the weighting coefficient for shape loss.

[0027] In a preferred embodiment of the above technical solution, S1 includes:

[0028] The formula for calculating the Intersection over Union (IoU) is:

[0029] Where B is the predicted bounding box and Bgt is the ground truth bounding box;

[0030] The target confidence loss is ;

[0031] Multi-class classification loss is

[0032]

[0033] Where i is the batch index, j is the category index, and y is the index. ij For category labels (1 indicates belonging to category j, 0 indicates not belonging), p ij For predicting probabilities, σ(·) is the Sigmoid function; t i It is the true confidence level, p i It is the prediction confidence level

[0034] Calculate the performance evaluation metric mAP.

[0035] Calculate AP for each class c:

[0036]

[0037] Where P cls (r) represents the interpolation precision corresponding to the recall rate r.

[0038] The average across all categories is mAP. ,in This represents the total number of categories.

[0039] In a preferred embodiment of the above technical solution, step S2 includes:

[0040] To perform disparity calculation, a binocular camera is used to capture the same scene from different perspectives. The corresponding pixels of the same spatial point are found in the left and right images, and the difference in the horizontal position of the point in the left and right images is calculated, i.e., disparity. The SGBM (Semi-Global Block Matching) algorithm is used for stereo matching.

[0041] The parallax formula is , where x left Let x be the horizontal coordinate of the point in the left image. rightLet Z be the horizontal coordinate of the point in the right image; depth estimation is performed using the principle of triangulation. Given the camera focal length f (intrinsic parameter), baseline T (distance between the optical centers of the two cameras), and parallax d, the depth Z is calculated using the following formula:

[0042] .

[0043] In a preferred embodiment of the above technical solution, step S3 includes:

[0044] A bilateral filter is used to process the Draw image, resulting in a preprocessed depth map, Dfiltered. The bilateral filter is a non-linear filter, and its output BF[D] at pixel p... p Represented as

[0045]

[0046] Where p(i,j) represents the pixel to be processed at coordinates (i,j), q represents any neighboring pixel within the filtering window centered on pixel p(i,j), used to traverse all pixels within the window for calculation; W p(i，j) As the normalization factor, It serves as a spatial domain kernel, used to suppress pixels that are spatially distant. This is a range kernel used to suppress pixels with large depth value differences. It is the original depth value at pixel p; It is the original depth value at the neighboring pixel q.

[0047] In a preferred embodiment of the above technical solution, step S3 includes:

[0048] For each target in the tracker output list, denoted as Track_ID and its ROI, perform the following operations:

[0049] S-1, performs pixel extraction based on coordinates from the preprocessed depth map D. filtered Extract the corresponding effective depth pixel set P n ;

[0050] S-2, perform GMM fitting process, assuming P n The pixels in the image are derived from a mixture of N Gaussian distributions, and a Gaussian Mixture Model (GMM) is used to analyze P. n By fitting the data, its probability density function is:

[0051]

[0052] Where π k The mixing coefficient (satisfying) ), For the k-th Gaussian component, μk For the mean, Σ k For covariance, in a one-dimensional scene, Σ k Simplified to variance Where N is the number of Gaussian components in the GMM, and its value is determined by parameter tuning based on actual scenario experiments—if P n The pixel depth distribution contains only the target and a small amount of background; typically, N can be 2 or 3. If P n The depth distribution has a lot of background interference and the target depth has multiple layers, so N takes a larger value.

[0053] S-3 describes the model selection and mean extraction process. The optimal number of components N (e.g., N ∈ {1, 2, 3}) is automatically determined using the Bayesian Information Criterion (BIC) or the Akaike Information Criterion (AIC). The optimal model, GMM, is selected by comparing the BIC scores of different N values. best Extracting GMM best All N means constitute a candidate observation means list M. i = {μ1, μ2, ..., μ N} i This list represents the N main depth distributions within the ROI, such as the depth of the target subject and the depth of the background.

[0054] In a preferred embodiment of the above technical solution, step S4 includes:

[0055] Define the state vector, KF i State vector X t This includes depth z and its rate of change v. d Two-dimensional vector:

[0056]

[0057] Where v d，t It represents the rate of change of the target depth at time t, and is used to describe the target's motion trend in the depth direction.

[0058] Establish a state transition model, assuming the target moves at a constant velocity during the time interval Δt between frames, and define the state transition matrix F:

[0059]

[0060] The state prediction equation is

[0061] , where w t This is process noise.

[0062] Perform measurement model calculations, directly observe depth, and define the measurement matrix H:

[0063] H = [1 0],

[0064] The measurement equation is Z. t = HX t + v t , where v t For measuring noise.

[0065] Establish the posterior covariance update equation.

[0066] Where I is the identity matrix, K t For Kalman gain, P t|t−1 Let be the a priori covariance.

[0067] In a preferred embodiment of the above technical solution, step S4 includes:

[0068] Combining GMM spatial robust observations with Kalman temporal predictions, the specific steps for Track_IDs that are already in the tracking state are as follows:

[0069] SA, the prediction process calls the corresponding KF i Perform the prediction step, starting from the prior state X t|t−1 Extract the predicted depth value d from predict =(X t|t−1 [0], the prediction equation is X t|t−1 = FX t−1|t−1 , where X t|t-1 X represents the prior state at time t (i.e., the initial state of frame t predicted by the motion model based on the optimal state of frame t-1), t-1|t-1 This represents the posterior state at time t-1 (i.e., the optimal estimate of the target state after updating the observations of frame t-1).

[0070] SB performs association operations, using d predict Solve the M output of GMM i = {μ1, ..., μ N The fuzziness of}, calculate d predict With M i The mean μ of each candidate k The Euclidean distance is used to select the closest mean μ. best Z, the effective observation value of this frame k,t :

[0071] ,in It is output by GMM. ={μ1, ..., μ N The mean of each Gaussian component in} is used to represent the possible depth result given by the GMM.

[0072] By leveraging temporal continuity (Kalman prediction) to intelligently select the cluster (target cluster) that best matches the target's motion trajectory from multiple depth clusters separated by the GMM, and automatically discard background cluster interference.

[0073] SC performs an update operation, using Z. k,t Correct the Kalman filter state and calculate the Kalman gain K. t

[0074]

[0075] Where R t For measuring noise covariance; H is the measurement matrix, used to map quantities in the "state vector space" to the "measurement value space"; It is the transpose of the measurement matrix H.

[0076] Calculate the optimal posterior state estimate X t|t :

[0077] ;

[0078] Thus, the posterior covariance is updated. .

[0079] The state output process extracts the updated posterior state X. t|t The depth component is used as the final smoothed depth of the target in this frame:

[0080] d smooth,i = (X t|t )[0].

[0081] The filter lifecycle management process begins by initializing the new Track_ID that appears for the first time, and then creating and initializing the KF. i The initial depth d0 is selected from the M output of the GMM. i (e.g., weight π) k Maximum or depth value μ k Minimum mean), initial velocity vd,0 = 0; termination process: Track_ID is confirmed lost by the tracker, and when multiple consecutive frames do not match, the corresponding KF is deleted. i The instance is used to release resources.

[0082] The present invention also discloses a computer system, comprising:

[0083] processor;

[0084] Memory used to store processor-executable instructions;

[0085] The processor is configured to implement the speed measurement method for campus scenes based on binocular vision as described in any one of claims 1 to 8 when executing the executable instructions.

[0086] The present invention also discloses a computer-readable storage medium, comprising:

[0087] A memory on which computer programs are stored;

[0088] A processor is configured to execute the program in the memory to implement the speed measurement method for campus scenes based on binocular vision as described in any one of claims 1 to 8.

[0089] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0090] Traditional speed measurement systems rely on the fusion of radar and cameras, resulting in high costs and complex deployments. This solution uses only binocular cameras for speed measurement, eliminating the need for radar and significantly reducing hardware costs and deployment barriers. Through a cascaded framework of "YOLOv8n + binocular stereo matching + DeepSORT multi-target tracking," it achieves simultaneous tracking of multiple vehicles and real-time speed calculation, solving the "who is speeding" problem and overcoming target loss and ID switching issues in complex scenarios such as occlusion and intersecting traffic. By introducing DeepSORT's Tentative / Confirmed trajectory state machine and combining it with a hierarchical association strategy of "Mahavioran distance + ReID appearance features + Hungarian matching," it effectively suppresses false detections and trajectory drift, improving anti-interference capabilities and speed measurement accuracy in complex campus traffic environments, thereby enhancing the robustness of the hierarchical data association and trajectory confirmation mechanism.

[0091] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0092] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0093] Figure 1 This is a schematic diagram of the overall invention;

[0094] Figure 2 This is a graph showing the experimental data of this invention;

[0095] Figure 3 This is a graph showing the experimental data of this invention. Detailed Implementation

[0096] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0097] like Figures 1 to 3 As shown, this invention discloses a method and system for speed measurement in a campus scene based on binocular vision, comprising the following steps:

[0098] Train a YOLO object detection neural network model to identify vehicle targets in images and obtain the two-dimensional coordinates and feature information of the target vehicles; use the DeepSORT multi-object tracking algorithm to track the vehicle targets; use the SGBM binocular stereo matching algorithm to obtain the depth information of the target vehicles, and then obtain the three-dimensional coordinates, and calculate the vehicle speed based on the three-dimensional coordinates.

[0099] First, the formula for extracting the structural features of the YOLOv8 network is calculated.

[0100]

[0101] Where the initial feature F0 = X is extracted input Subsequently, PANet is used for top-down and bottom-up fusion to output the final multi-scale features P for detection.

[0102]

[0103] Where Ci represents intermediate feature maps from different stages of the backbone network; the core role of Upsample here is to transmit the semantic information of high-level features and enhance low-level features; ⊕ indicates that the channel dimensions are concatenated and then reduced by 1 × 1 convolution.

[0104] Overall loss function ;

[0105] in , , These are the weights of the bounding box regression loss, the target confidence loss, and the classification loss, respectively. box L obj L cls These are the corresponding loss terms;

[0106] Bounding box regression loss (CIoU Loss):

[0107] ,

[0108] Shape loss weighting coefficient:

[0109] ,

[0110] Shape difference measurement:

[0111]

[0112] Where IoU is the intersection-union ratio, b is the center of the predicted bounding box, and b gt ρ is the center of the true bounding box. 2 (·) represents the squared Euclidean distance, c is the length of the diagonal of the smallest rectangle enclosing the predicted bounding box and the ground truth bounding box, and w and h are the width and height of the predicted bounding box, respectively. gt h gt The actual bounding box width and height are represented by v; v is the aspect ratio consistency loss term. It is the weighting coefficient for shape loss.

[0113] The formula for calculating the Intersection over Union (IoU) is:

[0114] Where B is the predicted bounding box and Bgt is the ground truth bounding box;

[0115] The target confidence loss is ;

[0116] Multi-class classification loss is

[0117]

[0118] Where i is the batch index, j is the category index, and y is the index. ij For category labels (1 indicates belonging to category j, 0 indicates not belonging), p ij For predicting probabilities, σ(·) is the Sigmoid function; t i It is the true confidence level, p i It is the prediction confidence level

[0119] Calculate the performance evaluation metric mAP.

[0120] Calculate AP for each class c:

[0121]

[0122] Where P cls (r) represents the interpolation precision corresponding to the recall rate r.

[0123] The average across all categories is mAP. ,in This represents the total number of categories.

[0124] Secondly, parallax calculation is performed by using a binocular camera to capture the same scene from different perspectives, finding the corresponding pixels of the same spatial point in the left and right images, and calculating the difference in the horizontal position of the point in the left and right images, i.e., parallax; the SGBM (Semi-Global Block Matching) algorithm is used for stereo matching.

[0125] The parallax formula is , where x left Let x be the horizontal coordinate of the point in the left image. right Let Z be the horizontal coordinate of the point in the right image; depth estimation is performed using the principle of triangulation. Given the camera focal length f (intrinsic parameter), baseline T (distance between the optical centers of the two cameras), and parallax d, the depth Z is calculated using the following formula:

[0126] ;

[0127] Then, multi-target robust depth information processing is performed. This method is used to process the original depth map Draw generated by the SGBM binocular matching algorithm, which is full of noise and artifacts, and outputs target depths with high accuracy and high temporal stability. The specific steps are as follows:

[0128] The depth map spatial preprocessing module is used to suppress high-frequency random noise in the original depth map Draw, smooth local disparity errors introduced by the matching algorithm, and at the same time protect the true depth edges between the target and the background.

[0129] In a specific embodiment, a bilateral filter is used to process the Draw image to obtain a preprocessed depth map, Dfiltered. The bilateral filter is a non-linear filter, and its output BF[D] at pixel p... p Represented as

[0130]

[0131] Where p(i,j) represents the pixel to be processed at coordinates (i,j), q represents any neighboring pixel within the filtering window centered on pixel p(i,j), used to traverse all pixels within the window for calculation; W p(i，j) As the normalization factor, It serves as a spatial domain kernel, used to suppress pixels that are spatially distant. This is a range kernel used to suppress pixels with large depth value differences. It is the original depth value at pixel p; It is the original depth value at the neighboring pixel q.

[0132] Unlike Gaussian blur, bilateral filters... This feature preserves the edge, which removes noise from the depth map while avoiding erroneous blurring of the vehicle outline and background depth values, providing a cleaner data source for subsequent ROI analysis.

[0133] A robust ROI observation module based on Gaussian mixture model is used for robust separation in ROIs contaminated by background or other objects, and extracts the true depth measurement value belonging to the target subject, replacing the traditional mean method.

[0134] For each target in the tracker output list, denoted as Track_ID and its ROI, perform the following operations:

[0135] S-1, performs pixel extraction based on coordinates from the preprocessed depth map D. filtered Extract the corresponding effective depth pixel set P n ;

[0136] S-2, perform GMM fitting process, assuming P n The pixels in the image are derived from a mixture of N Gaussian distributions, and a Gaussian Mixture Model (GMM) is used to analyze P. n By fitting the data, its probability density function is:

[0137]

[0138] Where π k The mixing coefficient (satisfying) ), For the k-th Gaussian component, μ k For the mean, Σ k For covariance, in a one-dimensional scene, Σ k Simplified to variance Where N is the number of Gaussian components in the GMM, and its value is determined by parameter tuning based on actual scenario experiments—if P n The pixel depth distribution contains only the target and a small amount of background; typically, N can be 2 or 3. If P n The depth distribution has a lot of background interference and the target depth has multiple layers, so N takes a larger value.

[0139] S-3 describes the model selection and mean extraction process. The optimal number of components N (e.g., N ∈ {1, 2, 3}) is automatically determined using the Bayesian Information Criterion (BIC) or the Akaike Information Criterion (AIC). The optimal model, GMM, is selected by comparing the BIC scores of different N values. best Extracting GMM best All N means constitute a candidate observation means list M. i = {μ1, μ2, ..., μ N} iThis list represents the N main depth distributions within the ROI, such as the depth of the target subject and the depth of the background.

[0140] A Kalman filter-based timing consistency module is used to maintain an independent Kalman filter instance KF for each tracked target Track_ID. i This enables deep temporal smoothing and utilizes historical information to assist current observations.

[0141] Define the state vector, KF i State vector X t This includes depth z and its rate of change v. d Two-dimensional vector:

[0142]

[0143] Where v d，t It represents the rate of change of the target depth at time t, and is used to describe the target's motion trend in the depth direction.

[0144] Establish a state transition model, assuming the target moves at a constant velocity during the time interval Δt between frames, and define the state transition matrix F:

[0145]

[0146] The state prediction equation is

[0147] , where w t This is process noise.

[0148] Perform measurement model calculations, directly observe depth, and define the measurement matrix H:

[0149] H = [1 0],

[0150] The measurement equation is Z. t = HX t + v t , where v t For measuring noise.

[0151] Establish the posterior covariance update equation.

[0152] Where I is the identity matrix, K t For Kalman gain, P t|t−1 Let be the a priori covariance.

[0153] The prediction-association-update closed-loop processing module combines GMM spatially robust observations with Kalman temporal predictions. The specific steps for Track_IDs already in the tracking state are as follows:

[0154] SA, the prediction process calls the corresponding KF i Perform the prediction step, starting from the prior state X t|t−1 Extract the predicted depth value d from predict =(X t|t−1 [0], the prediction equation is X t|t−1 = FX t−1|t−1 , where X t|t-1 X represents the prior state at time t (i.e., the initial state of frame t predicted by the motion model based on the optimal state of frame t-1), t-1|t-1 This represents the posterior state at time t-1 (i.e., the optimal estimate of the target state after updating the observations of frame t-1).

[0155] SB performs association operations, using d predict Solve the M output of GMM i = {μ1, ..., μ N The fuzziness of}, calculate d predict With M i The mean μ of each candidate k The Euclidean distance is used to select the closest mean μ. best Z, the effective observation value of this frame k,t :

[0156] ,in It is output by GMM. ={μ1, ..., μ N The mean of each Gaussian component in} is used to represent the possible depth result given by the GMM.

[0157] By leveraging temporal continuity (Kalman prediction) to intelligently select the cluster (target cluster) that best matches the target's motion trajectory from multiple depth clusters separated by the GMM, and automatically discard background cluster interference.

[0158] SC performs an update operation, using Z. k,t Correct the Kalman filter state and calculate the Kalman gain K. t

[0159]

[0160] Where R t For measuring noise covariance; H is the measurement matrix, used to map quantities in the "state vector space" to the "measurement value space"; It is the transpose of the measurement matrix H.

[0161] Calculate the optimal posterior state estimate X t|t :

[0162] ;

[0163] Thus, the posterior covariance is updated. .

[0164] The state output process extracts the updated posterior state X. t|t The depth component is used as the final smoothed depth of the target in this frame:

[0165] d smooth,i = (X t|t )[0].

[0166] The filter lifecycle management process begins by initializing the new Track_ID that appears for the first time, and then creating and initializing the KF. i The initial depth d0 is selected from the M output of the GMM. i (e.g., weight π) k Maximum or depth value μ k Minimum mean), initial velocity vd,0 = 0; termination process: Track_ID is confirmed lost by the tracker, and when multiple consecutive frames do not match, the corresponding KF is deleted. i The instance frees up resources and is able to effectively identify vehicle movement and license plate information.

[0167] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for speed measurement in a campus scene based on binocular vision, characterized in that, include: S1: Train the YOLOv8 model to recognize vehicles, track them using DeepSORT, and calculate the YOLOv8 loss and performance metrics; S2: Use a binocular camera to take pictures, calculate the parallax using the SGBM algorithm, and estimate the depth using the triangulation principle; S3: Preprocess the depth map, use GMM to fit and extract the mean, and select the optimal model to determine the depth; S4: Use Kalman filtering to maintain instances, combined with GMM observation updates, to manage the filter lifecycle.

2. The campus scene speed measurement method based on binocular vision according to claim 1, characterized in that, S1 includes: Calculation of YOLOv8 network structure feature extraction formula ； Where the initial feature F0 = X is extracted input Subsequently, PANet is used for top-down and bottom-up fusion to output the final multi-scale features P for detection. ； Where Ci represents intermediate feature maps from different stages of the backbone network; the core role of Upsample here is to transfer the semantic information of high-level features and enhance low-level features; ⊕ indicates that the channel dimensions are concatenated and then reduced by 1 × 1 convolution; Overall loss function ; in , , These are the weights of the bounding box regression loss, the target confidence loss, and the classification loss, respectively. box L obj L cls These are the corresponding loss terms; Bounding box regression loss (CIoU Loss): ， Shape loss weighting coefficient: ， Shape difference measurement: ； Where IoU is the intersection-union ratio, b is the center of the predicted bounding box, and b gt ρ is the center of the true bounding box. 2 (·) represents the squared Euclidean distance, c is the length of the diagonal of the smallest rectangle enclosing the predicted bounding box and the ground truth bounding box, and w and h are the width and height of the predicted bounding box, respectively. gt h gt The actual bounding box width and height; v is the aspect ratio consistency loss term. It is the weighting coefficient for shape loss.

3. The campus scene speed measurement method based on binocular vision according to claim 2, characterized in that, S1 includes: The formula for calculating the Intersection over Union (IoU) is: Where B is the predicted bounding box and Bgt is the ground truth bounding box; The target confidence loss is ; Multi-class classification loss is ； Where i is the batch index, j is the category index, and y is the index. ij For category labels (1 indicates belonging to category j, 0 indicates not belonging), p ij For predicting probabilities, σ(·) is the Sigmoid function; t i It is the true confidence level, p i It is the prediction confidence level Calculate the performance evaluation metric mAP. Calculate AP for each class c: ； Where P cls (r) represents the interpolation precision corresponding to the recall rate r; The average across all categories is mAP. ,in This represents the total number of categories.

4. The campus scene speed measurement method based on binocular vision according to claim 1, characterized in that, S2 includes: To perform disparity calculation, a binocular camera is used to capture the same scene from different perspectives. The corresponding pixels of the same spatial point are found in the left and right images, and the difference in the horizontal position of the point in the left and right images is calculated, i.e., disparity. The SGBM (Semi-Global Block Matching) algorithm is used for stereo matching. The parallax formula is , where x left Let x be the horizontal coordinate of the point in the left image. right Let Z be the horizontal coordinate of the point in the right image; depth estimation is performed using the principle of triangulation. Given the camera focal length f (intrinsic parameter), baseline T (distance between the optical centers of the two cameras), and parallax d, the depth Z is calculated using the following formula: 。 5. The campus scene speed measurement method based on binocular vision according to claim 1, characterized in that, S3 includes: A bilateral filter is used to process the Draw image, resulting in a preprocessed depth map, Dfiltered. The bilateral filter is a non-linear filter, and its output BF[D] at pixel p... p Represented as ； Where p(i,j) represents the pixel to be processed at coordinates (i,j), q represents any neighboring pixel within the filtering window centered on pixel p(i,j), used to traverse all pixels within the window for calculation; W p(i，j) As the normalization factor, It serves as a spatial domain kernel, used to suppress pixels that are spatially distant. This is a range kernel used to suppress pixels with large depth value differences. It is the original depth value at pixel p; It is the original depth value at the neighboring pixel q.

6. The campus scene speed measurement method based on binocular vision according to claim 5, characterized in that, S3 includes: For each target in the tracker output list, denoted as Track_ID and its ROI, perform the following operations: S-1, performs pixel extraction based on coordinates from the preprocessed depth map D. filtered Extract the corresponding effective depth pixel set P n ; S-2, perform GMM fitting process, assuming P n The pixels in the image are derived from a mixture of N Gaussian distributions, and a Gaussian Mixture Model (GMM) is used to analyze P. n The probability density function is obtained by fitting the data. ； Where π k The mixing coefficient (satisfying) ), For the k-th Gaussian component, μ k For the mean, Σ k For covariance, in a one-dimensional scene, Σ k Simplified to variance Where N is the number of Gaussian components in the GMM, and its value is determined by parameter tuning based on actual scenario experiments—if P n The pixel depth distribution contains only the target and a small amount of background; typically, N can be 2 or 3. If P n The background interference is high and the target depth has multiple layers, so N takes a larger value; S-3 describes the model selection and mean extraction process. The optimal number of components N (e.g., N ∈ {1, 2, 3}) is automatically determined using the Bayesian Information Criterion (BIC) or the Akaike Information Criterion (AIC). The optimal model, GMM, is selected by comparing the BIC scores of different N values. best Extracting GMM best All N means constitute a candidate observation means list M. i = {μ1, μ2, ..., μ N } i This list represents the N main depth distributions within the ROI, such as the depth of the target subject and the depth of the background.

7. The campus scene speed measurement method based on binocular vision according to claim 1, characterized in that, S4 includes: Define the state vector, KF i State vector X t This includes depth z and its rate of change v. d Two-dimensional vector: ； Where v d，t Represents the rate of change of the target depth at time t, used to describe the target's motion trend in the depth direction; Establish a state transition model, assuming the target moves at a constant velocity during the time interval Δt between frames, and define the state transition matrix F: ； The state prediction equation is , where w t This is process noise; Perform measurement model calculations, directly observe depth, and define the measurement matrix H: H =[1 0]， The measurement equation is Z. t = HX t + v t , where v t For measuring noise; Establish the posterior covariance update equation. Where I is the identity matrix, K t For Kalman gain, P t|t−1 Let be the a priori covariance.

8. The campus scene speed measurement method based on binocular vision according to claim 7, characterized in that, S4 includes: Combining GMM spatial robust observations with Kalman temporal predictions, the specific steps for Track_IDs that are already in the tracking state are as follows: SA, the prediction process calls the corresponding KF i Perform the prediction step, starting from the prior state X t|t−1 Extract the predicted depth value d predict =(X t|t−1 [0], the prediction equation is X t|t−1 = FX t−1|t−1 , where X t|t-1 X represents the prior state at time t (i.e., the initial state of frame t predicted by the motion model based on the optimal state of frame t-1), t-1|t-1 Represents the posterior state at time t-1 (i.e., the optimal estimate of the target state after updating the observations of frame t-1). SB performs association operations, using d predict Solve the M output of GMM i = {μ1, ..., μ N The fuzziness of}, calculate d predict With M i The mean μ of each candidate k The Euclidean distance is used to select the closest mean μ. best Z, the effective observation value of this frame k,t : ,in It is output by GMM. ={μ1, ..., μ N The mean of each Gaussian component in} is used to represent the possible depth result given by the GMM; By utilizing temporal continuity (Kalman prediction), the cluster that best matches the target's motion trajectory (target cluster) is intelligently selected from multiple depth clusters separated by GMM, and background cluster interference is automatically discarded; SC performs an update operation, using Z. k,t Correct the Kalman filter state and calculate the Kalman gain K. t ； Where R t For measuring noise covariance; H is the measurement matrix, used to map quantities in the "state vector space" to the "measurement value space"; It is the transpose of the measurement matrix H; Calculate the posterior optimal state estimate X t|t : ； Thus, the posterior covariance is updated. ; The state output process extracts the updated posterior state X. t|t The depth component is used as the final smoothed depth of the target in this frame: d smooth,i = (X t|t )[0]; The filter lifecycle management process begins by initializing the new Track_ID that appears for the first time, and then creating and initializing the KF. i The initial depth d0 is selected from the M output of the GMM. i (e.g., weight π) k Maximum or depth value μ k Minimum mean), initial velocity vd,0 =0; termination process: Track_ID is confirmed lost by the tracker, and when multiple consecutive frames do not match, the corresponding KF is deleted. i The instance is used to release resources.

9. A computer system, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the speed measurement method for campus scenes based on binocular vision as described in any one of claims 1 to 8 when executing the executable instructions.

10. A computer-readable storage medium, characterized in that, include: A memory on which computer programs are stored; A processor is configured to execute the program in the memory to implement the speed measurement method for campus scenes based on binocular vision as described in any one of claims 1 to 8.

Images