A method and device for detecting reverse driving of a highway vehicle

Abstract

This application provides a method and apparatus for detecting vehicles driving in the wrong direction on highways. The method acquires monitoring images of vehicles on the highway, inputs these images into an image detection model, tracks vehicle bounding box information based on vehicle category confidence, obtains vehicle trajectory data, determines vehicle motion vectors based on the trajectory data, samples regions of interest within lanes, inputs the coordinates of the sampled feature points into a motion vector estimation model to obtain displacement vectors of the sampled feature points, clusters the displacement vectors of the sampled feature points based on their reachability distance to obtain target clusters, uses the mean of the displacement vectors in the target clusters as background displacement vectors, and determines the vehicle driving in the wrong direction detection result based on the vehicle motion vector and the background displacement vector. This application improves the accuracy of vehicle driving in the wrong direction detection result and meets the needs for automatic real-time monitoring, identification, tracking, and detection of vehicle driving in the wrong direction in highway scenarios.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method and device for detecting vehicles driving in the wrong direction on highways. Background Technology

[0002] Driving against the flow of traffic is an extremely dangerous and rule-breaking behavior in public transportation, especially on highways. Driving against the flow of traffic significantly increases the probability of traffic accidents, often resulting in serious injuries and property damage. Furthermore, this behavior disrupts normal traffic order, causing congestion, affecting traffic flow efficiency, and placing greater management pressure and rescue difficulties on highway authorities. Therefore, developing a method for detecting vehicles driving against the flow of traffic on highways, enabling timely detection and appropriate measures to ensure safe and smooth traffic flow, is of significant practical importance.

[0003] Currently, the Chinese invention patent CN117557922A, entitled "Improved YOLOv8 UAV Aerial Target Detection Method," discloses: a data preprocessing module, a network structure module, and a training and output module. The data preprocessing module enhances the UAV aerial dataset and uniformly adjusts the image resolution. The network structure module expands the receptive field of the feature extraction and detection network model by introducing multi-branch dilated partial convolution (DPC) into the backbone network of YOLOv8 (YouOnly Look Once version 8), while embedding EMA (Efficient Multi-scale Attention) mechanism to optimize feature extraction capabilities. The training and output module performs end-to-end training of the detection network model using a multi-task loss function combining classification loss, localization loss, and confidence loss, and selects the optimal model through an early stopping mechanism. Although the aforementioned invention patent achieved high mean average precision (mAP) across all classes on datasets such as VisDrone, its application to the dynamic scenario of "real-time detection of UAVs driving in reverse at high speeds" reveals the following three core shortcomings: First, the invention patent lacks analysis and utilization of temporal information, and detection based solely on a single frame image cannot distinguish whether a vehicle is driving normally or in reverse. Second, the invention patent can only identify the bounding box and category label of the target, lacking the logic for determining whether the vehicle's behavior is illegal. Third, the complex modules introduced by the invention patent (e.g., DPC) increase the computational complexity and number of parameters of the model, making it difficult for the model to meet the real-time requirement of millisecond-level response to vehicle reversing behavior in highway scenarios in terms of inference speed.

[0004] Therefore, the technical content of this invention patent needs to be improved to meet the millisecond-level detection requirements for vehicles traveling in the wrong direction on highways. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, this application provides a method and device for detecting vehicles driving in the wrong direction on highways to solve the above-mentioned technical problems.

[0006] According to one aspect of the embodiments of this application, a method for detecting vehicles driving in the wrong direction on a highway is provided. The method includes: acquiring monitoring images of vehicles on a highway; inputting the monitoring images into an image detection model to obtain vehicle bounding box information, vehicle category confidence, and lane region of interest; training the image detection model on a pre-constructed detection model based on sample monitoring images; tracking the vehicle bounding box information according to the vehicle category confidence to obtain vehicle trajectory data; determining the vehicle motion vector according to the vehicle trajectory data; sampling the lane region of interest to obtain sampled feature points; and setting the coordinates of the sampled feature points... The motion vector estimation model is used to obtain the displacement vectors of the sampled feature points. Based on the reachability distance of the sampled feature points, the displacement vectors are clustered to obtain a target cluster. The mean of the displacement vectors in the target cluster is used as the background displacement vector. The target cluster is the cluster with the most displacement vectors among all clusters. The motion vector estimation model is determined based on the historical position coordinates and historical displacement vectors of the sampled feature points. The vehicle reverse driving detection result is determined based on the difference vector between the vehicle motion vector and the background displacement vector, and the angle between the vehicle motion vector and the background displacement vector.

[0007] According to one aspect of the embodiments of this application, a highway vehicle wrong-way driving detection device is provided. The device includes: an image acquisition module for acquiring monitoring images of vehicles on a highway; an image detection module for inputting the monitoring images into an image detection model to obtain vehicle bounding box information, vehicle category confidence, and lane region of interest; the image detection model is obtained by training a pre-constructed detection model based on sample monitoring images; a vehicle motion determination module for tracking the vehicle bounding box information according to the vehicle category confidence to obtain vehicle trajectory data; and determining the vehicle motion vector according to the vehicle trajectory data; and a background displacement determination module for sampling the lane region of interest to obtain sampled data. Feature points; the coordinate values ​​of the sampled feature points are input into the motion vector estimation model to obtain the displacement vectors of the sampled feature points; based on the reachability distance of the sampled feature points, the displacement vectors of the sampled feature points are clustered to obtain target clusters; the mean of the displacement vectors in the target clusters is used as the background displacement vector; the target cluster is the cluster with the most displacement vectors among all clusters; the motion vector estimation model is determined based on the historical position coordinates and historical displacement vectors of the sampled feature points; the reverse driving detection module is used to determine the vehicle reverse driving detection result based on the difference vector between the vehicle motion vector and the background displacement vector, and the angle between the vehicle motion vector and the background displacement vector.

[0008] The beneficial effects of this application are as follows: This application acquires monitoring images of vehicles on highways, inputs these images into an image detection model to obtain vehicle bounding box information, vehicle category confidence, and lane regions of interest. Based on the vehicle category confidence, the vehicle bounding box information is tracked to obtain vehicle trajectory data. Based on the vehicle trajectory data, vehicle motion vectors are determined. The lane regions of interest are sampled to obtain sampled feature points. The coordinate values ​​of the sampled feature points are input into a motion vector estimation model to obtain the displacement vectors of the sampled feature points. Based on the reachability distance of the sampled feature points, the displacement vectors of the sampled feature points are clustered to obtain target clusters. The mean of the displacement vectors in the target clusters is used as the background displacement vector. Based on the difference vector between the vehicle motion vector and the background displacement vector, and the angle between the vehicle motion vector and the background displacement vector, the vehicle reverse driving detection result is determined. In the process of determining the vehicle reverse driving detection result, the application fully utilizes the time-series information... The system obtains vehicle category confidence scores from different monitoring images, tracks vehicle bounding box information obtained from time-series information, and obtains vehicle trajectory data and vehicle movement vectors, thereby improving the accuracy of vehicle motion vectors. Furthermore, it determines a motion vector estimation model based on the historical position coordinates and historical displacement vectors of sampled feature points, and estimates the displacement vectors of sampled feature points using this model. This improves the accuracy of displacement vector estimation for sampled feature points, reducing estimation bias caused by relying solely on data from a single frame of monitoring image. Moreover, based on the improved accuracy of sampled feature point displacement vectors, the system also improves the accuracy of background displacement vectors, thus enhancing the accuracy of vehicle reverse-driving detection results. This meets the requirements for automatic real-time monitoring, identification, tracking, and detection of vehicle reverse-driving behavior in highway scenarios, significantly improving the real-time performance and effectiveness of highway safety management.

[0009] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application; Figure 2 This is a flowchart illustrating a method for detecting vehicles driving in the wrong direction on a highway, as shown in an exemplary embodiment of this application; Figure 3 This is a flowchart illustrating a method for detecting vehicles driving in the wrong direction on a highway, as shown in another exemplary embodiment of this application; Figure 4 This is an exemplary embodiment of the present application illustrating the architecture of a pre-built detection model; Figure 5 This is a flowchart illustrating a method for detecting vehicles traveling in the wrong direction on a highway, as shown in yet another exemplary embodiment of this application. Figure 6 This is a block diagram illustrating a highway vehicle wrong-way detection device, as shown in an exemplary embodiment of this application. Detailed Implementation

[0011] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0012] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0013] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0014] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application.

[0015] Reference Figure 1As shown, the system architecture may include a data acquisition device 101 and an electronic device 102. The electronic device 102 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, or a neural network computer. Technical personnel can use this electronic device 102 to acquire monitoring images of vehicles on a highway, input these images into an image detection model to obtain vehicle bounding box information, vehicle category confidence, and lane regions of interest. Based on the vehicle category confidence, the vehicle bounding box information is tracked to obtain vehicle trajectory data. Based on the vehicle trajectory data, vehicle motion vectors are determined. The lane regions of interest are sampled to obtain sampled feature points. The coordinates of these sampled feature points are input into a motion vector estimation model to obtain displacement vectors. Based on the reachability distance of the sampled feature points, the displacement vectors are clustered to obtain target clusters. The mean of the displacement vectors in the target clusters is used as the background displacement vector. Based on the difference vector between the vehicle motion vector and the background displacement vector, and the angle between the vehicle motion vector and the background displacement vector, the vehicle reverse driving detection result is determined. The acquisition device 101 is used to acquire monitoring images of vehicles on the highway and provide them to the electronic device 102 for processing.

[0016] Schematic illustration: After acquiring monitoring images of vehicles on the highway from the acquisition device 101, the electronic device 102 inputs the monitoring images into an image detection model to obtain vehicle bounding box information, vehicle category confidence, and lane region of interest. Based on the vehicle category confidence, it tracks the vehicle bounding box information to obtain vehicle trajectory data. Based on the vehicle trajectory data, it determines the vehicle motion vector, samples the lane region of interest to obtain sampled feature points, inputs the coordinates of the sampled feature points into a motion vector estimation model to obtain the displacement vectors of the sampled feature points, clusters the displacement vectors of the sampled feature points based on their reachability distance to obtain target clusters, uses the mean of the displacement vectors in the target clusters as the background displacement vector, and determines the vehicle reverse driving detection result based on the difference vector between the vehicle motion vector and the background displacement vector, as well as the angle between the vehicle motion vector and the background displacement vector. In the process of determining the vehicle reverse driving detection result, the electronic device 102 fully utilizes... By using vehicle category confidence scores obtained from time-series information (i.e., different monitoring images), vehicle bounding box information obtained from time-series information is tracked to obtain vehicle trajectory data and vehicle running vectors, thereby improving the accuracy of vehicle motion vectors. Furthermore, a motion vector estimation model is determined based on the historical position coordinates and historical displacement vectors of sampled feature points, and the displacement vectors of sampled feature points are estimated using the motion vector estimation model, which improves the accuracy of displacement vector estimation of sampled feature points and reduces the estimation bias caused by determining the displacement vectors of sampled feature points based solely on data obtained from a single frame of monitoring image. In addition, based on the improved accuracy of the displacement vectors of sampled feature points, the accuracy of background displacement vectors is also improved, thereby improving the accuracy of vehicle reverse driving detection results. This meets the needs of automatic real-time monitoring, identification, tracking, and detection of vehicle reverse driving behavior in highway scenarios, greatly improving the real-time performance and effectiveness of highway safety management.

[0017] The implementation details of the technical solutions in the embodiments of this application are described in detail below: Figure 2 This is a flowchart illustrating a method for detecting vehicles traveling in the wrong direction on a highway, as shown in an exemplary embodiment of this application. (Refer to...) Figure 2 As shown, the method for detecting vehicles driving in the wrong direction on highways includes at least steps S210 to S250, which are described in detail below: In step S210, monitoring images of vehicles on the highway are acquired. In one embodiment of this application, the monitoring images of vehicles on the highway are obtained by a drone equipped with a camera to capture monitoring video of vehicles on the highway, and then the monitoring video is sampled frame by frame. The frequency of frame sampling of the monitoring video is set according to the actual situation, for example, 25 FPS.

[0018] In step S220, the monitoring image is input into the image detection model to obtain vehicle bounding box information, vehicle category confidence, and lane region of interest. In one embodiment of this application, the image detection model is trained on a pre-built detection model based on sample monitoring images. The process of training the pre-built detection model based on sample monitoring images includes: labeling vehicle and lane information in the sample monitoring images to obtain sample monitoring images with labeled information; preprocessing the sample monitoring images with labeled information to obtain preprocessed sample monitoring images; the preprocessing operations include: image normalization, image size transformation, and image enhancement; and training the pre-built detection model using the preprocessed sample monitoring images to obtain the image detection model. In the above process, by labeling and preprocessing the sample monitoring images, the preprocessed sample image data becomes more standardized and effective, enabling the pre-built detection model to better learn the feature information of vehicles and lanes. During the training process, by continuously adjusting the parameters of the pre-built detection model, it can accurately identify vehicle bounding box information, vehicle category confidence, and lane region of interest from the sample monitoring images. As training progresses, the pre-built detection model's ability to recognize sample monitoring images under different scenarios and lighting conditions will gradually improve, thereby enabling the image detection model to have high accuracy and stability, and providing a reliable data foundation for subsequent vehicle reverse driving detection.

[0019] In step S230, the vehicle bounding box information is tracked according to the vehicle category confidence score to obtain vehicle trajectory data; the vehicle motion vector is determined based on the vehicle trajectory data. In one embodiment of this application, if the vehicle category confidence score includes: the vehicle category confidence score detected from the monitoring image at the current moment, denoted as the current vehicle category confidence score; if the vehicle bounding box information includes: the bounding box information detected from the monitoring image at the current moment, denoted as the current bounding box information; and the bounding box information detected from the monitoring image at the previous moment, denoted as the previous bounding box information; the current vehicle category confidence score corresponds to the current bounding box information; and the bounding box information includes: the bounding box center coordinates, the bounding box scale, the motion speed of the bounding box center coordinates, and the bounding box scale change rate, then the process of tracking the vehicle bounding box information according to the vehicle category confidence score to obtain vehicle trajectory data includes: inputting the previous bounding box information into the state transition equation to obtain the current bounding box prediction information; and inputting the current bounding box information into the state transition equation to obtain the current bounding box prediction information; and inputting the previous ... The predicted information is correlated with the confidence level of the current vehicle category to obtain the confidence level of the current bounding box prediction information. Based on a first preset confidence threshold, the bounding boxes corresponding to the current bounding box prediction information are divided into high-quality bounding boxes and low-quality bounding boxes. High-quality bounding boxes are those corresponding to bounding box prediction information with a confidence level greater than or equal to the first preset confidence threshold, while low-quality bounding boxes are those corresponding to bounding box prediction information with a confidence level less than the first preset confidence threshold. High-quality bounding boxes are matched with existing trajectories to obtain first matching data. Low-quality bounding boxes are matched with existing trajectories to obtain second matching data. The first and second matching data are combined to obtain combined trajectory data. A sliding window is used to perform weighted smoothing on the combined trajectory data to obtain vehicle trajectory data.

[0020] In one embodiment of this application, the process of matching high-quality bounding boxes with existing trajectories includes: if the distance between high-quality bounding boxes at different times is greater than a preset distance threshold, calculating the decision function value of the high-quality bounding boxes based on the feature similarity of the motion direction change angle, velocity change rate, and bounding box ID (Identification) of the high-quality bounding boxes at different times; if the decision function value of the high-quality bounding boxes is greater than the preset decision threshold, then it is determined that the high-quality bounding boxes at different times belong to the same bounding box; if the decision function value of the high-quality bounding boxes is less than or equal to the preset decision threshold, then it is determined that the high-quality bounding boxes at different times do not belong to the same bounding box; if the distance between high-quality bounding boxes at different times is less than or equal to the preset distance threshold, then calculating the first association cost value between the high-quality bounding boxes at different times through the first association cost function, determining whether the high-quality bounding boxes at different times are the same bounding box, and performing trajectory matching (i.e., if the first association cost value between high-quality bounding boxes at different times is less than or equal to the first preset association cost value, then it is determined that the high-quality bounding boxes at different times belong to the same bounding box; if the first association cost value between high-quality bounding boxes at different times is greater than the first preset association cost value, then it is determined that the high-quality bounding boxes at different times do not belong to the same bounding box). The distance between high-quality bounding boxes at different times is the Euclidean distance between the center points of the high-quality bounding boxes at different times.

[0021] In one embodiment of this application, the process of matching low-quality bounding boxes with existing trajectories includes: if the distance between low-quality bounding boxes at different times is greater than a preset distance threshold, calculating the decision function value of the low-quality bounding boxes based on the feature similarity of the motion direction change angle, velocity change rate, and bounding box ID of the low-quality bounding boxes at different times; if the decision function value of the low-quality bounding boxes is greater than the preset decision threshold, then it is determined that the low-quality bounding boxes at different times belong to the same bounding box; if the decision function value of the low-quality bounding boxes is less than or equal to the preset decision threshold, then it is determined that the low-quality bounding boxes at different times do not belong to the same bounding box; if the distance between low-quality bounding boxes at different times is less than or equal to the preset distance threshold, then calculating the second association cost value between low-quality bounding boxes at different times through the second association cost function, determining whether the low-quality bounding boxes at different times are the same bounding box, and performing trajectory matching (i.e., if the second association cost value between low-quality bounding boxes at different times is greater than the second preset association cost value, then it is determined that the low-quality bounding boxes at different times belong to the same bounding box; if the second association cost value between low-quality bounding boxes at different times is less than or equal to the second preset association cost value, then it is determined that the low-quality bounding boxes at different times do not belong to the same bounding box). The distance between low-quality bounding boxes at different times is the Euclidean distance between the center points of the low-quality bounding boxes at different times.

[0022] In one embodiment of this application, the formula for calculating the decision function value is as follows: Equation (1) in, Represents the value of the decision function. This indicates the angle of change in the direction of movement of the bounding box at different times. This represents the rate of change of the bounding box velocity at different times. This represents the similarity of appearance features between bounding boxes at different times. The weight representing the angle of change in the direction of motion. The weights representing the rate of change of velocity The weights represent the feature similarity. The angle of change in motion direction is the angle between the direction vectors of the bounding box at the previous time step and the current time step. The rate of change of velocity of the bounding boxes at different time steps is the difference between the motion rate of the current bounding box and the motion rate of the bounding box at the previous time step. The similarity of appearance features between bounding boxes at different time steps is the cosine similarity between the appearance feature vectors of the current bounding box and the appearance feature vectors of the bounding box at the previous time step. The weights represent the angles of change in motion direction. Weight of rate of change of velocity Weights of feature similarity The sum equals 1. The angle of change in the direction of movement of the bounding box at different times. Rate of change of velocity of the bounding box at different times All data have been normalized.

[0023] In one embodiment of this application, the formula for calculating the appearance feature similarity between bounding boxes at different times is as follows: Equation (2) in, This represents the similarity of appearance features between bounding boxes at different times. This represents the appearance feature vector of the bounding box at the current time. This represents the appearance feature vector of the bounding box at the previous time step. The appearance feature vectors of different bounding boxes are obtained by feature extraction of the monitoring images at different time steps by the backbone network module in the image detection model.

[0024] In one embodiment of this application, the expression for the bounding box information is as follows: Equation (3) in, Represents bounding box information, ( ) represents the center coordinates of the bounding box, ( ) represents the bounding box scale, ( () represents the movement speed of the bounding box center coordinates, ) represents the rate of change of the bounding box scale.

[0025] The state transition equation is based on the Extended Kalman Filter (EKF), and the calculation formula for the state transition equation is shown below: Equation (4) in, This represents the current bounding box prediction information, obtained through the bounding box information at time k-1. The prediction was obtained. Here is the state transition matrix. This is process noise.

[0026] In one embodiment of this application, the expression for the first association cost function is as follows: (5) in, Denotes the first association cost function. This represents the intersection-union ratio (IoU) between the currently predicted bounding box and the currently detected bounding box. This represents the normalized Euclidean distance between the currently predicted bounding box and the currently detected bounding box. This represents the cosine distance of appearance features between high-quality bounding boxes at different times. This represents the adjustable weighting coefficient of the intersection-union ratio term. This represents the adjustable weighting coefficient of the Euclidean distance term. The adjustable weight coefficient of the cosine distance term is represented. The current predicted bounding box is predicted by the bounding box information at time k-1 and formula (3). The current detected bounding box is detected by the image detection model on the current frame monitoring image.

[0027] In one embodiment of this application, the formula for calculating the cosine distance of appearance features between high-quality bounding boxes at different times is as follows: Equation (6) in, This represents the cosine distance of appearance features between high-quality bounding boxes at different times, that is, the cosine distance of appearance features between the high-quality bounding box at the current time and the high-quality bounding box at the previous time. The similarity in appearance between high-quality bounding boxes at different times is indicated by formula (2).

[0028] In one embodiment of this application, the expression for the second association cost function is as follows: Equation (7) in, This represents the second association cost function. This represents the intersection-union ratio (IoU) between the currently predicted bounding box and the currently detected bounding box. The similarity of appearance features between low-quality bounding boxes at different times is represented by formula (2). This represents a balance coefficient achieved between spatial overlap and similarity in appearance features.

[0029] In one embodiment of this application, the setting of formulas (5)-(7) significantly enhances the tracking continuity of the highway vehicle reverse driving detection system in complex scenarios such as image occlusion and image blurring, and improves the tracking accuracy of the highway vehicle reverse driving detection system in complex scenarios such as image occlusion and image blurring.

[0030] In one embodiment of this application, the formula for calculating vehicle trajectory data is as follows: Equation (8) in, Represents vehicle trajectory data. Indicates the combination trajectory data with the first Trajectory data with time interval i. Indicates the combination trajectory data with the first The weight of the trajectory data at time interval i is related to the weight of the combined trajectory data at time i. The confidence levels of trajectory data at time interval i are positively correlated. This represents the radius of the sliding window. Both the combined trajectory data and the vehicle trajectory data are represented by the center coordinates of the bounding box.

[0031] In one embodiment of this application, weighted smoothing of the combined trajectory data effectively suppresses bounding box jitter, providing continuous, stable, and spatiotemporally consistent vehicle trajectory data for subsequent determination of reverse driving behavior.

[0032] In step S240, the region of interest of the lane is sampled to obtain sampled feature points; the coordinate values ​​of the sampled feature points are input into the motion vector estimation model to obtain the displacement vectors of the sampled feature points; based on the reachability distance of the sampled feature points, the displacement vectors of the sampled feature points are clustered to obtain target clusters; the mean of the displacement vectors in the target clusters is used as the background displacement vector. In one embodiment of this application, the target cluster is the cluster with the most displacement vectors among all clusters; the motion vector estimation model is determined based on the historical position coordinates and historical displacement vectors of the sampled feature points, and the motion vector estimation model is determined based on the historical position coordinates and historical displacement vectors of the sampled feature points, and the displacement vectors of the sampled feature points are estimated through the motion vector estimation model, which improves the accuracy of the displacement vector estimation of the sampled feature points and reduces the estimation bias caused by determining the displacement vectors of the sampled feature points based only on data obtained from a single frame of monitoring image.

[0033] In step S250, the vehicle reverse driving detection result is determined based on the difference vector between the vehicle motion vector and the background displacement vector, and the angle between the vehicle motion vector and the background displacement vector. In one embodiment of this application, in the process of determining the vehicle reverse driving detection result, the vehicle category confidence obtained from time-series information (i.e., different monitoring images) is fully utilized to track the vehicle bounding box information obtained from the time-series information, thereby obtaining vehicle trajectory data and vehicle running vector, thus improving the accuracy of vehicle motion vector. Furthermore, a motion vector estimation model is determined based on the historical position coordinates and historical displacement vectors of the sampled feature points, and the displacement vector of the sampled feature points is estimated using the motion vector estimation model, thereby improving the accuracy of estimating the displacement vector of the sampled feature points and reducing the estimation bias caused by determining the displacement vector of the sampled feature points based solely on data obtained from a single frame of monitoring image. In addition, based on the improved accuracy of the displacement vector of the sampled feature points, the accuracy of the background displacement vector is also improved, thereby improving the accuracy of the vehicle reverse driving detection result. This meets the requirements for automatic real-time monitoring, identification, tracking, and detection of vehicle reverse driving behavior in highway scenarios, greatly improving the real-time performance and effectiveness of highway safety management.

[0034] In one embodiment of this application, the process of training a pre-built detection model based on sample monitoring images to obtain an image detection model includes: Vehicle and lane information in sample monitoring images are labeled to obtain labeled sample monitoring images. In one embodiment of this application, vehicle information includes: bounding box information for each vehicle instance, type of each vehicle instance, and road signage, etc. The bounding box information for each vehicle instance includes the coordinates of the top left corner of the detection box, width, height, etc. The type of each vehicle instance includes three vehicle categories: sedan, truck, and non-motorized vehicle. When the sample monitoring image is slightly occluded (e.g., the area of ​​the vehicle occluded by surrounding objects is less than 30% of the vehicle area), its complete outline is required to be inferred and marked based on the vehicle structure and context information. When the sample monitoring image is heavily occluded (e.g., the area of ​​the vehicle occluded by surrounding objects is greater than or equal to 30% of the vehicle area), the vehicle information is ignored and treated as part of the background content, which can effectively prevent data pollution. Lane information includes: normal driving roads (e.g., passable areas, including overtaking lanes, driving lanes, emergency lanes, and ramps), abnormal roads (e.g., irrelevant areas, including cross bridges, urban roads, and non-highway sections), and service area roads (e.g., toll stations, service area entrances and exits, and other transitional areas). This multi-category labeling provides rich contextual information for subsequent analysis of vehicle behavior characteristics under different road types.

[0035] Preprocessing is performed on sample monitoring images with labeled information to obtain preprocessed sample monitoring images. In one embodiment of this application, the preprocessing operations include: image normalization, image resizing, and image enhancement. Image resizing includes: image scaling, image cropping, image resampling, image interpolation, geometric transformation, etc. Image enhancement operations include: spatial domain filtering, frequency domain filtering, nonlocal mean denoising, mosaic processing, color perturbation, etc. Preprocessing sample monitoring images with labeled information helps improve the quality of the sample monitoring images and effectively enhances the generalization ability of the pre-built detection model.

[0036] The pre-built detection model is trained using pre-processed sample monitoring images to obtain an image detection model. In one embodiment of this application, the pre-built detection model includes: a backbone network module, a neck network module, a detection head, and a segmentation head; the backbone network module includes: a first convolutional layer, a space-to-depth convolution module (SPD-Conv), a cross-stage partial convolution with 2 convolutions and feature fusion (CSP Bottleneck with 2 Convolutions and FeatureFusion, C2f) module, and a global attention mechanism (GAM) module; the neck network module includes: layers P1, P2, P3, P4, and P5. Layers P1, P2, P3, P4, and P5 all perform convolution operations, upsampling operations, and concatenation operations. Layer P5 uses a convolution kernel of a preset size, which can be 10×10 or other sizes. The spatial-to-depth convolution module replaces the downsampling convolution in the backbone network module of the YOLOv8 architecture. By converting spatial dimension information to channel dimension, it achieves feature dimensionality reduction without information loss, effectively solving the problem of feature dilution for small targets (targets smaller than the preset scale) during convolution. A global attention mechanism module is embedded in the backbone network module. This module establishes long-range dependencies in both the channel and spatial dimensions, enhancing the pre-built detection model's attention to vehicle targets and effectively suppressing interference from complex backgrounds such as road reflections and guardrail shadows. A P2 layer (160×160 pixels) is introduced as a new detection scale in the neck network module, significantly enhancing the feature extraction capability for small vehicles at a distance. Simultaneously, considering the scarcity of extra-large targets from the drone's perspective, the detection scale of the P5 layer is reduced, with the P5 layer's convolution kernel set to 10×10 or other sizes. This achieves approximately 30% computational compression while maintaining the detection performance of the pre-built detection model.

[0037] In one embodiment of this application, if the lane region of interest includes a first preset region of interest and a second preset region of interest, then the process of sampling the lane region of interest to obtain sampled feature points includes: According to a first spatial sampling density, a first preset region of interest is sampled to obtain first sampled feature points. In one embodiment of this application, the average gradient magnitude of the first preset region of interest is less than the average gradient magnitude of a second preset region of interest. The first preset region of interest includes flat road surfaces, etc., and the second preset region of interest includes lane lines, road shoulders, etc. The first spatial sampling density is set according to the actual situation.

[0038] The second preset region of interest is sampled according to the second spatial sampling density to obtain the second sampled feature points. In one embodiment of this application, the first spatial sampling density is less than the second spatial sampling density, and the second spatial sampling density is set according to the actual situation. By adopting a non-uniform sampling method based on texture complexity (characterized by gradient magnitude) (sampling the first preset region of interest according to the first spatial sampling density and sampling the second preset region of interest according to the second spatial sampling density), the confidence and efficiency of background motion estimation are improved.

[0039] The first and second sampling feature points are combined to form the sampling feature points. In one embodiment of this application, before sampling the region of interest of the lane, since vehicle movement in the vehicle detection area can introduce abnormal optical flow interference, the optical flow feature points corresponding to the vehicle are set to 0, so that the optical flow feature points corresponding to the vehicle do not participate in the calculation of background optical flow, thus eliminating the influence of the moving subject on the background optical flow.

[0040] In one embodiment of this application, the process of inputting the coordinate values ​​of the sampled feature points into the motion vector estimation model to obtain the displacement vector of the sampled feature points includes: Gradient filtering is performed on the sampled feature points to obtain target feature points. In one embodiment of this application, the gradient of the target feature point is greater than a preset gradient threshold; the preset gradient threshold is set according to the actual situation, and the gradient of the sampled feature point is determined based on the eigenvalues ​​of the autocorrelation matrix of the region where the sampled feature point is located. The formula for calculating the gradient of the sampled feature point is as follows: Equation (9) in, Represents the gradient of the sampled feature points. This represents the first eigenvalue of the autocorrelation matrix of the region where the sampled feature points are located. This represents the second eigenvalue of the autocorrelation matrix of the region where the sampled feature points are located. This represents the empirical coefficient.

[0041] In one embodiment of this application, the Shi-Tomasi corner detection algorithm (i.e., the gradient screening of sampled feature points is performed based on the comparison between the calculation result of formula (7) and the preset gradient threshold) selects feature points with high gradient significance from the sampled feature points as target feature points.

[0042] The corner response values ​​of the target feature points are filtered to obtain filtered feature points. In one embodiment of this application, the corner response value of the filtered feature points is the maximum value within a preset image area. The process of filtering the corner response values ​​of the target feature points to obtain filtered feature points is completed by a FAST feature detector. The FAST feature detector is beneficial to improving the fast filtering speed and reducing the computational complexity of filtering target feature points to the order of O(n).

[0043] The coordinates of the selected feature points are input into the motion vector estimation model to obtain the displacement vectors of the selected feature points. In one embodiment of this application, the motion vector estimation model is determined based on the historical position coordinates and historical displacement vectors of the selected feature points. By determining the motion vector estimation model using the historical position coordinates and historical displacement vectors of the selected feature points, the number of feature points involved in the calculation is reduced. On the one hand, this helps to improve the calculation speed in the process of determining the motion vector estimation model. On the other hand, the selected feature points are those with high gradient magnitudes and large corner response values ​​obtained through screening. These feature points can better reflect significant motion information in the monitoring image. Therefore, the motion vector estimation model built based on these feature points has higher accuracy and robustness.

[0044] In one embodiment of this application, the calculation formula for the motion vector estimation model is as follows: Equation (10) in, Indicates the first The displacement vector of each selected feature point Denotes the affine transformation matrix. Represents the translation vector. Indicates the first The initial coordinates of the nth selected feature point (i.e., the nth feature point) The coordinates of each selected feature point in the monitoring image at the initial time.

[0045] In one embodiment of this application, the first The expression for the initial coordinates of the selected feature points is as follows: Equation (11) in, Indicates the first The initial coordinates of each selected feature point are in two-dimensional vector form. Indicates the first The X-axis coordinates of the initial coordinates of the selected feature points Indicates the first The Y-axis coordinate of the initial coordinates of the selected feature points.

[0046] In one embodiment of this application, the expression for the affine transformation matrix is ​​as follows: Equation (12) in, Denotes the affine transformation matrix. This represents the scaling factor of the X-axis in the affine transformation matrix. This represents the shearing factor in the affine transformation matrix that is related to the Y-axis coordinate and causes a change in the X-axis direction. This represents the shearing factor in the affine transformation matrix that is related to the X-axis coordinate and causes a change in the Y-axis direction. This represents the Y-axis scaling factor in the affine transformation matrix.

[0047] In one embodiment of this application, the expression for the translation vector is as follows: Equation (13) in, Represents the translation vector. This represents the translation along the X-axis in the affine transformation matrix. This represents the translation along the Y-axis in the affine transformation matrix.

[0048] In one embodiment of this application, before obtaining the motion vector estimation model, the historical position coordinates of the selected feature points are used as the initial coordinates of the selected feature points, and the historical displacement vectors of the selected feature points are used as the displacement vectors of the selected feature points. The combination of parameters in the affine transformation matrix and the translation vector is optimized by iterative weighted least squares method. In the process of optimizing the combination of parameters in the affine transformation matrix and the translation vector by iterative weighted least squares method, the parameter combination is adjusted by minimizing the photometric error, thereby obtaining the motion vector estimation model.

[0049] In one embodiment of this application, the formula for calculating photometric error is as follows: Equation (14) in, Indicates the first The historical displacement vector of each selected feature point Indicates the first Historical location coordinates of each selected feature point Indicates the number of feature points to be selected. Indicates the first The weights of each selection feature point This indicates the first frame of a historical surveillance image. The grayscale value of the pixel corresponding to each selected feature point This indicates the first frame of the previous monitoring image in a given historical frame. The grayscale value of the pixel corresponding to each selected feature point Indicates parameter combination, , This represents the scaling factor of the X-axis in the affine transformation matrix. This represents the shearing factor in the affine transformation matrix that is related to the Y-axis coordinate and causes a change in the X-axis direction. This represents the shearing factor in the affine transformation matrix that is related to the X-axis coordinate and causes a change in the Y-axis direction. This represents the Y-axis scaling factor in the affine transformation matrix. This represents the translation along the X-axis in the affine transformation matrix. This represents the translation along the Y-axis in the affine transformation matrix.

[0050] In one embodiment of this application, the optimization process of parameter combination suppresses the influence of outliers through an iterative reweighting mechanism, which significantly enhances the robustness of the motion vector estimation model to local deformation and illumination changes while ensuring computational efficiency.

[0051] In one embodiment of this application, the process of clustering the displacement vectors of sampled feature points based on their reachability distance to obtain a target cluster includes: Based on the displacement vectors of the selected feature points, the mean displacement vector of each feature point is determined. The Mahalanobis distance is then determined based on the displacement vectors and the mean displacement vector of each feature point. Finally, the displacement vectors of the selected feature points are filtered according to the comparison between the Mahalanobis distance and a preset distance threshold, resulting in the selected displacement vectors. In one embodiment of this application, the mean displacement vector of each feature point is the average displacement vector of all selected feature points. The formula for calculating the Mahalanobis distance is as follows: Equation (15) in, Represents Mahalanobis distance, Indicates the first The displacement vector of each selected feature point This represents the mean displacement vector of the selected feature points. This represents the mean of the displacement vectors of the selected feature points and the... The covariance matrix is ​​calculated from the difference vector of the displacement vectors of the selected feature points.

[0052] In one embodiment of this application, the preset distance threshold is set to 3. Based on the comparison result of the Mahalanobis distance and the preset distance threshold, the displacement vector of the selected feature point is filtered. The process of obtaining the filtered displacement vector includes: when the Mahalanobis distance calculated from the displacement vector of the selected feature point is greater than the preset distance threshold, the displacement vector of the selected feature point is used as the filtered displacement vector. Through the above settings, abnormal vectors caused by vehicle movement, occlusion, etc. can be deleted from the displacement vector of the selected feature point, thereby ensuring the effectiveness and accuracy of the filtered displacement vector and providing a reliable data foundation for the subsequent generation of target clusters.

[0053] The reachability distances of the feature points corresponding to the selected displacement vectors are calculated, and the selected displacement vectors are clustered based on these reachability distances to obtain target clusters. In one embodiment of this application, the reachability distance is used to characterize the shortest distance between each feature point corresponding to a selected displacement vector and other feature points corresponding to selected displacement vectors. After calculating the reachability distance of each feature point corresponding to a selected displacement vector, the feature points are sorted according to their reachability distances. The process of sorting each feature point corresponding to a selected displacement vector after calculating its reachability distance is implemented using the OPTICS (Ordering Points To Identify the Clustering Structure) algorithm. The sorting process reflects the density relationship between feature points corresponding to selected displacement vectors in different image regions; that is, the reachability distances of feature points corresponding to selected displacement vectors in image regions with high density are smaller, while the reachability distances of feature points corresponding to selected displacement vectors in image regions with low density are larger.

[0054] In one embodiment of this application, after sorting the feature points corresponding to each selected displacement vector, the selected displacement vectors are clustered in the motion direction-amplitude two-dimensional space to obtain a target cluster. The target cluster is the cluster with the most displacement vectors among all clusters, and the mean of the displacement vectors in the target cluster is used as the background displacement vector. The DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm is used to cluster the selected displacement vectors in the motion direction-amplitude two-dimensional space to obtain the target cluster.

[0055] In one embodiment of this application, the displacement vectors of the sampled feature points are clustered based on the reachability distance of the sampled feature points. This can effectively eliminate the parallax error introduced by the movement of the UAV, improve the accuracy of the background displacement vector, and provide accurate and stable kinematic analysis data for subsequent determination of reverse behavior.

[0056] In one embodiment of this application, if the region of interest for a lane includes a highway lane region of interest, then the process of determining the vehicle reverse driving detection result based on the difference vector between the vehicle motion vector and the background displacement vector, and the angle between the vehicle motion vector and the background displacement vector, includes: Based on vehicle bounding box information and highway lane region of interest (ROI) information, the spatial affiliation between the vehicle bounding box and the ROI is determined. In one embodiment of this application, the spatial affiliation includes: the vehicle bounding box spatially belonging to the ROI of the highway lane and the vehicle bounding box spatially not belonging to the ROI of the highway lane. By determining the spatial affiliation between the vehicle bounding box and the ROI of the highway lane, the spatial validity of the analyzed target vehicle is ensured. If the vehicle bounding box is not spatially belonging to the ROI of the highway lane, the vehicle is determined to be outside the control range of the highway vehicle wrong-way driving detection system. The highway vehicle wrong-way driving detection system will terminate the wrong-way driving determination process to prevent false alarms of wrong-way driving events in non-highway monitored areas and reduce the probability of false alarms in non-highway monitored areas.

[0057] If the spatial affiliation is such that the vehicle bounding box spatially belongs to the region of interest of a highway lane, then the ratio of the difference vector to the background displacement vector is denoted as the vector ratio. Based on the comparison result of the vector ratio and a preset ratio threshold, the lane type to which the highway lane region of interest belongs is determined. And based on the lane type, the angle between the vehicle motion vector and the background displacement vector, the vehicle reverse driving detection result is determined. In one embodiment of this application, the preset ratio threshold is set according to actual conditions, for example, 0.3. Lane types include: forward lanes and reverse lanes. The formula for calculating the vector ratio is as follows: Equation (16) in, Indicates the vector ratio. Represents the difference vector. Represents the background displacement vector. This represents the vehicle's motion vector. Calculating the difference vector helps eliminate observation biases introduced by the UAV's own motion.

[0058] In one embodiment of this application, if the information of the region of interest (ROI) for a highway lane includes: the center coordinates of the ROI, a mask image, and a region bounding box, and the information of the vehicle bounding box includes: the center coordinates of the vehicle bounding box, a region image, and boundary coordinates, then the process of determining the spatial membership between the vehicle bounding box and the ROI for the highway lane based on the vehicle bounding box information and the ROI information includes: If the distance between the center coordinates of the region of interest (ROI) of the highway lane and the center coordinates of the vehicle bounding box, the intersection-over-union (IoU) ratio between the mask and the region image, and the number of boundary coordinates located within the region box satisfy the spatial membership determination criteria, then the vehicle bounding box is determined to spatially belong to the ROI of the highway lane. In one embodiment of this application, the spatial membership determination criteria include: the distance between the center coordinates of the ROI of the highway lane and the center coordinates of the vehicle bounding box is less than a preset distance threshold, the IoU ratio between the mask and the region image is greater than a preset IoU threshold, and the number of boundary coordinates located within the region box is greater than a preset number threshold. The preset distance threshold, the preset IoU threshold, and the preset number threshold are set according to actual conditions, for example, 3.

[0059] If the distance between the center coordinates of the region of interest (ROI) of the highway lane and the center coordinates of the vehicle bounding box, the intersection-over-union (IoU) ratio between the mask image and the region image, and the number of boundary coordinates located within the region box do not meet the spatial membership determination criteria, then the vehicle bounding box is determined not to belong spatially to the ROI of the highway lane. In one embodiment of this application, the accuracy of determining the spatial membership between the vehicle bounding box and the ROI of the highway lane is improved by setting spatial membership determination criteria.

[0060] In one embodiment of this application, the process of determining the lane type of a highway lane's region of interest based on a comparison of the vector ratio with a preset ratio threshold includes: If the ratio comparison result is that the vector ratio is greater than a preset ratio threshold, then the subsequent first preset number of consecutive monitoring images of the vehicle on the highway are acquired. Furthermore, if the vector ratio obtained from each of the subsequent first preset number of consecutive monitoring images is greater than the preset ratio threshold, then the lane type is determined to be a forward lane. In one embodiment of this application, the first preset number is set according to actual conditions, for example, 80. In actual operation, a sliding window mechanism is used to select different monitoring images in chronological order. The size of the sliding window is set to 100. Within the sliding window, if the ratio comparison result calculated from a certain monitoring image is that the vector ratio is greater than the preset ratio threshold, then the forward count of the region of interest for the highway lane is 1. And if the vector ratio obtained from each of the subsequent 80 consecutive monitoring images is greater than the preset ratio threshold, then the lane type is determined to be a forward lane.

[0061] If the ratio comparison result is a negative value where the vector ratio is less than a preset ratio threshold, then the following first preset number of consecutive monitoring images of the vehicle on the highway are acquired. Furthermore, if the vector ratio obtained from each of the following first preset number of consecutive monitoring images is less than a preset ratio threshold, the lane type is determined to be a reverse lane. In one embodiment of this application, during actual operation, a sliding window mechanism is used to select different monitoring images in chronological order. The size of the sliding window is set to 100. Within the sliding window, if the ratio comparison result calculated from a certain monitoring image is a negative value where the vector ratio is less than a preset ratio threshold, the reverse count of the highway lane's region of interest is -1. And if the vector ratio obtained from each of the following 80 consecutive monitoring images is less than a preset ratio threshold, the lane type is determined to be a reverse lane.

[0062] In one embodiment of this application, the general method determines the lane type of the region of interest of the highway lane based on the comparison result of the vector ratio and the preset ratio threshold. This can effectively distinguish between forward lanes and reverse lanes, providing accurate lane type information for subsequent reverse driving detection.

[0063] In one embodiment of this application, the process of determining the vehicle reverse driving detection result based on the lane type and the angle between the vehicle motion vector and the background displacement vector includes: If the lane type is a forward lane, and the cosine of the angle between the vehicle's motion vector and the background displacement vector is greater than a preset angle threshold, and the magnitude of the vehicle's motion vector is greater than the product of the magnitude of the background displacement vector and a preset proportional threshold, then the risk of the vehicle driving in the wrong direction is considered a preliminary result of the judgment. In one embodiment of this application, the preset proportional threshold is set according to the actual situation, for example, 0.15, and the preset angle threshold is set according to the actual situation, for example, 0.85. Through the above judgment method, the accuracy of judging the risk of a vehicle driving in the wrong direction in a forward lane is improved.

[0064] If the lane type is a reverse lane, and the cosine of the angle between the vehicle's motion vector and the background displacement vector is less than the negative value of a preset angle threshold, and the magnitude of the vehicle's motion vector is greater than the product of the magnitude of the background displacement vector and a preset proportional threshold, then the risk of the vehicle driving in the wrong direction is considered a preliminary result of the reverse driving assessment. In one embodiment of this application, the accuracy of the assessment of the risk of a vehicle driving in the wrong direction in a reverse lane is improved through the above-mentioned assessment method.

[0065] If the initial determination of wrong-way driving indicates a risk of the vehicle driving in the wrong direction, then a second set of preset number of consecutive monitoring images of the vehicle on the highway are acquired. A vote is then taken between the initial determination of wrong-way driving obtained from the monitoring images and the initial determination of wrong-way driving obtained from each of the subsequent preset number of consecutive monitoring images, resulting in a voting result for each monitoring image within the current sliding window. In one embodiment of this application, the monitoring images and the subsequent preset number of consecutive monitoring images constitute all monitoring images within the current sliding window. The second preset number is set according to actual conditions, for example, 24 frames. In actual operation, the size of the current sliding window is set to 25 frames, meaning the monitoring image frame numbers within the current sliding window are: 0-24, 25-49, etc.

[0066] If the number of monitoring image frames in the current sliding window whose voting result indicates a risk of vehicle driving in the wrong direction exceeds a third preset threshold, then the vehicle driving in the wrong direction detection result is determined to be a vehicle driving in the wrong direction. In one embodiment of this application, the third preset threshold is set according to actual conditions, for example, set to 15. By adopting a voting mechanism to vote on the preliminary judgment result of driving in the wrong direction obtained from all monitoring images in the current sliding window, and judging the vehicle driving in the wrong direction detection result based on the voting result of each monitoring image in the current sliding window, it is beneficial to avoid misjudgment in a single frame and improve the accuracy of the vehicle driving in the wrong direction detection result.

[0067] Figure 3 This is a flowchart illustrating a highway vehicle reverse-driving detection method, as shown in another exemplary embodiment of this application. Figure 3As shown, the highway vehicle reverse driving detection method includes: (1) inputting a video stream into the highway vehicle reverse driving detection system; (2) verifying and parsing the video stream to obtain the parsed video stream; (3) sampling the parsed video stream at a frame rate of 25 FPS to obtain a monitoring image; (4) preprocessing the monitoring image to obtain a preprocessed image; (5) inputting the preprocessed image into an image detection model (YOLOv8 multi-task model) to obtain vehicle bounding box information, vehicle category confidence, and lane region of interest; (6) tracking the vehicle bounding box information according to the vehicle category confidence to obtain vehicle trajectory data; Based on vehicle trajectory data, determine vehicle motion vector; (7) track the lane region of interest to obtain lane region of interest tracking data; (8) sample the lane region of interest to obtain sampled feature points; (9) perform gradient filtering and corner response value filtering on the sampled feature points to obtain filtered feature points; (10) input the coordinate values ​​of the filtered feature points into the motion vector estimation model to obtain the displacement vector of the filtered feature points; use the mean of the displacement vectors in the target cluster as the background displacement vector; (11) fuse the lane region of interest tracking data, vehicle motion vector and background displacement vector; (12) if the vehicle bounding box is not spatially affiliated with the high If the region of interest (ROI) is within a highway lane, monitoring of vehicles on the highway continues, and the video stream is sampled to obtain new monitoring images. Backtracking detection is then performed based on these new and historical images. If a vehicle's bounding box belongs to the ROI of a highway lane, a determination is made regarding the risk of backtracking. If no risk is found, monitoring of vehicles on the highway continues, and the video stream is sampled to obtain new monitoring images. Backtracking detection is then performed based on these new and historical images. If a risk is found, a voting mechanism is used to obtain backtracking votes for each frame within the current sliding window. Results; If the number of monitoring image frames in the current sliding window that indicate a risk of vehicle reversing is less than or equal to the third preset threshold, then the vehicle reversing detection result is determined to be that the vehicle is not reversing. Then, the monitoring of vehicles on the highway continues, and the monitoring video stream is sampled to obtain new monitoring images. Reversing detection is performed based on the new monitoring images and historical monitoring images. If the number of monitoring image frames in the current sliding window that indicate a risk of vehicle reversing is greater than the third preset threshold, then the vehicle reversing detection result is determined to be that the vehicle is reversing. Then, the vehicle reversing detection result is encapsulated in a time-structured manner. (13) Report to the event management platform.

[0068] Figure 4 This is an exemplary embodiment of the present application illustrating the architecture of a pre-built detection model. Figure 4The pre-built detection model uses the YOLOv8 architecture as its base network and improves upon it. The pre-built detection model includes: a backbone network module, a neck network module, a detection head, and a segmentation head. The backbone network module includes: a first convolutional layer, a spatial-to-depth convolutional module, a cross-stage partial convolutional module with double convolution and feature fusion, and a global attention mechanism module. The neck network module includes: layers P1, P2, P3, P4, and P5. Layers P1, P2, P3, P4, and P5 all perform convolution, upsampling, and concatenation operations. The size of the convolution kernel in layer P1 is larger than that in layer P2, the size of the convolution kernel in layer P2 is larger than that in layer P3, the size of the convolution kernel in layer P3 is larger than that in layer P4, and the size of the convolution kernel in layer P4 is larger than that in layer P5. Layer P5 uses a pre-defined convolution kernel, which can be 10×10 or other sizes.

[0069] In one embodiment of this application, the process of training a pre-built detection model using pre-processed sample monitoring images to obtain an image detection model includes: inputting the pre-processed sample monitoring images into a first convolutional layer to obtain local feature maps; inputting the local feature maps into a depth convolutional module to obtain depth reconstructed feature maps; inputting the depth reconstructed feature maps into a cross-stage partial convolutional module with two convolutions and feature fusion to obtain multi-scale feature maps; inputting the multi-scale feature maps into a global attention mechanism module to obtain multi-scale global feature maps; inputting the multi-scale global feature maps into a P1 layer to obtain a first enhanced feature map; inputting the first enhanced feature map into a P2 layer to obtain a second enhanced feature map; inputting the second enhanced feature map into a P3 layer to obtain a third enhanced feature map; and upsampling the first enhanced feature map to obtain a first upsampled feature map. The first upsampled feature map and the third enhanced feature map are concatenated to obtain the first concatenated feature map. A 1×1 convolution operation is performed on the first concatenated feature map to obtain the first fused feature map. The first fused feature map is upsampled to obtain the second upsampled feature map. The second upsampled feature map and the second enhanced feature map are concatenated to obtain the second concatenated feature map. A 1×1 convolution operation is performed on the second concatenated feature map to obtain the second fused feature map. The second fused feature map is input into layer P4 to obtain the fourth enhanced feature map. The fourth enhanced feature map is input into layer P5 to obtain the fifth enhanced feature map. The fifth enhanced feature map is input into the detection head to obtain vehicle bounding box information, vehicle category confidence, and vehicle category. The fifth enhanced feature map is input into the segmentation head to obtain the lane region of interest, the mask map of the lane region of interest, and pixel-level classification results. The detection head performs bounding box regression, class classification, and non-maximum suppression (NMS) on the fifth enhanced feature map to obtain vehicle bounding box information, vehicle class confidence, and vehicle class. The segmentation head performs feature fusion, upsampling, and mask prediction on the fifth enhanced feature map to obtain lane region of interest, lane region of interest mask, and pixel-level classification results. Based on the output of the detection head and the vehicle corresponding bounding boxes in the labeled sample monitoring images, a CIoU (Complete Intersection over Union) loss function is established. Based on the output of the segmentation head and the lane corresponding regions in the labeled sample monitoring images, a binary cross-entropy loss function and a Dice (Dice Coefficient Loss) loss function are constructed. The pre-built detection model is trained with the goal of minimizing the CIoU loss function value, and the image detection model is trained with the goal of minimizing the sum of the binary cross-entropy loss function value and the Dice loss function value.

[0070] In one embodiment of this application, a pre-built detection model is trained with the goal of minimizing the CIoU loss function value. The strategy for training the pre-built detection model with the goal of minimizing the sum of the binary cross-entropy loss function value and the Dice loss function value includes: first, pre-training the pre-built detection model on large public datasets such as BDD100K and VisDrone to obtain a general feature representation, thus obtaining a first-stage detection model; then, using a carefully labeled self-built UAV highway dataset, the first-stage detection model is adaptively adjusted to the domain to obtain an image detection model. In the process of domain-adaptive adjustment of the first-stage detection model, data augmentation techniques such as Mosaic and MixUp were used to simulate various complex scenarios on highways. Based on the actual size and distribution characteristics of target objects in the images captured by drones, a set of anchor box sizes more suitable for complex scenarios was automatically generated using algorithms such as K-means clustering. The anchor box sizes of complex scenarios were used as annotation information in the sample monitoring images. The parameters in the first-stage detection model were adjusted with the goal of minimizing the sum of the CIoU loss function value, the binary cross-entropy loss function value, and the Dice loss function value. In the process of domain-adaptive adjustment of the first-stage detection model, a cosine annealing strategy was adopted for the learning rate strategy, and the optimizer used was the SGD (Stochastic Gradient Descent with Momentum) optimization algorithm with momentum. SGD+momentum is an optimization algorithm that combines the ideas of stochastic gradient descent (SGD) and momentum, aiming to accelerate model convergence and reduce oscillations during training.

[0071] Figure 5 This is a flowchart illustrating a method for detecting vehicles traveling in the wrong direction on a highway, as shown in another exemplary embodiment of this application. Figure 5As shown, the method for detecting vehicles traveling in the wrong direction on highways includes: (1) obtaining the vehicle motion vector and the background displacement vector; (2) the difference vector between the vehicle motion vector and the background displacement vector; (3) determining the spatial membership relationship between the vehicle bounding box and the highway lane region of interest based on the vehicle bounding box information and the highway lane region of interest information; (4) if the spatial membership relationship is that the vehicle bounding box does not belong to the highway lane region of interest in space, then no reverse driving judgment is made; if the spatial membership relationship is that the vehicle bounding box belongs to the highway lane region of interest in space, then the cosine value of the angle between the vehicle motion vector and the background displacement vector is calculated; (5) the ratio of the difference vector to the background displacement vector is recorded as the vector ratio; the lane type to which the highway lane region of interest belongs is determined based on the comparison result of the vector ratio and the preset ratio threshold; (6) if the lane type is a forward lane, the cosine value of the angle between the vehicle motion vector and the background displacement vector is greater than the preset angle threshold, and the magnitude of the vehicle motion vector is greater than the preset angle threshold. If the product of the magnitude of the background displacement vector and the preset ratio threshold is used as the preliminary judgment result of the vehicle having a risk of driving in the wrong direction; if the lane type is the reverse lane, the cosine value of the angle between the vehicle motion vector and the background displacement vector is less than the negative value of the preset angle threshold, and the magnitude of the vehicle motion vector is greater than the product of the magnitude of the background displacement vector and the preset ratio threshold, then the vehicle having a risk of driving in the wrong direction is used as the preliminary judgment result of driving in the wrong direction; (7) the preliminary judgment result of driving in the wrong direction obtained by the monitoring image and the preliminary judgment result of driving in the wrong direction obtained by the monitoring image of each frame in the subsequent second preset number of monitoring images are voted to obtain the voting result of driving in the wrong direction for each frame of the monitoring image in the current sliding window; (8) if the number of monitoring image frames in the current sliding window whose voting result of driving in the wrong direction is that the vehicle has a risk of driving in the wrong direction is greater than the third preset number threshold, then the vehicle driving in the wrong direction detection result is determined to be driving in the wrong direction; (9) if the vehicle driving in the wrong direction detection result is determined to be that the vehicle has not driven in the wrong direction, or the preliminary judgment result of driving in the wrong direction is that the vehicle does not have a risk of driving in the wrong direction, then the current judgment ends.

[0072] In one embodiment of this application, the cosine of the angle between the vehicle motion vector and the background displacement vector is greater than a preset angle threshold, and the magnitude of the vehicle motion vector is greater than the product of the magnitude of the background displacement vector and a preset proportional threshold, as a condition for determining the risk of driving in the wrong direction in the forward lane; the cosine of the angle between the vehicle motion vector and the background displacement vector is less than the negative value of the preset angle threshold, and the magnitude of the vehicle motion vector is greater than the product of the magnitude of the background displacement vector and a preset proportional threshold, as a condition for determining the risk of driving in the wrong direction in the reverse lane.

[0073] In one embodiment of this application, a highway vehicle wrong-way detection system includes: an image acquisition module for acquiring monitoring images of vehicles on the highway; an image detection module for inputting the monitoring images into an image detection model to obtain vehicle bounding box information, vehicle category confidence, and lane region of interest; a vehicle motion determination module for tracking the vehicle bounding box information based on the vehicle category confidence to obtain vehicle trajectory data; and determining the vehicle motion vector based on the vehicle trajectory data; and a background displacement determination module for sampling the lane region of interest to obtain sampled feature points; inputting the coordinate values ​​of the sampled feature points into a motion vector estimation model to obtain the displacement vectors of the sampled feature points; clustering the displacement vectors of the sampled feature points based on their reachability distance to obtain target clusters; and using the mean of the displacement vectors in the target clusters as the background displacement vectors; and the system further includes a wrong-way detection module. The vehicle in reverse detection module is used to determine the vehicle in reverse detection result based on the difference vector between the vehicle motion vector and the background displacement vector, as well as the angle between the vehicle motion vector and the background displacement vector. The highway vehicle in reverse detection system also includes: an event metadata module, used to record the UAV number, event ID, patrol road ID, timestamp, and the geographic latitude and longitude of the UAV when the event occurred; a vehicle metadata module, used to detect the vehicle category, track vehicle trajectory ID, and vehicle detection confidence information based on the YOLOv8 multi-task model; and an event evidence metadata module, used to integrate two keyframe images (50 frames apart and containing vehicle detection annotation information), a TXT (textfile) file record (containing JSON (JavaScriptObject Notation) format of all fields in the event metadata module and the vehicle metadata module), and an MP4 (MPEG-4 Part 14) file (containing 10 seconds of H.265 encoded video evidence (5 seconds before and after the event trigger)) triggered by the in reverse, and display the integrated results on the event platform.

[0074] When reporting vehicle wrong-way driving detection results to the event platform, a secure transmission channel based on the TLS (Transport Layer Security) 1.3 protocol is used, combined with application-layer encryption, forming a dual protection mechanism to ensure the security and integrity of the communication process. The reporting mechanism also supports breakpoint resumption; it automatically saves progress during network fluctuations and resumes transmission from the point of interruption upon recovery, ensuring the reliability of the reporting. Upon receiving an event alarm, the event platform automatically displays a pop-up window on the monitoring interface highlighting the location of the event, and also prominently annotates it on a map, pushing the alarm information to relevant regulatory personnel in real time. After the event is processed, the highway vehicle wrong-way driving detection system automatically archives and stores the report, evidence, and handling records, providing data support for subsequent management decisions and performance evaluations. The highway vehicle wrong-way driving detection system achieves full automation from event detection and reporting to response, with an average response time controlled within 5 seconds, greatly improving the real-time performance and efficiency of highway safety management.

[0075] This application accurately determines the vehicle's direction of travel (forward or backward) by calculating the cosine of the angle between the vehicle's motion vector and the background displacement vector. This effectively overcomes misjudgments caused by factors such as drone perspective jitter, scale changes, and background interference, improving the accuracy of backward vehicle identification to 86.0%, significantly outperforming methods based on fixed positions or single-frame images. Using YOLOv8n as the detection backbone, and combining sparse sampling, dense sampling, and target tracking, the application significantly reduces computational load while maintaining high accuracy, meeting the stringent real-time requirements of highway monitoring. Addressing challenges common in drone aerial photography such as image jitter, continuous scale changes, local defocusing, and lighting fluctuations, the application introduces Kalman filter prediction and a multi-frame voting mechanism, effectively improving the stability and reliability of the highway vehicle backward detection system in dynamic scenes and enhancing its adaptability in complex environments.

[0076] This application realizes the full-process automation of real-time identification, tracking and event reporting of vehicles traveling in the wrong direction in dynamic scenarios, and completes the closed-loop processing from detection and structure generation to safety warning, providing a high-precision, low-latency and practically applicable intelligent solution for highway safety management.

[0077] The following describes an embodiment of the apparatus of this application, which can be used to execute a method for detecting vehicles driving in the wrong direction on a highway, as described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of a method for detecting vehicles driving in the wrong direction on a highway described in the above embodiments of this application.

[0078] Figure 6 This is a block diagram illustrating a highway vehicle wrong-way detection device, as shown in an exemplary embodiment of this application.

[0079] like Figure 6As shown, this exemplary highway vehicle wrong-way detection device 600 includes: The image acquisition module 601 is used to acquire monitoring images of vehicles on highways.

[0080] The image detection module 602 is used to input the monitoring image into the image detection model to obtain vehicle bounding box information, vehicle category confidence, and lane region of interest.

[0081] The vehicle motion determination module 603 is used to track the vehicle bounding box information based on the vehicle category confidence level to obtain vehicle trajectory data; and to determine the vehicle motion vector based on the vehicle trajectory data.

[0082] The background displacement determination module 604 is used to sample the region of interest of the lane to obtain sampled feature points; input the coordinate values ​​of the sampled feature points into the motion vector estimation model to obtain the displacement vectors of the sampled feature points; cluster the displacement vectors of the sampled feature points according to the reachability distance of the sampled feature points to obtain target clusters; and use the mean of the displacement vectors in the target clusters as the background displacement vectors.

[0083] The reverse driving detection module 605 is used to determine the reverse driving detection result based on the difference vector between the vehicle motion vector and the background displacement vector, as well as the angle between the vehicle motion vector and the background displacement vector.

[0084] In one embodiment of this application, surveillance images of vehicles on highways are obtained by capturing surveillance video of vehicles on highways using a camera mounted on a drone, and then sampling the surveillance video frame by frame. The frequency of frame sampling of the surveillance video is set according to the actual situation, for example, 25 FPS.

[0085] In one embodiment of this application, the image detection model is trained on a pre-built detection model based on sample surveillance images. The process of training the pre-built detection model based on sample surveillance images includes: labeling vehicle information and lane information in the sample surveillance images to obtain sample surveillance images with labeled information; preprocessing the sample surveillance images with labeled information to obtain preprocessed sample surveillance images; the preprocessing operations include: image normalization, image size transformation, and image enhancement; and training the pre-built detection model using the preprocessed sample surveillance images to obtain the image detection model. In the above process, by labeling and preprocessing the sample surveillance images, the preprocessed sample image data becomes more standardized and effective, enabling the pre-built detection model to better learn the feature information of vehicles and lanes. During the training process, by continuously adjusting the parameters of the pre-built detection model, it can accurately identify vehicle bounding box information, vehicle category confidence, and lane regions of interest from the sample surveillance images. As training progresses, the pre-built detection model's ability to recognize sample monitoring images under different scenarios and lighting conditions will gradually improve, thereby enabling the image detection model to have high accuracy and stability, and providing a reliable data foundation for subsequent vehicle reverse driving detection.

[0086] In one embodiment of this application, if the vehicle category confidence score includes: the vehicle category confidence score detected from the monitoring image at the current moment, denoted as the current vehicle category confidence score; if the vehicle bounding box information includes: the bounding box information detected from the monitoring image at the current moment, denoted as the current bounding box information; and the bounding box information detected from the monitoring image at the previous moment, denoted as the previous bounding box information; the current vehicle category confidence score corresponds to the current bounding box information; and the bounding box information includes: the bounding box center coordinates, the bounding box scale, the movement speed of the bounding box center coordinates, and the bounding box scale change rate; then, the process of tracking the vehicle bounding box information based on the vehicle category confidence score to obtain vehicle trajectory data includes: inputting the previous bounding box information into the state transition equation to obtain the current bounding box prediction information; and inputting the current bounding box information into the state transition equation to obtain the current bounding box prediction information. The predicted information is correlated with the confidence level of the current vehicle category to obtain the confidence level of the current bounding box prediction information. Based on a first preset confidence threshold, the bounding boxes corresponding to the current bounding box prediction information are divided into high-quality bounding boxes and low-quality bounding boxes. High-quality bounding boxes are those corresponding to bounding box prediction information with a confidence level greater than or equal to the first preset confidence threshold, while low-quality bounding boxes are those corresponding to bounding box prediction information with a confidence level less than the first preset confidence threshold. High-quality bounding boxes are matched with existing trajectories to obtain first matching data. Low-quality bounding boxes are matched with existing trajectories to obtain second matching data. The first and second matching data are combined to obtain combined trajectory data. A sliding window is used to perform weighted smoothing on the combined trajectory data to obtain vehicle trajectory data.

[0087] In one embodiment of this application, the target cluster is the cluster with the most displacement vectors among all clusters; the motion vector estimation model is determined based on the historical position coordinates and historical displacement vectors of the sampled feature points, and the motion vector estimation model is determined based on the historical position coordinates and historical displacement vectors of the sampled feature points, and the displacement vectors of the sampled feature points are estimated by the motion vector estimation model, thereby improving the accuracy of the displacement vector estimation of the sampled feature points and reducing the estimation deviation caused by determining the displacement vectors of the sampled feature points based only on data obtained from a single frame of monitoring image.

[0088] In one embodiment of this application, during the process of determining the vehicle reverse driving detection result, the vehicle category confidence obtained from time-series information (i.e., different monitoring images) is fully utilized to track the vehicle bounding box information obtained from the time-series information, thereby obtaining vehicle trajectory data and vehicle running vectors, thus improving the accuracy of vehicle motion vectors. Furthermore, a motion vector estimation model is determined based on the historical position coordinates and historical displacement vectors of sampled feature points, and the displacement vectors of sampled feature points are estimated using the motion vector estimation model, thereby improving the accuracy of estimating the displacement vectors of sampled feature points and reducing the estimation bias caused by determining the displacement vectors of sampled feature points based solely on data obtained from a single frame of monitoring image. In addition, based on the improved accuracy of the displacement vectors of sampled feature points, the accuracy of background displacement vectors is also improved, thereby improving the accuracy of the vehicle reverse driving detection result. This meets the requirements for automatic real-time monitoring, identification, tracking, and detection of vehicle reverse driving behavior in highway scenarios, greatly improving the real-time performance and effectiveness of highway safety management.

[0089] It should be noted that the highway vehicle wrong-way detection device and the highway vehicle wrong-way detection method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the highway vehicle wrong-way detection device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0090] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for detecting vehicles driving in the wrong direction on highways, characterized in that, The method includes: Acquire surveillance images of vehicles on the highway; The surveillance image is input into the image detection model to obtain vehicle bounding box information, vehicle category confidence, and lane region of interest; the image detection model is obtained by training a pre-built detection model based on sample surveillance images; Based on the vehicle category confidence level, the vehicle bounding box information is tracked to obtain vehicle trajectory data; based on the vehicle trajectory data, the vehicle motion vector is determined. The region of interest (ROI) of the lane is sampled to obtain sampled feature points; the coordinate values ​​of the sampled feature points are input into a motion vector estimation model to obtain the displacement vectors of the sampled feature points; based on the reachability distance of the sampled feature points, the displacement vectors of the sampled feature points are clustered to obtain target clusters; the mean of the displacement vectors in the target clusters is used as the background displacement vector; the target cluster is the cluster with the most displacement vectors among all clusters; the motion vector estimation model is determined based on the historical position coordinates and historical displacement vectors of the sampled feature points. The vehicle reverse driving detection result is determined based on the difference vector between the vehicle motion vector and the background displacement vector, and the angle between the vehicle motion vector and the background displacement vector. If the region of interest for the lane includes a highway lane region of interest, then the process of determining the vehicle reverse driving detection result based on the difference vector between the vehicle motion vector and the background displacement vector, and the angle between the vehicle motion vector and the background displacement vector, includes: Based on the vehicle bounding box information and the highway lane region of interest information, the spatial membership relationship between the vehicle bounding box and the highway lane region of interest is determined; If the spatial affiliation is such that the vehicle bounding box spatially belongs to the region of interest of the highway lane, then the ratio of the difference vector to the background displacement vector is recorded as the vector ratio; based on the comparison result of the vector ratio with a preset ratio threshold, the lane type to which the region of interest of the highway lane belongs is determined; and based on the lane type, the angle between the vehicle motion vector and the background displacement vector, the vehicle reverse driving detection result is determined; the lane type includes: forward lane and reverse lane.

2. The method for detecting vehicles driving in the wrong direction on highways according to claim 1, characterized in that, The process of training a pre-built detection model based on sample surveillance images to obtain the image detection model includes: The vehicle and lane information in the sample monitoring images are labeled to obtain sample monitoring images with labeled information; The sample monitoring images with labeled information are preprocessed to obtain preprocessed sample monitoring images; the preprocessing operations include: image normalization, image size transformation, and image enhancement. The pre-built detection model is trained using the pre-processed sample monitoring images to obtain the image detection model. The pre-built detection model includes: a backbone network module, a neck network module, a detection head, and a segmentation head. The backbone network module includes: a first convolutional layer, a spatial-to-depth convolutional module, a cross-stage partial convolutional module with two convolutions and feature fusion, and a global attention mechanism module. The neck network module includes: layers P1, P2, P3, P4, and P5. Layers P1, P2, P3, P4, and P5 all perform convolution operations, upsampling operations, and concatenation operations. Layer P5 uses a convolutional kernel of a preset size.

3. The method for detecting vehicles driving in the wrong direction on highways according to claim 1, characterized in that, If the lane region of interest includes a first preset region of interest and a second preset region of interest, then the process of sampling the lane region of interest to obtain sampled feature points includes: According to the first spatial sampling density, the first preset region of interest is sampled to obtain the first sampled feature points; the average gradient magnitude of the first preset region of interest is less than the average gradient magnitude of the second preset region of interest. The second preset region of interest is sampled according to the second spatial sampling density to obtain second sampled feature points; the first spatial sampling density is less than the second spatial sampling density. The first sampling feature point and the second sampling feature point are combined to form the sampling feature point.

4. The method for detecting vehicles driving in the wrong direction on highways according to claim 1 or 3, characterized in that, The process of inputting the coordinate values ​​of the sampled feature points into the motion vector estimation model to obtain the displacement vector of the sampled feature points includes: Gradient filtering is performed on the sampled feature points to obtain target feature points; the gradient of the target feature point is greater than a preset gradient threshold; the gradient of the sampled feature point is determined based on the eigenvalues ​​of the autocorrelation matrix of the region where the sampled feature point is located. The corner response values ​​of the target feature points are filtered to obtain filtered feature points; the corner response values ​​of the filtered feature points are the maximum values ​​within a preset image area. The coordinate values ​​of the selected feature points are input into the motion vector estimation model to obtain the displacement vector of the selected feature points; the motion vector estimation model is determined based on the historical position coordinates and historical displacement vector of the selected feature points.

5. The method for detecting vehicles driving in the wrong direction on highways according to claim 4, characterized in that, The process of clustering the displacement vectors of the sampled feature points based on their reachability distances to obtain the target cluster includes: Based on the displacement vectors of the selected feature points, the mean displacement vector of the selected feature points is determined; based on the displacement vectors of the selected feature points and the mean displacement vector of the selected feature points, the Mahalanobis distance is determined; and based on the distance comparison result between the Mahalanobis distance and a preset distance threshold, the displacement vectors of the selected feature points are filtered to obtain the selected displacement vectors. Calculate the reachability distance of the feature points corresponding to the selected displacement vectors, and cluster the selected displacement vectors according to the reachability distance of the feature points corresponding to the selected displacement vectors to obtain the target cluster.

6. The method for detecting vehicles driving in the wrong direction on highways according to claim 1, characterized in that, If the information of the region of interest (ROI) of the highway lane includes: the center coordinates of the ROI, a mask image, and a region bounding box, and the information of the vehicle bounding box includes: the center coordinates of the vehicle bounding box, a region image, and boundary coordinates, then the process of determining the spatial membership between the vehicle bounding box and the ROI of the highway lane based on the vehicle bounding box information and the information of the ROI of the highway lane includes: If the distance between the center coordinates of the region of interest of the highway lane and the center coordinates of the vehicle bounding box, the intersection-over-union ratio (IoU) between the mask and the region image, and the number of boundary coordinates located within the region box satisfy the spatial membership determination conditions, then the vehicle bounding box is determined to spatially belong to the region of interest of the highway lane. The spatial membership determination conditions include: the distance between the center coordinates of the region of interest of the highway lane and the center coordinates of the vehicle bounding box is less than a preset distance threshold, the IoU between the mask and the region image is greater than a preset IoU threshold, and the number of boundary coordinates located within the region box is greater than a preset number threshold. If the distance between the center coordinates of the region of interest of the highway lane and the center coordinates of the vehicle bounding box, the intersection-over-union ratio between the mask and the region image, and the number of boundary coordinates located within the region box do not satisfy the spatial membership determination conditions, then it is determined that the vehicle bounding box does not spatially belong to the region of interest of the highway lane.

7. The method for detecting vehicles driving in the wrong direction on highways according to claim 1, characterized in that, The process of determining the lane type of the highway lane's region of interest based on the comparison result of the vector ratio and the preset ratio threshold includes: If the ratio comparison result is that the vector ratio is greater than the preset ratio threshold, then the subsequent consecutive first preset number of monitoring images of the vehicle on the highway are obtained, and if the vector ratio obtained by judging each monitoring image in the subsequent consecutive first preset number of monitoring images is greater than the preset ratio threshold, then the lane type is determined to be a forward lane. If the ratio comparison result is a negative value where the vector ratio is less than the preset ratio threshold, then the subsequent first preset number of consecutive monitoring images of the vehicle on the highway are obtained. If the vector ratio obtained by judging each monitoring image in the subsequent first preset number of consecutive monitoring images is less than the negative value of the preset ratio threshold, then the lane type is determined to be a reverse lane.

8. The method for detecting vehicles driving in the wrong direction on highways according to claim 1, characterized in that, The process of determining the vehicle reverse driving detection result based on the lane type and the angle between the vehicle motion vector and the background displacement vector includes: If the lane type is a forward lane, the cosine of the angle between the vehicle motion vector and the background displacement vector is greater than a preset angle threshold, and the magnitude of the vehicle motion vector is greater than the product of the magnitude of the background displacement vector and a preset ratio threshold, then the risk of the vehicle driving in the wrong direction is taken as the preliminary result of the judgment. If the lane type is a reverse lane, the cosine of the angle between the vehicle motion vector and the background displacement vector is less than the negative value of the preset angle threshold, and the magnitude of the vehicle motion vector is greater than the product of the magnitude of the background displacement vector and the preset ratio threshold, then the risk of the vehicle driving in the wrong direction is taken as the preliminary result of the reverse driving judgment. If the preliminary determination result of wrong-way driving indicates that the vehicle poses a risk of wrong-way driving, then a second set number of subsequent consecutive monitoring images of the vehicle on the highway are acquired. A vote is then taken between the preliminary determination result of wrong-way driving obtained from the monitoring images and the preliminary determination result of wrong-way driving obtained from each of the subsequent second set number of consecutive monitoring images to obtain the wrong-way driving voting result for each monitoring image within the current sliding window. The monitoring images and the subsequent second set number of consecutive monitoring images constitute all monitoring images within the current sliding window. If the number of monitoring image frames in the current sliding window indicating a risk of vehicle driving in the wrong direction is greater than a third preset threshold, then the vehicle driving in the wrong direction detection result is determined to be a vehicle driving in the wrong direction.

9. A highway vehicle wrong-way driving detection device, characterized in that, The device includes: Image acquisition module, used to acquire monitoring images of vehicles on highways; The image detection module is used to input the monitoring image into the image detection model to obtain vehicle bounding box information, vehicle category confidence, and lane region of interest; the image detection model is obtained by training a pre-built detection model based on sample monitoring images; The vehicle motion determination module is used to track the vehicle bounding box information based on the vehicle category confidence level to obtain vehicle trajectory data; and to determine the vehicle motion vector based on the vehicle trajectory data. The background displacement determination module is used to sample the region of interest of the lane to obtain sampled feature points; input the coordinate values ​​of the sampled feature points into the motion vector estimation model to obtain the displacement vectors of the sampled feature points; cluster the displacement vectors of the sampled feature points according to their reachability distance to obtain target clusters; use the mean of the displacement vectors in the target clusters as the background displacement vectors; the target clusters are the clusters with the most displacement vectors among all clusters; the motion vector estimation model is determined based on the historical position coordinates and historical displacement vectors of the sampled feature points. The reverse driving detection module is used to determine the vehicle reverse driving detection result based on the difference vector between the vehicle motion vector and the background displacement vector, and the angle between the vehicle motion vector and the background displacement vector; If the region of interest for the lane includes a highway lane region of interest, then the process of determining the vehicle reverse driving detection result based on the difference vector between the vehicle motion vector and the background displacement vector, and the angle between the vehicle motion vector and the background displacement vector, includes: Based on the vehicle bounding box information and the highway lane region of interest information, the spatial membership relationship between the vehicle bounding box and the highway lane region of interest is determined; If the spatial affiliation is such that the vehicle bounding box spatially belongs to the region of interest of the highway lane, then the ratio of the difference vector to the background displacement vector is recorded as the vector ratio; based on the comparison result of the vector ratio with a preset ratio threshold, the lane type to which the region of interest of the highway lane belongs is determined; and based on the lane type, the angle between the vehicle motion vector and the background displacement vector, the vehicle reverse driving detection result is determined; the lane type includes: forward lane and reverse lane.

Images