Highway vehicle detection method and system based on computer vision

Abstract

The invention relates to the technical field of electric digital data processing, and discloses an expressway vehicle detection method and system based on computer vision, and the method comprises the steps: environment classification and recognition: recognizing the updated environment classification level of a certain expressway section position planned to drive according to the position; data acquisition and preprocessing: sequentially performing three links of data acquisition, data screening and data labeling, and preprocessing the acquired data according to different environmental classification levels of the highway section to obtain initial image data; selecting a vehicle detection model according to the current grade of the environment classification of a certain highway section, and if the current grade of the environment classification is a fourth-grade environment, performing detection by adopting an existing vehicle detection model; if the environment classification is a specific classification, adopting an improved detection model; and detecting and identifying vehicles on the highway according to the selected model, and feeding back a detection result. The method can give consideration to the detection precision and the calculation amount, and provides the optimal detection according to the actual situation.

Description

Technical Field

[0001] The invention relates to the technical field of electronic digital data processing, and in particular to a highway vehicle detection method and system based on computer vision. Background Art

[0002] Highway vehicle detection plays an important role in ensuring traffic safety, maintaining traffic order, strengthening vehicle management and promoting the development of intelligent transportation systems. Therefore, in order to better carry out highway vehicle detection, various highway vehicle detection methods are constantly being innovated.

[0003] Among them, the vehicle detection method based on machine vision is one of the most commonly used methods in the existing highway vehicle detection methods. For example, "CN112069969B A highway monitoring video cross-mirror vehicle tracking method and system" provides a vehicle detection method that relies on computer machine vision image technology to achieve cross-mirror tracking and reduce the difficulty of tracking implementation. This solution replaces the backbone network in Y0L0v3 with a variant of DenseNet with better learning ability, VoVNet, to obtain a feature extraction network layer. At the same time, according to the size of the vehicle captured in the highway monitoring video image, the three scales of large, medium and small in the YOLOv3 detection layer are reduced to two scales of small and medium, and an improved Y0L0 target detection model is obtained, which makes the model smaller, faster in calculation speed and reduces the amount of calculation, and can obtain a vehicle detection image containing a complete vehicle rectangular frame more quickly.

[0004] However, like most existing solutions, the current highway car detection method overly relies on the calculation itself and the recognition of the vehicle itself under ideal conditions, while ignoring the influence of environmental factors on the original collected image, resulting in the need to spend a lot of calculations to improve the accuracy of recognition. Taking "CN112069969B A highway monitoring video cross-mirror vehicle tracking method and system" as an example, although the system can obtain good detection results under ideal conditions, in actual environments, the situation close to the ideal state is rare or even almost non-existent. In a complex highway environment, the driving state and characteristics of the vehicle will be affected by many factors, such as vehicle occlusion, light changes, road curvature, etc., which makes it difficult for existing detection methods to accurately identify and track each vehicle, and it is easy to miss and misdetect. Moreover, for some subtle features and special situations of some vehicles, such as the license plate number of the vehicle, the damage of the vehicle, etc., it is often difficult to effectively identify them with existing methods without increasing the amount of calculation.

[0005] In short, it is difficult for existing detection methods to balance the contradiction between recognition accuracy and computational complexity in complex practical environments. With the rapid development of highway traffic, higher requirements are placed on the accuracy and real-time performance of vehicle detection. It is urgent to develop new highway vehicle detection methods and systems that can meet the requirements in order to improve the efficiency and safety of highway traffic management. Summary of the invention

[0006] The present invention aims to provide a highway vehicle detection method based on computer vision, so as to achieve a balance between the contradiction between recognition accuracy and computational complexity in the detection of highway vehicles in a complex actual environment.

[0007] In order to achieve the above object, the present invention adopts the following technical scheme: Solution 1: A highway vehicle detection method based on computer vision, comprising the following steps: Step 1: judging the road section environment classification, according to the location of the highway section, identifying the updated value of the environment classification on the day, and judging the environment classification level; the environment classification includes the fourth level environment, the third level environment, the second level environment and the first level environment from low to high driving danger, among which the third level environment, the second level environment and the first level environment belong to specific classifications; Step 2: Data collection and preprocessing. After data collection, data screening and data labeling, the collected data is preprocessed according to the different environmental classification levels of the highway section to obtain preliminary image data. Step 3: Model selection: select different vehicle detection models according to the different environmental classification levels of the highway section. If the environmental classification level is the fourth level, use the existing vehicle detection model for detection. If the environmental classification level is a specific classification, use the improved detection model. Step 4: detection feedback: detecting and identifying vehicles on the highway according to the selected model, and feeding back the detection results; The steps for constructing the improved detection model are as follows: First, select the YOLOv8n target detection model as the basic model and replace its backbone network with ECA_ESNet to form the first model; Then, the ECA attention mechanism and SimAM attention mechanism are introduced into the first model to form the second model; Finally, in the second model, BiFPN is used to fuse feature maps at different levels to form an improved detection model.

[0008] Preferably, the data collection step includes: Using highway data collection equipment to collect original data to obtain first data; The environmental classification includes seven parameter items, namely, altitude, longitude and latitude coordinates, visibility, road slipperiness, real-time traffic flow, and road curvature. After removing the two fixed items, namely, altitude and longitude and latitude coordinates, the environmental classification level of the corresponding expressway section is dynamically adjusted according to the values ​​of the four changing items, namely, visibility, road slipperiness, real-time traffic flow, and road curvature, which are updated in time on the day and compared with the preset safety range values. If the environment is classified as the fourth level environment, the first data is equal to the second data; if the environment is classified as the third level environment, the second level environment or the first level environment, the data is collected again at the key sections to obtain the re-collected data, and the re-collected data is added to the first data to obtain the second data.

[0009] Preferably, the third-level environment is that the updated value of one of the four change items is not within the safe range, and the length of the critical section is more than one-tenth of the length of the corresponding highway section; the second-level environment is that the updated values ​​of two of the four change items are not within the safe range, and the length of the critical section is more than one-eighth of the length of the corresponding highway section; the first-level environment is that the updated values ​​of three of the four change items are not within the safe range, and the length of the critical section is more than one-sixth of the length of the corresponding highway section; the highway section refers to the continuous section between any two highway ramps planned or currently being traveled, and the critical section is one or more high-risk areas in the highway section that are most likely to cause traffic accidents.

[0010] Preferably, the data screening step comprises: Effective data screening: According to the current environment classification of the highway section, a picture is captured every N frames of the collected second data to obtain the screening data. At the same time, images with obvious vehicle features and clear driving directions are screened out, and images that cannot be accurately detected due to vehicle occlusion and poor angles are removed; among them, N for the fourth-level environment is 6, N for the third-level environment is 4, N for the second-level environment is 3, and N for the first-level environment is 2; Image quality assessment: Perform quality assessment on the video images in the filtered data, select images with good quality through image clarity, contrast, and degree of unobstructedness, and use existing image enhancement algorithms to process blurred or poorly lit images to form third data.

[0011] Preferably, the data annotation step is: using a professional annotation tool to annotate the third data image to obtain preliminary image data, dividing vehicles into three categories: cars, buses, and trucks during annotation, and marking the location, size, and travel direction of the vehicles.

[0012] Preferably, in ECA_ESNet, ESNet Block is divided into two types: stride 1 and stride 2, with a total of four branches, consisting of depthwise separable convolution, Ghost Block, SE Block and channels shuffle modules, with a convolution kernel size of 3x3 and a stride of 2.

[0013] Preferably, the processing process of the ECA attention mechanism is: for an input size of , the feature map with the number of channels C, firstly perform global average pooling to compress the feature map size to 1*1, and then perform one-dimensional convolution with a convolution kernel size of K. The function relationship between the value of K and the number of channels C is: ,in , c is the number of channels; The feature map after one-dimensional convolution is activated to obtain the weight of each channel of the feature map X, and the weight is multiplied by the initial feature map X to obtain the final result X'.

[0014] Preferably, the processing process of the SimAM attention mechanism is: through the energy function

[0015] To evaluate the importance of each neuron and add regularization to the energy function; where t represents the index of the neuron, w t and b t are the weights and biases of neuron transformation, y t and y0 take the values ​​1 and -1 respectively, which are used for the calculation of energy function. i is the neighboring neuron of the input feature, and M is the total number of neurons set based on the channel dimension of the current feature map; Compute analytical solution , , in, , , Get the weight of the neuron; Finally, through

[0016] The processed feature map is obtained, where E is the sum of all energies.

[0017] Solution 2: The present invention also provides a highway vehicle detection system based on computer vision, which is used to implement the above-mentioned highway vehicle detection method based on computer vision, including: The road section judgment module is used to identify the updated value of the environmental classification on the day according to the location of the highway section and judge the environmental classification level; The data acquisition module is connected to the road section judgment module, and collects image or video data of the highway according to the environmental classification level obtained by judgment; if the environmental classification is a specific classification, in addition to the first data obtained by routine data collection, additional data collection of key sections is required to collect data again to supplement the data and obtain second data; The data preprocessing module is connected to the data acquisition module, and screens the image or video data of the second data according to the rule of intercepting one picture every N frames, and then selects the image with good quality through image quality evaluation, and enhances the blurred or poorly lit images to obtain the third data; the third data is annotated with vehicle type, position, size, and driving direction to obtain preliminary image data; wherein the N value decreases as the environmental classification level increases; The target detection module is connected to the data preprocessing module. If the environment classification level is the fourth level environment, the existing vehicle detection model is used for detection; if the environment classification is a specific classification, an improved detection model is used; the improved detection model is based on YOLOv8n, and its backbone network is replaced with ECA_ESNet, the ECA and SimAM attention mechanisms are introduced, and the weighted bidirectional feature pyramid BiFPN is used for feature fusion.

[0018] The advantages of the present invention are: 1. Excellent environmental adaptability and targeted data collection.

[0019] Accurate environmental classification: This method dynamically divides the highway section environment into four danger levels based on multiple parameters such as altitude, longitude and latitude coordinates, visibility, road slipperiness, real-time traffic flow, and road curvature. This allows the system to fully consider the impact of different environments on vehicle detection, providing an accurate environmental basis for subsequent steps, making the detection process more in line with actual conditions.

[0020] On-demand data collection: Differentiated data collection strategies are implemented according to the environmental classification level. For the fourth-level safety environment, only the conventionally collected data is used; while for the first, second, and third-level environments with higher levels of danger, additional data is added at key sections. This on-demand collection method ensures that there is enough data to support detection in complex environments, while avoiding excessive collection in safe environments, effectively balancing the amount of data and detection accuracy.

[0021] 2. Refinement of data preprocessing.

[0022] Hierarchical data screening: Set different frame interception intervals (N values) according to the environmental classification level. The more complex the environment, the smaller the N value, that is, the denser the collected data. At the same time, images with obvious vehicle features and clear driving directions are screened out, and interfering images are removed, which greatly reduces the amount of invalid data processing and improves the efficiency and accuracy of subsequent detection.

[0023] High-quality image evaluation and enhancement: The quality of the screened images is evaluated, and high-quality images are selected through indicators such as clarity and contrast, and blurred or poorly lit images are enhanced. This improves the overall quality of the image data and provides a better data foundation for subsequent model detection.

[0024] 3. Flexibility in model selection and model performance advantages.

[0025] Flexible model adaptation: Select the appropriate detection model according to the environmental classification level. Use the existing mature model in the safe fourth-level environment, and use the improved detection model in the complex specific classification environment. This flexible selection strategy not only ensures the detection efficiency in simple environments, but also takes advantage of the improved model in complex environments to improve the accuracy of detection.

[0026] 4. Improved model performance.

[0027] Optimize network structure: Improve the detection model based on YOLOv8n, and replace the backbone network with ECA_ESNet. ECA_ESNet combines the advantages of ShuffleNet V2, SE module and Ghost module, and makes extensive use of deep separable convolution. It can identify vehicles more accurately under limited computational workload, improve feature extraction capabilities and the lightweight level of the model.

[0028] Introducing attention mechanism: Introducing ECA and SimAM attention mechanisms. The ECA mechanism dynamically adjusts channel relationships through adaptive one-dimensional convolution to focus on important features; the SimAM mechanism integrates channel and spatial attention to deeply capture the deep features of the vehicle and reduce interference information. The two work together to significantly improve the feature extraction accuracy of the model.

[0029] Efficient feature fusion: BiFPN is used to fuse feature maps at different levels, and cross-scale feature interaction is achieved through weighted and bidirectional connections. This fully utilizes the detailed information of shallow features and the semantic information of deep features, and enhances the model's ability to detect targets of different sizes.

[0030] 5. The comprehensive advantages of the holistic approach.

[0031] This detection method forms a complete and mutually collaborative system from environment classification, data collection, preprocessing to model selection and detection feedback. Each step is closely integrated, fully considering the complexity and diversity of the highway environment, effectively controlling the data processing volume and computing cost while ensuring the detection accuracy, achieving a good balance between detection efficiency and accuracy, and providing an efficient and reliable solution for highway vehicle detection.

[0032] The present invention can fully consider the impact of the environment on the driving status and characteristics of vehicles in the complex actual environment of highways, optimize and improve the existing detection methods, balance the contradiction between recognition accuracy and calculation amount, meet the detection requirements of accuracy and real-time, and can accurately identify each vehicle without missed detection and false detection, which helps to improve the efficiency and safety of highway traffic management. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flow chart of Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0034] The following is further described in detail through specific implementation methods: Embodiment 1 This embodiment is basically as shown in the attached Figure 1 Shown: A highway vehicle detection method based on computer vision, including the following: The first step is environmental classification, data collection and preprocessing, which goes through three steps in sequence: data collection, data screening and data labeling. According to the different environmental classifications of the highway, different collected data are preprocessed to obtain preliminary image data.

[0035] (I) Data collection: First, use existing highway data collection equipment (such as drones with high-definition shooting capabilities, vehicle-mounted cameras, and high-definition cameras fixed on the roadside as data collection equipment) to collect raw data and obtain first data. Then, by reading the fixed items in the preset highway environment classification, identifying and comparing the changed items, the environmental classification level of the highway is determined, and the first data of the highway area in a specific classification is supplemented to obtain second data.

[0036] Among them, the environmental classification includes altitude, longitude and latitude coordinates, visibility, road slipperiness, real-time traffic flow, and road curvature. The environmental classification is based on the four variable items of visibility, road slipperiness, real-time traffic flow, and road curvature left after removing the two fixed items of altitude and longitude and latitude coordinates from the above seven parameters. According to the values ​​updated in time on the day and the preset safety range value, the environmental classification level of the corresponding highway section is dynamically adjusted. The environmental classification includes a total of four levels, from low to high in terms of driving danger, they are the fourth level environment, the third level environment, the second level environment, and the first level environment.

[0037] For the fourth level environment, no additional supplement is required based on the existing data collection, that is, if the environment of a certain highway section on that day is classified as the fourth level environment, the first data set is the second data set, which is the object data of vehicle detection and identification in subsequent steps. In this embodiment, the fourth level environment is that the updated values ​​of the four change items on that day are all within the safe range.

[0038] For the third-level environment, additional supplements are required on the basis of existing data collection, that is, if the environment of a certain highway section on that day is classified as a third-level environment, it is necessary to use the existing collection equipment to collect data again in the key section, and add this part of the re-collected data to the original first data set to form a second data set. The second data set is the object data of vehicle detection and identification in subsequent steps. In this embodiment, the third-level environment is that one of the four change items has a value updated on the day that is not within the safe range. The length of the key section of the third-level environment is more than one-tenth of the length of a certain highway section. Among them, the key section is one or more high-risk areas in a certain highway section where traffic accidents are most likely to occur. A certain highway section refers to a continuous section between any two highway ramps.

[0039] For the second-level environment, additional supplements are required on the basis of existing data collection. That is, if the environment of a certain highway section on that day is classified as a second-level environment, it is necessary to use the existing collection equipment to collect data again at the key section, and add this part of the re-collected data to the original first data set to form a second data set. The second data set is the object data of vehicle detection and identification in subsequent steps. In this embodiment, the second-level environment is that the values ​​of the day updated by 2 of the 4 change items are not within the safe range. The length of the key section of the second-level environment is more than one-eighth of the length of a certain highway section.

[0040] For the first-level environment, additional supplements are required on the basis of existing data collection. That is, if the environment of a certain highway section on that day is classified as the first-level environment, it is necessary to use the existing collection equipment to collect data again at the key section, and add this part of the re-collected data to the original first data set to form a second data set. The second data set is the object data of vehicle detection and identification in subsequent steps. In this embodiment, the first-level environment is that 3 of the 4 change items have updated values ​​on the day that are not within the safe range. The length of the key section of the first-level environment is more than one-sixth of the length of a certain highway section.

[0041] Visibility, road slipperiness, real-time traffic flow, and road curvature are measured and updated daily using existing equipment and measurement methods. Among them, visibility can be measured using a forward scattering visibility meter or a laser visibility meter. Visibility measurements in severe weather such as fog and rainstorms can automatically record data every 5 minutes. Road slipperiness can be measured using a friction coefficient meter or a road humidity sensor, and data is uploaded at a set time interval of 10 minutes. Real-time traffic flow is statistically analyzed using ETC data, and data is uploaded every 1 hour. Road curvature is collected through on-board measurement equipment. High-precision inertial measurement units (IMUs) and global navigation satellite systems (GNSS) are installed on vehicles. During vehicle driving, these devices can measure the vehicle's position, posture, and motion status in real time. By processing and analyzing these data, the curvature of the vehicle's driving path is calculated, thereby obtaining road curvature data. Road curvature is used in daily road inspections or specific measurement tasks. The frequency of data collection is set as needed, and data is collected every 2 seconds.

[0042] This embodiment performs data collection and classification processing according to the actual environmental conditions of a certain highway section and the difficulty of high-speed driving of vehicles according to the current environmental classification level. For situations where the driving difficulty is high and the risk is relatively high, that is, the third-level environment, the second-level environment, and the first-level environment, the first data obtained by routine collection is collected again and supplemented to obtain the second data; while for the fourth-level environment within the safety range, no additional supplementary data is required, and the first data obtained by routine collection can be used. This embodiment allows the processing accuracy and the processing data volume to be taken into account from the beginning of data collection, and can accurately determine which situations require additional supplementary collection and which situations do not. For key sections that require additional supplementary collection, the length can also be accurately controlled. On the premise of achieving recognition accuracy, the length of the recognition section will not be increased.

[0043] (ii) Data screening: performing effective data screening and image quality assessment on the second data in sequence to obtain third data.

[0044] Effective data screening: According to the current environmental classification of a certain highway section, a picture is captured every N frames of the collected second data to obtain screened data to reduce the amount of data and improve processing efficiency. At the same time, images with obvious vehicle features and clear driving directions are screened out, and images that cannot be accurately detected due to vehicle occlusion, poor angles, etc. are removed. Among them, N is 6 for the fourth-level environment, 4 for the third-level environment, 3 for the second-level environment, and 2 for the first-level environment.

[0045] Image quality assessment: Perform quality assessment on the video images in the screening data. According to convention, select images with good quality through indicators such as image clarity, contrast, and degree of unobstructedness. Use image enhancement algorithms to process blurry or poorly lit images to improve image quality and form third data.

[0046] (iii) Data labeling: use professional labeling tools, such as LabelImg, to label the third data image to obtain preliminary image data. During labeling, vehicles are divided into three categories: cars, buses, and trucks in accordance with existing regulations, and the location, size, direction of travel, and other information of the vehicles are accurately marked. To ensure the accuracy of the labeling, this embodiment adopts a multi-person labeling and cross-validation method. Different labelers label the same image, and then compare and check the labeling results, discuss and correct any inconsistencies, and finally determine the accurate labeling results.

[0047] In the second step, a vehicle detection model is selected according to the current level of the environmental classification of a certain highway section. If the current level of the environmental classification is the fourth level environment, an existing vehicle detection model is used for detection, such as the YOLOv3 target detection model; if the environmental classification is a specific classification, an improved detection model is used.

[0048] (i) Select the YOLOv8n target detection model as the basic model, and replace the backbone network of the basic model to form the first model.

[0049] Network structure: Replace the backbone network of YOLOv8n with ECA_ESNet. ECA_ESNet is improved on the basis of ShuffleNet V2. It introduces SE module and Ghost module in GhostNet, and uses a lot of deep separable convolution to extract and fuse information from different channels. YOLOv8n has a good balance between detection speed and accuracy, and can meet the real-time requirements of highway vehicle detection. After selecting YOLOv8n as the basic model, by replacing the backbone network with ECA_ESNet, the model is more efficient in feature extraction, can capture key features in complex environments, and has stronger robustness and generalization ability in model performance, which is more suitable for target recognition in complex environments.

[0050] Parameter optimization: In ECA_ESNet, the structure of ESNet Block is optimized. ESNet Block is divided into two types: stride 1 and stride 2, with a total of four branches, consisting of depthwise separable convolution, Ghost Block, SEBlock and channels shuffle modules. By adjusting the parameters of these modules, such as convolution kernel size, step size, etc., the feature extraction capability and lightweight performance of the network are improved. The convolution kernel size in this embodiment is 3x3 and the step size is 2, so that the optimized ESNet Block and ECA_ESNet are more conducive to accurately identifying and detecting vehicles on highways under the premise of limited computational effort.

[0051] (ii) The attention mechanism is introduced to form the second model.

[0052] 1. ECA attention mechanism: In ECA_ESNet, the ECA attention mechanism is introduced to dynamically adjust the relationship between channels, so that the model can pay more attention to important feature information and improve the accuracy of feature extraction.

[0053] In this embodiment, the ECA module input size is , the feature map with the number of channels C, firstly performs global average pooling to compress the feature map size to 1*1. Then, a one-dimensional convolution with a convolution kernel size of K is performed. The function relationship between the value of K and the number of channels C is: ,in , c is the number of channels.

[0054] The feature map after one-dimensional convolution is further activated to obtain the weight of each channel of the feature map X. Finally, the weight is multiplied by the initial feature map X, that is, the weight is used as an excitation on X to obtain the final result X'.

[0055] The ECA attention mechanism is a lightweight attention mechanism module that is optimized based on the SE attention mechanism. It uses efficient one-dimensional convolution to achieve local cross-channel interaction, thereby improving feature extraction capabilities. At the same time, an adaptive channel dimension function is used to determine the size of the one-dimensional convolution kernel, allowing the model to more accurately identify and weight important features.

[0056] 2. SimAM attention mechanism: In the C2f module of YOLOv8n, the SimAM attention mechanism is embedded. This mechanism combines channel attention and spatial attention, which can better capture the deep feature space information of the vehicle and reduce the impact of interference features.

[0057] The SimAM module of this embodiment calculates the similarity between channels through convolution, then converts the similarity into weights and applies them to the original feature map.

[0058] To evaluate the importance of each neuron, and The values ​​are 1 and -1, and regularization is added to the energy function. Where t represents the index of the neuron, w t and b t are the weights and biases of neuron transformation, y t and y0 take the values ​​1 and -1 respectively, which are used for the calculation of energy function. i are the neighboring neurons of the input feature, and M is the total number of neurons set based on the channel dimension of the current feature map.

[0059] Analytical solution by calculation , , in, , , Get the weights of the neurons.

[0060] Finally, through

[0061] The processed feature map is obtained, where E is the sum of all energies.

[0062] SimAM is a parameter-free 3D attention mechanism that combines channel attention and spatial attention mechanisms. It introduces the concept of neurons, infers three-dimensional weights for the current neurons, and reversely optimizes the neurons, thereby assigning unique weights to neurons in each channel and spatial position, and flexibly extracting features.

[0063] (III) Feature fusion improvement: In the second model, BiFPN is used to fuse feature maps at different levels to form an improved detection model. Through weighted feature fusion and bidirectional connection, cross-scale connection is achieved, and the detailed information of shallow features and the semantic information of deep features are fully utilized to improve the model's detection capabilities for small and large targets.

[0064] BiFPN is an improvement on the traditional feature pyramid network (FPN). By adding weighted feature fusion and bidirectional connections, a bidirectional channel is constructed to achieve cross-scale connections.

[0065] This embodiment sets up an improved detection model and collects data for different environmental classifications, so that the model can maintain high detection accuracy and robustness while reducing the amount of calculation, and better meet the real-time requirements of highway vehicle detection and target recognition needs in complex environments.

[0066] In the third step, the improved detection model is used to process the preliminary image data of specific categories, detect and identify vehicles on the highway, and feedback the detection results.

[0067] This embodiment uses the ECA attention mechanism, embeds the SimAM attention mechanism in the C2f module, and optimizes the ESNet Block structure when improving the YOLOv8n model. These improvement measures can help the model capture features more accurately and improve detection accuracy.

[0068] This embodiment integrates different improved modules (such as attention mechanism, optimized network structure, etc.), thereby improving detection accuracy and reducing computational complexity without significantly increasing computational complexity. This integration method enables each module to cooperate with each other, giving full play to their respective advantages, and bringing better results for highway vehicle detection.

[0069] Embodiment 2 Different from the first embodiment, after the vehicle is identified and detected by the improved detection model, the present embodiment also calculates the vehicle's motion trajectory by a multi-target tracking algorithm, and then judges the vehicle's motion trajectory to see whether the vehicle is driving in violation of regulations or has any abnormal conditions.

[0070] The multi-target tracking algorithm in this embodiment: the Byte Track algorithm is used to track multiple targets of the vehicle. The Byte Track algorithm is simple and efficient, and can quickly and accurately track the movement trajectory of the vehicle.

[0071] 1. Constructing the correlation matrix Distance feature calculation 1. Anchor point distance feature: Calculate the Euclidean distance between the anchor point of the tracked target in the previous frame and the anchor point of the newly detected target in the current frame in the preliminary image data, and penalize it according to the target's movement speed to improve the calculation accuracy of the anchor point distance and avoid the influence of the target's displacement outside the speed direction vector on the matching result.

[0072] 2. Bounding box direction angle distance feature: Calculate the cosine similarity between the direction vector formed by the anchor point of the target tracked in the previous frame in the preliminary image data and the anchor point of the target detected in the current frame and the direction vector of the target movement in the previous frame, define the direction distance based on the cosine similarity, and determine whether the target's movement direction is consistent.

[0073] 3. Bounding box size distance feature: Calculate the distance between the bounding box size and the main direction of the tracked target in the previous frame and the detected target in the current frame in the preliminary image data, and perform different calculation methods according to the size of the main direction angle to ensure that the similarity of the target can be accurately judged even in the presence of occlusion.

[0074] 4. Point cloud quantity difference distance feature: Calculate the point cloud number difference between the tracked object and the new object, and perform normalization to obtain the point cloud quantity difference distance, which reflects the change in the number of targets.

[0075] 5. Center of mass offset distance feature: Calculate the Euclidean distance between the two-dimensional center of mass coordinates of two objects as the center of mass offset distance feature to determine the position change of the target.

[0076] 5. Bounding box IOU distance feature: Calculate the IOU of the tracking target bounding box of the previous frame of the preliminary image data and the new target bounding box of the current frame, and calculate the distance between the two bounding boxes based on the IOU as the bounding box IOU distance feature.

[0077] Correlation value calculation: The distances of the above six dimensions are weighted and summed according to the set weights to obtain the final correlation value. The smaller the correlation value, the more likely the two targets are the same object.

[0078] (II) Multi-target matching: The Hungarian Algorithm is used to solve the correlation matrix and achieve multi-target matching. In the matching process, each target is regarded as a vertex, and the matching relationship between targets is regarded as the edge connecting the vertices. By finding the maximum weighted match, the newly detected target in the current frame is optimally matched with the tracked target in the previous frame.

[0079] (III) Abnormal behavior detection: After the Byte Track algorithm obtains the motion trajectory and feature information of each target vehicle, it conducts speeding detection, wrong-way detection, parking detection and compaction line detection on each target vehicle in accordance with existing regulations to determine whether the target vehicle has abnormal behavior and whether it has violated the regulations, and then provides feedback on the detection and judgment results.

[0080] Speeding detection: The vehicle speed is calculated by calculating the frame difference between the vehicle entering and leaving the detection area, combined with the video frame rate and the actual distance of the detection area. The calculated speed is compared with the speed limit to achieve speeding detection.

[0081] Reverse detection: Determine whether the vehicle is traveling in the opposite direction based on the vehicle's up and down direction and direction of movement. In the first frame of the video, select two points on the central isolation belt to construct a direction vector, and calculate the cross product of the vehicle's center of mass and the direction vector. Then, by obtaining the continuous center of mass coordinates of the vehicle, construct a movement direction vector, calculate the angle with the direction vector of the central isolation belt, and comprehensively determine whether the vehicle is traveling in the opposite direction.

[0082] Parking detection: Get the center of mass coordinates of the vehicle in real time, and determine whether the center of mass coordinates are getting smaller and smaller in consecutive frames. If the conditions are met, the vehicle is considered to be parked.

[0083] Solid line detection: grayscale transformation of the input video image is performed, and the straight line segment is detected using the LSD algorithm. By judging the distance between the vehicle's position information and the detected solid line, it is determined whether the vehicle has crossed the solid line.

[0084] Through the added ECA attention mechanism, SimAM attention mechanism, ESNet Block structure optimization, multi-target tracking and abnormal behavior recognition, these improvements have integrated the advantages of multiple modules without adding too much extra computational effort, but can improve the detection accuracy of highway vehicle detection methods and systems based on computer vision, while reducing the amount of computation. It comprehensively considers factors such as power consumption, network communication conditions, driving abnormalities, road abnormalities, and possible natural disasters on highways, providing strong support for the safe operation of highways.

[0085] Embodiment 3 This embodiment is based on the highway vehicle detection method in the first embodiment, and provides a highway vehicle detection system for implementing the method. The highway vehicle detection system based on computer machine vision in this embodiment includes: The section judgment module is used to determine the latitude and longitude coordinates and altitude of the current position according to a certain highway section (i.e., the current highway section) and the preset environmental classification fixed items, and read the current value of the change item in the environmental classification updated on the same day according to the various locations pre-recorded on the certain highway section planned to drive, whether it is within the safe range value, so as to judge the environmental classification level of the current highway section; wherein, the environmental classification levels are fourth-level environment, third-level environment, second-level environment, and first-level environment from low to high in terms of driving difficulty, driving difficulty, and danger level, wherein the third-level environment to the first-level environment are specific classifications; The data acquisition module is used to collect images or video data of the highway according to the determined environmental classification level. The data acquisition module includes high-definition cameras mounted on drones and vehicles and high-intensity cameras fixed on the roadside. The cameras have the ability to collect clear images under different weather and lighting conditions. If the environment is classified as a specific classification, in addition to routine data collection, additional data collection of key sections is required to supplement it. If the environment is classified as a fourth-level environment, routine data collection is only required. A data preprocessing module is connected to the data acquisition module and is used to screen, annotate and enhance the collected image or video data, remove blurred, severely blocked or poor quality images, screen according to the rule of capturing one picture every N frames, and use professional annotation tools to annotate information such as vehicle category, location, size, and driving direction; as the environmental classification level increases, the value of N gradually decreases; The target detection module is connected to the data preprocessing module. If the environment is judged to be a fourth-level environment, the existing conventional vehicle detection method is used for subsequent detection. If the environment classification is judged to be a specific classification, an improved detection model is used for vehicle detection. The improved detection model is based on YOLOv8n, and its backbone network is replaced with ECA_ESNet, and the ECA and SimAM attention mechanisms are introduced, and the weighted bidirectional feature pyramid BiFPN is used for feature fusion; The tracking and detection module is connected to the target detection module. It adopts a multi-target tracking algorithm based on Byte Track. It constructs a correlation matrix including anchor point distance features, bounding box direction angle distance features, bounding box size distance features, point cloud quantity difference distance features, centroid offset distance features and bounding box IOU distance features, and uses the Hungarian Algorithm algorithm to perform multi-target matching. The abnormal behavior detection module is connected to the tracking module and is used to detect abnormal behaviors of the vehicle, including speeding, reverse driving, parking, and compacting the line, etc. The speed is calculated based on the time and distance the vehicle passes through the preset detection area to determine speeding; the reverse driving is determined by using the vector cross product property and the angle between the moving direction vector; the parking is determined based on the change in the coordinates of the vehicle's center of mass; the solid line of the road is detected using the LSD algorithm, and the compacting line is determined by the distance between the vehicle's center of mass and the solid line; The system optimization and communication module is connected to the above modules, optimizes the algorithm to reduce the amount of calculation, uses low-power hardware devices to reduce power consumption, establishes a local cache mechanism and multi-network fusion technology to ensure the stability of network communication, and encrypts the transmitted data to ensure security.

[0086] Except for the road section judgment module and the target detection module, the remaining modules of this embodiment can be replaced by existing functional modules. This solution can take into account both the amount of calculation and the detection accuracy. By giving priority to judging the environmental classification of the highway section, the complex actual environment is simplified in a targeted manner. For the environmental specific classification sections with relatively large data collection volume, the detection model is improved to accurately complete vehicle detection and identification without increasing the amount of calculation too much. Combined with the tracking detection module, the abnormal behavior detection module and the system optimization and communication module, it can not only effectively complete all the specified detection items, but also comprehensively provide energy-saving, stable, accurate and efficient optimal detection solutions.

[0087] The above is only an embodiment of the present invention, and the common knowledge such as the known specific technical solutions and / or characteristics in the solution is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the technical solution of the present invention, several modifications and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A highway vehicle detection method based on computer vision, characterized in that: The following steps are involved: Step 1: judging the road section environment classification, according to the location of the highway section, identifying the updated value of the environment classification on the day, and judging the environment classification level; the environment classification includes the fourth level environment, the third level environment, the second level environment and the first level environment from low to high driving danger, among which the third level environment, the second level environment and the first level environment belong to specific classifications; Step 2: Data collection and preprocessing. After data collection, data screening and data labeling, the collected data is preprocessed according to the different environmental classification levels of the highway section to obtain preliminary image data. Step 3: Model selection: select different vehicle detection models according to the different environmental classification levels of the highway section. If the environmental classification level is the fourth level, use the existing vehicle detection model for detection. If the environmental classification level is a specific classification, use the improved detection model. Step 4: detection feedback: detecting and identifying vehicles on the highway according to the selected model, and feeding back the detection results; The steps for constructing the improved detection model are as follows: First, select the YOLOv8n target detection model as the basic model and replace its backbone network with ECA_ESNet to form the first model; Then, the ECA attention mechanism and SimAM attention mechanism are introduced into the first model to form the second model; Finally, in the second model, BiFPN is used to fuse feature maps at different levels to form an improved detection model.

2. The highway vehicle detection method based on computer vision according to claim 1 is characterized in that: The data collection step comprises: Using highway data collection equipment to collect original data to obtain first data; The environmental classification includes seven parameter items, namely, altitude, longitude and latitude coordinates, visibility, road slipperiness, real-time traffic flow, and road curvature. After removing the two fixed items, namely, altitude and longitude and latitude coordinates, the environmental classification level of the corresponding expressway section is dynamically adjusted according to the values ​​of the four changing items, namely, visibility, road slipperiness, real-time traffic flow, and road curvature, which are updated in time on the day and compared with the preset safety range values. If the environment is classified as the fourth level environment, the first data is equal to the second data; if the environment is classified as the third level environment, the second level environment or the first level environment, the data is collected again at the key sections to obtain the re-collected data, and the re-collected data is added to the first data to obtain the second data.

3. The highway vehicle detection method based on computer vision according to claim 2, characterized in that: The third-level environment is that the updated value of one of the four change items on the day is outside the safe range, and the length of the critical section is more than one-tenth of the length of the corresponding highway section; the second-level environment is that the updated values ​​of two of the four change items on the day are outside the safe range, and the length of the critical section is more than one-eighth of the length of the corresponding highway section; the first-level environment is that the updated values ​​of three of the four change items on the day are outside the safe range, and the length of the critical section is more than one-sixth of the length of the corresponding highway section; the highway section refers to the continuous section between any two highway ramps planned or currently being driven, and the critical section is one or more high-risk areas in the highway section where traffic accidents are most likely to occur.

4. The highway vehicle detection method based on computer vision according to claim 2 is characterized in that: The data screening step includes: Effective data screening: According to the current environment classification of the highway section, a picture is captured every N frames of the collected second data to obtain the screening data. At the same time, images with obvious vehicle features and clear driving directions are screened out, and images that cannot be accurately detected due to vehicle occlusion and poor angles are removed; among them, N for the fourth-level environment is 6, N for the third-level environment is 4, N for the second-level environment is 3, and N for the first-level environment is 2; Image quality assessment: Perform quality assessment on the video images in the filtered data, select images with good quality through image clarity, contrast, and degree of unobstructedness, and use existing image enhancement algorithms to process blurred or poorly lit images to form third data.

5. The method for detecting highway vehicles based on computer vision according to claim 4, characterized in that: The data annotation step is: using a professional annotation tool to annotate the third data image to obtain preliminary image data, dividing vehicles into three categories: cars, buses, and trucks during annotation, and marking the location, size, and travel direction of the vehicles.

6. The highway vehicle detection method based on computer vision according to claim 1, characterized in that: In ECA_ESNet, ESNet Block is divided into two types: stride 1 and stride 2, with a total of four branches, consisting of depthwise separable convolution, Ghost Block, SE Block and channels shuffle modules, with a convolution kernel size of 3x3 and a stride of 2.

7. The highway vehicle detection method based on computer vision according to claim 1, characterized in that: The processing process of the ECA attention mechanism is: for input size , the feature map with the number of channels C, firstly perform global average pooling to compress the feature map size to 1*1, and then perform one-dimensional convolution with the convolution kernel size K. The function relationship between the value of K and the number of channels C is ,in , c is the number of channels; The feature map after one-dimensional convolution is activated to obtain the weight of each channel of the feature map X, and the weight is multiplied by the initial feature map X to obtain the final result X'.

8. The method for detecting highway vehicles based on computer vision according to claim 1, characterized in that: The processing process of the SimAM attention mechanism is as follows: through the energy function To evaluate the importance of each neuron and add regularization to the energy function; where t represents the index of the neuron, w t and b t are the weights and biases of neuron transformation, y t and y0 take the values ​​1 and -1 respectively, which are used for the calculation of energy function. i is the neighboring neuron of the input feature, and M is the total number of neurons set based on the channel dimension of the current feature map; Compute analytical solution ， ， in, ， ， Get the weight of the neuron; Finally, through The processed feature map is obtained, where E is the sum of all energies.

9. A highway vehicle detection system based on computer vision, characterized in that: The method for detecting vehicles on a highway based on computer vision as claimed in claim 1 comprises: The road section judgment module is used to identify the updated value of the environmental classification on the day according to the location of the highway section and judge the environmental classification level; The data acquisition module is connected to the road section judgment module, and collects image or video data of the highway according to the environmental classification level obtained by judgment; if the environmental classification is a specific classification, in addition to the first data obtained by routine data collection, additional data collection of key sections is required to collect data again to supplement the data and obtain second data; The data preprocessing module is connected to the data acquisition module, and screens the image or video data of the second data according to the rule of capturing one picture every N frames, and then selects images with good quality through image quality assessment, and enhances blurred or poorly lit images to obtain third data; the third data is annotated with vehicle type, position, size, and driving direction to obtain preliminary image data; wherein the N value decreases as the environmental classification level increases; The target detection module is connected to the data preprocessing module. If the environment classification level is the fourth level environment, the existing vehicle detection model is used for detection; if the environment classification is a specific classification, an improved detection model is used; the improved detection model is based on YOLOv8n, and its backbone network is replaced with ECA_ESNet, the ECA and SimAM attention mechanisms are introduced, and the weighted bidirectional feature pyramid BiFPN is used for feature fusion.

Images