A method and system for counting traffic flow at intersections based on monocular camera

Abstract

The present invention provides a method and system for counting vehicle flow at an intersection based on a monocular camera, and belongs to the field of image processing technology. The method comprises: collecting video data, wherein the video data comprises a plurality of continuous video frames; setting the number of lanes and the position of lanes in the video frame according to preset lane information; inputting the set video frame into a vehicle target detection model based on Yolov7‑tiny for detection, and outputting a detection frame of the vehicle; inputting the vehicle image in the detection frame into a vehicle detail recognition model based on an improved Resnet‑18 for detection, and outputting a ground projection of the vehicle; according to the detection frame, outputting a vehicle tracking trajectory through a vehicle tracking algorithm based on ByteTrack; and performing traffic statistics on the vehicles in each lane according to the lane position, the ground projection of the vehicle, and the vehicle tracking trajectory.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for counting traffic flow at an intersection based on a monocular camera. Background Art

[0002] With the rapid development of deep learning technology, its application in the fields of video surveillance and intelligent transportation has become a trend. In particular, in the intelligent transportation system, vehicle recognition technology, as one of the core functions, can realize multiple functions such as traffic flow statistics, intelligent control of traffic signals, congestion warning, and cross-line warning. Vehicle recognition technology based on deep learning can greatly simplify the algorithm process, improve the performance and efficiency of the algorithm, and thus promote the intelligent development of the intelligent transportation system.

[0003] Existing traffic flow statistics methods usually use the CamShift algorithm for target tracking, which relies on color histograms to describe targets and tracks them through adaptive windows. However, the CamShift algorithm has some shortcomings when tracking targets: first, it lacks the necessary template update mechanism; second, due to the fixed window width, tracking will fail when the target scale changes; third, when the target speed is fast, the tracking effect is not good; fourth, the histogram features lack spatial information when describing the target color features.

[0004] In addition, conventional methods often use electronic counters to count vehicles, which results in higher costs. Summary of the invention

[0005] The main purpose of the present invention is to provide a method and system for counting traffic flow at intersections based on a monocular camera, so as to solve the problems existing in the prior art when tracking vehicle targets, such as lack of necessary template update mechanism, tracking failure caused by target scale change, poor tracking effect when the target moves at a fast speed, lack of spatial information in histogram features when describing target color features, and high cost caused by using electronic counters for vehicle counting.

[0006] In order to achieve the above object, according to one aspect of the present invention, a method for counting traffic flow at an intersection based on a monocular camera is provided, comprising:

[0007] S1: collecting video data, where the video data includes a plurality of continuous video frames;

[0008] S2: setting the number of lanes and the positions of lanes in the video frame according to preset lane information;

[0009] S3: Input the set video frame into the vehicle target detection model based on Yolov7-tiny for detection, and output the vehicle detection frame;

[0010] S4: inputting the vehicle image in the detection frame into a vehicle detail recognition model based on an improved Resnet-18 for detection, and outputting the vehicle ground projection;

[0011] S5: Outputting a vehicle tracking trajectory according to the detection frame through a vehicle tracking algorithm based on ByteTrack;

[0012] S6: Perform flow statistics on vehicles in each lane according to the lane position, the vehicle ground projection and the vehicle tracking trajectory.

[0013] Furthermore, the S4 specifically includes:

[0014] S401: Replace some pooling layers in the backbone network of Resnet-18;

[0015] S402: The head part of Resnet-18 is formed by stacking multiple convolutional layers, and a vehicle detail recognition model based on the improved Resnet-18 is constructed;

[0016] S403: inputting the vehicle image in the detection frame into the vehicle detail recognition model for detection, and outputting the vehicle wheel line;

[0017] S404: Generate the ground projection of the vehicle according to the wheel line of the vehicle and the bottom edge of the detection frame.

[0018] Furthermore, the S403 specifically includes:

[0019] S4031: capturing a vehicle image in a video frame according to the detection frame;

[0020] S4032: Adjusting the vehicle image to a target size;

[0021] S4033: inputting the resized vehicle image into the vehicle detail recognition model for detection, and outputting vehicle viewing angle information and wheel line coordinate information;

[0022] S4034: Constructing a vehicle wheel line according to the vehicle viewing angle information and the wheel line coordinate information.

[0023] Furthermore, the vehicle detail recognition model is trained in the following manner:

[0024] With the goal of minimizing the error between the predicted wheel line coordinates and the actual wheel line coordinates, a mean square error loss function is constructed to train the vehicle detail recognition model.

[0025] Furthermore, the S5 specifically includes:

[0026] S501: when the track is empty and the confidence of the detection box exceeds a first preset confidence, a first track is created, the tracking state of the first track is Tracked, and the activation state is True;

[0027] S502: when the confidence of the detection box that does not match any track exceeds a second preset confidence, a second track is created, and the tracking state of the second track is Tracked, and the activation state is False;

[0028] S503: Merge the tracks whose activation status is True and whose tracking status is Lost;

[0029] S504: Perform Kalman filter prediction on the current trajectory and perform vehicle tracking;

[0030] S505: updating the tracking status of the vehicle tracking trajectory of each vehicle;

[0031] S506: Outputting the updated vehicle tracking trajectory of each vehicle.

[0032] Further, the S504 specifically includes:

[0033] S5041: Obtain the prediction box of each vehicle through Kalman filter prediction;

[0034] S5042: performing a first match between the detection box whose confidence is greater than the first preset confidence and the prediction box;

[0035] S5043: performing a second match on the prediction box that failed to be matched successfully for the first time and the detection box whose confidence is less than the first preset confidence;

[0036] S5044: Perform a third match on the prediction box whose activation state is False and the detection box whose first match was unsuccessful and whose confidence is greater than the second preset confidence.

[0037] Furthermore, the S505 specifically includes:

[0038] S5051: when the duration of the tracking state of the track being Lost exceeds a preset duration, the tracking state of the track is changed from Lost to Removed, and the track is deleted;

[0039] S5052: When the tracking state of a track changes from Lost to Tracked or Removed, duplicate tracks with the same ID are deleted.

[0040] Furthermore, after S6, the method further includes:

[0041] S7: When the vehicle is in multiple lanes at the same time, according to the area ratio of the ground projection of the vehicle on each lane, the lane with the largest area ratio is used as the lane where the vehicle is located, and a crossing lane warning signal is issued;

[0042] S8: Update lane information through the SuperPoint model.

[0043] Furthermore, the S8 specifically includes:

[0044] S801: extracting feature points of two video frames respectively through the SuperPoint model;

[0045] S802: Calculating the homography matrix of the two video frames according to the feature points;

[0046] S803: Perform perspective transformation on the lane coordinates according to the homography matrix, and update the lane information to correct the coordinates of the camera offset.

[0047] According to one aspect of the present invention, a traffic flow statistics system for intersections based on a monocular camera is provided, comprising:

[0048] memory and one or more processors;

[0049] One or more application programs are stored in the memory, and the one or more application programs are suitable for being executed by the one or more processors to implement the above-mentioned monocular camera-based intersection traffic flow statistics method.

[0050] By applying the technical solution of the present invention, the set video frame is input into a vehicle target detection model based on Yolov7-tiny for detection, the detection frame of the vehicle is output, and according to the detection frame, the vehicle tracking trajectory is output through a vehicle tracking algorithm based on ByteTrack, which provides a necessary template update mechanism when tracking the vehicle target, and can provide a better tracking effect when facing a target with a faster moving speed. The vehicle image in the detection frame is input into a vehicle detail recognition model based on an improved Resnet-18 for detection, and the vehicle ground projection is output. It can adapt to targets of different scales and will not cause tracking failure due to changes in the target scale. The vehicle features can also be extracted, which makes up for the problem that the histogram features lack spatial information when describing the target color features. According to the lane position, the vehicle ground projection and the vehicle tracking trajectory, the vehicles on each lane are counted for traffic, and the video data is fully utilized for vehicle detection and counting, which avoids the investment of additional hardware, thereby greatly reducing the cost.

[0051] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0053] Figure 1 A schematic diagram of a flow chart of a method for counting traffic flow at an intersection based on a monocular camera provided in an embodiment of the present invention is shown;

[0054] Figure 2 A schematic structural diagram of a monocular camera-based intersection traffic flow statistics system provided in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0055] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0056] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0057] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the present invention described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0058] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0059] Reference Manual Attached Figure 1 , shows a flow chart of a method for counting traffic flow at an intersection based on a monocular camera provided in an embodiment of the present invention.

[0060] Reference Manual Attached Figure 2 , shows a structural schematic diagram of a traffic flow statistics system for an intersection based on a monocular camera provided in an embodiment of the present invention.

[0061] The embodiment of the present invention provides a method for counting traffic flow at an intersection based on a monocular camera, comprising:

[0062] S1: Collect video data, where the video data includes a plurality of continuous video frames.

[0063] Specifically, a monocular camera is used to collect multiple continuous video frames at the intersection.

[0064] In the present invention, the use of a monocular camera to collect continuous video frames has the advantages of simplified equipment configuration, reduced cost, real-time data acquisition, strong adaptability and data processing consistency. This method can provide a stable and reliable solution for traffic monitoring by reducing hardware complexity and cost while maintaining efficient real-time traffic data acquisition and processing capabilities.

[0065] S2: According to the preset lane information, the number of lanes and the lane positions are set in the video frame.

[0066] It should be noted that the preset lane information specifically includes the ID number of each lane and the coordinates of the area corner points.

[0067] In the present invention, by presetting lane information (such as ID serial number and regional corner point coordinates), the position and range of each lane can be clearly defined, thereby improving the accuracy of vehicle detection and traffic statistics. The preset lane information provides a standardized lane definition, ensuring the consistency and reliability of lane detection under different times and conditions. By presetting the position and ID of the lane, the vehicle lane allocation and traffic statistics can be quickly performed, reducing the computational burden during dynamic detection.

[0068] S3: Input the set video frame into the vehicle target detection model based on Yolov7-tiny for detection, and output the vehicle detection frame.

[0069] Among them, Yolov7-tiny is an efficient target detection model and a lightweight version of the YOLO (You Only Look Once) series. It aims to improve detection speed and efficiency while maintaining good detection accuracy by optimizing the network structure and reducing the number of model parameters. YOLOv7-tiny is particularly suitable for devices with limited computing resources or application scenarios that require fast real-time processing. It can quickly locate the detection box (Bounding Box) of the target object in the image and classify and identify it.

[0070] In the present invention, Yolov7-tiny is a lightweight version of the YOLO series, which optimizes the network structure and reduces the model parameters, so that it can run efficiently on devices with limited computing resources. This makes real-time processing of video data more feasible. Yolov7-tiny can quickly locate the target object in the image and provide a real-time detection frame. Such speed is crucial for processing vehicle detection in continuous video frames, especially in application scenarios that require instant response, such as traffic monitoring. Yolov7-tiny optimizes computing efficiency, enabling it to perform well in application scenarios that require rapid processing of a large number of video frames, meeting the needs of real-time vehicle detection.

[0071] S4: Input the vehicle image in the detection frame into the vehicle detail recognition model based on the improved Resnet-18 for detection, and output the vehicle ground projection.

[0072] Among them, ResNet-18 is a deep convolutional neural network (CNN) architecture, belonging to the ResNet (ResidualNetwork) series. It solves the gradient vanishing and degradation problems in deep network training by introducing "residual connections", allowing the network to be trained and learned more effectively. ResNet-18 consists of an 18-layer deep network, including multiple residual blocks, each of which passes the input directly to the output through a shortcut connection, thereby enhancing the network's expressiveness and training stability. This architecture is particularly suitable for computer vision tasks such as image classification and target detection.

[0073] In the present invention, the deep convolutional network architecture of ResNet-18 can effectively extract detail features in the image. In vehicle detail recognition, the wheel line and other important features of the vehicle can be accurately captured to improve the accuracy of ground projection. The deep network and residual connection of ResNet-18 enable the model to adapt to targets of different scales, ensuring the accuracy and robustness of vehicle detail recognition under various vehicle speeds and distances. Vehicle detail recognition based on the improved ResNet-18 can not only provide accurate vehicle ground projection, but also better handle complex background and lighting conditions, thereby improving the reliability of vehicle detection and tracking.

[0074] In a possible implementation, S4 specifically includes sub-steps S401 to S404:

[0075] S401: Replace some pooling layers in the backbone network of Resnet-18.

[0076] S402: The head part of Resnet-18 is composed by stacking multiple convolutional layers, and a vehicle detail recognition model based on the improved Resnet-18 is constructed.

[0077] In the present invention, by replacing some pooling layers and rebuilding the network head, the improved ResNet-18 can better extract the detail features in the vehicle image. This enhances the model's ability to recognize details such as vehicle wheel lines, improves the accuracy of vehicle details, and can accurately extract the vehicle's detail features, including wheel lines and perspective information. This high-precision detail recognition is crucial to the accuracy of the vehicle's ground projection.

[0078] S403: Input the vehicle image in the detection frame into the vehicle detail recognition model for detection, and output the vehicle wheel line.

[0079] In a possible implementation, S403 specifically includes sub-steps S4031 to S4034:

[0080] S4031: Capturing a vehicle image in a video frame according to the detection frame.

[0081] S4032: Adjust the vehicle image to a target size.

[0082] Optionally, resize the vehicle image to 96×96 pixels.

[0083] In the present invention, the vehicle image is adjusted to the target size (such as 96×96 pixels) before being input into the model, which can unify the scale of the input data and reduce the impact of changes caused by different sizes on the model performance. This simplifies model training and application and improves processing efficiency.

[0084] S4033: Input the resized vehicle image into the vehicle detail recognition model for detection, and output the vehicle viewing angle information and wheel line coordinate information.

[0085] Specifically, the resized vehicle image is input into the vehicle detail recognition model for detection, and the vehicle view information and wheel line coordinate information are output through five times downsampling of Resnet-18.

[0086] The vehicle perspective information is [0,7], which means that the vehicle perspective information is divided into 8 discrete angle values, from 0 to 7, a total of 8 perspectives, that is, (0, 1, 2, 3, 4, 5, 6, 7) 7 perspective information, each of which corresponds to a different direction or angle of the vehicle under the camera's perspective. This discretized perspective information helps to accurately describe the vehicle's posture and direction, and provides necessary data for subsequent vehicle ground projection and other analyses.

[0087] The wheel line coordinate information includes the lower coordinate (X, Y) and upper coordinate (X, Y) of the vehicle wheel line, which define the position of the tire contact point line of the vehicle on the ground. The wheel line refers to the tire contact point line of the vehicle on the ground. When the vehicle is at an odd angle, the wheel line refers to the tire contact point line on the side of the vehicle.

[0088] In the present invention, by providing vehicle view information ([0,7]), the model can process vehicle images at different angles, which helps to accurately identify the vehicle wheel line and ground projection under multiple view conditions. The discretization of the view ensures an accurate description of the vehicle's posture and direction. The wheel line coordinate information output by the model, including the upper and lower coordinates of the wheel line, can accurately describe the tire contact point of the vehicle on the ground. These precise wheel line locations provide the necessary data for generating the vehicle ground projection, improving the accuracy of the overall detection.

[0089] Optionally, the vehicle detail recognition model is trained in the following manner:

[0090] With the goal of minimizing the error between the predicted wheel line coordinates and the actual wheel line coordinates, a mean square error loss function is constructed to train the vehicle detail recognition model.

[0091] Specifically, during the encoding process, the coordinate information of the vehicle wheel line is converted into a format that facilitates model learning and prediction. This process involves converting the coordinates of the wheel line into the distance from the center point of the vehicle image to the foot of the wheel line (i.e., the point perpendicular to the wheel line), which are the lateral and longitudinal offsets of the upper left corner of the vehicle image. When calculating the loss, the wheel line coordinate information output by the model is activated and transformed to obtain the polar coordinates of the foot of the wheel line, and then these polar coordinates are converted into Euclidean coordinates, and the mean square error (MSELoss) between the predicted coordinates and the actual coordinates is calculated. The mean square error loss function is used to measure the difference between the predicted value and the actual value, thereby optimizing the parameters of the model.

[0092] In this invention, the model is optimized by minimizing the error between the predicted wheel line coordinates and the actual wheel line coordinates, using the mean square error (MSELoss) loss function to ensure the accuracy of the model in vehicle detail recognition. The encoding and decoding process of the wheel line coordinates (from polar coordinates to Euclidean coordinates) enhances the learning ability and prediction accuracy of the model.

[0093] S4034: Construct the vehicle wheel line according to the vehicle viewing angle information and the wheel line coordinate information.

[0094] S404: Generate a ground projection of the vehicle according to the wheel line of the vehicle and the bottom edge of the detection frame.

[0095] In the present invention, the vehicle ground projection is generated based on the detected wheel line and the bottom edge of the detection frame, ensuring the accurate mapping of the vehicle position and the tire contact point. This is crucial for subsequent lane flow statistics and vehicle crossing warning.

[0096] S5: Output the vehicle tracking trajectory according to the detection box through the ByteTrack-based vehicle tracking algorithm.

[0097] Among them, ByteTrack is an advanced vehicle tracking algorithm designed to improve the accuracy and efficiency of target tracking. It tracks targets in video sequences based on target detection boxes and solves common problems in target tracking, such as target loss, occlusion, and confusion between similar targets. ByteTrack achieves high accuracy and real-time performance by combining efficient target detection models (such as the YOLO series) with optimized tracking algorithms. It uses technologies such as Kalman filters and Hungarian algorithms for prediction and matching, ensuring that the motion trajectory of each target can be stably tracked even in complex scenes.

[0098] In a possible implementation, S5 specifically includes sub-steps S501 to S506:

[0099] S501: When the track is empty and the confidence of the detection box exceeds a first preset confidence, a first track is newly created, and the tracking state of the first track is Tracked, and the activation state is True.

[0100] S502: When the confidence of the detection box that does not match any track exceeds a second preset confidence, a second track is newly created, and the tracking state of the second track is Tracked, and the activation state is False.

[0101] S503: Merge the tracks whose activation status is True and whose tracking status is Lost.

[0102] S504: Perform Kalman filter prediction on the current trajectory and perform vehicle tracking.

[0103] In a possible implementation, S504 specifically includes sub-steps S5041 to S5044:

[0104] S5041: Obtain the prediction box of each vehicle through Kalman filter prediction.

[0105] S5042: Perform a first match between the detection box whose confidence is greater than a first preset confidence and the prediction box.

[0106] S5043: Perform a second match on the predicted box that failed to be matched successfully at the first time and the detection box whose confidence is less than the first preset confidence.

[0107] S5044: Perform a third match on the prediction box whose activation state is False and the detection box whose first match was unsuccessful and whose confidence is greater than the second preset confidence.

[0108] S505: Update the tracking status of the vehicle tracking trajectory of each vehicle.

[0109] In a possible implementation, S505 specifically includes sub-steps S5051 to S5052:

[0110] S5051: When the duration of the tracking state of the track being Lost exceeds a preset duration, the tracking state of the track is changed from Lost to Removed, and the track is deleted.

[0111] S5052: When the tracking state of a track changes from Lost to Tracked or Removed, duplicate tracks with the same ID are deleted.

[0112] S506: Outputting the updated vehicle tracking trajectory of each vehicle.

[0113] It should be noted that those skilled in the art can set the first preset reliability, the second preset reliability and the preset duration according to actual needs, and the present invention is not limited thereto.

[0114] In the present invention, the detection frame is tracked using the ByteTrack algorithm, which can provide accurate vehicle trajectories. This precise tracking can effectively reduce false detection and missed detection and improve the overall tracking quality. By setting different confidence thresholds (first preset confidence and second preset confidence) and defining a preset duration, the system can flexibly adapt to different scene requirements. This flexibility enables the tracking algorithm to work efficiently in a variety of environments and handle various complex situations. A new trajectory is dynamically created according to the confidence of the detection frame, and the trajectory is predicted through Kalman filtering to ensure real-time tracking of the vehicle. This method can handle the occlusion and speed changes that may occur during the movement of the vehicle and provide continuous tracking data. Through multiple rounds of matching strategies (first matching, second matching, and third matching), the mismatch problem between the prediction frame and the detection frame is effectively handled, and the matching accuracy is improved. The use of Kalman filtering further enhances the ability to predict the vehicle's motion state. The management of the tracking state (such as the state from Lost to Removed) avoids repeated calculation and storage of the trajectory and optimizes the performance of the tracking system. By deleting invalid or duplicate trajectories, the system burden can be reduced and the processing efficiency can be improved.

[0115] S6: Count the vehicle flow in each lane based on the lane position, vehicle ground projection, and vehicle tracking trajectory.

[0116] Specifically, in traffic flow statistics, the system first determines the lane where each vehicle is located based on the preset lane position and the ground projection of the vehicle. The ground projection of the vehicle is obtained by mapping the wheel line information of the vehicle to the actual position on the ground. These projected positions are compared with the boundaries of the lane to determine whether the vehicle is located in a specific lane area. By analyzing the vehicle tracking trajectory, the system is able to track the movement path of each vehicle and count the number of vehicles in each lane. Specifically, the system records the total volume of traffic in each lane and updates the number of vehicles currently present in real time.

[0117] In the present invention, by comparing the ground projection of the vehicle with the preset lane position, the system can accurately assign each vehicle to the correct lane. This precise lane assignment ensures the accuracy of traffic flow statistics and reduces statistical errors caused by misjudgment of vehicle positions. Combining vehicle tracking trajectories and lane information, the system can update the number of vehicles in the lane in real time. This real-time update provides dynamic traffic flow data, which helps to understand and manage traffic conditions in a timely manner.

[0118] In a possible implementation manner, after S6, the method further includes:

[0119] S7: When a vehicle is in multiple lanes at the same time, the lane with the largest area ratio is used as the lane where the vehicle is located, based on the area ratio of the vehicle's ground projection on each lane, and a cross-lane warning signal is issued.

[0120] Optionally, the lane crossing warning signal includes: a red warning frame, which is used to mark the position of the vehicle with a red frame on the monitoring screen to show that the vehicle is crossing the lane line. A warning sound effect, which is used to issue a sound alarm to remind relevant personnel to pay attention to the situation where the vehicle crosses the lane line.

[0121] In the present invention, when a vehicle is in multiple lanes at the same time, the main lane of the vehicle is determined based on the area ratio of the ground projection on each lane. This method can more accurately assign vehicles to the correct lanes, thereby improving the accuracy of traffic flow statistics. By issuing a cross-lane warning signal, the situation of vehicles crossing the lane line can be discovered and handled in a timely manner. This is very important for monitoring and managing traffic anomalies, and helps to take measures as early as possible, such as adjusting traffic signals or conducting on-site intervention. By marking the vehicle position with a red warning box on the monitoring screen, relevant personnel can intuitively see the vehicle's position and the situation of crossing the lane line. This visual warning helps to identify and respond to traffic problems more quickly.

[0122] S8: Update lane information through the SuperPoint model.

[0123] Among them, the SuperPoint model is a deep learning model for feature point extraction and description. It can automatically detect significant feature points from an image and generate descriptors for these feature points. This model can not only efficiently identify key points in an image, but also find the correspondence between these points between images of different perspectives or scales. SuperPoint uses convolutional neural networks (CNNs) to detect and describe feature points, thereby providing high-precision feature point information in tasks such as image matching and object recognition. This makes it play an important role in the field of computer vision, especially in applications such as image registration and scene reconstruction.

[0124] In a possible implementation, S8 specifically includes sub-steps S801 to S803:

[0125] S801: extracting feature points of two video frames respectively through the SuperPoint model.

[0126] S802: Calculate the homography matrix of the two video frames according to the feature points.

[0127] S803: Perform perspective transformation on the lane coordinates according to the homography matrix, and update the lane information to correct the coordinates of the camera offset.

[0128] In the present invention, by extracting feature points through the SuperPoint model and calculating the homography matrix, perspective transformation can be performed accurately. This helps to accurately update the lane coordinates and ensure that the lane information is consistent with the actual video scene, thereby improving the accuracy of the lane information. Implementing a perspective transformation of the lane coordinates can effectively correct the coordinate offset caused by changes in camera position or viewing angle. This correction can ensure the consistency of lane information at different times and scenes, and reduce data deviations caused by camera movement or changes in installation angles. Regularly updating lane information can ensure the stability and reliability of the system in a dynamic environment. For example, changes in traffic flow, road conditions or other external factors may cause changes in lane information. By updating the lane information, the system can adapt to these changes and provide stable performance.

[0129] From the above description, it can be seen that the monocular camera-based intersection traffic flow statistics method provided by the present invention inputs the set video frame into the vehicle target detection model based on Yolov7-tiny for detection, outputs the vehicle detection frame, and outputs the vehicle tracking trajectory based on the detection frame through the vehicle tracking algorithm based on ByteTrack, provides the necessary template update mechanism when tracking the vehicle target, and can also provide better tracking effect when facing targets with faster moving speeds, inputs the vehicle image in the detection frame into the vehicle detail recognition model based on the improved Resnet-18 for detection, outputs the vehicle ground projection, can adapt to targets of different scales, will not cause tracking failure due to changes in target scale, and can also extract vehicle features, making up for the problem that the histogram feature lacks spatial information when describing the target color feature, and performs traffic statistics on the vehicles on each lane according to the lane position, the vehicle ground projection and the vehicle tracking trajectory, making full use of the video data for vehicle detection and counting, avoiding the investment in additional hardware, and thus greatly reducing the cost.

[0130] The embodiment of the present invention provides a monocular camera-based intersection traffic flow counting system 30 , including: a memory 303 and one or more processors 301 .

[0131] One or more application programs are stored in the memory 303 , and the one or more application programs are suitable for being executed by the one or more processors 301 to implement the monocular camera-based intersection traffic flow statistics method described in the method embodiment.

[0132] The monocular camera-based intersection traffic flow statistics system 30 includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302.

[0133] The structure of the monocular camera-based intersection vehicle flow statistics system 30 does not constitute a limitation on the embodiment of the present invention.

[0134] Processor 301 may be a CPU, a general purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0135] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI bus or an EISA bus, etc. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0136] The memory 303 can be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disk storage (including a compressed optical disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0137] The monocular camera-based intersection traffic flow counting system 30 provided in an embodiment of the present invention can implement each step in the above-mentioned monocular camera-based intersection traffic flow counting method and can obtain the same technical effect. To avoid repetition, the present invention will not go into details.

[0138] From the above description, it can be seen that in the monocular camera-based intersection traffic flow statistics system provided by the present invention, when the tested device receives the test message and the test video, it can synchronously process the received test message and the test video based on the change of the video frame number, and discard the repeated video frames. There is no need to receive all the physical quantities of the real vehicle data and the video data, which reduces the processor load and does not require a high-performance processor and its accessories. The cost of intersection traffic flow statistics based on the monocular camera is reduced, which is conducive to the application of intersection traffic flow statistics based on the monocular camera and improves vehicle safety.

[0139] Unless otherwise specifically stated, the relative arrangement of the parts and steps described in these embodiments, numerical expressions and numerical values ​​do not limit the scope of the present invention. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be regarded as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0140] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" may include both "above" and "below". The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0141] In the description of the present invention, it is necessary to understand that the directions or positional relationships indicated by directional words such as "front, back, up, down, left, right", "lateral, vertical, perpendicular, horizontal" and "top, bottom" are usually based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description. Unless otherwise specified, these directional words do not indicate or imply that the devices or elements referred to must have a specific direction or be constructed and operated in a specific direction. Therefore, they cannot be understood as limiting the scope of protection of the present invention. The directional words "inside and outside" refer to the inside and outside relative to the contours of each component itself.

[0142] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for counting traffic flow at intersections based on a monocular camera, characterized in that: include: S1: collecting video data, where the video data includes a plurality of continuous video frames; S2: setting the number of lanes and the positions of lanes in the video frame according to preset lane information; S3: Input the set video frame into the vehicle target detection model based on Yolov7-tiny for detection, and output the vehicle detection frame; S4: inputting the vehicle image in the detection frame into a vehicle detail recognition model based on an improved Resnet-18 for detection, and outputting the vehicle ground projection; S5: Outputting a vehicle tracking trajectory according to the detection frame through a vehicle tracking algorithm based on ByteTrack; S6: Performing traffic statistics on vehicles in each lane according to the lane position, the ground projection of the vehicle and the vehicle tracking trajectory; The S4 specifically includes: S401: Replace some pooling layers in the backbone network of Resnet-18; S402: The head part of Resnet-18 is formed by stacking multiple convolutional layers, and a vehicle detail recognition model based on the improved Resnet-18 is constructed; S403: inputting the vehicle image in the detection frame into the vehicle detail recognition model for detection, and outputting the vehicle wheel line; S404: generating a ground projection of the vehicle according to the wheel line of the vehicle and the bottom edge of the detection frame; The S403 specifically includes: S4031: capturing a vehicle image in a video frame according to the detection frame; S4032: Adjusting the vehicle image to a target size; S4033: inputting the resized vehicle image into the vehicle detail recognition model for detection, and outputting vehicle viewing angle information and wheel line coordinate information; S4034: Constructing a vehicle wheel line according to the vehicle viewing angle information and the wheel line coordinate information.

2. The method for counting traffic flow at intersections based on a monocular camera according to claim 1, characterized in that: The training method of the vehicle detail recognition model is specifically as follows: With the goal of minimizing the error between the predicted wheel line coordinates and the actual wheel line coordinates, a mean square error loss function is constructed to train the vehicle detail recognition model.

3. The method for counting traffic flow at intersections based on a monocular camera according to claim 1, characterized in that: The S5 specifically includes: S501: when the track is empty and the confidence of the detection box exceeds a first preset confidence, a first track is created, the tracking state of the first track is Tracked, and the activation state is True; S502: when the confidence of the detection box that does not match any track exceeds a second preset confidence, a second track is created, and the tracking state of the second track is Tracked, and the activation state is False; S503: Merge the tracks whose activation status is True and whose tracking status is Lost; S504: Perform Kalman filter prediction on the current trajectory and perform vehicle tracking; S505: updating the tracking status of the vehicle tracking trajectory of each vehicle; S506: Outputting the updated vehicle tracking trajectory of each vehicle.

4. The method for counting traffic flow at intersections based on a monocular camera according to claim 3, characterized in that: The S504 specifically includes: S5041: Obtain the prediction box of each vehicle through Kalman filter prediction; S5042: performing a first match between the detection box whose confidence is greater than the first preset confidence and the prediction box; S5043: performing a second match on the prediction box that failed to be matched successfully for the first time and the detection box whose confidence is less than the first preset confidence; S5044: Perform a third match on the prediction box whose activation state is False and the detection box whose first match was unsuccessful and whose confidence is greater than the second preset confidence.

5. The method for counting traffic flow at intersections based on a monocular camera according to claim 3, characterized in that: The S505 specifically includes: S5051: when the duration of the tracking state of the track being Lost exceeds a preset duration, the tracking state of the track is changed from Lost to Removed, and the track is deleted; S5052: When the tracking state of a track changes from Lost to Tracked or Removed, duplicate tracks with the same ID are deleted.

6. The method for counting traffic flow at intersections based on a monocular camera according to claim 1, characterized in that: After S6, the method further includes: S7: When the vehicle is in multiple lanes at the same time, according to the area ratio of the ground projection of the vehicle on each lane, the lane with the largest area ratio is used as the lane where the vehicle is located, and a crossing lane warning signal is issued; S8: Update lane information through the SuperPoint model.

7. The method for counting traffic flow at intersections based on a monocular camera according to claim 6, characterized in that: The S8 specifically includes: S801: extracting feature points of two video frames respectively through the SuperPoint model; S802: Calculating the homography matrix of the two video frames according to the feature points; S803: Perform perspective transformation on the lane coordinates according to the homography matrix, and update the lane information to correct the coordinates of the camera offset.

8. A traffic flow statistics system for intersections based on a monocular camera, characterized in that: include: memory and one or more processors; One or more applications are stored in the memory, and the one or more applications are suitable for being executed by the one or more processors to implement the intersection traffic flow statistics method based on a monocular camera as described in any one of claims 1 to 7.

Images