Traffic flow monitoring method based on automatic road detection and related equipment

Abstract

The invention discloses a traffic flow monitoring method based on automatic road detection and related equipment, and relates to the technical field of intelligent traffic, and the method comprises the steps: obtaining the video image data of a to-be-monitored road; based on the video image data, automatically identifying and segmenting at least one road area through a road detection model; determining a minimum enclosing rectangle of each road area; according to the minimum enclosing rectangle of each road area, generating at least one candidate virtual detection line in the corresponding road area; determining an effective virtual detection line from the generated candidate virtual detection lines; detecting and tracking a track of a vehicle in the video image data based on the video image data; and counting the traffic flow according to the interaction relationship between the track of the vehicle and the effective virtual detection line. According to the invention, a continuous and stable monitoring function can be maintained when the visual angle of the camera is changed or the camera faces a complex road structure, and high-quality traffic flow statistical data can be provided in various dynamic scenes such as urban complex intersections and the like.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a method and related equipment for monitoring traffic flow based on automatic road detection. Background Technology

[0002] With the rapid development of smart cities and intelligent transportation systems, accurate and real-time monitoring of road traffic flow has become a core requirement for traffic management, planning, and decision-making. Video image-based monitoring methods have become the mainstream technology due to their low cost, rich information, and flexible deployment. Among these methods, the "virtual detection line" technology is widely used. Its principle is to set a virtual detection line at a specific location in the video frame and count vehicles by analyzing moving targets crossing that line. Currently, the application of this technology heavily relies on initial manual configuration and subsequent manual maintenance, leading to widespread problems in efficiency, accuracy, and adaptability, and failing to meet the actual needs of large-scale, automated intelligent monitoring.

[0003] Currently, the industry widely adopts a static virtual detection line setting scheme that relies on manual configuration. During the system deployment phase, technicians need to manually configure each individual camera video stream in the backend management interface. This includes two steps: the first step is lane line marking, where technicians need to manually draw the boundaries of each lane based on the road conditions in the video footage; the second step is virtual detection line drawing, where technicians manually draw one or more virtual detection lines within the marked lanes based on experience, and need to determine the specific location, angle, and number of virtual detection lines. This method forms the basis of current traffic flow monitoring solutions.

[0004] This existing technology has several significant drawbacks. First, it suffers from low deployment efficiency and high costs. In large-scale monitoring scenarios requiring hundreds or thousands of cameras, manually configuring each camera individually requires enormous manpower and time, resulting in a lengthy initial deployment cycle and hindering rapid deployment and expansion. Second, monitoring accuracy is highly susceptible to human factors and exhibits poor stability. The quality of virtual detection line settings heavily relies on the personal experience and subjective judgment of technicians; improper settings can easily lead to missed or false detections, severely impacting the accuracy of counting data and making it difficult to perform unified quantitative analysis and comparison of monitoring data from different intersections. Third, the system has poor adaptability and is difficult to maintain. When cameras change position or angle due to natural aging, strong winds, or manual adjustments, the original configuration immediately becomes invalid, requiring technicians to intervene and recalibrate and redraw. This reactive maintenance mode leads to monitoring data interruptions, significantly increasing the burden of later operation and maintenance. Fourth, it struggles to handle complex road scenarios. In complex road structures such as curved lanes, merging areas, and large intersections, manual settings struggle to achieve globally optimal planning, easily leading to duplicate or uncounted vehicles.

[0005] In summary, existing technologies are essentially static configuration models, unable to adapt to the dynamically changing traffic monitoring environments of the real world. The contradiction lies in the conflict between the lag and subjectivity of relying heavily on human intervention and the demand for large-scale, automated, and highly adaptable intelligent monitoring. Therefore, there is an urgent need in this field for a novel traffic flow monitoring technology capable of automatically sensing the road environment, intelligently generating optimal detection schemes, and adapting to changes in perspective, in order to fundamentally solve the aforementioned problems. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to address the shortcomings of the prior art, specifically by providing a traffic flow monitoring method and related equipment based on automatic road detection, as detailed below: 1) In a first aspect, the present invention provides a traffic flow monitoring method based on automatic road detection, the specific technical solution of which is as follows: Acquire video image data of the road to be monitored; Based on video image data, at least one road area is automatically identified and segmented using a road detection model; Determine the minimum bounding rectangle for each road region; Based on the minimum bounding rectangle of each road region, generate at least one candidate virtual detection line within the corresponding road region; Determine the valid virtual detection lines from the generated candidate virtual detection lines; Based on video image data, detect and track the trajectory of vehicles in the video image data; Traffic flow is counted based on the interaction between vehicle trajectories and effective virtual detection lines.

[0007] The beneficial effects of the traffic flow monitoring method based on automatic road detection provided by this invention are as follows: By automatically identifying and segmenting road areas using a road detection model, the process of manually drawing lane boundaries is completely replaced, enabling rapid deployment in large-scale road network monitoring scenarios, significantly improving deployment efficiency and reducing labor costs. By intelligently generating candidate virtual detection lines through analysis of the minimum bounding rectangle of the road area, the subjective judgment relying on personal experience in traditional methods is eliminated, allowing virtual detection lines to be set based on unified geometric features, thereby improving the accuracy and objectivity of traffic flow monitoring. Furthermore, by dynamically determining valid virtual detection lines from the candidate lines, counting is performed based on the interaction between vehicle trajectories and valid virtual detection lines, giving it adaptive capabilities. This allows it to maintain continuous and stable monitoring functions even when the camera's perspective changes or when facing complex road structures, significantly reducing the difficulty and burden of later maintenance. Ultimately, this method expands the reliable application range of traffic flow monitoring technology, enabling it to provide high-quality traffic flow statistics in various dynamic scenarios such as complex urban intersections.

[0008] Based on the above scheme, the traffic flow monitoring method based on automatic road detection of the present invention can be further improved as follows.

[0009] Furthermore, based on the minimum bounding rectangle of each road region, at least one candidate virtual detection line is generated within the corresponding road region, including: Based on the comparison between the aspect ratio of the minimum bounding rectangle of each road region and a preset threshold, at least one candidate virtual detection line is generated within the corresponding road region.

[0010] The beneficial effect of adopting the above-mentioned further scheme is that by comparing the aspect ratio of the minimum bounding rectangle of each road area with a preset threshold, automated intelligent judgment of road direction is achieved. Based on this objective geometric comparison result, one or two candidate virtual detection lines can be adaptively generated in the corresponding road area. This process completely replaces the subjective judgment and manual drawing steps that rely on human experience in traditional methods, making the generation of candidate virtual detection lines based on unified and quantifiable rules, thus laying a reliable foundation for subsequent high-precision traffic flow monitoring in the initial stage.

[0011] Furthermore, valid virtual detection lines are determined from the generated candidate virtual detection lines, including: When a candidate virtual detection line is generated, it is directly used as the valid virtual detection line.

[0012] When two candidate virtual detection lines are generated, one of them is selected as the valid virtual detection line based on the traffic flow data passing through each candidate virtual detection line within the statistical period.

[0013] The beneficial effects of adopting the above-mentioned further scheme are as follows: By employing a differentiated determination strategy based on the number of candidate virtual detection lines, intelligent selection of effective virtual detection lines is achieved. When only one candidate virtual detection line is generated, it is directly confirmed, ensuring the monitoring efficiency of roads with clear directional directions. When two candidate virtual detection lines are generated, a dynamic optimization process based on traffic flow data within the statistical period is introduced, enabling the selection of effective virtual detection lines to truly reflect the actual road usage and the mainstream traffic direction, thereby improving the accuracy of traffic flow statistics and the scene adaptability in complex or irregular road scenarios.

[0014] Furthermore, based on the interaction between vehicle trajectories and effective virtual detection lines, traffic flow is counted, including: Connect the center points of vehicles in consecutive video frames in the video image data to form a trajectory line segment. Determine whether there is a geometric intersection between the trajectory line segment and the valid virtual detection line. If there is, determine that the vehicle has crossed the valid virtual detection line once, and increment the traffic flow count by one.

[0015] The beneficial effects of adopting the above-mentioned further scheme are as follows: By connecting the center points of vehicles in consecutive video frames to form trajectory segments, and determining whether these trajectory segments have geometric intersections with valid virtual detection lines, crossing events are identified. This method judges based on the precise path of continuous vehicle movement, replacing the traditional simple intersection detection that relies on single-frame vehicle detection boxes. Counting based on the geometric relationship between trajectory segments and virtual detection lines can more accurately reflect the actual crossing behavior of vehicles, effectively reducing false counts and missed counts caused by vehicle shaking, changes in bounding box size, or brief proximity that prevents actual crossing, thereby improving the accuracy and reliability of traffic flow statistics.

[0016] 2) In a second aspect, the present invention also provides a traffic flow monitoring system based on automatic road detection, the specific technical solution of which is as follows: It includes a video image data acquisition module, a recognition and segmentation module, a minimum bounding rectangle determination module, a candidate virtual detection line generation module, an effective virtual detection line determination module, a detection and tracking module, and a traffic flow counting module; The video image data acquisition module is used to: acquire video image data of the road to be monitored; The recognition and segmentation module is used to automatically identify and segment at least one road region based on video image data using a road detection model. The minimum bounding rectangle determination module is used to: determine the minimum bounding rectangle for each road area; The candidate virtual detection line generation module is used to generate at least one candidate virtual detection line within the corresponding road area based on the minimum bounding rectangle of each road area. The effective virtual detection line determination module is used to: determine the effective virtual detection line from the generated candidate virtual detection lines; The detection and tracking module is used to: detect and track the trajectory of vehicles in video image data based on video image data; The traffic flow counting module is used to count traffic flow based on the interaction between vehicle trajectories and effective virtual detection lines.

[0017] Based on the above solution, the traffic flow monitoring system based on automatic road detection of the present invention can be further improved as follows.

[0018] Furthermore, the candidate virtual detection line generation module is specifically used for: Based on the comparison between the aspect ratio of the minimum bounding rectangle of each road region and a preset threshold, at least one candidate virtual detection line is generated within the corresponding road region.

[0019] Furthermore, the effective virtual detection line determination module is specifically used to: when a candidate virtual detection line is generated, directly use the candidate virtual detection line as the effective virtual detection line.

[0020] When two candidate virtual detection lines are generated, one of them is selected as the valid virtual detection line based on the traffic flow data passing through each candidate virtual detection line within the statistical period.

[0021] Furthermore, the traffic flow counting module is specifically used to: connect the center points of vehicles in consecutive video frames in the video image data to form a trajectory line segment, determine whether there is a geometric intersection between the trajectory line segment and the valid virtual detection line, and if there is, determine that the vehicle has crossed the valid virtual detection line once, and increment the traffic flow count by one.

[0022] 3) In a third aspect, the present invention also provides an electronic device, the electronic device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor, so as to enable the electronic device to implement any of the above-mentioned traffic flow monitoring methods based on automatic road detection.

[0023] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned traffic flow monitoring methods based on automatic road detection.

[0024] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below: Figure 1 This is one of the flowcharts illustrating a traffic flow monitoring method based on automatic road detection according to an embodiment of the present invention; Figure 2 This is a second schematic flowchart of a traffic flow monitoring method based on automatic road detection according to an embodiment of the present invention; Figure 3 This is a diagram illustrating the effect of traffic flow monitoring. Figure 4 This is a schematic diagram of a traffic flow monitoring system based on automatic road detection according to an embodiment of the present invention. Detailed Implementation

[0026] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0027] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0028] like Figure 1 As shown in the figure, a traffic flow monitoring method based on automatic road detection according to an embodiment of the present invention includes the following steps: S1. Obtain video image data of the road to be monitored. The specific implementation process is as follows: S10. Install fixed or pan-tilt-zoom (PTZ) cameras at the locations on the roads requiring monitoring, ensuring that the camera coverage completely encompasses the target road area. The cameras must have image resolution and frame rate that meet the monitoring requirements, such as 1080P or higher resolution, and capture video at a rate of 25 frames per second. The cameras should be connected to the monitoring system via a network, and system configuration of network and video parameters such as IP address, video encoding format, and transmission protocol should be completed to ensure stable video stream output.

[0029] S11. Receive video stream data from the camera in real time via standard network protocols. This process is typically implemented based on streaming media transmission protocols such as RTSP or HTTP. A video stream request command is sent to the camera, and the camera responds by continuously pushing the compressed video stream to the designated network port. A video stream decapsulation module is deployed at the network receiving end to parse and reassemble the incoming data packets, restoring a continuous sequence of compressed video frames.

[0030] S12. The built-in video decoder is invoked to decode the received compressed video frames, converting the encoded data into a processable raw image pixel matrix. The decoded image data may undergo color space conversion, such as from YUV to RGB format, depending on the requirements of subsequent processing modules. Simultaneously, image scaling may be performed to uniformly adjust the resolution to the input size required by the road detection model, ensuring processing efficiency and consistency.

[0031] S13. The decoded and converted continuous video image data is temporarily stored in a first-in-first-out (FIFO) image data buffer. The buffer management module is responsible for coordinating the rate difference between the high-speed writing of video data and the relatively low-speed reading of subsequent processing modules to prevent data loss or backlog. According to a preset sampling strategy, such as frame-by-frame processing or frame extraction at fixed time intervals, the video image data is sequentially retrieved from the buffer, ready to be delivered to the subsequent road detection model for analysis.

[0032] S14. After the above-described acquisition, transmission, decoding, buffering, and scheduling processes, a series of video image data of the monitored road, arranged in chronological order and with a unified format, is finally obtained. This video image data constitutes the original input source for all subsequent analysis and processing. Through a standardized data interface, the video image data is passed to the next step in the technical solution, namely the road detection model, thereby initiating the automated road recognition and segmentation process. The entire acquisition process ensures the real-time performance, integrity, and availability of the video image data, providing a stable and reliable data foundation for the entire traffic flow monitoring method.

[0033] S2. Based on video image data, at least one road region is automatically identified and segmented using a road detection model. The specific implementation process is as follows: S20. Load the pre-trained road detection model from the storage medium. This road detection model is a pixel-level semantic segmentation model based on a deep convolutional neural network, such as using architectures like U-Net or DeepLabv3+. During loading, the model's weight parameters and structure definition file are read into memory, and the computing environment required for model operation is initialized, such as specifying whether to use CPU or GPU for computation and allocating the corresponding computing resources. After initialization, the road detection model enters a ready state, waiting to receive input video image data.

[0034] S21. To match the input format specified by the road detection model, a series of preprocessing operations are performed on this video image data. Preprocessing includes scaling the video image data to the fixed width and height required by the model, for example, 512×512 pixels. Next, pixel values ​​of the video image data are normalized, linearly transforming the original RGB pixel values ​​from 0 to 255 to a floating-point range of 0 to 1, or standardizing the mean and standard deviation. The preprocessed video image data is organized into a three-dimensional tensor, whose dimensions represent batch size, image height, image width, and color channels, and then fed into the input interface of the road detection model.

[0035] S22. The road detection model performs a series of forward propagation calculations on the input 3D tensor. This process includes feature extraction through multiple convolutional layers, activation function layers, and pooling layers to obtain visual features at different scales from the video image data. Subsequently, the model fuses the extracted high-level semantic features with low-level detail features through structures such as upsampling layers and skip connections, ultimately outputting a probability map or feature map associated with the input size. In this output map, each pixel location contains the prediction confidence of one or more categories, with the most crucial category being "road region".

[0036] S23. Receive the raw results output by the road detection model. For each pixel, select the confidence level corresponding to the "road region" category and apply a preset confidence threshold for binarization. When the confidence level of a pixel is higher than the threshold, the pixel is determined to belong to the road region; otherwise, it belongs to the background. This process generates a binary road region segmentation mask, where white pixels represent identified road regions and black pixels represent non-road regions. Next, image morphological operations, such as closing operations, are used to process the binary mask to eliminate small noise points and fill in any holes that may exist inside the road regions, making the segmented road regions more complete and smoother.

[0037] S24. Perform connected component analysis on the processed binary road region segmentation mask. By scanning the mask, all connected white pixel sets are identified as independent connected components, each corresponding to an independent road region instance. For each identified road region instance, the coordinates of its outer contour pixels are extracted. These contour coordinate points are arranged in clockwise or counterclockwise order to form an ordered set of points. This ordered set of points is a precise digital representation of the shape of a road region. Finally, a list is output, where each element represents a detected road region, and each road region is described by an ordered set of points, thus completing the entire process of automatically identifying and segmenting at least one road region from video image data.

[0038] S3. Determine the minimum bounding rectangle for each road region; S4. Based on the minimum bounding rectangle of each road region, generate at least one candidate virtual detection line within the corresponding road region. Specifically, based on the comparison result of the aspect ratio of the minimum bounding rectangle of each road region with a preset threshold, generate at least one candidate virtual detection line within the corresponding road region. The specific implementation process is as follows: S40. For each ordered set of points obtained from the road detection model, calculate its minimum bounding rectangle. The minimum bounding rectangle is the rectangle that completely encloses all contour points of the road area and has the smallest area. Using a rotating caliper algorithm or similar geometric calculation methods, traverse the boundary of the ordered set of points to find the rectangle that satisfies the minimum area condition. This rectangle is defined by four parameters: center point coordinates, width, height, and rotation angle. After calculation, divide the rectangle's width by its height to obtain the aspect ratio of the minimum bounding rectangle. The width and height refer to the lengths of the two adjacent sides of the rectangle, regardless of direction; the aspect ratio is always a value greater than or equal to 1.

[0039] S41. Read a fixed value from the preset configuration parameters as a preset threshold. This preset threshold is an empirical value, such as 3, used to distinguish between roads with a clear direction and roads with irregular orientations. Compare the aspect ratio of the minimum bounding rectangle of each road region calculated to this preset threshold. The comparison operation is a simple conditional judgment, determining whether the aspect ratio of the minimum bounding rectangle is greater than the preset threshold. This comparison result will determine the strategy for subsequently generating candidate virtual detection lines.

[0040] S42. When the aspect ratio of the minimum bounding rectangle of a road area is greater than a preset threshold, the road is determined to be a long, narrow road with a clear direction. In this case, a candidate virtual detection line is generated. The generation method is as follows: Locate the two long sides of the minimum bounding rectangle of the road area. For each long side, calculate the coordinates of its midpoint, which is the arithmetic mean of the coordinates of the two endpoints of the long side. After the calculation, obtain the coordinates of the midpoints of the two long sides. Finally, connect the midpoints of the two long sides to form a straight line segment. This straight line segment is defined as the unique candidate virtual detection line generated within the road area and is associated with and stored in relation to the current road area.

[0041] S43. When the aspect ratio of the minimum bounding rectangle of a road area is less than or equal to a preset threshold, the road is determined to be an undefined or irregular road, such as a curved intersection or a merging area. In this case, two candidate virtual detection lines need to be generated. The first candidate virtual detection line is generated in exactly the same way as in S42, that is, by connecting the midpoints of the two long sides of the minimum bounding rectangle. The second candidate virtual detection line is generated by locating the two short sides of the minimum bounding rectangle and calculating the coordinates of the midpoint of each short side. Then, the midpoints of these two short sides are connected to form another straight line segment. This straight line segment formed by connecting the midpoints of the short sides is defined as the second candidate virtual detection line. Finally, the road area is associated with two candidate virtual detection lines, namely the line connecting the midpoints of the long sides and the line connecting the midpoints of the short sides, for dynamic selection in subsequent steps.

[0042] S44. After the above judgment and generation steps, at least one candidate virtual detection line is generated for each road area. These candidate virtual detection lines exist in the form of geometric line segments, defined by the two-dimensional coordinates of the start and end points. All generated candidate virtual detection lines, along with the identification information of their respective road areas, are packaged and output to the next processing unit. This completes the process of intelligently generating the initial detection scheme based on the geometric features of the minimum bounding rectangle, laying the foundation for subsequent dynamic optimization of virtual detection lines and traffic flow statistics.

[0043] S5. Determine the valid virtual detection line from the generated candidate virtual detection lines. Specifically, when one candidate virtual detection line is generated, it is directly used as the valid virtual detection line. When two candidate virtual detection lines are generated, based on the traffic flow data passing through each candidate virtual detection line within the statistical period, one of the two candidate virtual detection lines is selected as the valid virtual detection line. The specific implementation process is as follows: S50. Receive input from the previous step. The input is a list, where each entry corresponds to a road region and one or two candidate virtual detection lines generated for that road region. Process sequentially starting from the first entry in the list, independently executing subsequent judgment and selection logic for each road region.

[0044] S51. For the road area currently being processed, read the number of candidate virtual detection lines associated with it. This is a basic counting operation. Based on the counting result, proceed to different processing branches. If the count result is 1, proceed to the process of directly determining the valid virtual detection lines. If the count result is 2, proceed to the process of determining the valid virtual detection lines based on statistical optimization.

[0045] S52. When it is confirmed that the current road area is associated with only one candidate virtual detection line, the processing is very simple and direct. Mark this unique candidate virtual detection line as the valid virtual detection line for the current road area. Store this association—that is, the road area identifier and the geometric coordinates of this valid virtual detection line—in the configuration database or the list of active detection lines in memory. After storage is complete, the processing of this road area ends immediately, and the next road area in the list begins processing.

[0046] S53. When it is confirmed that the current road area is associated with two candidate virtual detection lines, a dynamic selection process is initiated. An independent vehicle crossing counter is initialized for each of these two candidate virtual detection lines, and the count value is set to zero. Simultaneously, a timer is started, with a duration equal to a preset statistical period, such as five minutes. Before the timer expires, the parallel statistical phase begins. In this phase, both candidate virtual detection lines are allowed to be active simultaneously, and the subsequent traffic flow counting module is used to independently accumulate and count the vehicles crossing each candidate virtual detection line.

[0047] S54. During the initial statistical period, the vehicle detection and tracking module operates continuously. Whenever a vehicle is detected crossing the first candidate virtual detection line within the current road area, the counter value corresponding to the first candidate virtual detection line is incremented by one. Similarly, whenever a vehicle is detected crossing the second candidate virtual detection line, the counter value corresponding to the second candidate virtual detection line is incremented by one. This counting process is completely parallel; the counting of the two candidate virtual detection lines does not affect each other. The status of the timer is continuously monitored.

[0048] S55. When the timer for the statistical period expires, a signal indicating the end of the period is received. Immediately read the accumulated counter values ​​of the two candidate virtual detection lines during the period; these values ​​represent the traffic flow data passing through each candidate virtual detection line. Compare the two traffic flow data. Select the candidate virtual detection line with the larger traffic flow data as the winner. If the traffic flow data of the two candidate virtual detection lines are exactly equal, then make the final decision according to predefined rules, such as selecting the candidate virtual detection line generated first or the candidate virtual detection line generated at the midpoint of the longer side.

[0049] S56. Based on the comparison results in S55, the selected candidate virtual detection line is officially designated as the valid virtual detection line for the current road area, and its coordinate information is stored in the active detection line list. Simultaneously, all monitoring and counting activities for the unselected candidate virtual detection line are immediately stopped, and the temporary resources allocated to it, such as counters, are released. Thereafter, traffic flow statistics will only be based on this determined valid virtual detection line.

[0050] S57. Repeat S51 to S56 until all road regions in the input list have been processed. Finally, output a unified list containing the confirmed valid virtual detection line for each road region. This output list signifies the completion of the selection process from candidate virtual detection lines to valid virtual detection lines, allowing the final traffic flow counting stage to begin.

[0051] S6. Based on video image data, detect and track the trajectory of vehicles in the video image data. The specific implementation process is as follows: S60. Load a pre-trained vehicle detection model, typically a deep learning-based convolutional neural network such as YOLO or Faster R-CNN, specifically designed to identify multiple vehicle targets in an image. Simultaneously, initialize a multi-object tracking algorithm, such as SORT or DeepSORT based on Kalman filtering and Hungarian matching. The vehicle detection model is responsible for locating all vehicles in a single image, while the multi-object tracking algorithm is responsible for associating detection results of the same vehicle in different images to form a continuous motion trajectory. Allocate computational resources to the vehicle detection model and configure initial parameters for the multi-object tracking algorithm, such as the maximum number of lost frames for trajectory prediction and the distance threshold for association matching.

[0052] S61. Extract a frame of video image data from the video stream and input it into the initialized vehicle detection model. The vehicle detection model performs forward inference calculations on this frame of video image data and outputs a set of vehicle detection boxes. Each vehicle detection box contains the coordinate information of a rectangular region, i.e., the position of the vehicle in the upper left and lower right corners of the image, as well as a confidence score for the vehicle category and the specific vehicle category label. Apply a confidence threshold to filter these original vehicle detection boxes, retaining only vehicle detection boxes with a confidence score higher than the threshold, to eliminate noise results caused by model false detections.

[0053] S62. For each valid vehicle detection box obtained, two calculations are performed. The first calculation is to extract the vehicle's appearance features. Based on the coordinates of the vehicle detection box, the corresponding image region is cropped from the original video image data, and this image region is input into a feature extraction network to obtain a high-dimensional feature vector, which is used to uniquely represent the appearance of the vehicle. The second calculation is the geometric center point. Using the coordinates of the upper left and lower right corners of the vehicle detection box, the coordinates of the center point of the rectangle are calculated. This center point coordinate is a two-dimensional coordinate, representing the approximate position of the vehicle in the current frame of video image data.

[0054] S63. The multi-object tracking algorithm is invoked to process the detection results of the current frame. The multi-object tracking algorithm maintains a list of active trajectories. Each trajectory in the list records the continuous motion state of a vehicle in a previous frame, including its historical center point sequence, historical appearance features, and the current frame position predicted using Kalman filtering. The algorithm performs a matching association operation: it calculates the association cost between all vehicle detection boxes in the current frame and all existing predicted trajectories. The association cost typically consists of two parts: the geometric distance between the predicted positions of the vehicle detection boxes and the predicted positions of the trajectories, and the appearance similarity distance between the appearance features of the vehicle detection boxes and the historical appearance features of the trajectories. The algorithm uses strategies such as Hungarian matching to find an optimal pairing scheme that minimizes the total association cost, thereby matching the vehicle detection boxes of the current frame to existing trajectories.

[0055] S64. Based on the matching results obtained in S63, update the trajectory list maintained by the multi-object tracking algorithm. For trajectories that successfully match vehicle detection boxes in the current frame, perform the following update operations: add the center point coordinates of the vehicle detection box to the end of the historical center point sequence of this trajectory; update the appearance model of the trajectory with the latest appearance features of the vehicle detection box; and simultaneously correct the state of the Kalman filter with the actual position of the vehicle detection box to make its prediction more accurate. For vehicle detection boxes in the current frame that fail to match any existing trajectory, it is considered that they may represent a newly appearing vehicle. Create a new trajectory for each such vehicle detection box, assign it a globally unique trajectory identifier, and initialize its center point, appearance features, and other information into this new trajectory.

[0056] S65. The multi-target tracking algorithm includes trajectory management logic. For an existing trajectory, if a vehicle detection box fails to match in several consecutive frames of video image data, the trajectory is marked as "lost." When the number of frames in the "lost" state for a trajectory exceeds the preset maximum allowable value, it is determined that the vehicle has left the monitoring area or is continuously occluded. Therefore, this trajectory is removed from the active trajectory list, marking the end of tracking. Active trajectory information is output in real time. For each active trajectory, its core information is the vehicle's trajectory, i.e., a sequence of center point coordinates arranged in chronological order. This sequence clearly depicts the vehicle's continuous movement path in the video image data. These trajectory data, along with their unique trajectory identifiers, are passed to the next traffic flow statistics module.

[0057] S7. Based on the interaction between the vehicle's trajectory and the effective virtual detection line, traffic flow is counted. Specifically, the center points of the vehicle in consecutive video frames in the video image data are connected to form a trajectory line segment. It is determined whether the trajectory line segment and the effective virtual detection line have a geometric intersection. If they do, it is determined that the vehicle has crossed the effective virtual detection line once, and the traffic flow count is incremented by one. The specific implementation process is as follows: S70. Receive real-time data from the vehicle detection and tracking module. For each tracked vehicle, the data includes a globally unique trajectory identifier and the latest information about that vehicle's trajectory. The latest information critically includes the vehicle's center point coordinates in the current frame of video image data, as well as the center point coordinates stored in the trajectory history in the previous frame of video image data. Ensure that these coordinate data are valid, non-empty, and correspond to the continuous motion state of the same vehicle.

[0058] S71. For each vehicle to be judged for crossing the line, perform a geometric construction operation. Read the coordinates of the vehicle's center point in the previous frame of video image data and denote this coordinate point as point A. Then read the coordinates of the vehicle's center point in the current frame of video image data and denote this coordinate point as point B. In the two-dimensional image coordinate plane, connect point A and point B with a straight line segment. This straight line segment connecting point A and point B is defined as the trajectory line segment formed by the vehicle in the time interval between two adjacent frames. The trajectory line segment has directionality, pointing from point A, representing the past time, to point B, representing the present time.

[0059] S72. From the established configuration, obtain the geometric definition of the valid virtual detection line for which intersection point determination is required. A valid virtual detection line is stored as a straight line segment, uniquely determined by the coordinates of its start and end points. Retrieve the start and end coordinates of this valid virtual detection line from memory or the configuration database.

[0060] S73. A rigorous geometric calculation process is performed to determine whether the constructed trajectory segment intersects the valid virtual detection line read in S72 at a single point on the two-dimensional plane. The calculation process does not rely on the vehicle's bounding box but is based on the mathematical definition of a line segment. The trajectory segment and the valid virtual detection line are represented as parametric equations or calculated using the vector cross product principle. The judgment logic needs to simultaneously check two conditions: the first condition is that the trajectory segment and the valid virtual detection line are not parallel to each other, and there is a possibility of intersection; the second condition is that the calculated potential intersection point must be located within the range of both the trajectory segment and the valid virtual detection line, and cannot be a point on the extension of their respective lines. This calculation process ultimately outputs a Boolean result: "Geometric intersection exists" or "Geometric intersection does not exist".

[0061] S74. Perform different operations based on the Boolean result output by S73. If the judgment result is "Geometric intersection exists", a valid vehicle crossing event is triggered. This event is associated with the valid virtual detection line and the specific lane information and recorded. Subsequently, the traffic flow counter bound to the valid virtual detection line is incremented by one. This counter is maintained independently for each valid virtual detection line and is used to accumulate the total number of vehicles crossing the line. If the judgment result is "No geometric intersection exists", no counting operation is performed, and the vehicle is not judged as crossing in this detection cycle.

[0062] S75. To prevent duplicate counting caused by the same vehicle's trajectory repeatedly intersecting with the same valid virtual detection line in consecutive frames, a state management mechanism is introduced. When a vehicle is determined to have crossed a valid virtual detection line once, a "counted" association marker is established between the vehicle's trajectory identifier and this valid virtual detection line. In subsequent frames of video image data, even if the vehicle's trajectory segment geometrically intersects the valid virtual detection line again, this marker is checked and the event is ignored, without repeatedly increasing the traffic flow count. Only when the vehicle's trajectory completely moves away from the valid virtual detection line and subsequently approaches it again from the other side will the marker be cleared, allowing a new, valid crossing count to be performed.

[0063] S76. After completing the trajectory segment judgment and count update for all vehicles, the latest traffic flow count data is summarized. This data can be archived by time granularity and transmitted to the upper-level traffic management platform or data visualization interface via network interface or message queue for real-time display or further analysis. Then, the next frame of video image data is processed, and the entire process is executed cyclically to achieve continuous, real-time monitoring of traffic flow.

[0064] like Figure 2 As shown, the technical effects of the present invention will be further illustrated through the following embodiments, specifically including the following steps: Step S401: Automatic road detection and representation. Specifically, the road detection model is triggered periodically according to a preset time cycle. The video image data of the current moment is input into the road detection model for processing. The road detection model outputs pixel-level road segmentation results for the current video image data. To facilitate subsequent calculations, this invention represents and stores each segmented passable road area using a data structure called an "ordered point set." This ordered point set is a sequence of polygon vertices that accurately depicts the shape and direction of the road.

[0065] Step S402: Road Scene Adaptation and Filtering. Specifically, based on the actual monitoring scenario requirements, target roads to be monitored are filtered from all road areas detected in Step S401. For example, in a highway monitoring scenario, a set of main road areas representing the up and down directions is specifically filtered out, while non-mainstream monitoring areas such as ramps and emergency lanes are filtered out. This step ensures that system resources are focused on key traffic flow channels.

[0066] Step S403: Dynamic virtual detection line generation, specifically: For each road area selected in step S402, perform the following operations to generate a virtual detection line.

[0067] The first operation is to calculate the minimum bounding rectangle for each road region. Based on the "ordered set of points" of the road region, the minimum bounding rectangle that can completely enclose the region is found.

[0068] The second operation is road direction determination and initial selection of virtual detection lines. The aspect ratio of the minimum bounding rectangle is calculated and compared with a preset threshold. The determination result falls into two categories: Scenario 1, a road with a clear direction. When the aspect ratio of the minimum bounding rectangle is greater than the preset threshold, the road is determined to be a long, narrow road with a clear direction. In this case, the line connecting the midpoints of the long side of the rectangle is automatically selected as the only candidate virtual detection line, and this line is marked as L1.

[0069] Scenario 2: Irregular Roads. When the aspect ratio of the smallest bounding rectangle is not greater than a preset threshold, the road is determined to be an irregularly oriented road. In this case, two candidate virtual detection lines are generated simultaneously: the first is line L1 connecting the midpoints of the longer sides, and the second is line L2 connecting the midpoints of the shorter sides.

[0070] The third operation is the dynamic optimization of the virtual detection line. For the road in scenario two above, where two candidate virtual detection lines are generated, the number of vehicles crossing both candidate virtual detection lines L1 and L2 is counted in parallel within a subsequent predetermined statistical period. After the statistical period ends, the candidate virtual detection line with the larger traffic flow data is automatically selected as the final effective virtual detection line for that road, and the counting of the other line is stopped. This dynamic optimization process ensures that the virtual detection line can adapt to the actual road usage.

[0071] Step S404: The vehicle passes through the virtual inspection line for inspection, specifically: The vehicle detection and tracking module continuously acquires the continuous trajectory coordinates of each vehicle. During line-crossing detection, the following precise judgment is performed: For each tracked vehicle, the center point of the vehicle in two consecutive frames of video image data is taken, and these two points are connected to form a trajectory line segment. It is then determined whether this trajectory line segment geometrically intersects with the valid virtual detection line of the current road area. If the judgment result is that a geometric intersection exists, the vehicle is determined to have validly crossed the virtual detection line once, and the traffic flow count is incremented by one. If the judgment result is that no geometric intersection exists, no counting operation is performed.

[0072] Furthermore, based on video image data, vehicles are detected in real time and their movement trajectories are tracked, providing continuous vehicle trajectory coordinate data for the line crossing detection in step S404.

[0073] Step S405: Timed Re-detection. Specifically, to balance operational efficiency and adaptability to scene changes, this invention employs a low-frequency, timed triggering mechanism for road re-detection. It checks whether a preset re-detection time interval has been reached. When the time interval is reached, the automatic road detection and representation, along with subsequent processes, restart from step S401. For example, a complete road re-detection can be triggered once per minute, instead of detecting every frame of video image data. This method effectively addresses the slow shift in camera perspective caused by strong winds or collisions, updating the positions of the road and virtual detection lines, while avoiding the enormous computational overhead of continuously running deep learning models, ensuring long-term operational stability.

[0074] The entire process, from the beginning, through steps S401 to S405, and combined with continuous vehicle detection and tracking, forms a complete, automated, and adaptive closed-loop traffic flow monitoring process. The traffic flow monitoring effect is as follows: Figure 3 As shown.

[0075] This invention utilizes a deep learning model to automatically identify and segment road regions in video image data, and uses an "ordered point set" data structure for digital representation, replacing manual visual identification and drawing. A virtual detection line intelligent generation algorithm based on geometric features analyzes the geometric features of the minimum bounding rectangle of each road region, especially the aspect ratio, to intelligently determine the mainstream direction of vehicles and automatically generate initial virtual detection lines accordingly. This is the decision logic for achieving full-process automation. For roads deemed irregular, multiple candidate virtual detection lines are set in parallel, and dynamic competition and selection are performed using actual traffic flow statistics within a predetermined statistical period. This process enables the system to adapt to real-world road usage, ensuring long-term monitoring accuracy. The computationally intensive road detection and representation tasks are run at low frequency with timed triggering, decoupling them from the computationally intensive vehicle tracking and line-crossing detection tasks which run at high frequency. This architecture ensures system adaptability to environmental changes while maintaining overall computational efficiency. Compared with existing technologies, the advantages of this invention are reflected in the following dimensions: In terms of automation and efficiency, traditional technologies rely entirely on manual labor, requiring technicians to manually calibrate lanes and draw virtual detection lines for each camera's video stream. This process is time-consuming and has extremely low deployment efficiency. This invention achieves fully automated processing, eliminating the need for any manual intervention from road recognition to virtual detection line setup. This results in rapid deployment of large-scale road networks within minutes, significantly reducing labor and time costs and making previously difficult-to-achieve large-scale monitoring possible. Regarding monitoring accuracy and objectivity, the virtual detection line setup in traditional technologies is highly subjective, relying on the personal experience of technicians, which can easily lead to missed or false detections, making accuracy difficult to guarantee and unstable. This invention is completely objective, based on a unified geometric algorithm and dynamic optimization using real traffic flow data, eliminating the uncertainty of human judgment. This improves the accuracy and reliability of traffic flow statistics, providing a high-quality data foundation for traffic management and decision-making. Finally, in terms of adaptability and robustness, traditional technologies are extremely fragile. If a camera shifts due to external force, the entire system immediately fails, requiring manual recalibration and incurring high maintenance costs. This invention is highly adaptive. Through a timed re-detection process, the system can automatically sense scene changes and self-correct; the dynamic virtual detection line mechanism can adapt to the actual traffic flow on complex roads. This results in the system's ability to operate stably without interruption, significantly reducing the burden of later operation and maintenance, and demonstrating strong adaptability to complex outdoor application environments. In terms of handling complex scenes, traditional technologies have limited capabilities. In complex scenarios such as curved roads and large intersections, it is difficult to manually set up reasonable virtual detection lines, resulting in poor monitoring performance. This invention can intelligently optimize, automatically finding the best detection solution for complex scenarios through a dynamic competition and selection process. This expands the application boundaries of traffic flow monitoring technology, enabling it to maintain high accuracy in various scenarios such as highways and complex urban intersections, thus broadening its application range.

[0076] In the above embodiments, although the steps are numbered S1, S2, etc., they are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation. The scheme after adjusting the order is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0077] like Figure 4 As shown, an embodiment of the present invention provides a traffic flow monitoring system 200 based on automatic road detection, which includes a video image data acquisition module 201, a recognition and segmentation module 202, a minimum bounding rectangle determination module 203, a candidate virtual detection line generation module 204, an effective virtual detection line determination module 205, a detection and tracking module 206, and a traffic flow counting module 207. The video image data acquisition module 201 is used to: acquire video image data of the road to be monitored; The recognition and segmentation module 202 is used to: automatically identify and segment at least one road region based on video image data and a road detection model; The minimum bounding rectangle determination module 203 is used to: determine the minimum bounding rectangle of each road area; The candidate virtual detection line generation module 204 is used to: generate at least one candidate virtual detection line in the corresponding road area based on the minimum bounding rectangle of each road area; The effective virtual detection line determination module 205 is used to: determine the effective virtual detection line from the generated candidate virtual detection lines; The detection and tracking module 206 is used to: detect and track the trajectory of a vehicle in video image data based on video image data; The traffic flow counting module 207 is used to count traffic flow based on the interaction between the vehicle trajectory and the effective virtual detection line.

[0078] Optionally, in the above technical solution, the candidate virtual detection line generation module 204 is specifically used to: generate at least one candidate virtual detection line in the corresponding road area based on the comparison result of the aspect ratio of the minimum bounding rectangle of each road area with a preset threshold.

[0079] Optionally, in the above technical solution, the effective virtual detection line determination module 205 is specifically used to: when a candidate virtual detection line is generated, directly use the candidate virtual detection line as the effective virtual detection line.

[0080] When two candidate virtual detection lines are generated, one of them is selected as the valid virtual detection line based on the traffic flow data passing through each candidate virtual detection line within the statistical period.

[0081] Optionally, in the above technical solution, the traffic flow counting module 207 is specifically used to: connect the center points of vehicles in consecutive video frames in the video image data to form a trajectory line segment, determine whether there is a geometric intersection between the trajectory line segment and the valid virtual detection line, and if there is, determine that the vehicle has crossed the valid virtual detection line once, and increment the traffic flow count by one.

[0082] It should be noted that the beneficial effects of the traffic flow monitoring system 200 based on automatic road detection provided in the above embodiments are the same as those of the traffic flow monitoring method based on automatic road detection described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0083] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned traffic flow monitoring methods based on automatic road detection.

[0084] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described traffic flow monitoring methods based on automatic road detection.

[0085] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0086] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for monitoring traffic flow based on automatic road detection, characterized in that, include: Acquire video image data of the road to be monitored; Based on the video image data, at least one road area is automatically identified and segmented using a road detection model; Determine the minimum bounding rectangle for each of the road regions; Based on the minimum bounding rectangle of each road region, at least one candidate virtual detection line is generated within the corresponding road region; Determine the valid virtual detection lines from the generated candidate virtual detection lines; Based on the video image data, the trajectory of the vehicle in the video image data is detected and tracked; Traffic flow is counted based on the interaction between the vehicle's trajectory and the effective virtual detection line.

2. The traffic flow monitoring method based on automatic road detection according to claim 1, characterized in that, Based on the minimum bounding rectangle of each road region, at least one candidate virtual detection line is generated within the corresponding road region, including: Based on the comparison between the aspect ratio of the minimum bounding rectangle of each road region and a preset threshold, at least one candidate virtual detection line is generated in the corresponding road region.

3. The traffic flow monitoring method based on automatic road detection according to claim 2, characterized in that, Determine the valid virtual detection lines from the generated candidate virtual detection lines, including: When a candidate virtual detection line is generated, the candidate virtual detection line is directly used as the effective virtual detection line. When two candidate virtual detection lines are generated, one of the two candidate virtual detection lines is selected as the valid virtual detection line based on the traffic flow data passing through each candidate virtual detection line within the statistical period.

4. The traffic flow monitoring method based on automatic road detection according to claim 3, characterized in that, Based on the interaction between the vehicle's trajectory and the effective virtual detection line, traffic flow is counted, including: Connect the center points of vehicles in consecutive video frames in the video image data to form a trajectory line segment. Determine whether the trajectory line segment has a geometric intersection with the effective virtual detection line. If it does, determine that the vehicle has crossed the effective virtual detection line once, and increment the traffic flow count by one.

5. A traffic flow monitoring system based on automatic road detection, characterized in that, It includes a video image data acquisition module, a recognition and segmentation module, a minimum bounding rectangle determination module, a candidate virtual detection line generation module, an effective virtual detection line determination module, a detection and tracking module, and a traffic flow counting module; The video image data acquisition module is used to: acquire video image data of the road to be monitored; The recognition and segmentation module is used to: automatically identify and segment at least one road region based on the video image data using a road detection model; The minimum bounding rectangle determination module is used to: determine the minimum bounding rectangle of each road area; The candidate virtual detection line generation module is used to: generate at least one candidate virtual detection line in the corresponding road area based on the minimum bounding rectangle of each road area; The effective virtual detection line determination module is used to: determine the effective virtual detection line from the generated candidate virtual detection lines; The detection and tracking module is used to: detect and track the trajectory of the vehicle in the video image data based on the video image data; The traffic flow counting module is used to count traffic flow based on the interaction between the vehicle's trajectory and the effective virtual detection line.

6. A traffic flow monitoring system based on automatic road detection according to claim 5, characterized in that, The candidate virtual detection line generation module is specifically used for: Based on the comparison between the aspect ratio of the minimum bounding rectangle of each road region and a preset threshold, at least one candidate virtual detection line is generated in the corresponding road region.

7. A traffic flow monitoring system based on automatic road detection according to claim 6, characterized in that, The effective virtual detection line determination module is specifically used to: when a candidate virtual detection line is generated, directly use the candidate virtual detection line as the effective virtual detection line; When two candidate virtual detection lines are generated, one of the two candidate virtual detection lines is selected as the valid virtual detection line based on the traffic flow data passing through each candidate virtual detection line within the statistical period.

8. A traffic flow monitoring system based on automatic road detection according to claim 7, characterized in that, The traffic flow counting module is specifically used to: connect the center points of vehicles in consecutive video frames in the video image data to form a trajectory line segment, determine whether the trajectory line segment has a geometric intersection with the effective virtual detection line, and if so, determine that the vehicle has crossed the effective virtual detection line once and increment the traffic flow count by one.

9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the traffic flow monitoring method based on automatic road detection as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements a traffic flow monitoring method based on automatic road detection as described in any one of claims 1 to 4.

Images