Traffic flow counting method and system based on unmanned aerial vehicle vision technology

Abstract

This invention relates to the field of traffic flow statistics technology, specifically to a method and system for traffic flow statistics based on UAV vision technology. The invention divides the detection boxes output by the target detection model into blocks to obtain detection box sub-regions. Then, it analyzes the intersection-union ratio (IU / U) of these sub-regions and performs pixel-level segmentation of the image within each region to determine the occlusion type of the detection box, thereby determining the final weight of each region. Finally, it fuses the final weights and IU / U of each region to obtain the overall IU / U of the detection boxes, enabling target tracking and traffic flow statistics. This invention effectively reduces inaccurate recognition information caused by vehicle occlusion, improves robustness in complex scenarios, increases the utilization rate of low-scoring detection boxes, and thus improves the accuracy of traffic flow statistics results.

Description

Technical Field

[0001] This invention relates to the field of traffic flow statistics technology, specifically to a traffic flow statistics method and system based on UAV vision technology. Background Technology

[0002] Using drones can acquire traffic flow data at higher altitudes, ensuring coverage of the detection range and improving the accuracy and precision of data collection, which has important practical significance in urban traffic management.

[0003] Existing technologies often use the Byte Track algorithm (Byte-based Association for Multi-Object Tracking, an online multi-object tracking algorithm based on detection) for vehicle tracking. The core idea of ​​the Byte Track algorithm is to distinguish between high-confidence and low-confidence detection boxes and use Kalman filtering to predict the position and size of the bounding box in the next frame to determine the tracking position. However, when the vehicle is at a distance, the detection boxes only cover part of the vehicle body, and the positions of detection boxes in adjacent frames jump significantly, making accurate association prediction by Kalman filtering impossible. The most common solution to this problem is to fuse the IoU (Intersection over Union) cost from the Byte Track algorithm with appearance cosine weighted calculation. However, during the fusion process, in the presence of partial occlusion, the detection boxes may still be able to roughly locate the target, especially if the detection model is functioning correctly and can handle partial occlusion. In this case, IoU can still serve as an effective association indicator, while appearance features may be unreliable due to the loss of key features in the occluded parts, leading to inaccurate traffic flow statistics. Summary of the Invention

[0004] To address the above-mentioned technical problems, the present invention aims to provide a traffic flow statistics method and system based on UAV vision technology.

[0005] According to a first aspect of the present invention, a traffic flow statistics method based on UAV vision technology is provided, and the specific technical solution adopted is as follows:

[0006] Acquire vehicle images under UAV visual tracking and input the vehicle images into the target detection model;

[0007] The detection bounding box output by the target detection model is divided into blocks to obtain detection bounding box sub-regions;

[0008] Analyze the intersection-union ratio (IU) between the detected bounding box region and the predicted bounding box of the predicted motion trajectory in the next frame, and output the IU value of the detected bounding box region;

[0009] The image within the detection box area is segmented at the pixel level to identify the vehicle body and occlusions within the detection box area and determine the occlusion type of the detection box.

[0010] Based on the occlusion type of the detection frame, and combined with the intersection-union ratio and area of ​​the detection frame sub-regions, the final weight of the detection frame region is determined;

[0011] The final weights and intersection-union ratios of the detection frame regions are fused to obtain the overall intersection-union ratio of the detection frame, which is then used for target tracking and traffic flow statistics.

[0012] In some embodiments of the present invention, the detection box output by the target detection model is divided into blocks to obtain detection box sub-regions, including:

[0013] The detection box output by the target detection model is horizontally divided into three sub-regions: left, middle, and right.

[0014] Based on the vehicle's direction of movement, the left and right sub-regions are dynamically expanded to obtain three detection frame regions: left, center, and right.

[0015] In some embodiments of the present invention, the left and right sub-regions are dynamically expanded according to the vehicle's direction of motion to obtain three detection frame regions: left, center, and right, including:

[0016] When the vehicle moves to the right, the right sub-region expands its coverage area to the right by an additional 'a'% of its original length. When the vehicle moves to the left, the left sub-region expands its coverage area to the left by an additional 'a'% of its original length, resulting in three detection box sub-regions: left, center, and right.

[0017] In some embodiments of the present invention, the cross-union ratio (CUB) between the detected bounding box region and the predicted bounding box of the predicted motion trajectory of the next frame is analyzed, and the CUB value of the detected bounding box region is output, including:

[0018] Predict the motion trajectory of the next frame to obtain the prediction box;

[0019] The ratio of the pixel-level overlap area between the detection box region and the prediction box to the total coverage area is calculated, and a penalty factor is introduced to obtain the intersection-union ratio of the detection box region.

[0020] In some embodiments of the present invention, predicting the motion trajectory of the next frame to obtain a prediction box includes:

[0021] The motion trajectory of the next frame is predicted based on Kalman filtering, and a uniform velocity model of Taylor third-order expanded extended Kalman filtering is introduced to realize third-order motion compensation for motion trajectory prediction, thus obtaining the prediction box.

[0022] In some embodiments of the present invention, pixel-level segmentation is performed on the image within the detection frame region to identify the vehicle body and occluders within the detection frame region, including:

[0023] The image within the detection box area is segmented at the pixel level to obtain multiple entity image blocks;

[0024] For each entity image block in the current image block sub-region, traverse other detection box regions to determine whether similar entities exist;

[0025] If similar entities exist, the occlusion detection logic is set as follows:

[0026] If the area of ​​the entity image block is greater than the area of ​​the similar entity, and the features of the two match, then the entity image block is determined to be an occluder within the current detection box area;

[0027] If the area of ​​the entity image block is smaller than the area of ​​the similar entity, then the entity image block is determined to be the vehicle body within the current detection frame area.

[0028] In some embodiments of the present invention, the occlusion type includes unilateral occlusion, central occlusion, bilateral occlusion, full occlusion, and no occlusion, wherein:

[0029] When one side is blocked, the obstruction covers the right and middle detection frame areas of the detection frame, and the left detection frame area is the vehicle body; or, the obstruction covers the left and middle detection frame areas of the detection frame, and the right detection frame area is the vehicle body.

[0030] When the middle part is obstructed, the obstruction covers the middle detection frame area of ​​the detection frame, while the left and right detection frame areas are the main body of the vehicle;

[0031] When both sides are blocked, the obstruction covers the left and right detection frame areas of the detection frame, while the middle detection frame area is the vehicle body;

[0032] When fully obscured, the main body of the vehicle within the detection frame is largely obscured by the obstruction, with only fragments visible.

[0033] When there is no obstruction, the entire vehicle body within the detection frame is visible.

[0034] In some embodiments of the present invention, the final weight of the detection frame region is determined based on the occlusion type of the detection frame, combined with the intersection-union ratio and area of ​​the detection frame sub-regions, including:

[0035] Based on the occlusion type of the detection frame, determine the detection frame sub-area covered by the occlusion object;

[0036] For the detection frame area covered by the obstruction, the intersection-union ratio and area of ​​the detection frame area are combined to perform weight allocation and determine the final weight of the detection frame area;

[0037] For an unobstructed detection frame region, the final weight of the detection frame region is determined to be 1.

[0038] According to a second aspect of the present invention, a traffic flow statistics system based on UAV vision technology is provided, comprising: a memory and a processor, wherein:

[0039] The memory is used to store program code;

[0040] The processor is configured to read program code stored in the memory and execute the method described in the first aspect of the present invention.

[0041] In some embodiments of the present invention, the processor includes:

[0042] The vehicle image acquisition module is used to acquire vehicle images under UAV visual tracking and input the vehicle images into the target detection model;

[0043] The detection box segmentation module is used to segment the detection boxes output by the target detection model into sub-regions of the detection boxes;

[0044] The intersection-over-union (IoU) analysis module is used to analyze the IoU relationship between the detected bounding box region and the predicted bounding box of the predicted motion trajectory of the next frame, and output the IoU value of the detected bounding box region.

[0045] The weight allocation module is used to perform pixel-level segmentation of the image within the detection box region, identify the vehicle body and occlusions in the detection box region, determine the occlusion type of the detection box, and determine the final weight of the detection box region based on the occlusion type of the detection box, combined with the intersection-union ratio and area of ​​the detection box region.

[0046] The target tracking module is used to fuse the final weights and intersection-union ratios of the detection box sub-regions to obtain the overall intersection-union ratio of the detection box, perform target tracking, and complete traffic flow statistics.

[0047] Compared with existing technologies, the traffic flow statistics method and system based on UAV vision technology provided by this invention have the following beneficial effects:

[0048] This invention segments vehicle images from a drone's perspective by dividing the detection boxes output by the target detection model into blocks. Even if the vehicle is mostly occluded, the unoccluded parts can be identified. The vehicle information can then be analyzed to identify the vehicle. By dividing the detection boxes into blocks, the matching of each block can be evaluated more precisely, reducing the problem of inaccurate recognition information caused by vehicle occlusion.

[0049] The core purpose of this invention, by analyzing the intersection-union ratio (IoU) of the detection bounding box regions, is to address the problem of missing local information in complex scenarios, resolve the deficiencies in overall IoU under occlusion, preserve matching information of locally visible regions, adapt to changes in vehicle movement direction, and improve robustness in complex scenarios. In addition, calculating the IoU of the detection bounding box regions can enhance tracking continuity under partial occlusion, reduce false matching, improve the utilization rate of low-scoring detection boxes, and optimize trajectory association in conjunction with motion prediction.

[0050] This invention performs pixel-level segmentation of the image within the detection box region, identifies the vehicle body and occlusions within the detection box region, determines the occlusion type of the detection box, and provides pixel-level confidence weights for the Byte Track algorithm.

[0051] This invention determines the final weight of a detection box region based on the occlusion type of the detection box, combined with the intersection-union ratio and area of ​​the detection box sub-regions. The core objective of weight allocation is to quantify local confidence, solve the problem of information weight imbalance in occluded scenarios, and amplify the contribution of high-confidence regions through weight allocation.

[0052] This invention fuses the final weights and intersection-union ratios of the detection box regions to obtain the overall intersection-union ratio of the detection boxes, enabling target tracking and traffic flow statistics. This solves the problem that when the overall IoU drops below 0.3, the Byte Track algorithm discards the low-scoring detection box, and vehicle tracking can only rely on the vehicle's appearance features, which may be unreliable due to the loss of key features in occluded parts.

[0053] This invention segments the visible region, retains positional information under partial occlusion, and performs local IoU calculation and fusion. It decomposes the vehicle target matching problem in occluded scenarios into independent evaluations of multiple detection box regions, and then dynamically fuses the results based on visibility and motion consistency. Its core value lies in providing a more robust and adaptive matching metric than the overall IoU in complex scenarios such as occlusion, deformation, and abrupt motion changes, ultimately improving the stability and accuracy of vehicle tracking from a UAV perspective. Attached Figure Description

[0054] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a basic flowchart illustrating a traffic flow statistics method based on UAV vision technology, provided in one embodiment of the present invention.

[0056] Figure 2 This is a schematic diagram of the basic components of a traffic flow statistics system based on UAV vision technology, provided as an embodiment of the present invention. Detailed Implementation

[0057] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the traffic flow statistics method and system based on UAV vision technology proposed by the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms such as “comprising,” “including,” or any other variations thereof are intended to cover a non-exclusive inclusion, such that a circuit structure, article, or device comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such article or device. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of additional identical elements in the article or device that includes the element.

[0059] The following description, in conjunction with the accompanying drawings, details a specific scheme for a traffic flow statistics method based on UAV vision technology provided by the present invention.

[0060] Please see Figure 1 This illustrates the basic flow of a traffic flow statistics method based on UAV vision technology provided by an embodiment of the present invention.

[0061] like Figure 1 As shown, an embodiment of the present invention provides a traffic flow statistics method based on UAV vision technology, which specifically includes:

[0062] S100: Acquire vehicle images under UAV visual tracking and input the vehicle images into the target detection model.

[0063] A drone-based aerial video traffic flow statistics system is used to count traffic flow over roads with high traffic volume using drone vision technology. First, vehicle images are acquired via drone visual tracking. Specifically, the drone is equipped with an RGB-D camera capable of simultaneously acquiring color and depth vehicle images. Depth vehicle image information is typically obtained through techniques such as structured light and time-of-flight analysis. It combines color and depth data to provide richer information for visual localization, enabling accurate target recognition and positioning. The acquired vehicle images are then input into a target detection model, such as the YOLOX model (You Only Look Once, an anchor-bound target detection model). The YOLOX model outputs traffic flow information from the drone's vision. Throughout the traffic flow statistics process, the Byte Track algorithm (Byte-based Association for Multi-Object Tracking, a detection-based online multi-object tracking algorithm) is used for target tracking.

[0064] S200: Divide the detection box output by the target detection model into blocks to obtain the detection box sub-regions.

[0065] In the Byte Track algorithm's process of tracking traffic flow information, various vehicle information is selected and located using bounding boxes to achieve target tracking. However, in reality, due to the large amount of traffic flow information and the presence of obstacles or other obstructions in the vehicle images, the vehicle information within the bounding boxes is incomplete. Furthermore, due to the specific nature of the Byte Track algorithm, combined with the drone's traffic flow tracking perspective, it may directly discard low-scoring bounding boxes. In this case, vehicle tracking will rely solely on the vehicle's appearance features. For example, from a drone's top-down view, a vehicle may be more than 50% obscured by other objects, causing the overall IoU (Intersection over Union) to drop below 0.3. This leads the Byte Track algorithm to discard the low-scoring bounding box, and vehicle tracking will rely solely on the vehicle's appearance features, reducing tracking accuracy. Even with partial occlusion (overall IoU above 0.3), the bounding boxes may still be able to roughly locate the target, especially if the detection model is functioning correctly and can handle partial occlusion. In this case, IoU can still serve as a valid correlation indicator, while appearance features may be unreliable due to the loss of key features in the occluded portion, ultimately leading to a decrease in vehicle tracking accuracy. Segmenting the detection bounding boxes allows for the segmentation of vehicle images from a drone's perspective. Even if a vehicle is largely occluded, the unoccluded portions can be identified, enabling vehicle recognition through analysis of vehicle information. Segmenting the detection bounding boxes allows for a more detailed evaluation of the matching within each segment, reducing inaccuracies caused by vehicle occlusion.

[0066] Based on the above analysis, in an embodiment of the present invention, the detection bounding box output by the target detection model is divided into blocks to obtain detection bounding box sub-regions. Further methods include:

[0067] First, the detection box output by the target detection model is horizontally divided into three sub-regions: left, middle, and right. The baseline length of each sub-region is one-third of the length of the detection box before expansion (rounded down).

[0068] Then, based on the vehicle's direction of movement, the left and right sub-regions are dynamically expanded to obtain three detection box sub-regions: left, center, and right. Specifically, when the vehicle moves to the right, the right sub-region extends its coverage to the right by an additional 'a%' (where 'a' can be 10%) of the original detection box length. , ( (This indicates the length of the detection box before expansion), and the length of the right sub-region increases. This is used to pre-cover areas where the front of the vehicle might appear. For example, when the vehicle moves to the right, the right-side expansion can capture the front of the vehicle that is about to enter the field of view in advance, mitigating the jumping of detection boxes caused by rapid movement. This division allows for effective matching through the calculation of IoU in local sub-regions even in occluded scenarios (such as when the rear of the vehicle is visible but the front is occluded), avoiding trajectory interruptions caused by the discarding of detection boxes due to low overall IoU in traditional algorithms. When the vehicle moves to the left, the left sub-region is extended to the left by an additional coverage area of ​​the original length a% (a% can be 10%), with the extended length being... This indicates that the length of the left subregion has increased. The system captures any potentially occluded portions of the vehicle's rear. Finally, by combining the baseline and extended lengths, three detection bounding boxes (left, center, and right) are obtained, and the YOLOX object detection model outputs segmented images. Coordinates of the left, center, and right detection box regions .

[0069] S300: Analyzes the cross-union ratio (CUB) between the detected bounding box region and the predicted bounding box of the predicted motion trajectory in the next frame, and outputs the CUB value of the detected bounding box region.

[0070] In the Byte Track algorithm, IoU (Intersection over Union) is the ratio of the intersection area to the union area of ​​two bounding boxes, used to measure the degree of overlap between the two boxes. In target tracking, IoU is often used to determine whether bounding boxes in consecutive frames belong to the same target. In occlusion situations, incomplete bounding box coverage leads to inaccurate IoU calculations and thus association failures. In UAV vision-based traffic flow statistics, the core purpose of determining and calculating the IoU of sub-regions of bounding boxes is to address the problem of missing local information in complex scenes. Calculating the IoU of sub-regions of bounding boxes helps to address the shortcomings of the overall IoU under occlusion, preserves matching information in locally visible areas, adapts to changes in vehicle movement direction, and improves robustness in complex scenes. Calculating the IoU of bounding box regions can enhance tracking continuity under partial occlusion, reduce mismatches, improve the utilization rate of low-scoring bounding boxes, and optimize trajectory association in conjunction with motion prediction.

[0071] Based on the above analysis, in an embodiment of the present invention, the cross-union ratio (CUB) of the detected bounding box region is output by analyzing the cross-union ratio (CUB) relationship between the detected bounding box region and the predicted bounding box of the predicted motion trajectory in the next frame. Further, it includes:

[0072] First, the motion trajectory of the next frame is predicted to obtain the predicted bounding box. Specifically, the motion trajectory of the next frame is predicted based on Kalman filtering, and a uniform velocity model of the extended Kalman filter is introduced by Taylor third-order expansion to realize third-order motion compensation for motion trajectory prediction, improve the trajectory prediction accuracy from the perspective of UAV shooting, and reduce the problem of trajectory interruption in Kalman filtering prediction in the Byte Track algorithm, thus obtaining the predicted bounding box.

[0073] Then, the ratio of the pixel-level overlap area between the detected bounding box region and the predicted bounding box to the total coverage area is calculated, and a penalty factor is introduced to obtain the intersection-union ratio of the detected bounding box region. Specifically, the calculation of IoU for the detection bounding box regions is achieved through two stages: block matching and deformation penalty. First, the basic intersection-union ratio (IoU) is calculated between the left, center, and right detection bounding box regions and the predicted bounding boxes of the next frame's trajectory predicted based on Kalman filtering. This is achieved by dividing the pixel-level overlap area (intersection) between the detection bounding box region and the predicted bounding box by the total coverage area (union, i.e., the area of ​​the predicted bounding box plus the area of ​​the detection bounding box region minus the intersection), yielding a preliminary overlap rate that quantifies the matching degree of local regions. However, the basic IoU has some limitations, such as potential divergence during training. To address these issues, existing technologies introduce a penalty factor based on the basic IoU. The penalty factor takes into account the aspect ratio of the predicted bounding box fitting the aspect ratio of the target bounding box. Specifically, the penalty factor ensures that the aspect ratio of the predicted bounding box is closer to that of the ground truth bounding box, thereby improving detection accuracy. Therefore, a penalty factor is introduced based on the basic IoU. This yields the IoU values ​​corresponding to the left, middle, and right detection box regions.

[0074] Combining the ratio of the pixel-level overlap area (intersection) between the detected bounding box and the predicted bounding box to the total coverage area (union) with the penalty factor, the IoU expression formula for the detected bounding box region is obtained as follows:

[0075]

[0076] In the formula, This represents the intersection-union ratio (IU) of the detection bounding boxes, including the IU of the detection bounding boxes on the left. The intersection-union ratio of the detection frame area on the middle side The intersection-union ratio of the detection box area on the right. ; This represents the pixel-level overlap area (intersection), including the pixel-level overlap area (intersection) between the detection box sub-regions on the left, middle, and right sides and the prediction box. This represents the total coverage area (union), which includes the total coverage area (union) of the detection box sub-regions on the left, center, and right sides and the prediction box, respectively. This represents the penalty factor.

[0077] The final output is the IoU value corresponding to the left, center, and right detection box regions. .

[0078] In addition, before analyzing the cross-union ratio (CUB) between the detection box region and the predicted box of the next frame's motion trajectory, perspective projection correction is performed on the detection box based on existing camera imaging geometric model technology. This can eliminate the problem of detection box deformation caused by changes in UAV pose (such as pitch and yaw), improve cross-frame geometric consistency, stabilize the aspect ratio of the corrected detection box, make IoU calculation more accurate, and reduce ID switching caused by deformation.

[0079] S400: Perform pixel-level segmentation on the image within the detection box area, identify the vehicle body and occlusions within the detection box area, and determine the occlusion type of the detection box.

[0080] Since each detection box contains both the vehicle body and occluders, and the size of the occluded area varies, it is not possible to directly assign weights. Therefore, pixel-level segmentation is required to identify the vehicle body and occluders within the detection box, providing pixel-level confidence weights for the Byte Track algorithm.

[0081] Therefore, in embodiments of the present invention, by performing pixel-level segmentation of the image within the detection frame region, the vehicle body and occlusions within the detection frame region are identified, and the occlusion type of the detection frame is determined. Further, it includes:

[0082] First, the image within the detection bounding box region is segmented pixel-wise to obtain multiple entity image blocks. Specifically, the input image blocks are... and detection frame area At this point, each segmented detection box (detection box sub-region) will contain different subjects. The existing algorithm SOLOv2 (Dynamic and Fast Instance Segmentation) is used to segment the image within each detection box (segmented image). It is divided into multiple entity image blocks.

[0083] Then, after segmenting into multiple entity image blocks, it's necessary to determine whether the segmented object within the detection box of each entity image block is the main body of the vehicle or an occluder. The principle is that when the area of ​​the occluder is larger than that of the main body of the vehicle, the detection box regions of multiple entity image blocks overlap. The detection box region containing the segmented occluder must contain another detection box region whose content is similar to the occluder. In other words, this occluder has an object within another detection box region that shares similarity features with it. For example, if car A is in front of car B, and car A is an occluder of car B, then within another detection box region, there must be an object similar to car A; they could be two parts of the same vehicle. This similarity is used to determine whether the segmented object is the main body of the vehicle or an occluder.

[0084] Therefore, for each entity image block, other detection box regions are traversed to determine whether similar entities exist. Specifically, for each entity image block in the current image block sub-region, other detection box regions are traversed, and color is compared using cosine similarity, and texture features are compared using Euclidean distance to determine whether similar entities exist.

[0085] If similar entities exist, the occlusion detection logic is set as follows:

[0086] If the area of ​​an entity image patch is larger than the area of ​​a similar entity, and the features of the two match, then the entity image patch is determined to be an occluder within the current detection box area.

[0087] If the area of ​​the entity image block is smaller than the area of ​​a similar entity, then the entity image block is determined to be the vehicle body within the current detection box area.

[0088] If no similar entity exists, the entity image block is determined to be the vehicle body within the current detection box area.

[0089] Throughout the process, the main body of the vehicle (the main visible area of ​​the vehicle) is selected from the multiple entities generated by segmentation (such as the front of the vehicle, the rear of the vehicle, and fragments of obstructions), and obstructions (such as other vehicles and roadblocks) are identified. Finally, the obstructions are associated with the complete vehicle to which they belong.

[0090] Finally, determine the occlusion type of the detection box. Specifically, input the IoU value corresponding to the sub-region of the detection box. Based on the arrangement of IoU values ​​corresponding to the occlusion objects and the detection frame regions, the relationship between the occlusion objects and the three detection frame regions is determined, thus determining the occlusion type of the detection frame. Occlusion types include unilateral occlusion, central occlusion, bilateral occlusion, full occlusion, and no occlusion. Specifically: Unilateral occlusion occurs when the occlusion object covers the right and middle detection frame regions, with the vehicle body on the left; or, the occlusion object covers the left and middle detection frame regions, with the vehicle body on the right. Central occlusion occurs when the occlusion object covers the middle detection frame region, with the vehicle body on the left and right. Bilateral occlusion occurs when the occlusion object covers the left and right detection frame regions, with the vehicle body (e.g., the vehicle body) in the middle. Full occlusion occurs when the vehicle body within the detection frame is largely obscured by the occlusion object (e.g., a car completely obscured by a truck), with only fragments visible. In this case, area correction is used, combined with historical trajectories, to predict the vehicle body. No occlusion occurs when the entire vehicle body within the detection frame is visible.

[0091] S500: Based on the occlusion type of the detection box, combined with the intersection-union ratio and area of ​​the detection box sub-regions, determine the final weight of the detection box region.

[0092] Different occlusion types correspond to different weight allocation methods. Therefore, in embodiments of the present invention, weight allocation is performed based on the occlusion type of the detection box, combined with the intersection-union ratio and area of ​​the detection box sub-regions, to determine the final weight of the detection box region. Further details include:

[0093] First, determine the area of ​​the detection frame covered by the obstruction based on the type of obstruction. Specifically, when there is unilateral obstruction, the obstruction covers the right and middle areas of the detection frame, while the left area is the vehicle body; or, the obstruction covers the left and middle areas of the detection frame, while the right area is the vehicle body. When there is middle obstruction, the obstruction covers the middle area of ​​the detection frame, while the left and right areas are the vehicle body. When there is bilateral obstruction, the obstruction covers the left and right areas of the detection frame, while the middle area is the vehicle body. When there is full obstruction, the vehicle body within the detection frame is largely obscured by the obstruction, with only fragments visible. When there is no obstruction, the entire vehicle body within the detection frame is visible.

[0094] Then, for the detection frame regions covered by occlusions, weights are allocated based on the intersection-union ratio (IoU) and area of ​​the detection frame regions to determine the final weights of the detection frame regions. Specifically, based on the IoU value of each detection frame region covered by an occlusion, a pre-assigned weight is given. The higher the IoU value of a detection frame region, the higher its weight proportion. The proportion of the IoU value of a detection frame region to the sum of the IoU values ​​of the three detection frame regions is defined as the pre-assigned weight coefficient. At this point, the pre-assigned weight coefficients for the detection frame regions covered by occlusions for each occlusion type are:

[0095]

[0096] In the formula, Indicates the pre-assigned weight coefficient; , , These represent the IoU values ​​corresponding to the left, middle, and right detection box regions, respectively. This represents the sum of the IoU values ​​corresponding to the left, middle, and right detection box regions.

[0097] The final weight of each detection frame region is obtained by redistributing the area proportions and pre-assigned weight coefficients of each detection frame region:

[0098]

[0099] In the formula, This represents the final weight of the detection box region covered by each occluded object in the detection box corresponding to the i-th occlusion type; This indicates the length of the adjusted detection frame area; Indicates the width of the detection frame area; This indicates the total length of the adjusted (expanded) detection frame; This indicates the width of the detection box.

[0100] For unobstructed detection bounding box regions, the final weight of the detection bounding box region is determined to be 1. That is, the initial IOU value can be used for unobstructed detection bounding box regions without adjustment.

[0101] Finally, the weight set of each detection box region is output.

[0102] By allocating weights as described above, the weight of occluded detection bounding box regions is reduced, while the weight of unoccluded detection bounding box regions is increased.

[0103] S600: The final weights and intersection-union ratios of the detection box regions are fused to obtain the overall intersection-union ratio of the detection boxes, enabling target tracking and traffic flow statistics.

[0104] The final weights and intersection-union ratios of the detection box regions are fused to obtain the overall intersection-union ratio of the detection boxes, which is then used for target tracking and traffic flow statistics. Specifically, the final weights of each output detection box region are fused with the IoU values ​​of each detection box region using the existing Hungarian algorithm. This involves weighting the IoU values ​​of each detection box region using the final weights to obtain a weighted IoU value, maximizing the average weighted IoU value of the left, middle, and right detection box regions, thus maximizing the overall IoU value of the detection box (increasing the proportion of the IoU value of local regions with high IoU values ​​in the overall IoU value, thereby maximizing the overall IoU value). Then, the IoU values ​​(weighted IoU values) of the three fused detection box regions are added together and averaged to obtain the maximum overall IoU value of the detection box. This maximum overall IoU value is then used in the Byte Track algorithm for subsequent calculations. This solves the problem that when the overall IoU drops below 0.3, the Byte Track algorithm directly discards low-scoring detection boxes. In this case, vehicle tracking would rely solely on the vehicle's appearance features, which may be unreliable due to the loss of key features caused by occlusion.

[0105] Based on the same inventive concept as the above method, this embodiment also provides a traffic flow statistics system based on UAV vision technology.

[0106] Please see Figure 2 This illustrates the basic components of a traffic flow statistics system based on UAV vision technology provided by an embodiment of the present invention.

[0107] like Figure 2 As shown, a traffic flow statistics system based on UAV vision technology includes: a memory 10 and a processor 20, wherein:

[0108] Memory 10 is used to store program code;

[0109] The processor 20 is used to read the program code stored in the memory 10 and execute it to acquire vehicle images under UAV visual tracking, and input the vehicle images into the target detection model; divide the detection boxes output by the target detection model into blocks to obtain detection box sub-regions; analyze the intersection-union ratio (IU / U) relationship between the detection box sub-regions and the predicted boxes of the predicted motion trajectory of the next frame, and output the IU / U value of the detection box region; perform pixel-level segmentation on the image within the detection box region, identify the vehicle body and occlusions in the detection box region, and determine the occlusion type of the detection box; determine the final weight of the detection box region based on the occlusion type of the detection box, combined with the IU / U value and area of ​​the detection box region; fuse the final weight of the detection box region and the IU / U value to obtain the overall IU / U value of the detection box, perform target tracking, and complete traffic flow statistics.

[0110] Furthermore, the processor 20 includes a vehicle image acquisition module 21, a detection box segmentation module 22, an intersection-over-union (IoU) analysis module 23, a weight allocation module 24, and a target tracking module 25. Wherein:

[0111] The vehicle image acquisition module 21 is used to acquire vehicle images under UAV visual tracking and input the vehicle images into the target detection model;

[0112] The detection box segmentation module 22 is used to segment the detection boxes output by the target detection model into sub-regions of the detection boxes;

[0113] The intersection-over-union (IoU) analysis module 23 is used to analyze the IoU relationship between the detected bounding box region and the predicted bounding box of the predicted motion trajectory of the next frame, and output the IoU value of the detected bounding box region.

[0114] The weight allocation module 24 is used to perform pixel-level segmentation of the image within the detection box region, identify the vehicle body and occluders in the detection box region, determine the occlusion type of the detection box, and determine the final weight of the detection box region based on the occlusion type of the detection box, combined with the intersection-union ratio and area of ​​the detection box region.

[0115] The target tracking module 25 is used to fuse the final weight and intersection-union ratio of the detection box region to obtain the overall intersection-union ratio of the detection box, perform target tracking, and complete traffic flow statistics.

[0116] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0117] The various embodiments in this specification are described in a progressive manner. For the same or similar parts between the various embodiments, please refer to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A traffic flow statistics method based on UAV vision technology, characterized in that, The method includes: Acquire vehicle images under UAV visual tracking and input the vehicle images into the target detection model; The detection bounding box output by the target detection model is divided into blocks to obtain detection bounding box sub-regions; Analyze the intersection-union ratio (IU) between the detected bounding box region and the predicted bounding box of the predicted motion trajectory in the next frame, and output the IU value of the detected bounding box region; The image within the detection box area is segmented at the pixel level to identify the vehicle body and occlusions within the detection box area and determine the occlusion type of the detection box. Based on the occlusion type of the detection frame, and combined with the intersection-union ratio and area of ​​the detection frame sub-regions, the final weight of the detection frame region is determined; The final weights and intersection-union ratios of the detection frame regions are fused to obtain the overall intersection-union ratio of the detection frame, which is then used for target tracking and traffic flow statistics. The detection bounding box output by the target detection model is divided into blocks to obtain detection bounding box sub-regions, including: The detection box output by the target detection model is horizontally divided into three sub-regions: left, middle, and right. Based on the vehicle's direction of movement, the left and right sub-regions are dynamically expanded to obtain three detection box regions: left, center, and right. The pre-assigned weight coefficients for the sub-regions of the detection frame covered by occluded objects in the detection frame corresponding to each occlusion type are as follows: In the formula, Indicates the pre-assigned weight coefficient; , , These represent the IoU values ​​corresponding to the left, middle, and right detection box regions, respectively. This represents the sum of the IoU values ​​corresponding to the left, middle, and right detection box regions.

2. The traffic flow statistics method based on UAV vision technology according to claim 1, characterized in that, Based on the vehicle's direction of movement, the left and right sub-regions are dynamically expanded to obtain three detection box regions: left, center, and right, including: When the vehicle moves to the right, the right sub-region expands its coverage area to the right by an additional 'a'% of its original length. When the vehicle moves to the left, the left sub-region expands its coverage area to the left by an additional 'a'% of its original length, resulting in three detection box sub-regions: left, center, and right.

3. The traffic flow statistics method based on UAV vision technology according to claim 1, characterized in that, Analyze the intersection-over-union (IoU) ratio between the detected bounding box region and the predicted bounding box of the predicted motion trajectory in the next frame, and output the IoU value of the detected bounding box region, including: Predict the motion trajectory of the next frame to obtain the predicted bounding box; The ratio of the pixel-level overlap area between the detection box region and the prediction box to the total coverage area is calculated, and a penalty factor is introduced to obtain the intersection-union ratio of the detection box region.

4. The traffic flow statistics method based on UAV vision technology according to claim 3, characterized in that, Predict the motion trajectory of the next frame to obtain the predicted bounding box, including: The motion trajectory of the next frame is predicted based on Kalman filtering, and a uniform velocity model of Taylor third-order expanded extended Kalman filtering is introduced to realize third-order motion compensation for motion trajectory prediction, thus obtaining the prediction box.

5. The traffic flow statistics method based on UAV vision technology according to claim 1, characterized in that, Perform pixel-level segmentation on the image within the detection box region, and identify the vehicle body and occluders within the detection box region, including: The image within the detection box area is segmented at the pixel level to obtain multiple entity image blocks; For each entity image block in the current image block sub-region, traverse other detection box regions to determine whether similar entities exist; If similar entities exist, the occlusion detection logic is set as follows: If the area of ​​the entity image block is greater than the area of ​​the similar entity, and the features of the two match, then the entity image block is determined to be an occluder within the current detection box area; If the area of ​​the entity image block is smaller than the area of ​​the similar entity, then the entity image block is determined to be the vehicle body within the current detection frame area.

6. The traffic flow statistics method based on UAV vision technology according to claim 5, characterized in that, The occlusion types include unilateral occlusion, central occlusion, bilateral occlusion, full occlusion, and no occlusion, wherein: When one side is blocked, the obstruction covers the right and middle detection frame areas of the detection frame, and the left detection frame area is the vehicle body; or, the obstruction covers the left and middle detection frame areas of the detection frame, and the right detection frame area is the vehicle body. When the middle part is obstructed, the obstruction covers the middle detection frame area of ​​the detection frame, while the left and right detection frame areas are the main body of the vehicle; When both sides are blocked, the obstruction covers the left and right detection frame areas of the detection frame, while the middle detection frame area is the vehicle body; When fully obscured, the main body of the vehicle within the detection frame is largely obscured by the obstruction, with only fragments visible. When there is no obstruction, the entire vehicle body within the detection frame is visible.

7. The traffic flow statistics method based on UAV vision technology according to claim 1, characterized in that, Based on the occlusion type of the detection frame, and combined with the intersection-union ratio and area of ​​the detection frame sub-regions, the final weight of the detection frame region is determined, including: Based on the occlusion type of the detection frame, determine the detection frame sub-area covered by the occlusion object; For the detection frame area covered by the obstruction, the intersection-union ratio and area of ​​the detection frame area are combined to perform weight allocation and determine the final weight of the detection frame area; For an unobstructed detection frame region, the final weight of the detection frame region is determined to be 1.

8. A traffic flow statistics system based on UAV vision technology, characterized in that, The system includes: a memory and a processor, wherein: The memory is used to store program code; The processor is configured to read program code stored in the memory and execute the method as described in any one of claims 1 to 7.

9. The traffic flow statistics system based on UAV vision technology according to claim 8, characterized in that, The processor includes: The vehicle image acquisition module is used to acquire vehicle images under UAV visual tracking and input the vehicle images into the target detection model; The detection box segmentation module is used to segment the detection boxes output by the target detection model into sub-regions of the detection boxes; The intersection-over-union (IoU) analysis module is used to analyze the IoU relationship between the detected bounding box region and the predicted bounding box of the predicted motion trajectory of the next frame, and output the IoU value of the detected bounding box region. The weight allocation module is used to perform pixel-level segmentation of the image within the detection box region, identify the vehicle body and occlusions in the detection box region, determine the occlusion type of the detection box, and determine the final weight of the detection box region based on the occlusion type of the detection box, combined with the intersection-union ratio and area of ​​the detection box region. The target tracking module is used to fuse the final weight and intersection-union ratio of the detection box region to obtain the overall intersection-union ratio of the detection box, perform target tracking, and complete traffic flow statistics. The detection bounding box output by the target detection model is divided into blocks to obtain detection bounding box sub-regions, including: The detection box output by the target detection model is horizontally divided into three sub-regions: left, middle, and right. Based on the vehicle's direction of movement, the left and right sub-regions are dynamically expanded to obtain three detection box regions: left, center, and right. The pre-assigned weight coefficients for the sub-regions of the detection frame covered by occluded objects in the detection frame corresponding to each occlusion type are as follows: In the formula, Indicates the pre-assigned weight coefficient; , , These represent the IoU values ​​corresponding to the left, middle, and right detection box regions, respectively. This represents the sum of IoU values ​​corresponding to the left, middle, and right detection box regions.

Images