Vehicle tracking method and device, electronic equipment and storage medium

Abstract

The invention provides a vehicle tracking method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a vehicle image of a target vehicle when it is monitored that the target vehicle is located in front of and behind a road stop line and is shielded, the vehicle image comprises a first vehicle image when the target vehicle is in front of and behind a road stop line and is not shielded and a second vehicle image reproduced after shielding; carrying out feature extraction on the vehicle image, and obtaining a vehicle target feature according to the correlation among a plurality of vehicle local features obtained through feature extraction and a vehicle global feature; based on the first target feature and the second target feature, the shielded target vehicle is tracked, the problem that vehicle tracking is lost under the conditions of posture change and strong shielding in a traditional scheme is solved, the target vehicle can be accurately tracked in a complex and changeable traffic environment, and the vehicle tracking accuracy is improved. Data support can be provided for repairing of the vehicle motion trail under the shielding condition, and powerful support is provided for violation judgment based on the continuous motion trail.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more particularly to a vehicle tracking method, apparatus, electronic device, and storage medium. Background Technology

[0002] Vehicle tracking is a crucial technology in fields such as intelligent traffic monitoring, autonomous driving, and video analytics, playing a significant role in road safety, traffic management, and the monitoring of illegal activities. However, in practical applications, especially in complex and ever-changing traffic scenarios, vehicle occlusion is extremely common, particularly when large vehicles obstruct smaller vehicles in front for extended periods, posing a serious challenge to traditional vehicle tracking methods.

[0003] Existing vehicle tracking technologies have achieved some success in handling short-term, short-range occlusion. These methods typically involve modeling the occluder to determine the area where the target may reappear, or combining short-term tracking and appearance feature matching. However, these methods have significant limitations when dealing with long-term, complete occlusion scenarios. Specifically, methods based on occlusion modeling are limited by the size and range of the occlusion, making it difficult to effectively handle long-term, large-scale occlusion caused by large vehicles. Furthermore, methods based on short-term tracking and appearance feature matching suffer from significantly reduced accuracy in trajectory prediction based on appearance matching due to the diversity of vehicle trajectories within the occluded area and the significant differences in vehicle posture before and after occlusion. Summary of the Invention

[0004] This invention provides a vehicle tracking method, apparatus, electronic device, and storage medium to address the shortcomings of existing vehicle tracking methods under occlusion conditions, which are only applicable to short-term, short-distance occlusion scenarios and cannot cope with long-term, completely occluded driving scenarios. It can accurately track target vehicles under strong occlusion conditions in complex and ever-changing traffic environments, ensuring the continuity and effectiveness of vehicle tracking and assisting in the identification and judgment of traffic violations.

[0005] This invention provides a vehicle tracking method, comprising: When a target vehicle is detected to be in front of or behind the road stop line and is obscured, a vehicle image of the target vehicle is acquired. The vehicle image includes a first vehicle image when the target vehicle is in front of or behind the road stop line and is not obscured, and a second vehicle image that reappears after being obscured. Feature extraction is performed on the vehicle image to obtain the global vehicle features and multiple local vehicle features of the target vehicle. Based on the correlation between the local vehicle features and the global vehicle features, feature fusion is performed to obtain the vehicle target features. The vehicle target features include a first target feature corresponding to the first vehicle image and a second target feature corresponding to the second vehicle image. Based on the first target feature and the second target feature, vehicle tracking is performed on the occluded target vehicle.

[0006] According to a vehicle tracking method provided by the present invention, the step of extracting features from the vehicle image to obtain global vehicle features and multiple local vehicle features of the target vehicle includes: Global feature extraction is performed on the vehicle image to obtain the vehicle's global features; Key point detection is performed on the vehicle image to obtain key point detection results; Based on the location of each key point in the key point detection results and the global features of the vehicle, local features are extracted to obtain the local features of the vehicle for each key point.

[0007] According to a vehicle tracking method provided by the present invention, the key point detection result further includes the confidence level of the position of each key point; The process of fusing features based on the correlation between local features of each vehicle and the global features of the vehicle to obtain vehicle target features includes: Based on the confidence level of the position of each key point, the occluded key points are filtered out from the key points to obtain each target key point; Based on the local vehicle features of each target key point, feature stitching is performed, and the features obtained by feature stitching are divided into feature blocks to obtain multiple local vehicle sub-features. Based on the correlation between the local sub-features of each vehicle and the global features of the vehicle, feature fusion is performed to obtain the target features of the vehicle.

[0008] According to a vehicle tracking method provided by the present invention, the step of performing feature fusion based on the correlation between local sub-features of each vehicle and the global features of the vehicle to obtain vehicle target features includes: The local sub-features of any vehicle are concatenated with the local sub-features of other vehicles, and the concatenated features are convolved to obtain the local fusion features corresponding to the local sub-features of any vehicle. Based on any vehicle local sub-feature and its corresponding local fusion feature, determine the local related features corresponding to any vehicle local sub-feature; Based on the local related features corresponding to the local sub-features of each vehicle and the global features of the vehicle, feature fusion is performed to obtain the vehicle target features.

[0009] According to a vehicle tracking method provided by the present invention, the method further includes, after tracking the occluded target vehicle based on the first target feature and the second target feature, the following steps: If the target vehicle is tracked, determine the position of the target vehicle when it is in front of or behind the road stop line and is obscured, and the position where it reappears after being obscured; Based on the road traffic information of the monitored road where the target vehicle is located, the position of the target vehicle when it is obscured and the position when it reappears after being obscured, a vehicle motion trajectory model is applied to predict the trajectory and obtain the motion trajectory of the target vehicle when it is obscured. The vehicle trajectory model is obtained by fitting historical vehicle movement information and road traffic information of the monitored road.

[0010] According to a vehicle tracking method provided by the present invention, the historical vehicle motion information includes the motion trajectories of vehicles in each lane under unobstructed conditions; The vehicle trajectory model for any lane on the monitored road is determined based on the following steps: Based on the trajectory of the first vehicle in any lane and the road traffic information of any lane, curve fitting is performed to obtain the initial motion curve of the vehicle in any lane. Based on the motion trajectories of other vehicles in any lane, the initial motion curve of the vehicle in any lane is corrected to obtain the vehicle motion trajectory model of any lane.

[0011] According to a vehicle tracking method provided by the present invention, the motion trajectory of the target vehicle is obtained when it is obscured, and the method further includes: The initial tracking parameters are determined based on the position of the target vehicle when it is obscured. Based on the initial tracking parameters and the reappearance position of the target vehicle after occlusion, the motion trajectory of the target vehicle under occlusion conditions is updated to obtain the target motion trajectory under occlusion conditions.

[0012] The present invention also provides a vehicle tracking device, comprising: The image acquisition unit is used to acquire a vehicle image of the target vehicle when the target vehicle is detected to be in front of or behind the road stop line and is obscured. The vehicle image includes a first vehicle image of the target vehicle when it is in front of or behind the road stop line and is not obscured, and a second vehicle image that reappears after being obscured. An image processing unit is used to extract features from the vehicle image to obtain global vehicle features and multiple local vehicle features of the target vehicle, and to perform feature fusion based on the correlation between the local vehicle features and the global vehicle features to obtain vehicle target features. The vehicle target features include a first target feature corresponding to the first vehicle image and a second target feature corresponding to the second vehicle image. A vehicle tracking unit is used to track the occluded target vehicle based on the first target feature and the second target feature.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle tracking method as described above.

[0014] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle tracking method as described above.

[0015] The vehicle tracking method, apparatus, electronic device, and storage medium provided by this invention extract global vehicle features and multiple local vehicle features from vehicle images before and after occlusion. Based on the correlation between the local vehicle features and the global vehicle features, feature fusion is performed, and vehicle tracking is performed based on the vehicle target features obtained after fusion. This overcomes the problem of vehicle tracking loss under attitude changes and strong occlusion in traditional solutions. It can accurately track target vehicles in complex and ever-changing traffic environments and can effectively address the problem of target tracking being difficult in long-term, completely occluded scenarios. It has high robustness and real-time performance, and can provide data support for repairing vehicle motion trajectories under occlusion, providing strong support for violation judgment based on continuous motion trajectories. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the vehicle tracking method provided in an embodiment of the present invention; Figure 2 This is an example diagram illustrating the process of a vehicle being obscured, as provided in an embodiment of the present invention. Figure 3 This is an example diagram of the key point detection process provided in an embodiment of the present invention; Figure 4 This is an example diagram of monitoring traffic rules on roads provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the vehicle tracking device provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0019] Currently, most vehicle tracking methods in road traffic monitoring under occlusion conditions are designed for short-term, short-distance occlusion scenarios. These methods typically model the occluder to determine the area where the target may reappear, or combine short-term tracking, trajectory prediction, and appearance feature matching to achieve vehicle tracking. However, they have significant shortcomings when dealing with long-term, complete occlusion scenarios. Specifically, methods for modeling occluders are limited by the size and range of the occlusion, making it difficult to effectively handle long-term, large-scale occlusion caused by large vehicles; while methods based on trajectory prediction and appearance feature matching suffer from significantly reduced appearance matching effectiveness and trajectory prediction failure due to the diversity of vehicle trajectories within the occlusion area and the significant differences in vehicle posture before and after occlusion.

[0020] In particular, when large vehicles such as trucks and long trailers approach key traffic nodes (such as stop lines at intersections), smaller vehicles ahead of them may be completely obscured while waiting for the traffic light, making it impossible to directly observe their specific positions and subsequent trajectories in the image. Current methods for handling short-term, short-distance obstruction scenarios are simply inadequate for such situations, failing to effectively track the complete trajectory of smaller vehicles that have been completely obscured for extended periods after the obstruction is lifted. This not only limits the practicality and accuracy of vehicle tracking technology but also results in a lack of necessary evidence in areas such as traffic violation monitoring and accident liability determination. Therefore, how to achieve vehicle tracking under prolonged, complete obstruction conditions has become an urgent problem to be solved.

[0021] To address this issue, the present invention provides a vehicle tracking method that extracts local and global vehicle features from vehicle images before and after occlusion. By fusing local and global features, it solves the problem of vehicle tracking loss under pose changes and strong occlusion conditions. This method can accurately match and track target vehicles in complex and ever-changing traffic environments, exhibiting high robustness and real-time performance. It can provide data support for the repair of vehicle motion trajectories under occlusion conditions and provide strong technical support for the construction and application of intelligent transportation systems. Figure 1 This is a flowchart illustrating the vehicle tracking method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes: Step 110: When the target vehicle is detected to be in front of or behind the road stop line and is obscured, acquire the vehicle image of the target vehicle. The vehicle image includes a first vehicle image when the target vehicle is in front of or behind the road stop line and is not obscured, and a second vehicle image that reappears after being obscured. Step 120: Extract features from the vehicle image to obtain the global features of the target vehicle and multiple local features of the vehicle. Based on the correlation between the local features of the vehicle and the global features of the vehicle, feature fusion is performed to obtain the vehicle target features. The vehicle target features include the first target features corresponding to the first vehicle image and the second target features corresponding to the second vehicle image. Step 130: Based on the first target features and the second target features, perform vehicle tracking on the occluded target vehicle.

[0022] Specifically, considering that the current methods used for vehicle tracking on the road under occlusion conditions, such as occlusion modeling or appearance feature matching, are only applicable to short-term, short-distance occlusion scenarios and cannot cope with long-term, complete occlusion scenarios, the vehicle tracking is easily lost in such scenarios. In this embodiment of the invention, a vehicle re-identification is proposed by using the global and local features of the vehicle in the vehicle images before and after occlusion, thereby achieving accurate vehicle tracking under strong occlusion or even complete occlusion.

[0023] Understandably, in actual driving, when a vehicle enters the monitoring range, if there are both large and small vehicles in the same lane, the large vehicle in front of the stop line will completely block the view of the small vehicle in front of it. Figure 2 This is an example diagram illustrating the process of a vehicle being obscured, as provided in an embodiment of the present invention. Figure 2 As shown, from the moment a small vehicle stops in front of the road stop line and is blocked by a large vehicle entering in the same lane, it takes a long time for the small vehicle to completely leave the blocked area. During this period, the vehicle is completely blocked for a long time, which makes it impossible to detect the small vehicle for a long time. Prolonged blocking will cause tracking to be interrupted, making it impossible to determine the vehicle's movement trajectory and resulting in loss of vehicle tracking.

[0024] This invention proposes extracting local vehicle features and fusing them based on the correlation between multiple local features and global vehicle features to address the problem of vehicle re-identification under pose changes and strong occlusion conditions. Compared to traditional methods that extract features from the entire vehicle region using convolutional neural networks and then perform matching based on the extracted overall features, this invention offers better accuracy and superior performance.

[0025] Specifically, when a target vehicle is detected as being obscured before or after the road stop line, where "before or after" can be understood as a certain distance (e.g., 2 meters, 3 meters, etc.) in front of and behind the stop line, the image captured by the surveillance camera near the stop line determines that the target vehicle to be tracked has entered the monitoring field of view and is obscured before or after the stop line. At this point, the image of the target vehicle before the obscuration, i.e., the image of the vehicle when it was not obscured before or after the stop line, is obtained. This is called the first vehicle image. By comparing the vehicle images before and after the obscuration within the most recent time, vehicle re-identification is achieved. That is, after obtaining the first vehicle image, to achieve vehicle re-identification, avoid vehicle tracking loss, and ensure the continuity of vehicle tracking, this embodiment of the invention also needs to obtain the vehicle image after the obscuration, i.e., the image of the vehicle that begins to leave the obscured area. This is called the second vehicle image.

[0026] It is worth noting that, since vehicle tracking is a continuous process, in order to ensure the continuity of tracking and the accuracy of vehicle re-identification, in this embodiment of the invention, after the target vehicle is detected to be obscured, since it is difficult to know exactly when and where the target vehicle will leave the obscured area, in order to ensure the accuracy and completeness of tracking, the target vehicle can be continuously monitored to capture it leaving the obscured area in a timely manner, reproduce the image after the field of view, and use it as the second vehicle image for comparison and judgment with the first vehicle image, so as to achieve the re-identification of the target vehicle.

[0027] For example, based on the lane where the target vehicle is located when it is obscured, and the lane guidance of that lane, it can be determined which direction and lane the target vehicle will enter next. Then, road images can be collected from surveillance cameras in the direction and lane it may go, and the second vehicle image can be selected from them.

[0028] For example, when the monitoring range of the surveillance cameras around / near the current road stop line is large enough, the second vehicle image can be obtained by filtering the image captured by the surveillance camera after it is blocked. The reason is that when the monitoring range is large enough and the driving direction has not changed, it can monitor the situation after the blocked target vehicle leaves the blocked area and reproduce the monitoring field of vision.

[0029] After determining the first and second vehicle images, before performing vehicle re-identification to achieve vehicle tracking, this embodiment of the invention further processes these images to extract features and then performs vehicle re-identification based on these features. Specifically, considering the low accuracy and high recognition difficulty of traditional appearance feature matching and overall matching based on convolutional neural networks, this embodiment proposes extracting global and local vehicle features. Vehicle re-identification is achieved by fusing features based on the correlation between multiple local vehicle features and global vehicle features. Compared to whole-vehicle matching, local features contain richer detailed information, enabling accurate differentiation of vehicles with similar appearance features. Combined with global features, this provides a more complete and comprehensive reflection of the target vehicle's information, including not only global and local features but also local detailed information, thus enabling accurate vehicle re-identification.

[0030] In detail, in this embodiment of the invention, when processing vehicle images, feature extraction can be performed on the acquired vehicle images first. That is, feature extraction can be performed on the first vehicle image and the second vehicle image respectively to extract their respective global and local features, thereby obtaining global vehicle features and local vehicle features. Specifically, a feature extraction network can be used to extract features from the first vehicle image and the second vehicle image respectively, to extract local detail information and global vehicle information, thereby obtaining the corresponding global vehicle features and multiple local vehicle features for each of the first and second vehicle images. To ensure the relevance and effectiveness of feature extraction, the feature extraction network can be pre-trained based on sample data under occlusion conditions.

[0031] Furthermore, after obtaining the global features and multiple local features of the target vehicle, in order to ensure the accuracy of identification, in this embodiment of the invention, these two can be fused, that is, the feature information contained in these two can be fused to obtain a fused feature that can more completely, comprehensively and accurately reflect the information of the target vehicle, namely the vehicle target feature.

[0032] Specifically, a feature fusion network can be used here to fuse multiple local and global vehicle features to integrate local details from the local features and global vehicle information from the global features, thereby obtaining more accurate and reliable vehicle target features. However, considering that the multiple local vehicle features obtained through feature extraction are "fragmented" and lack correlation with each other, in order to make the fused features more discriminative and capable of more accurate vehicle tracking, this embodiment of the invention can utilize the relationships between local vehicle features to perform feature fusion, thereby obtaining more complete and comprehensive information reflecting the target vehicle and more discriminative vehicle target features. That is, vehicle target features can be obtained based on the correlation between multiple local vehicle features and global vehicle features. Specifically, this can be achieved by using an attention mechanism to associate multiple local vehicle features and then combining them with global vehicle features to obtain vehicle target features.

[0033] The above fusion process yields a first target feature corresponding to the first vehicle image and a second target feature corresponding to the second vehicle image. The first target feature is a fused feature obtained by fusing global and local vehicle features from the first vehicle image, while the second target feature is a fused feature obtained by fusing global and local vehicle features from the second vehicle image.

[0034] After this, since the obtained vehicle target features can clearly and completely reflect the information of the target vehicle, vehicle re-identification can be performed directly based on this target vehicle. That is, vehicle matching can be performed based on the first and second target features in the vehicle target features to achieve vehicle tracking. Specifically, this can be done by calculating the feature similarity of the target vehicle before and after occlusion based on the first and second target features, and determining whether re-identification is successful based on the feature similarity. For example, cosine similarity, Euclidean distance, etc., can be used to determine whether it is the same vehicle before and after occlusion, thereby determining whether vehicle re-identification is successful and whether the target vehicle is successfully tracked.

[0035] It should be noted that when a successful match is determined based on the first vehicle image and the second vehicle image acquired at a certain moment, the vehicle re-identification can be confirmed as successful, and the target vehicle can be successfully tracked. Conversely, when a successful match is not achieved, the vehicle target feature can be matched with the first vehicle image based on the second vehicle image obtained through continuous monitoring (such as a vehicle image with a smaller occlusion area after it begins to leave the occlusion area) until a successful match is achieved.

[0036] It is worth noting that, in addition to the vehicle re-identification method based on vehicle target features mentioned above, vehicle re-identification can also be performed based on license plate numbers in this embodiment of the invention. That is, since the license plate number is the vehicle identification number and is unique, if the license plate number can be identified before and after occlusion, denoted as LastPlate and NowPlate respectively, vehicle re-identification can be performed based on the license plate number. Specifically, a field-by-field matching method can be used on LastPlate and NowPlate to calculate the number of identical characters between them, denoted as PlateSameNum. If PlateSameNum equals the total number of characters in the license plate number (7 characters for gasoline vehicles and 8 characters for new energy vehicles), then the match is considered successful; otherwise, the match is considered unsuccessful. For example, when the license plate number before and after occlusion is identified as LastPlate = "1234567" and NowPlate = "1234567", PlateSameNum can be determined to be 7, thus confirming a successful match.

[0037] As a preferred embodiment of the invention, vehicle re-identification is performed by combining license plate number with vehicle target features. That is, if the license plate number can be identified both before and after occlusion, vehicle re-identification can be performed based on the license plate number first. If the license plate number before and after occlusion matches successfully, vehicle tracking can be performed based on this. Otherwise, if the match fails, vehicle target features corresponding to the vehicle images before and after occlusion can be used for vehicle re-identification to achieve vehicle tracking. In this way, vehicle tracking efficiency can be improved while ensuring tracking accuracy.

[0038] In this embodiment of the invention, vehicle re-identification is performed based on vehicle images before and after occlusion. If the license plate number can be identified, accurate re-identification is performed based on the license plate number. If the license plate number cannot be identified or the license plate number matching is unsuccessful, re-identification is performed based on vehicle target features. This effectively improves the matching effect of target vehicles under various postures and realizes vehicle tracking before and after occlusion.

[0039] The vehicle tracking method provided by this invention extracts local and global vehicle features from vehicle images before and after occlusion, fuses these features, and tracks the vehicle based on the fused target features. This overcomes the problem of vehicle tracking loss under pose changes and strong occlusion in traditional solutions, enabling accurate tracking of target vehicles in complex and ever-changing traffic environments. It effectively addresses the issue of target tracking being difficult in long-term, completely occluded scenarios, exhibiting high robustness and real-time performance. Furthermore, it provides data support for repairing vehicle motion trajectories under occlusion and provides strong support for violation judgment based on continuous motion trajectories.

[0040] Based on the above embodiments, in step 120, feature extraction is performed on the vehicle image to obtain the global features and multiple local features of the target vehicle, including: Global feature extraction is performed on the vehicle image to obtain the vehicle's global features; Key point detection is performed on the vehicle image to obtain the key point detection results; Based on the location of each key point in the key point detection results and the global features of the vehicle, local features are extracted to obtain the local features of the vehicle for each key point.

[0041] Specifically, step 120, which involves extracting features from the vehicle image to obtain the global features and multiple local features of the target vehicle, includes the following steps: After obtaining the vehicle images, keypoint localization can be performed on the first and second vehicle images respectively to determine the positions of keypoints of the target vehicle in the first vehicle image and the positions of keypoints of each vehicle in the second vehicle image, thereby obtaining keypoint detection results. Specifically, a keypoint detection network can be used for keypoint detection; that is, the first and second vehicle images are input into the keypoint detection network for fixed-point detection, thereby obtaining the keypoint detection results output by the network. The position and number of fixed points can be set according to actual conditions and needs; this embodiment of the invention does not impose specific limitations on this. Figure 3 This is an example diagram of the key point detection process provided in the embodiments of the present invention, such as... Figure 3 As shown, it can detect 10 fixed points on the roof, windows, and front / rear of the vehicle to obtain key point detection results.

[0042] Here, the keypoint detection result can include only the location of the keypoints of the vehicle in the vehicle image, or it can include both the location of the keypoints and the confidence level, that is, the confidence level of each pixel in the vehicle image as a keypoint of the vehicle. In this case, the keypoint detection result is actually the probability distribution / heatmap of the keypoints. The probability distribution includes the location of each possible keypoint and its corresponding probability. The location of the keypoint and its confidence level can be determined by different colors in the heatmap.

[0043] Simultaneously, global feature extraction can be performed to obtain global vehicle features. That is, global feature extraction can be performed on the vehicle image to extract global vehicle information, thereby obtaining global vehicle features. Specifically, this is achieved by using a feature extraction network to perform convolution operations on the vehicle image to extract features related to the vehicle's position, thus obtaining global vehicle features.

[0044] Then, based on the extracted global vehicle features and the keypoint detection results, local features can be extracted to obtain the vehicle's local features. That is, the local vehicle features of each keypoint can be directly calculated based on the position of each keypoint in the keypoint detection results and the global vehicle features.

[0045] Specifically, this could involve performing a multiplication operation based on the positions of each keypoint in the keypoint detection results and the vehicle's global features, using the vehicle's global features... By multiplying the values ​​of each key point by the values ​​of the key points, we can obtain the local vehicle features at each key point. The calculation formula is shown below:

[0046] in, A heatmap showing the heat of each key point.

[0047] Based on the above embodiments, the key point detection results also include the confidence level of the position of each key point; In step 120, feature fusion is performed based on the correlation between local features of each vehicle and the global features of the vehicle to obtain the vehicle target features, including: Based on the confidence level of each key point's location, occluded key points are filtered out from each key point to obtain each target key point; Based on the local vehicle features of each target key point, feature stitching is performed, and the features obtained by feature stitching are divided into feature blocks to obtain multiple local vehicle sub-features; Based on the correlation between local sub-features of each vehicle and the global features of the vehicle, feature fusion is performed to obtain the target features of the vehicle.

[0048] To ensure the accuracy of vehicle re-identification and avoid the impact of pose changes and occlusion on target vehicle tracking, thereby achieving accurate and reliable vehicle tracking, this embodiment of the invention proposes an adaptive feature segmentation method. This method segments the local features of the vehicle into blocks and fuses the segmented local sub-features and global features of the vehicle to obtain the target vehicle features.

[0049] In detail, this process can begin by filtering keypoints based on the confidence level of their positions in the keypoint detection results. This removes occluded keypoints, leaving only the unoccluded keypoints, referred to as target keypoints. Specifically, based on the confidence level of each keypoint's position obtained from keypoint detection, occluded keypoints are removed. This is done by using the confidence level as a reference, removing keypoints with a confidence level below a threshold (e.g., removing keypoints with a confidence level below 0.6), and retaining the remaining keypoints to obtain the target keypoints. The threshold for removing occluded keypoints can be set according to actual circumstances and requirements.

[0050] Next, feature stitching is performed based on the local vehicle features of each target key point. Then, the stitched features are divided into blocks to obtain multiple local vehicle sub-features. That is, the local vehicle features of each target key point retained in the previous step are stitched together to form a whole. This whole is then divided into several sub-blocks, resulting in multiple local vehicle sub-features after block division. Here, the feature block division uses an equal division method. The number of sub-features after block division can be set according to actual conditions and requirements, for example, it can be 3 blocks, 5 blocks, 6 blocks, etc. This embodiment of the invention does not impose a specific limitation.

[0051] Then, feature fusion can be performed on the local sub-features of each vehicle and the global features of the vehicle to obtain the target vehicle features. That is, feature fusion can be performed based on the correlation between the local sub-features of each vehicle and the global features of the vehicle to obtain the target vehicle features.

[0052] Specifically, this can be achieved by associating the fragmented local sub-features of the vehicle after segmentation using the attention concept, and then combining these with global vehicle features to obtain the target vehicle feature. That is, global max pooling can be performed on each segmented local sub-feature to highlight key information, followed by feature association. Specifically, local sub-features are fused based on the correlation between key information, and finally, the target vehicle feature is obtained by combining it with global vehicle features.

[0053] Based on the above embodiments, feature fusion is performed based on the correlation between local sub-features of each vehicle and the global features of the vehicle to obtain vehicle target features, including: The local sub-features of any vehicle are concatenated with the local sub-features of other vehicles, and the concatenated features are convolved to obtain the local fusion features corresponding to the local sub-features of any vehicle. Based on any vehicle local sub-feature and its corresponding local fusion feature, determine the local related features corresponding to any vehicle local sub-feature; The vehicle target features are obtained by fusing features based on the local related features corresponding to the local sub-features of each vehicle and the global features of the vehicle.

[0054] Specifically, the process of fusing features based on the correlation between local sub-features of each vehicle and global vehicle features to obtain the target features of the vehicle may include: Considering that after obtaining the local sub-features of each vehicle through block operation, each local sub-feature of the vehicle is completely isolated and has no relation to each other, but for the key points of the target vehicle, there is a connection between them, therefore, in this embodiment of the invention, the relationship between the local sub-features of the vehicle obtained after block division can be established based on the key points to obtain local related features, so that the features of each part are more identifiable.

[0055] In detail, for any vehicle local sub-feature, in this embodiment of the invention, the vehicle local sub-feature and other vehicle local sub-features other than the vehicle local sub-feature can be convolved to obtain the target vehicle local sub-feature corresponding to the vehicle local sub-feature, and the target vehicle local sub-features corresponding to the other vehicle local sub-features.

[0056] Next, the target vehicle local sub-features corresponding to the local sub-features of this vehicle, as well as the target vehicle local sub-features corresponding to the local sub-features of other vehicles, can be concatenated, and the concatenated features can be further convolved to obtain the local fusion features corresponding to the local sub-features of this vehicle.

[0057] Then, based on the local sub-features of the vehicle and the local fusion features corresponding to the local sub-features of the vehicle, the local related features corresponding to the local sub-features of the vehicle can be determined. Specifically, the corresponding positions of the target vehicle local sub-features and the local fusion features corresponding to the local sub-features of the vehicle are added together, and the added feature is used as the local related features corresponding to the local sub-features of the vehicle. Repeating the above process can obtain the local related features corresponding to each local sub-feature of the vehicle.

[0058] Then, feature fusion can be performed based on the local related features corresponding to each vehicle's local sub-features and the vehicle's global features to obtain the vehicle target features. Specifically, this can involve concatenating the local related features corresponding to each vehicle's local sub-features and the vehicle's global features, and using the concatenated features as the vehicle target features. After obtaining the vehicle target features, vehicle re-identification can be performed based on these vehicle target features to achieve vehicle tracking.

[0059] It should be noted that in this embodiment of the invention, both image processing and vehicle re-identification can be implemented through a vehicle re-identification model. Specifically, image processing can be performed using the keypoint detection network, feature extraction network, and feature fusion network within the vehicle re-identification model, and vehicle re-identification can be performed using the vehicle re-identification network, ultimately yielding the re-identification result output by the model. Here, the vehicle re-identification model can be built upon a deep neural network; it can be trained based on classification loss and triplet loss, where anchor points belong to the same vehicle as positive samples and different vehicles than negative samples. Backpropagation is used to update parameters during model training.

[0060] The above process will be explained below using an example where a vehicle has five local sub-features: First, perform a convolution operation on the local sub-feature P1 of the vehicle to obtain the feature. Convolution operations are also performed on sub-blocks P2-P5 to obtain features. ; Next, and The concatenation is performed, and then convolution is performed again to obtain C1; Subsequently, C1 and Adding the corresponding positions together yields the local correlation features of P1; Then, repeat the above process to obtain the local correlation features of P2-P5, ultimately resulting in 5 sets of local correlation features. Each set of features has a dimension of 1x1xd (height x width x number of channels). Finally, the five sets of local related features can be concatenated with the vehicle global features (feature dimension 1x1xd). The concatenated features are used as the vehicle target features (feature dimension 1x1x6d). Based on these vehicle global features, Euclidean distance can be used to calculate similarity, and finally determine whether the vehicles participating in the matching are the same vehicle.

[0061] The method provided in this invention performs vehicle re-identification based on vehicle target features extracted from vehicle images before and after occlusion. This method can effectively address the difficulty of vehicle tracking in long-term, completely occluded scenarios, accurately track target vehicles in complex and ever-changing traffic environments, and effectively improve the robustness of vehicle tracking under pose changes and strong occlusion conditions.

[0062] Furthermore, considering the increasing traffic volume and the resulting severity of road traffic problems, electronic traffic enforcement cameras are currently used to adjudicate and record vehicle violations to effectively deter such behavior. According to traffic management regulations, when automatically recording vehicle violations, information on two to three locations during the violation should be recorded to form an ideal chain of evidence. For example, regarding the violation of not driving in the designated lane, the ideal chain of evidence required by the traffic management department should include three images: a capture image of the vehicle before crossing the stop line, a capture image of the vehicle completely crossing the stop line, and a capture image of the vehicle completely crossing the stop line but not located within the designated area. Therefore, after re-identification, to determine whether a violation has occurred due to occlusion, this embodiment of the invention also requires repairing the target vehicle's trajectory under occlusion conditions to ensure the continuity of the vehicle's trajectory.

[0063] That is, in step 130, based on the first target features and the second target features, vehicle tracking is performed on the occluded target vehicle, and then the process further includes: If the target vehicle is tracked, determine the position of the target vehicle when it is in front of or behind the road stop line and is obscured, as well as the position where it reappears after being obscured; Based on the road traffic information of the monitored road where the target vehicle is located, the location of the target vehicle when it is obscured and the location where it reappears after being obscured, and the vehicle motion trajectory model, trajectory prediction is performed to obtain the motion trajectory of the target vehicle when it is obscured. The vehicle trajectory model is obtained by fitting historical vehicle movement information and road traffic information from the monitored roads.

[0064] For vehicle tracking, noise accumulates quadratically over time when a match fails, and the longer the target vehicle is lost, the more noise accumulates, causing the tracked trajectory to deviate significantly from the vehicle's actual movement trajectory. Trajectory tracking has a large error. To ensure the authenticity of the movement trajectory, in this embodiment of the invention, after vehicle re-identification, trajectory prediction requires a comprehensive understanding of the spatiotemporal dynamics of the environment, including the past states and interaction patterns of all observable vehicles, road scene data, and surrounding area data. Based on this, trajectory prediction is performed to obtain an accurate and complete movement trajectory of the target vehicle under occlusion conditions. Specifically, a vehicle movement trajectory model on the monitored road can be pre-fitted using historical vehicle movement information and road traffic information of the monitored road where the target vehicle is located. Then, trajectory prediction is performed based on this vehicle movement trajectory model to obtain the movement trajectory of the target vehicle when it is occluded.

[0065] Specifically, in this embodiment of the invention, when a target vehicle is successfully matched through vehicle re-identification, the position of the target vehicle when it is obscured before or after the road stop line, and the position where it reappears after being obscured, are determined. Here, the position of the target vehicle when it is obscured before or after the road stop line can be determined by the first vehicle image, while the position where it reappears after being obscured can be determined by the second vehicle image when the match is successful.

[0066] Simultaneously, road traffic information of the monitored road where the target vehicle is located can be obtained. This road traffic information can include both road information and traffic information. Road information can include road topology, such as the location of each lane and pedestrian crossing on the monitored road. This information can be used to pre-filter stationary vehicles, pedestrians, and non-motorized vehicles in the images captured by the surveillance cameras, preventing them from affecting subsequent vehicle tracking. Traffic information can include traffic rules and traffic light status. Here, traffic rules refer to lane guidance for each lane on the monitored road. Figure 4 This is an example diagram of traffic rules on monitored roads provided in an embodiment of the present invention, such as... Figure 4 As shown, based on the arrows indicating the stop line, it can be determined that the leftmost lane allows left turns and straight-ahead travel, the middle lane only allows straight-ahead travel, and the rightmost lane allows right turns and straight-ahead travel. Traffic light status refers to the status of the traffic lights at the intersection ahead. If the light is red and the duration is 90 seconds, then the target vehicle must be stopped within that time period. In this case, the target vehicle's position and tracking parameters do not need to be updated during trajectory prediction to prevent tracking deviation or loss.

[0067] Road traffic information can be obtained through surveillance cameras or manually configured. Additionally, traffic light status can be acquired in real time. After obtaining this information, it is converted into a digital model for computer processing during trajectory prediction. This prior knowledge not only filters out interference, improving the efficiency of target vehicle trajectory prediction, but also significantly enhances the accuracy of the predicted trajectory.

[0068] Furthermore, after obtaining the road traffic information of the monitored road, as well as the location of the target vehicle when it was obscured and the location where it reappeared after being obscured, in this embodiment of the invention, the vehicle motion trajectory model on the monitored road can be applied to predict the trajectory based on this road traffic information and the location of the target vehicle when it was obscured and the location where it reappeared after being obscured. That is, based on this vehicle motion trajectory model, combined with the road traffic information of the monitored road, the location of the target vehicle when it was obscured and the location where it reappeared after being obscured, trajectory prediction is performed to obtain the motion trajectory of the target vehicle when it is obscured.

[0069] Specifically, this can be achieved by determining the lanes of the target vehicle before and after the occlusion, based on its initial and subsequent reappearance positions. The vehicle trajectory model for this lane is then determined from the vehicle trajectory model of the monitored road. Starting from the target vehicle's position before or after the stop line and when it is occluded, timestamps are input into this trajectory model to obtain the possible locations of the target vehicle within the occluded area. These locations are used as fitted trajectory points. These trajectory points are then concatenated to obtain the target vehicle's trajectory under occlusion conditions. In other words, by concatenating the timestamps corresponding to the target vehicle's positions from before or after the stop line to when it exits the occluded area, the entire trajectory of the target vehicle, including the trajectory under occlusion conditions, is obtained. Here, the process of trajectory prediction using the vehicle trajectory model is essentially a maintenance and repair process for the target vehicle's trajectory, based on which its trajectory under occlusion conditions can be obtained.

[0070] It's important to note that before applying a vehicle trajectory model for trajectory prediction, it's typically necessary to pre-construct a vehicle trajectory model for the monitored road. This model can be obtained by fitting historical vehicle movement information and road traffic information of the monitored road. Specifically, this involves pre-acquiring historical vehicle movement information and road traffic information, and then fitting vehicle trajectories based on this information to obtain the vehicle trajectory model for the monitored road. Historical vehicle movement information refers to the movement information of each vehicle on the monitored road where the target vehicle is located within a historical period, under unobstructed conditions. This can include the vehicle's trajectory on the monitored road, such as the vehicle's initial entry into the monitoring range, its position before the stop line, its position halfway across the stop line, and its position when it exits the obstructed area.

[0071] Taking a three-lane road as an example, the process of constructing a vehicle motion trajectory model includes: First, the permissible motion direction of each lane can be determined based on road traffic information. For example, the motion directions of the three lanes are left turn, straight, and right turn. During the operation of the monitoring camera, the historical motion of vehicles in the three lanes can be recorded when there is no obstruction. This can be used as historical vehicle motion information. Then, the motion trajectory can be fitted based on this historical vehicle motion information and road traffic information to obtain the vehicle motion trajectory model of each lane on the monitored road.

[0072] It should be noted that if a lane in reality contains multiple permitted directions of movement, such as the left lane which allows both left turns and right turns, then vehicle trajectory models can be constructed separately based on the historical movement of vehicles in that lane.

[0073] The method provided in this invention addresses the problems of vehicle tracking loss and track trajectory interruption during long-term vehicle tracking. It proposes constructing a motion trajectory model using historical vehicle movement information from monitored roads after successful matching. Based on this, it combines road traffic information, the target vehicle's position when obscured, and its reappearance position after obscuration to predict the trajectory, thereby recovering the motion trajectory during the period of lost vehicle tracking. This ensures the accuracy of trajectory prediction even under obscuration conditions, resulting in a complete and accurate motion trajectory. This provides data support for identifying and judging traffic violations, reducing missed violations and ensuring the accuracy of violation judgment.

[0074] Based on the above embodiments, historical vehicle motion information includes the motion trajectory of vehicles in each lane under unobstructed conditions. The vehicle trajectory model for monitoring any lane on a road is determined based on the following steps: Based on the trajectory of the first vehicle in the lane and the road traffic information of the lane, curve fitting is performed to obtain the initial motion curve of the vehicle in the lane. Based on the motion trajectories of other vehicles in the lane, the initial motion curve of the vehicle in the lane is corrected to obtain the vehicle motion trajectory model of the lane.

[0075] Specifically, the historical vehicle movement information monitored on the road includes the movement trajectories of vehicles in each lane under unobstructed conditions over a historical period. This information can be obtained from publicly available data on traffic management websites or through manual recording; this embodiment of the invention does not impose specific limitations on this method. Here, the movement trajectory of a vehicle in any lane can be represented as a dataset, which includes data up to time step t-1. Motion data for individual vehicles:

[0076] in, Indicates the first The location where the vehicle just entered the monitoring range; Indicates the first The position of the vehicle before the road stop line; Indicates the first The obstruction position of a vehicle is 1, which is the position where half of the road stop line has been crossed; Indicates the first The obstruction point 2 of the vehicle is the position after the entire vehicle has crossed the road stop line; Indicates the first The obstruction position of the vehicle is 3, which is the position after the front of the vehicle has passed over the pedestrian crossing; Indicates the first The obstruction position of the vehicle is 4, which is the position after the rear of the vehicle has passed over the pedestrian crossing. Indicates the first The position of each vehicle when it leaves the obstructed area; Indicates the first The location of each vehicle when it crosses the opposite pedestrian crossing or is about to leave the monitoring range.

[0077] Therefore, when determining the vehicle trajectory model for any lane, it can be based on the trajectory of the vehicle in that lane. Using the motion data of each vehicle and the road traffic information of that lane, a curve is fitted. Specifically, based on the road traffic information of that lane, the motion trajectory of the first vehicle in that lane is first used... A curve is fitted using the least squares method, and the curve coefficients are determined by minimizing the sum of squared errors. The fitting formula is as follows:

[0078] in, Let f(xi) be the actual trajectory point, and f(xi) be the fitted trajectory point.

[0079] Next, the trajectory of the second vehicle is used to correct the curve fitted in the previous step. Input x, output the fitting result f(x), average it with the true value y, and refit the curve for y at time t. This process is similar to the backpropagation process of a convolutional neural network.

[0080] Then, based on the third, fourth, and subsequent vehicles in that lane... The vehicle's trajectory is continuously corrected to obtain a vehicle trajectory model for that lane.

[0081] Based on the above embodiments, the motion trajectory of the target vehicle when it is obscured is obtained, and the process further includes: Determine the initial tracking parameters based on the position of the target vehicle when it is obscured; Based on the initial tracking parameters and the reappearance position of the target vehicle after occlusion, the motion trajectory of the target vehicle under occlusion conditions is updated to obtain the target motion trajectory under occlusion conditions.

[0082] Specifically, considering that the longer the vehicle tracking is lost during the tracking process, the greater the accumulated error will be, the tracking parameters selected during trajectory repair should retain the characteristics of the original tracking trajectory as much as possible, without accumulating too much error. In this embodiment of the invention, the tracking model parameters saved before the frame locking is executed after tracking is lost are selected. Because the road stop line is used as a positional condition, the continuous drift of the tracking frame is prevented in time, ensuring that the tracking error is minimized. The tracking parameters at this position are saved, including the mean LastMean and the variance LastCov.

[0083] In detail, the initial tracking parameters are first determined based on the target vehicle's position when it is obscured. Then, based on the initial tracking parameters and the target vehicle's re-emergence position after obscuration, the motion trajectory of the target vehicle under obscured conditions is updated. That is, the LastRect position before the road stop line and the NowRect position when the target vehicle is re-detected are used as the endpoints for trajectory repair. The model parameter update process is as follows: After obtaining the motion trajectory of the occluded area based on the vehicle motion trajectory model and timestamps, the tracking parameters saved in the state before the lock frame is triggered, which is before the road stop line is blocked, are used as the initial values, i.e., the initial tracking parameters. The tracking parameters at each trajectory point in the predicted motion trajectory are updated, starting from LastRect, calculating the tracking parameters at each trajectory point until NowRect is updated. At this point, the motion trajectory of the target vehicle from before the road stop line to after it leaves the occluded area, as well as the latest tracking parameters NowParam, which includes the mean NowMean and the variance NowCov, can be obtained. When the target vehicle is detected again, NowParam is used to continuously update the tracking parameters and motion trajectory of the target vehicle to ensure the continuity of the trajectory. The target motion trajectory obtained in this way can approximate the real trajectory.

[0084] In this embodiment of the invention, the parameter update for the motion trajectory can make the motion trajectory of the repaired vehicle closer to the real motion trajectory, effectively smoothing the error accumulated by the target vehicle during the tracking loss stage, preventing the tracking box from moving in a random direction, and improving the accuracy of violation judgment.

[0085] It should be noted that, compared to current vehicle tracking methods that match license plates and vehicle appearance features, the vehicle tracking method provided in this invention can reduce the missed detection rate in road traffic scenarios by nearly 30%. Furthermore, it can effectively address scenarios involving long-term, long-distance complete occlusion in road traffic, resolving issues such as significant changes in vehicle posture before and after occlusion, obvious differences in appearance features, and low similarity.

[0086] The method provided in this invention can quickly and effectively predict the movement trajectory of vehicles within an occluded area, and can ensure the accuracy of the movement trajectory. It can make the predicted trajectory very close to the actual trajectory of the vehicle, further improving the robustness of tracking.

[0087] The vehicle tracking device provided by the present invention is described below. The vehicle tracking device described below and the vehicle tracking method described above can be referred to in correspondence.

[0088] Figure 5 This is a schematic diagram of the vehicle tracking device provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the device includes: The image acquisition unit 510 is used to acquire a vehicle image of the target vehicle when the target vehicle is detected to be in front of or behind the road stop line and is obscured. The vehicle image includes a first vehicle image when the target vehicle is in front of or behind the road stop line and is not obscured, and a second vehicle image that reappears after being obscured. Image processing unit 520 is used to extract features from the vehicle image to obtain global vehicle features and multiple local vehicle features of the target vehicle, and to perform feature fusion based on the correlation between the local vehicle features and the global vehicle features to obtain vehicle target features. The vehicle target features include a first target feature corresponding to the first vehicle image and a second target feature corresponding to the second vehicle image. The vehicle tracking unit 530 is used to track the occluded target vehicle based on the first target feature and the second target feature.

[0089] The vehicle tracking device provided by this invention extracts global vehicle features and multiple local vehicle features from vehicle images before and after occlusion. It then performs feature fusion based on the correlation between the local vehicle features and the global vehicle features, and tracks the vehicle based on the fused target features. This overcomes the problem of vehicle tracking loss under pose changes and strong occlusion in traditional solutions. It can accurately track target vehicles in complex and ever-changing traffic environments and effectively address the problem of target tracking being difficult in long-term, completely occluded scenarios. It has high robustness and real-time performance, and can provide data support for repairing vehicle motion trajectories under occlusion conditions, providing strong support for violation judgment based on continuous motion trajectories.

[0090] Based on the above embodiments, the image processing unit 520 is used for: Global feature extraction is performed on the vehicle image to obtain the vehicle's global features; Key point detection is performed on the vehicle image to obtain key point detection results; Based on the location of each key point in the key point detection results and the global features of the vehicle, local features are extracted to obtain the local features of the vehicle for each key point.

[0091] Based on the above embodiments, the key point detection results also include the confidence level of the position of each key point; Image processing unit 520 is used for: Based on the confidence level of the position of each key point, the occluded key points are filtered out from the key points to obtain each target key point; Based on the local vehicle features of each target key point, feature stitching is performed, and the features obtained by feature stitching are divided into feature blocks to obtain multiple local vehicle sub-features. Based on the correlation between the local sub-features of each vehicle and the global features of the vehicle, feature fusion is performed to obtain the target features of the vehicle.

[0092] Based on the above embodiments, the image processing unit 520 is used for: The local sub-features of any vehicle are concatenated with the local sub-features of other vehicles, and the concatenated features are convolved to obtain the local fusion features corresponding to the local sub-features of any vehicle. Based on any vehicle local sub-feature and its corresponding local fusion feature, determine the local related features corresponding to any vehicle local sub-feature; Based on the local related features corresponding to the local sub-features of each vehicle and the global features of the vehicle, feature fusion is performed to obtain the vehicle target features.

[0093] Based on the above embodiments, the device further includes a trajectory prediction unit, used for: If the target vehicle is tracked, determine the position of the target vehicle when it is in front of or behind the road stop line and is obscured, and the position where it reappears after being obscured; Based on the road traffic information of the monitored road where the target vehicle is located, the position of the target vehicle when it is obscured and the position when it reappears after being obscured, a vehicle motion trajectory model is applied to predict the trajectory and obtain the motion trajectory of the target vehicle when it is obscured. The vehicle trajectory model is obtained by fitting historical vehicle movement information and road traffic information of the monitored road.

[0094] Based on the above embodiments, the historical vehicle movement information includes the movement trajectories of vehicles in each lane under unobstructed conditions; the vehicle movement trajectory model for any lane on the monitored road is determined based on the following steps: Based on the trajectory of the first vehicle in the lane and the road traffic information of any lane, curve fitting is performed to obtain the initial vehicle motion curve of the lane. Based on the motion trajectories of other vehicles in the lane, the initial motion curve of the vehicle in the lane is corrected to obtain the vehicle motion trajectory model of the lane.

[0095] Based on the above embodiments, the trajectory prediction unit is also used for: The initial tracking parameters are determined based on the position of the target vehicle when it is obscured. Based on the initial tracking parameters and the reappearance position of the target vehicle after occlusion, the motion trajectory of the target vehicle under occlusion conditions is updated to obtain the target motion trajectory under occlusion conditions.

[0096] Figure 6An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a vehicle tracking method. This method includes: when a target vehicle is detected to be positioned before or after a road stop line and is obscured, acquiring a vehicle image of the target vehicle, the vehicle image including a first vehicle image when the target vehicle is before or after the road stop line and is not obscured, and a second vehicle image reappearing after obscuration; extracting features from the vehicle image to obtain global vehicle features and multiple local vehicle features of the target vehicle, and performing feature fusion based on the correlation between the local vehicle features and the global vehicle features to obtain vehicle target features, the vehicle target features including a first target feature corresponding to the first vehicle image and a second target feature corresponding to the second vehicle image; and tracking the obscured target vehicle based on the first target feature and the second target feature.

[0097] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0098] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the vehicle tracking method provided by the above methods, the method comprising: when a target vehicle is detected to be in front of or behind a road stop line and is obscured, acquiring a vehicle image of the target vehicle, the vehicle image comprising a first vehicle image of the target vehicle in front of or behind a road stop line and not obscured, and a second vehicle image reappearing after obscuration; extracting features from the vehicle image to obtain global vehicle features and multiple local vehicle features of the target vehicle, and performing feature fusion based on the correlation between the local vehicle features and the global vehicle features to obtain vehicle target features, the vehicle target features comprising a first target feature corresponding to the first vehicle image and a second target feature corresponding to the second vehicle image; and tracking the obscured target vehicle based on the first target feature and the second target feature.

[0099] In another aspect, the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the vehicle tracking methods provided by the methods described above. The method includes: when a target vehicle is detected to be positioned before or after a road stop line and is obscured, acquiring a vehicle image of the target vehicle, the vehicle image including a first vehicle image of the target vehicle when it is positioned before or after the road stop line and is not obscured, and a second vehicle image reappearing after obscuration; extracting features from the vehicle image to obtain global vehicle features and multiple local vehicle features of the target vehicle, and performing feature fusion based on the correlation between the local vehicle features and the global vehicle features to obtain vehicle target features, the vehicle target features including a first target feature corresponding to the first vehicle image and a second target feature corresponding to the second vehicle image; and tracking the obscured target vehicle based on the first target feature and the second target feature.

[0100] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0101] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle tracking method, characterized in that, include: When a target vehicle is detected to be in front of or behind the road stop line and is obscured, a vehicle image of the target vehicle is acquired. The vehicle image includes a first vehicle image when the target vehicle is in front of or behind the road stop line and is not obscured, and a second vehicle image that reappears after being obscured. Feature extraction is performed on the vehicle image to obtain the global vehicle features and multiple local vehicle features of the target vehicle. Based on the correlation between the local vehicle features and the global vehicle features, feature fusion is performed to obtain the vehicle target features. The vehicle target features include a first target feature corresponding to the first vehicle image and a second target feature corresponding to the second vehicle image. Based on the first target feature and the second target feature, vehicle tracking is performed on the occluded target vehicle.

2. The vehicle tracking method according to claim 1, characterized in that, The feature extraction of the vehicle image yields global vehicle features and multiple local vehicle features of the target vehicle, including: Global feature extraction is performed on the vehicle image to obtain the vehicle's global features; Key point detection is performed on the vehicle image to obtain key point detection results; Based on the location of each key point in the key point detection results and the global features of the vehicle, local features are extracted to obtain the local features of the vehicle for each key point.

3. The vehicle tracking method according to claim 2, characterized in that, The key point detection results also include the confidence level of the position of each key point; The process of fusing features based on the correlation between local features of each vehicle and the global features of the vehicle to obtain vehicle target features includes: Based on the confidence level of the position of each key point, the occluded key points are filtered out from the key points to obtain each target key point; Based on the local vehicle features of each target key point, feature stitching is performed, and the features obtained by feature stitching are divided into feature blocks to obtain multiple local vehicle sub-features. Based on the correlation between the local sub-features of each vehicle and the global features of the vehicle, feature fusion is performed to obtain the target features of the vehicle.

4. The vehicle tracking method according to claim 3, characterized in that, The process of fusing features based on the correlation between local sub-features of each vehicle and the global features of the vehicle to obtain vehicle target features includes: The local sub-features of any vehicle are concatenated with the local sub-features of other vehicles, and the concatenated features are convolved to obtain the local fusion features corresponding to the local sub-features of any vehicle. Based on any vehicle local sub-feature and its corresponding local fusion feature, determine the local related features corresponding to any vehicle local sub-feature; Based on the local related features corresponding to the local sub-features of each vehicle and the global features of the vehicle, feature fusion is performed to obtain the vehicle target features.

5. The vehicle tracking method according to any one of claims 1 to 4, characterized in that, The step of tracking the occluded target vehicle based on the first target feature and the second target feature further includes: If the target vehicle is tracked, determine the position of the target vehicle when it is in front of or behind the road stop line and is obscured, and the position where it reappears after being obscured; Based on the road traffic information of the monitored road where the target vehicle is located, the position of the target vehicle when it is obscured and the position when it reappears after being obscured, a vehicle motion trajectory model is applied to predict the trajectory and obtain the motion trajectory of the target vehicle when it is obscured. The vehicle trajectory model is obtained by fitting historical vehicle movement information and road traffic information of the monitored road.

6. The vehicle tracking method according to claim 5, characterized in that, The historical vehicle motion information includes the movement trajectories of vehicles in each lane under unobstructed conditions; The vehicle trajectory model for any lane on the monitored road is determined based on the following steps: Based on the trajectory of the first vehicle in any lane and the road traffic information of any lane, curve fitting is performed to obtain the initial motion curve of the vehicle in any lane. Based on the motion trajectories of other vehicles in any lane, the initial motion curve of the vehicle in any lane is corrected to obtain the vehicle motion trajectory model of any lane.

7. The vehicle tracking method according to claim 5, characterized in that, After obtaining the motion trajectory of the target vehicle when it is obscured, the process further includes: The initial tracking parameters are determined based on the position of the target vehicle when it is obscured. Based on the initial tracking parameters and the reappearance position of the target vehicle after occlusion, the motion trajectory of the target vehicle under occlusion conditions is updated to obtain the target motion trajectory under occlusion conditions.

8. A vehicle tracking device, characterized in that, include: The image acquisition unit is used to acquire a vehicle image of the target vehicle when the target vehicle is detected to be in front of or behind the road stop line and is obscured. The vehicle image includes a first vehicle image when the target vehicle is in front of or behind the road stop line and is not obscured, and a second vehicle image that reappears after being obscured. An image processing unit is used to extract features from the vehicle image to obtain global vehicle features and multiple local vehicle features of the target vehicle, and to perform feature fusion based on the correlation between the local vehicle features and the global vehicle features to obtain vehicle target features. The vehicle target features include a first target feature corresponding to the first vehicle image and a second target feature corresponding to the second vehicle image. A vehicle tracking unit is used to track the occluded target vehicle based on the first target feature and the second target feature.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the vehicle tracking method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the vehicle tracking method as described in any one of claims 1 to 7.

Images