Video stream-based vehicle driving trajectory tracking method and system

Abstract

This invention relates to the field of vehicle recognition technology, specifically to a method and system for tracking vehicle driving trajectories based on video streams. The method includes: acquiring vehicle regions from each frame of video captured by each camera within a target area; calculating the information coefficients of each feature point within each vehicle region; dividing each vehicle region into blocks; locating the corresponding target region block in a matching frame where the target vehicle appears; calculating the matching degree between any target region block in the matching frame and each unmatched region block in the unmatched frame; if the maximum matching degree is greater than a preset threshold, incorporating the corresponding unmatched region block into the continuous path sequence of the target vehicle and using it as a new matching frame; repeating the matching process until all frames are matched; integrating the continuous path sequence, mapping it to a real geographic coordinate system, and generating the driving trajectory of the target vehicle. Using this invention, the vehicle driving trajectory tracking technology is more effective and less prone to trajectory interruption or loss.

Description

Technical Field

[0001] This invention relates to the field of vehicle recognition technology, and more specifically to a method and system for tracking vehicle driving trajectories based on video streams. Background Technology

[0002] With the deepening of smart city construction and the widespread application of intelligent transportation systems, video surveillance networks have densely covered major urban roads and key nodes, generating massive amounts of video stream data. This data contains rich information on vehicle behavior, and how to automatically and accurately extract the dynamic driving trajectories of vehicles has become a crucial and highly challenging task in the fields of computer vision and intelligent transportation. Achieving efficient vehicle trajectory tracking has significant practical value for intelligent traffic management, public safety monitoring, and traffic behavior analysis.

[0003] In existing vehicle trajectory tracking methods, cross-camera matching typically relies on comparing feature vectors at the vehicle level. However, in real-world scenarios, vehicle appearance features often change significantly due to factors such as occlusion, lighting variations, and perspective shifts, making matching methods based on global features unstable. Significant deviations in feature representation can easily lead to matching failures, resulting in trajectory interruptions or loss. Summary of the Invention

[0004] To address the shortcomings of existing vehicle trajectory tracking technologies, which often result in ineffective trajectory tracking and track interruptions or loss, this invention aims to provide a video stream-based vehicle trajectory tracking method. The specific technical solution adopted is as follows:

[0005] The vehicle regions in each frame of video captured by each camera within the target area are obtained, the information coefficients of each feature point within each vehicle region are calculated, and then each vehicle region is divided into region blocks.

[0006] In the matching frame where the target vehicle appears, locate the corresponding target region block, and perform a matching process between any target region block in the matching frame and each region block to be matched in the matching frame. The matching process includes calculating the matching degree based on the information vector and position reasonableness coefficient of the target region block and the region block to be matched.

[0007] If the maximum matching degree is greater than the preset threshold, the corresponding unmatched area block is included in the continuous path sequence of the target vehicle and used as a new matching frame. The matching process is repeated until all frames are matched.

[0008] The continuous path sequence is integrated and mapped to a real geographic coordinate system to generate the driving trajectory of the target vehicle.

[0009] Furthermore, before acquiring the vehicle area from each frame of the video captured by each camera within the target area, the process further includes:

[0010] Ensure that the clocks of all cameras within the target area are synchronized, so that the videos captured by each camera have timestamps with a unified time base;

[0011] The video is preprocessed, including: eliminating lens distortion and performing color correction.

[0012] Furthermore, the vehicle area for acquiring each frame of video captured by each camera within the target area includes:

[0013] Optical flow between consecutive frames is obtained, the optical flow is optimized, and the pixels in each frame are clustered based on the optical flow direction to obtain pixel groups with the same optical flow direction. Each pixel group corresponds to a vehicle region.

[0014] Furthermore, the process of obtaining the information coefficients includes:

[0015] Determine a reference feature point for the current feature point, wherein the current feature point and the reference feature point together enclose a reference region;

[0016] Obtain the spatial distance and color difference between the current feature point and any of the reference feature points, as well as the maximum spatial scale within the reference region;

[0017] The information coefficient of the current feature point is obtained based on the spatial distance, the color difference, and the maximum spatial scale.

[0018] Furthermore, the process of dividing the region into blocks includes:

[0019] A local point set is constructed based on the feature points. The construction process of the local point set includes: for the vehicle area that does not contain the assigned feature points, the feature points corresponding to the maximum and second largest distance values ​​from the center of the area are assigned to the local point set; the rationality of adding each unassigned feature point to the local point set is calculated; if the rationality value is greater than a preset rationality threshold, the unassigned feature points are assigned to the local point set.

[0020] If there are still unassigned feature points when the construction process ends, the construction process is repeated to construct a new local point set until each feature point has its unique local point set to which it belongs, and one local point set corresponds to one region block.

[0021] Furthermore, the process of obtaining the rationality includes:

[0022] Obtain the area enclosed by the outer contour line after the unassigned feature points are assigned to the local point set, and obtain the geometric compactness based on the area and the area of ​​its smallest circumscribed quadrilateral.

[0023] The average difference between the information coefficients of each feature point in the local point set and the mean information coefficient, as well as the difference between the information coefficients of the unassigned feature points and the mean information coefficient, are obtained. Information consistency is obtained based on the average difference and the difference.

[0024] The rationality is determined based on the geometric compactness and the information consistency.

[0025] Furthermore, the process of selecting the frame to be matched includes:

[0026] When the matching frame is a non-ending frame in its video, the frame to be matched is the adjacent frame to be matched of the matching frame.

[0027] When the matching frame is the end frame of its own video, the frame to be matched also includes the start frame of any unmatched video.

[0028] Furthermore, the process of obtaining the matching degree includes:

[0029] When the matching frame is a non-end frame in its video, obtain the cosine similarity of the information vectors between any target region block of the matching frame and any region block in an adjacent frame to be matched.

[0030] The inverse proportional value of the ratio of the distance between the center of the target region block and the center of any region block in the adjacent frame to be matched to the diameter of the target region block is used as the first position rationality coefficient;

[0031] The matching degree is calculated based on the cosine similarity and the first position reasonable coefficient.

[0032] Furthermore, the process of obtaining the matching degree also includes:

[0033] When the matching frame is the end frame of its video, obtain the cosine similarity of the information vectors between any target region block of the matching frame and any region block in the start frame of any unmatched video.

[0034] The inverse proportional value of the time difference between the actual appearance time of the target vehicle in the unmatched video and the estimated appearance time interval is calculated as the second position reasonable coefficient.

[0035] The matching degree is calculated based on the cosine similarity and the second position reasonable coefficient.

[0036] This invention also provides a vehicle trajectory tracking system based on video streams, the system comprising:

[0037] The region block division module is used to obtain the vehicle region in each frame of the video captured by each camera in the target area, calculate the information coefficient of each feature point in each vehicle region, and then divide each vehicle region into region blocks.

[0038] The matching calculation module is used to locate the corresponding target region block in the matching frame where the target vehicle appears, and to perform a matching process between any target region block in the matching frame and each region block to be matched in the matching frame. The matching process includes calculating the matching degree based on the information vector and position reasonableness coefficient of the target region block and the region block to be matched.

[0039] The matching determination and repetition module is used to include the corresponding unmatched area block into the continuous path sequence of the target vehicle and use it as a new matching frame if the maximum matching degree is greater than a preset threshold. The matching process is repeated until all frames are matched.

[0040] An integration module is used to integrate the continuous path sequence, map it to a real geographic coordinate system, and generate the driving trajectory of the target vehicle.

[0041] The present invention has the following beneficial effects:

[0042] First, the vehicle regions in each frame of the video captured by each camera within the target area are obtained. The information coefficients of each feature point within each vehicle region are calculated, and then each vehicle region is divided into region blocks. By analyzing these region blocks and utilizing the stability and complementarity of local features, the dependence of existing vehicle trajectory tracking technologies on the overall global appearance is effectively reduced, thereby significantly improving their robustness and trajectory continuity in complex scenarios.

[0043] Secondly, in the matching frame where the target vehicle appears, the corresponding target region block is located. A matching process is then performed between any target region block in the matching frame and each region block to be matched in the matching frame. This matching process includes calculating a matching degree based on the information vectors and positional reasonableness coefficients of the target region block and the region blocks to be matched. The higher the matching degree, the more likely the target region block and the region blocks to be matched belong to the target vehicle.

[0044] Furthermore, if the maximum matching degree is greater than a preset threshold, the corresponding unmatched area block is included in the continuous path sequence of the target vehicle and used as a new matching frame. This matching process is repeated until all frames are matched. Each frame of the video captured by each camera within the target area needs to undergo this matching process.

[0045] Finally, the continuous path sequences are integrated and mapped to a real geographic coordinate system to generate the driving trajectory of the target vehicle. The continuous path sequence of each region represents the spatiotemporal trajectory of the corresponding vehicle component under the view of each camera within the target area. By fusing these local continuous path sequences, a complete and coherent global driving trajectory of the vehicle can be synthesized.

[0046] In summary, this invention effectively overcomes association errors caused by the variability of features in some regions by focusing on stable and highly discriminative local features for matching, thereby generating more accurate and continuous vehicle driving trajectories. Attached Figure Description

[0047] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of the vehicle trajectory tracking method based on video stream provided in the first embodiment of the present invention;

[0049] Figure 2 A flowchart illustrating the process of obtaining the information coefficients provided in the second embodiment of the present invention;

[0050] Figure 3 This is a flowchart of the vehicle area division block provided in the third embodiment of the present invention;

[0051] Figure 4 A flowchart illustrating the process of obtaining the rationality provided in the fourth embodiment of the present invention;

[0052] Figure 5 A flowchart illustrating the matching degree acquisition process provided in the fifth embodiment of the present invention;

[0053] Figure 6 A flowchart illustrating another process for obtaining the matching degree provided in the sixth embodiment of the present invention;

[0054] Figure 7 This is a schematic diagram of a vehicle trajectory tracking system based on video stream provided in the seventh embodiment of the present invention. Detailed Implementation

[0055] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the video stream-based vehicle trajectory tracking method and system proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0057] The specific solution of the vehicle trajectory tracking method based on video stream provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0058] Please see Figure 1 The diagram illustrates a flowchart of a video stream-based vehicle trajectory tracking method provided in the first embodiment of the present invention, the method comprising:

[0059] S101. Obtain the vehicle region in each frame of the video captured by each camera in the target area, calculate the information coefficient of each feature point in each vehicle region, and then divide each vehicle region into region blocks.

[0060] Before acquiring the vehicle area from each frame of video captured by each camera within the target area, the process also includes:

[0061] Ensure that the clocks of all cameras within the target area are synchronized, so that the videos captured by each camera have timestamps with a unified time base;

[0062] The video is preprocessed, including: eliminating lens distortion and performing color correction.

[0063] The vehicle area for acquiring each frame of video captured by each camera within the target area includes:

[0064] Optical flow between consecutive frames is obtained, the optical flow is optimized, and the pixels in each frame are clustered based on the optical flow direction to obtain pixel groups with the same optical flow direction. Each pixel group corresponds to a vehicle region.

[0065] In detail, the core objective of this invention is to perform cross-camera correlation matching of vehicle identities based on vehicle driving videos captured by multiple cameras, thereby constructing a complete and continuous driving trajectory of the vehicle within the monitored area, i.e., the target area. To achieve this objective, the video segments captured by each camera must first be processed. Specific steps include:

[0066] a. Video Acquisition: Acquire video data synchronously collected by all cameras within the target area, and ensure that the timestamps of each camera are accurately synchronized;

[0067] b. Time synchronization: Based on the timestamp information carried by the video, video streams from different cameras are aligned to the same time base to achieve time synchronization between multiple video streams;

[0068] c. Video preprocessing: Perform preprocessing operations on each video stream, mainly including: using camera intrinsic parameters to correct lens distortion, and performing color correction on videos from different cameras or at different times to reduce the interference of lighting changes on appearance feature extraction.

[0069] d. Target Separation: For each camera's video feed, vehicle targets are separated using optical flow trajectory analysis. Since pixels belonging to the same vehicle exhibit consistent optical flow trends within the same video, a pre-defined 3×3 convolution kernel (i.e., a pre-defined 3×3 convolution kernel with a specific orientation, such as a variant of a horizontal or vertical Prewitt filter) is first used to enhance the optical flow response in the vehicle's direction of travel. Then, all pixels are clustered based on the optical flow direction to obtain pixel groups with consistent optical flow directions. Each pixel group corresponds to an independent vehicle target. Pixels with unchanged optical flow in the video are identified as background pixels and excluded.

[0070] At this point, the region corresponding to each vehicle in each video has been obtained. Further analysis is needed to match vehicles between different cameras.

[0071] For any given camera, since the camera is fixed, the perspective of the vehicle captured by that camera remains basically unchanged. For example, if the road is a one-way straight road from east to west and the camera is installed on one side of the road, then the vehicles captured will all be traveling east-west from a top-down perspective. The perspective of the vehicles captured by the cameras is basically the same, meaning that the exposed surface of the vehicle under each camera can be determined.

[0072] The visual composition of different vehicle surfaces varies, resulting in highly distinctive information distribution patterns. Therefore, these distribution differences can be used to identify different vehicle surfaces from different camera perspectives.

[0073] First, corner detection is performed on each vehicle region in each frame, and the obtained corner points are defined as feature points of that region. Existing corner detection algorithms, such as the Harris Corner Detector, are used. Since there are distinct contour edges between different structural blocks on the vehicle surface, the richer the components and contour variations on the surface, the more feature points are typically detected. Subsequently, based on the spatial distribution of feature points and the differences in their neighboring pixels, their information complexity is evaluated and quantified as an information coefficient. This coefficient reflects the complexity and uniqueness of the vehicle structure near the feature point. The calculation principle is: the closer a feature point is to its neighboring feature points, and the greater the difference in visual content within its neighborhood, the richer its information content is considered.

[0074] The process of obtaining the information coefficients will be described in detail in the second embodiment, and will not be repeated here.

[0075] To address the vehicle matching deviation issue caused by occlusion in multi-camera perspectives, this invention does not directly rely on the vehicle feature vector, which is susceptible to occlusion. Instead, it utilizes the distribution characteristics of vehicle surface information for matching to generate trajectory segments from each camera. The specific implementation is as follows:

[0076] Since the same vehicle may appear as different surfaces in different cameras, and the information distribution patterns of each surface vary significantly, the vehicle region is first segmented to achieve accurate cross-view matching. Segmentation is based on the distribution of feature points and the differences in their information content (information coefficients), thus dividing the vehicle image into several visual units to approximately represent its different surfaces (such as the front, sides, and rear). This strategy aims to ensure that subsequent matching can target local features of the same vehicle surface, thereby improving the robustness of matching under occlusion conditions.

[0077] The process of dividing the vehicle area into blocks will be described in detail in the third embodiment, and will not be repeated here.

[0078] S102. In the matching frame where the target vehicle appears, locate the corresponding target region block, and perform a matching process between any target region block in the matching frame and each region block to be matched in the matching frame. The matching process includes calculating the matching degree based on the information vector and position reasonableness coefficient of the target region block and the region block to be matched.

[0079] As the name suggests, the matching frame is a single frame used for matching. First, it's necessary to determine which vehicle in the video is the target vehicle, i.e., the vehicle whose driving trajectory needs to be obtained. The matching frame needs to display as much surface information as possible about the target vehicle, i.e., include as many faces of the vehicle as possible. The initial matching frame and the target vehicle appearing within it can be specified by the user.

[0080] The frame to be matched is a frame of image used to match the matching frame.

[0081] The process of selecting the frame to be matched includes:

[0082] When the matching frame is a non-ending frame in its video, the frame to be matched is the adjacent frame to be matched of the matching frame.

[0083] When the matching frame is the end frame of its own video, the frame to be matched also includes the start frame of any unmatched video.

[0084] Specifically, when matching occurs within the same video, i.e., when the matching frame is a non-ending frame in its video, the frame to be matched can be the adjacent previous frame and the adjacent next frame to be matched. After the matching frame is determined, the matching process is bidirectional. When the frame to be matched fails to match the matching frame, the frame to be matched becomes an unmatched frame. Next, the adjacent frames to be matched with the unmatched frame are then matched with the matching frame. When the frame to be matched successfully matches the matching frame, the frame to be matched becomes a matched frame, and the matching process is repeated, continuing to match with adjacent frames to be matched. If every frame after a certain matching frame in a video fails to match, then the last matching frame in the video is the ending frame.

[0085] When performing cross-video matching, i.e., when the matching frame is the end frame of its own video, the frames to be matched also include the start frame of any unmatched video. In other words, when the matching frame is the end frame of its own video, it can not only be matched with the previous adjacent frame to be matched within the same video, but also with the start frame of any unmatched video across videos. Unmatched videos are those that have not undergone a matching process.

[0086] When any target region block in the matching frame is successfully matched with a region block in the frame to be matched, i.e. when the frame to be matched is successfully matched with the matching frame, it can be known that the remaining region blocks of the vehicle region to which the frame to be matched belong also become the new target region blocks.

[0087] Each region block corresponds to an information vector. The direction of the information vector is determined by the spatial gradient direction of the information coefficients of all feature points within the region block, while its magnitude is determined by the average information coefficient of all feature points within the region block.

[0088] Since the same vehicle surface may appear in the images of multiple cameras, cross-camera tracking can be achieved by matching corresponding regions in different videos. The specific implementation involves the following two steps:

[0089] Step 1: Trajectory construction within a single camera;

[0090] For any given region, its continuous motion path within a single camera can be constructed by tracing its position sequence in the video timeline. Considering the continuity of vehicle motion between adjacent frames, the trajectories of the region in the same video can be chained together by calculating the similarity, i.e., the matching degree, between the position of the region (represented by the center position of the feature points within the region) and its information vector.

[0091] When the matching frame is a non-ending frame in its own video, the process of obtaining the matching degree will be described in detail in the fifth embodiment, and will not be repeated here.

[0092] Step 2: Cross-camera trajectory association.

[0093] For different cameras, due to differences in perspective and coordinate system, it is not possible to directly determine whether a region belongs to the same vehicle based on its spatial location. Therefore, it is necessary to introduce spatiotemporal rationality constraints for comprehensive judgment. That is, the matched region should not only conform to the logical sequence of vehicle movement in terms of occurrence time, but its movement path should also have spatial continuity in the world coordinate system.

[0094] When the matching frame is the end frame in its own video, the process of obtaining the matching degree will be described in detail in the sixth embodiment, and will not be repeated here.

[0095] S103. If the maximum matching degree is greater than the preset threshold, the corresponding unmatched area block is included in the continuous path sequence of the target vehicle and used as a new matching frame. The matching process is repeated until all frames are matched.

[0096] The preset threshold can be set independently according to the actual situation, preferably 0.6.

[0097] A successful match is defined as a maximum matching degree that is greater than a preset threshold. The region block to be matched corresponding to the maximum matching degree is the target region block corresponding to the target vehicle.

[0098] Regardless of whether the match is successful or not, each frame of each video needs to be matched to obtain the path sequence of the target area block under all cameras. The path sequence of a target area block represents the path state of a certain part of the corresponding target vehicle across all cameras throughout the entire timeline.

[0099] S104. Integrate the continuous path sequence, map it to the real geographic coordinate system, and generate the driving trajectory of the target vehicle.

[0100] Based on step S103, continuous path sequences for each target region block were obtained. According to the prior installation location of the cameras, these paths can be transformed from image coordinates to real-world coordinates, i.e., the true geographic coordinate system. For the same target vehicle, the trajectories of different regions may be incomplete due to occlusion, but the paths between different regions are spatially complementary. By fusing multiple region block paths belonging to the same target vehicle, the occluded road segments can be completed, ultimately forming a complete and continuous driving trajectory for the target vehicle. Thus, vehicle trajectory tracking based on multi-camera video streams was achieved.

[0101] Figure 2 The flowchart below shows the process of obtaining the information coefficients according to the second embodiment of the present invention. The process of obtaining the information coefficients includes:

[0102] S201. Determine a reference feature point for the current feature point, wherein the current feature point and the reference feature point together form a reference region.

[0103] The reference feature points can be the n other feature points that are closest to the current feature point for comparison. The number of n can be set independently according to the actual situation, preferably 10.

[0104] S202. Obtain the spatial distance, color difference, and maximum spatial scale within the reference area between the current feature point and any of the reference feature points.

[0105] This represents the spatial distance between the r-th feature point and the i-th reference point within the m-th vehicle region in the f-th frame.

[0106] This represents the color difference between the r-th feature point and the i-th reference point within the m-th vehicle region in the f-th frame.

[0107] This represents the maximum distance between any two feature points within the m-th vehicle region in the f-th frame.

[0108] S203. Obtain the information coefficient of the current feature point based on the spatial distance, the color difference, and the maximum spatial scale.

[0109] The information coefficient can be expressed by the formula:

[0110] ;

[0111] Among them, the The information coefficient of the r-th feature point within the m-th vehicle region in the f-th frame is described. This represents the inverse proportional normalization function.

[0112] Similarly, the information coefficient of each feature point can be obtained. The information coefficient represents the complexity of the information of each feature point, which in turn represents the reliability of the neighboring region of each feature point as a feature matching object.

[0113] Figure 3 The flowchart below shows the process of dividing the region block according to the third embodiment of the present invention. The process of dividing the region block includes:

[0114] S301. Construct a local point set based on the feature points. The construction process of the local point set includes: for the vehicle area that does not contain the assigned feature points, assign the feature points corresponding to the maximum and second largest distance values ​​from the center of the area to the local point set; calculate the rationality of adding each unassigned feature point to the local point set; if the rationality value is greater than a preset rationality threshold, then the unassigned feature points are assigned to the local point set.

[0115] The center of the region is determined by the average of the horizontal and vertical coordinates of all feature points in the vehicle region that does not contain assigned feature points. That is, the horizontal coordinate of the center of the region is the average of the horizontal coordinates of all feature points in the vehicle region that does not contain assigned feature points, and the vertical coordinate of the center of the region is the average of the vertical coordinates of all feature points in the vehicle region that does not contain assigned feature points.

[0116] Taking the m-th vehicle region in the f-th frame as an example, the feature points in the m-th vehicle region are divided into blocks, and a set of points with similar information content within the region is constructed. The specific steps are as follows:

[0117] We start by constructing a local point set from the feature points that are farthest from the center of the vehicle region and the second farthest from the center of the vehicle region in the m-th vehicle region. Feature points belonging to the same local point set are approximately considered to be points on the same surface of the vehicle.

[0118] Each unassigned feature point can be considered as a candidate for inclusion in the local point set. Priority is given to the feature point with the smallest average distance to all feature points within the local point set. To determine the rationality of the local point set after a new feature point (candidate point) is added, two conditions must be met simultaneously: first, the new feature point must be spatially and stylistically consistent with the existing local point set, meaning it must be close in distance and have minimal information difference; second, the contour formed by the new feature point must still conform to the prior knowledge of regular vehicle surface boundaries. Therefore, by jointly analyzing the information difference and contour morphology, the rationality of adding a new feature point can be determined.

[0119] The process of obtaining the rationality mentioned will be described in detail in the fourth embodiment, and will not be repeated here.

[0120] The reasonableness threshold can be set independently according to the actual situation, preferably 0.7.

[0121] If the value of the reasonableness is greater than a preset reasonableness threshold, the unassigned feature point is assigned to the local point set. If the value of the reasonableness is less than or equal to the preset reasonableness threshold, the unassigned feature point is no longer assigned to the local point set.

[0122] S302. When there are still unassigned feature points after the construction process ends, repeat the construction process to construct a new local point set until each feature point has its unique local point set to which it belongs, and one local point set corresponds to one region block.

[0123] The vehicle area is further divided into several region blocks, which correspond to different faces of a car as captured by the camera. This process breaks down a vehicle into a sum of several point sets. Because the vehicle appears differently in the images from different cameras, matching based on region blocks yields more accurate results even when the vehicle is partially occluded.

[0124] Figure 4 This is a flowchart of the reasonableness acquisition process provided in the fourth embodiment of the present invention. The reasonableness acquisition process includes:

[0125] S401. Obtain the area enclosed by the outer contour line after the unassigned feature points are assigned into the local point set, and obtain the geometric compactness based on the area and the area of ​​its smallest circumscribed quadrilateral.

[0126] It represents the area enclosed by the outer contour line when the r-th feature point in the m-th vehicle region is added to the local point set.

[0127] Let represent the area of ​​the smallest circumscribed quadrilateral enclosed by the outer contour lines when the r-th feature point in the m-th vehicle region is added to the local point set.

[0128] The geometric compactness can be expressed as: The more regular and compact the contour of the local point set (e.g., close to a rectangle), the larger the value of geometric compactness (approaching 1). Therefore, the larger the value of geometric compactness, the more regular the contour, and the more reasonable it is to assign the unassigned feature points into the local point set.

[0129] S402. Obtain the average difference between the information coefficients of each feature point in the local point set and the mean information coefficient, as well as the difference between the information coefficients of the unassigned feature points and the mean information coefficient, and obtain information consistency based on the average difference and the difference.

[0130] The mean information coefficient refers to the average information coefficient of each feature point in the local point set (the feature points in the local point set before the unassigned feature points were assigned to the local point set, that is, excluding the unassigned feature points).

[0131] The average difference can be expressed as: .

[0132] Among them, the This represents the information coefficient of any feature point in the local point set before the addition of the r-th feature point in the m-th vehicle region. This represents the average information coefficient of each feature point in the local point set before the r-th feature point in the m-th vehicle region is added to the local point set. This indicates that the absolute value is being calculated. This represents the function for calculating the average value. This represents the information coefficient and the average information coefficient of all feature points in the local point set before the addition of the r-th feature point in the m-th vehicle region. The average of the absolute values ​​of the differences between them.

[0133] The difference can be expressed as: , wherein This represents the information coefficient of the r-th feature point in the m-th vehicle region.

[0134] The consistency of information can be represented as: If the difference (denominator) is much smaller than the average difference (numerator), then the ratio will be large. This indicates that the unassigned feature points closely "fit" the local point set, making it highly plausible.

[0135] S403. Determine the rationality based on the geometric compactness and the information consistency.

[0136] The rationale can be expressed by the following formula:

[0137] ;

[0138] Among them, the This indicates the rationale for adding the r-th feature point in the m-th vehicle region to the local point set. This represents the normalization function, preferably a linear normalization function.

[0139] When special circumstances exist At this time, in order to avoid the denominator being 0, the stated It can be replaced with The It can be set independently according to the actual situation, preferably... .

[0140] Figure 5 The flowchart below shows the matching degree acquisition process provided in the fifth embodiment of the present invention. The matching degree acquisition process includes:

[0141] S501. When the matching frame is a non-ending frame in its video, obtain the cosine similarity of the information vectors between any target region block of the matching frame and any region block in an adjacent frame to be matched.

[0142] The cosine similarity of the information vectors between any target region block g in the matching frame and any region block g' in an adjacent frame to be matched is expressed as cosine similarity.

[0143] S502. Calculate the inverse proportional value of the ratio of the distance between the center of the target region block and the center of any region block in the adjacent frame to be matched to the diameter of the target region block as a first position reasonable coefficient.

[0144] The rationality coefficient of the first position can be expressed by the formula:

[0145] ;

[0146] Among them, the Indicates the center of the target region block g. With any of the region blocks in the adjacent frame to be matched center The distance between them, wherein the distance is a non-negative number, the The diameter of the target region block g is represented by the... The first positional reasonableness coefficient represents the relationship between any target region block g in the matching frame and any region block g' in an adjacent frame to be matched. Indicates An exponential function with base 0.

[0147] S503. Calculate the matching degree based on the cosine similarity and the first position reasonable coefficient.

[0148] The matching degree can be expressed by the formula:

[0149] ;

[0150] Among them, the This represents the matching degree between any target region block g in the matching frame and any region block g' in an adjacent frame to be matched.

[0151] Figure 6The flowchart below shows another process for obtaining the matching degree provided in the sixth embodiment of the present invention. The process for obtaining the matching degree includes:

[0152] S601. When the matching frame is the end frame of its video, obtain the cosine similarity of the information vectors between any target region block of the matching frame and any region block in the start frame of any unmatched video.

[0153] The cosine similarity of the information vectors between any target region block g in the matched frame and any region block g' in the starting frame of any unmatched video is expressed as a factor of cosine similarity.

[0154] S602. Calculate the inverse proportional value of the time difference between the actual appearance time of the target vehicle in the unmatched video and the estimated appearance time interval as the second position reasonable coefficient.

[0155] The second position rationality coefficient can be expressed by the formula as follows:

[0156] ;

[0157] Among them, the This represents the time when the target area block g disappears from the x-th camera and reappears at the x'-th camera. This represents the estimated time interval during which the target area block moves from the x-th camera to the x'-th camera. This is used to quantify the time deviation between the actual occurrence time and the estimated occurrence time interval. The actual occurrence duration is the time it takes for the target region block g to disappear from the x-th camera and reappear at the x'-th camera; it is zero within the estimated occurrence time interval; outside the estimated occurrence time interval, it is the absolute value of the time distance between the current time point and the nearest endpoint of the estimated occurrence time interval. The second positional plausibility coefficient represents the relationship between any target region block g in the matched frame and any region block g' in the starting frame of any unmatched video. Represented by natural constant An exponential function with base 0.

[0158] in other words, This indicates the actual time it takes for the target vehicle to move between the two cameras. This represents the estimated time range. "Reasonable location" means that the vehicle's actual time falls within the estimated range, in which case the time deviation is 0; if it falls outside the range, the deviation is the difference between the actual time and the nearest endpoint of the range. For example, if the estimated vehicle time is 5-6 minutes: if the actual time is 4 minutes, the deviation is 1 minute; if it is 5 minutes and 30 seconds, the deviation is 0.

[0159] The The detailed acquisition process is as follows:

[0160] Since the camera installation location is prior information, knowing the position of a region block within a camera allows us to calculate its actual spatial coordinates. When a region block leaves a camera, the vehicle's appearance time at other cameras can be estimated using the vehicle's average speed and the path length between the leaving camera and other cameras. The calculation method is as follows:

[0161] ;

[0162] Among them, the This represents the path length (prior information) between the x-th camera and the x'-th camera. Let x represent the average vehicle speed as seen from the x-th camera. This represents the preset speed fluctuation amount, which can be taken as... The obtained The result is an interval, representing the estimated time interval for the target area block to move from the x-th camera to the x'-th camera.

[0163] S603. Calculate the matching degree based on the cosine similarity and the second position reasonable coefficient.

[0164] The matching degree can be expressed by the formula:

[0165] ;

[0166] Among them, the This represents the matching degree between any target region block g in the matching frame and any region block g' in an adjacent frame to be matched.

[0167] Figure 7 This is a schematic diagram of a vehicle trajectory tracking system based on video stream provided in the seventh embodiment of the present invention. The system includes:

[0168] The region block division module 701 is used to obtain the vehicle region in each frame of the video captured by each camera in the target area, calculate the information coefficient of each feature point in each vehicle region, and then divide each vehicle region into region blocks.

[0169] The matching calculation module 702 is used to locate the corresponding target region block in the matching frame where the target vehicle appears, and to perform a matching process between any target region block in the matching frame and each region block to be matched in the matching frame. The matching process includes calculating the matching degree based on the information vector and position reasonableness coefficient of the target region block and the region block to be matched.

[0170] The matching determination and repetition module 703 is used to include the corresponding unmatched area block into the continuous path sequence of the target vehicle and use it as a new matching frame if the maximum matching degree is greater than a preset threshold, and repeat the matching process until all frames are matched.

[0171] The integration module 704 is used to integrate the continuous path sequence, map it to the real geographic coordinate system, and generate the driving trajectory of the target vehicle.

[0172] The technical features and effects of the vehicle driving trajectory tracking system based on video stream proposed in this embodiment of the invention are the same as those of the method proposed in this embodiment of the invention, and will not be repeated here.

[0173] The present invention has the following beneficial effects:

[0174] First, the vehicle regions in each frame of the video captured by each camera within the target area are obtained. The information coefficients of each feature point within each vehicle region are calculated, and then each vehicle region is divided into region blocks. By analyzing these region blocks and utilizing the stability and complementarity of local features, the dependence of existing vehicle trajectory tracking technologies on the overall global appearance is effectively reduced, thereby significantly improving their robustness and trajectory continuity in complex scenarios.

[0175] Secondly, in the matching frame where the target vehicle appears, the corresponding target region block is located. A matching process is then performed between any target region block in the matching frame and each region block to be matched in the matching frame. This matching process includes calculating a matching degree based on the information vectors and positional reasonableness coefficients of the target region block and the region blocks to be matched. The higher the matching degree, the more likely the target region block and the region blocks to be matched belong to the target vehicle.

[0176] Furthermore, if the maximum matching degree is greater than a preset threshold, the corresponding unmatched area block is included in the continuous path sequence of the target vehicle and used as a new matching frame. This matching process is repeated until all frames are matched. Each frame of the video captured by each camera within the target area needs to undergo this matching process.

[0177] Finally, the continuous path sequences are integrated and mapped to a real geographic coordinate system to generate the driving trajectory of the target vehicle. The continuous path sequence of each region represents the spatiotemporal trajectory of the corresponding vehicle component under the view of each camera within the target area. By fusing these local continuous path sequences, a complete and coherent global driving trajectory of the vehicle can be synthesized.

[0178] In summary, this invention effectively overcomes association errors caused by the variability of features in some regions by focusing on stable and highly discriminative local features for matching, thereby generating more accurate and continuous vehicle driving trajectories.

[0179] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0180] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A vehicle trajectory tracking method based on video stream, characterized in that, The method includes: The process involves acquiring vehicle regions from each frame of video captured by each camera within a target area, calculating the information coefficients of each feature point within each vehicle region, and then dividing each vehicle region into region blocks. The acquisition of the information coefficients includes: determining a reference feature point for the current feature point, with the current feature point and the reference feature point together forming a reference region; acquiring the spatial distance, color difference, and maximum spatial scale between the current feature point and any of the reference feature points; and acquiring the information coefficients of the current feature point based on the spatial distance, color difference, and maximum spatial scale. In the matching frame where the target vehicle appears, the corresponding target region block is located, and a matching process is performed between any target region block in the matching frame and each region block to be matched in the matching frame. The matching process includes calculating the matching degree based on the information vector and position reasonableness coefficient of the target region block and the region blocks to be matched. The direction of the information vector is the spatial change gradient direction of the information coefficient of all feature points in the region block, and the magnitude of the information vector is the average information coefficient of all feature points in the region block. If the maximum matching degree is greater than the preset threshold, the corresponding unmatched area block is included in the continuous path sequence of the target vehicle and used as a new matching frame. The matching process is repeated until all frames are matched. The continuous path sequence is integrated and mapped to a real geographic coordinate system to generate the driving trajectory of the target vehicle.

2. The vehicle trajectory tracking method based on video stream as described in claim 1, characterized in that, Before acquiring the vehicle area from each frame of video captured by each camera within the target area, the process also includes: Ensure that the clocks of all cameras within the target area are synchronized, so that the videos captured by each camera have timestamps with a unified time base; The video is preprocessed, including: eliminating lens distortion and performing color correction.

3. The vehicle trajectory tracking method based on video stream as described in claim 1, characterized in that, The vehicle area for acquiring each frame of video captured by each camera within the target area includes: Optical flow between consecutive frames is obtained, the optical flow is optimized, and the pixels in each frame are clustered based on the optical flow direction to obtain pixel groups with the same optical flow direction. Each pixel group corresponds to a vehicle region.

4. The vehicle trajectory tracking method based on video stream as described in claim 1, characterized in that, The process of dividing the region into blocks includes: A local point set is constructed based on the feature points. The construction process of the local point set includes: for the vehicle area that does not contain the assigned feature points, the feature points corresponding to the maximum and second largest distance values ​​from the center of the area are assigned to the local point set; the rationality of adding each unassigned feature point to the local point set is calculated; if the rationality value is greater than a preset rationality threshold, the unassigned feature points are assigned to the local point set. When there are still unassigned feature points after the construction process ends, the construction process is repeated to construct a new local point set until each feature point has its unique local point set to which it belongs, and one local point set corresponds to one region block. The process of obtaining the rationality includes: Obtain the area enclosed by the outer contour line after the unassigned feature points are assigned to the local point set, and obtain the geometric compactness based on the area and the area of ​​its smallest circumscribed quadrilateral. The average difference between the information coefficients of each feature point in the local point set and the mean information coefficient, as well as the difference between the information coefficients of the unassigned feature points and the mean information coefficient, are obtained. Information consistency is obtained based on the average difference and the difference. The rationality is determined based on the geometric compactness and the information consistency.

5. The vehicle trajectory tracking method based on video stream as described in claim 1, characterized in that, The process of selecting the frame to be matched includes: When the matching frame is a non-ending frame in its video, the frame to be matched is the adjacent frame to be matched of the matching frame. When the matching frame is the end frame of its own video, the frame to be matched also includes the start frame of any unmatched video.

6. The vehicle trajectory tracking method based on video stream as described in claim 5, characterized in that, The process of obtaining the matching degree includes: When the matching frame is a non-end frame in its video, obtain the cosine similarity of the information vectors between any target region block of the matching frame and any region block in an adjacent frame to be matched. The inverse proportional value of the ratio of the distance between the center of the target region block and the center of any region block in the adjacent frame to be matched to the diameter of the target region block is used as the first position rationality coefficient; The matching degree is calculated based on the cosine similarity and the first position reasonable coefficient.

7. The vehicle trajectory tracking method based on video stream as described in claim 5, characterized in that, The process of obtaining the matching degree also includes: When the matching frame is the end frame of its video, obtain the cosine similarity of the information vectors between any target region block of the matching frame and any region block in the start frame of any unmatched video. The inverse proportional value of the time difference between the actual appearance time of the target vehicle in the unmatched video and the estimated appearance time interval is calculated as the second position reasonable coefficient. The matching degree is calculated based on the cosine similarity and the second position reasonable coefficient.

8. A vehicle trajectory tracking system based on video stream, characterized in that, The system includes: The region block division module is used to acquire vehicle regions in each frame of video captured by each camera within a target area, calculate the information coefficients of each feature point within each vehicle region, and then divide each vehicle region into region blocks. The process of acquiring the information coefficients includes: determining a reference feature point for the current feature point, wherein the current feature point and the reference feature point together form a reference region; acquiring the spatial distance, color difference, and maximum spatial scale within the reference region between the current feature point and any of the reference feature points; and acquiring the information coefficients of the current feature point based on the spatial distance, the color difference, and the maximum spatial scale. The matching calculation module is used to locate the corresponding target region block in the matching frame where the target vehicle appears, and to perform a matching process between any target region block in the matching frame and each region block to be matched in the matching frame. The matching process includes calculating the matching degree based on the information vector and positional rationality coefficient of the target region block and the region blocks to be matched. The direction of the information vector is the spatial gradient direction of the information coefficient of all feature points in the region block, and the magnitude of the information vector is the average information coefficient of all feature points in the region block. The matching determination and repetition module is used to include the corresponding unmatched area block into the continuous path sequence of the target vehicle and use it as a new matching frame if the maximum matching degree is greater than a preset threshold. The matching process is repeated until all frames are matched. An integration module is used to integrate the continuous path sequence, map it to a real geographic coordinate system, and generate the driving trajectory of the target vehicle.

Images