A vehicle precise tracking method and system

Abstract

The present invention relates to a method and system for accurately tracking a vehicle. The method comprises the following steps: obtaining aerial video and flight strip features of an unmanned aerial vehicle (UAV); performing real-time monitoring of a road by the UAV; segmenting the aerial video according to the flight strip features to obtain a plurality of video frames; performing feature weighting on the plurality of video frames to obtain a mutually optimized training set; iteratively adjusting the conditional probabilities in the mutually optimized training set to obtain an initial conditional probability; obtaining an optimal conditional probability when the initial conditional probability meets a preset iteration condition; optimizing the plurality of video frames according to the optimal conditional probability to obtain an optimization result; identifying a vehicle according to the plurality of video frames and the optimization results to obtain an identification result; and performing coordinate transformation on the identification result to obtain an accurately tracking result of the vehicle; the aerial video can also be segmented and iteratively adjusted to obtain an optimal optimization result, and then the optimized result is used to identify the vehicle, thereby improving the accuracy of vehicle identification.

Description

Technical Field

[0001] The present invention relates to the field of smart highway technology, and in particular to a vehicle precision tracking method and system. Background Art

[0002] With the continuous development of society and the economy, people's travel needs and the scale of freight transportation are constantly expanding, and traffic volume on highways has shown a trend of rapid annual growth. The increasing number of vehicles has led to frequent road congestion and an increased probability of traffic accidents. Traditional operation and management models are unable to effectively cope with this large-scale and high-density traffic flow. There is an urgent need to use intelligent platforms to optimize traffic organization, improve road traffic efficiency, ensure smooth vehicle travel, and reduce the time wasted and economic losses caused by traffic congestion and other problems.

[0003] Highway traffic is high, and accidents often have serious consequences. Traditional safety management methods, which rely primarily on manual inspections and post-event handling, are significantly delayed and unable to monitor road conditions in real time, provide timely warnings, or provide rapid responses.

[0004] Therefore, it is urgent to propose a vehicle precision tracking method and system to solve the technical problems that the existing technology relies on manual inspection and post-processing, which has obvious lags in safety management methods, cannot monitor road conditions in real time, and may have low accuracy due to manual inspection and post-processing. Summary of the Invention

[0005] In view of this, it is necessary to provide a vehicle precision tracking method and system to solve the technical problems in the existing technology that the safety management means relying on manual inspection and post-processing have obvious lags, cannot monitor road conditions in real time, and may have low accuracy due to manual inspection and post-processing.

[0006] In order to solve the above problems, the present invention provides a vehicle accurate tracking method, comprising:

[0007] Acquire an aerial video and flight strip features of the UAV, and segment the aerial video according to the flight strip features to obtain multiple video frames;

[0008] Performing feature weighting on the multiple video frames to obtain a mutually optimized training set; the mutually optimized training set includes multiple samples and a conditional probability of each sample;

[0009] Iteratively adjusting the conditional probability in the mutual optimization training set to obtain an initial conditional probability, obtaining an optimal conditional probability when the initial conditional probability meets a preset iteration condition, and optimizing the multiple video frames according to the optimal conditional probability to obtain an optimization result;

[0010] The vehicle is identified according to the multiple video frames, the flight path features and the optimization result to obtain an identification result, and the identification result is coordinate-converted to obtain an accurate vehicle tracking result.

[0011] In a possible implementation, segmenting the aerial video according to the flight strip features to obtain multiple video frames includes:

[0012] Reading the aerial video to obtain multiple initial video frames;

[0013] Set the number of tracking point detections and motion detections to determine whether the current initial video frame is the last video frame;

[0014] When the current initial video frame is not the last video frame, performing tracking point detection on the current initial video frame according to the flight strip feature to obtain a detection result;

[0015] When the detection result indicates that a tracking point exists, the number of tracking point detections is updated; when the updated number of tracking point detections satisfies a preset detection condition, the motion direction is obtained, the number of tracking point detections is cleared, and the number of motion detections is updated;

[0016] When the updated number of motion detections meets a preset judgment condition, a judgment result is obtained according to the motion direction, and the number of motion detections is cleared;

[0017] When the judgment result is that the motion direction has not changed, determining the frame number of the current initial video frame, determining the next initial video frame as the current initial video frame, and judging whether it is the last video frame;

[0018] When the current initial video frame is the last video frame, the aerial video is segmented according to all frame numbers to obtain multiple video frames.

[0019] In a possible implementation, performing feature weighting on the multiple video frames to obtain a mutually optimized training set includes:

[0020] Set the video stabilization factor, shooting distance factor, and image quality factor;

[0021] Performing feature weighting on the multiple video frames based on a preset algorithm and the video stabilization factor, the shooting distance factor, and the image quality factor to obtain a weight for each video frame;

[0022] The video frames whose weights do not meet a preset threshold are removed from the multiple video frames to obtain a mutual optimization training set. In one possible implementation, the mutual optimization training set also includes the posterior probability of each sample; the iterative adjustment of the conditional probability in the mutual optimization training set to obtain an initial conditional probability, and when the initial conditional probability meets a preset iteration condition, obtaining an optimal conditional probability includes:

[0023] Predicting the mutual optimization training set to obtain the optimization accuracy of each sample;

[0024] The most effective sample in the mutual optimization training set is determined according to the optimization accuracy and the posterior probability of each sample, and the conditional probability of the most effective sample is determined as the initial conditional probability.

[0025] In a possible implementation, after iteratively adjusting the conditional probabilities in the mutual optimization training set to obtain initial conditional probabilities, the method further includes:

[0026] Determine whether the initial condition probability is greater than the initial condition probability of the previous round;

[0027] If yes, the most effective sample in the mutual optimization training set is deleted to obtain a modified mutual optimization training set, and the next round of iterative adjustment is performed;

[0028] If not, the initial conditional probability of the previous round is determined to be the optimal conditional probability.

[0029] In a possible implementation, determining the most effective sample in the mutual optimization training set according to the optimization accuracy and the posterior probability of each sample includes:

[0030] Calculating the optimized accuracy and the posterior probability of each sample to obtain a difference;

[0031] The sample with the maximum absolute value of the difference is determined as the most effective sample in the mutual optimization training set.

[0032] In a possible implementation, optimizing the multiple video frames according to the optimal conditional probability to obtain an optimization result includes:

[0033] Optimizing the optimizer according to the optimal conditional probability to obtain a target optimizer;

[0034] The multiple video frames are optimized according to the target optimizer to obtain an optimization result.

[0035] In one possible implementation, identifying the vehicle based on the multiple video frames, the flight path features, and the optimization result to obtain a recognition result, and performing coordinate transformation on the recognition result to obtain a precise vehicle tracking result includes:

[0036] Processing the optimization result based on the binocular parallax principle to obtain a depth map;

[0037] Iteratively calculating the optimization result and the flight path characteristics based on the preset algorithm to obtain an optimal key frame orientation result;

[0038] Performing target detection on the plurality of video frames to obtain two-dimensional coordinates of the vehicle in an image plane;

[0039] Coordinate transformation is performed according to the depth map, the optimal key frame orientation result and the two-dimensional coordinates of the image plane to obtain a precise vehicle tracking result.

[0040] In a possible implementation, processing the optimization result based on the binocular parallax principle to obtain a depth map includes:

[0041] Processing the optimization result based on the binocular parallax principle to obtain parallax;

[0042] The parallax is calculated based on the triangulation principle and camera calibration parameters to obtain the depth value corresponding to each pixel and generate a depth map.

[0043] On the other hand, the present invention also provides a vehicle precision tracking system, comprising:

[0044] A video segmentation module is used to obtain the aerial video and flight strip features of the UAV, and segment the aerial video according to the flight strip features to obtain multiple video frames;

[0045] A feature weighting module is used to perform feature weighting on the multiple video frames to obtain a mutually optimized training set; the mutually optimized training set includes multiple samples and a conditional probability of each sample;

[0046] an iterative adjustment module, configured to iteratively adjust the conditional probabilities in the mutual optimization training set to obtain initial conditional probabilities, obtain optimal conditional probabilities when the initial conditional probabilities meet preset iteration conditions, and optimize the multiple video frames according to the optimal conditional probabilities to obtain optimization results;

[0047] The result determination module is used to identify the vehicle according to the multiple video frames, the flight path characteristics and the optimization result to obtain an identification result, and to perform coordinate conversion on the identification result to obtain an accurate vehicle tracking result.

[0048] The beneficial effects of the present invention are as follows: the aerial video and flight path features of the UAV are obtained, so that the road can be monitored in real time; further, the aerial video is segmented and processed according to the flight path features to obtain multiple video frames; the multiple video frames are feature-weighted to obtain a mutually optimized training set; the mutually optimized training set includes multiple samples and the conditional probability of each sample; the conditional probability in the mutually optimized training set is iteratively adjusted to obtain an initial conditional probability, and when the initial conditional probability meets the preset iteration condition, the optimal conditional probability is obtained, and the multiple video frames are optimized according to the optimal conditional probability to obtain an optimization result, which can improve the accuracy of the video frame; the vehicle is identified according to the multiple video frames and the optimization result to obtain a recognition result, and the recognition result is coordinate-converted to obtain a vehicle accurate tracking result; the present invention improves the accuracy of vehicle identification by performing vehicle identification on the video frame after the accuracy is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic flow chart of an embodiment of the vehicle precise tracking method provided by the present invention;

[0050] Figure 2 A schematic structural diagram of an embodiment of an application scenario of the vehicle tracking system provided by the present invention;

[0051] Figure 3 A schematic diagram of an embodiment of the aerial video segmentation process provided by the present invention;

[0052] Figure 4 For the present invention Figure 1 A schematic flow chart of an embodiment of step S104;

[0053] Figure 5 A schematic structural diagram of an embodiment of the vehicle precision tracking system provided by the present invention;

[0054] Figure 6 This is a schematic structural diagram of an embodiment of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0055] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0056] like Figure 1 As shown, a specific embodiment of the present invention discloses a vehicle precision tracking method, comprising:

[0057] S101, obtaining an aerial video and flight strip features of a UAV, and segmenting the aerial video according to the flight strip features to obtain multiple video frames;

[0058] S102, performing feature weighting on multiple video frames to obtain a mutually optimized training set; the mutually optimized training set includes multiple samples and a conditional probability of each sample;

[0059] S103, iteratively adjusting the conditional probability in the mutual optimization training set to obtain an initial conditional probability, obtaining an optimal conditional probability when the initial conditional probability meets a preset iteration condition, and optimizing multiple video frames based on the optimal conditional probability to obtain an optimization result;

[0060] S104: Identify the vehicle based on the multiple video frames, the flight path features, and the optimization results to obtain a recognition result, and perform coordinate conversion on the recognition result to obtain an accurate vehicle tracking result.

[0061] It should be understood that: it can be applied to vehicle tracking systems, which can receive information from drones, such as Figure 2 As shown, the drone can receive signals from the Beidou satellite to orient the drone and obtain its real-time location. The drone can take aerial videos of vehicles driving on the road, and the vehicle tracking system can obtain the precise location of the vehicle through the aerial videos.

[0062] In a specific embodiment of the present invention, an aerial video of a drone can be obtained, and the drone's flight trajectory data (such as GPS coordinates, flight altitude, heading angle) and geographic information of the shooting area (topography, landmarks, etc.) can also be obtained as relevant information on flight strip features. The flight strip features can be obtained through the Beidou positioning system or other methods. The specific settings can be made according to actual conditions, and this embodiment of the solution is not limited here. After obtaining the aerial video, the aerial video can be preprocessed. The preprocessing can include removing noise by filtering, ensuring image color consistency through color correction, and adjusting the resolution as needed for data processing. The preprocessed aerial video is then segmented to obtain multiple video frames. A preset algorithm can also be set. The preset algorithm can be a Horn-Schunck algorithm. The Horn-Schunck algorithm can be used to perform feature weighting processing on multiple video frames. The feature weighting process in the Horn-Schunck algorithm can be a conventional operation, and this embodiment of the present invention is not limited here. After feature weighting, a mutual optimization training set can be obtained. The mutual optimization training set can include multiple samples retained by specific optimizer learning and the conditional probability of each sample. In order to improve progress, the conditional probability in the mutual optimization training set can be iteratively adjusted to obtain the adjusted initial conditional probability, and then the initial conditional probability can be judged. When the initial conditional probability meets the preset iteration condition, the optimal conditional probability can be obtained, and then multiple video frames are optimized according to the optimal conditional probability to obtain the optimization result; then the vehicle can be identified based on the multiple video frames and the optimization results to obtain the recognition result, and then the recognition result can be coordinate transformed to obtain the vehicle accurate tracking result.

[0063] Compared with the prior art, the present embodiment provides a method for obtaining aerial video and flight strip features of a drone, which can monitor roads in real time; further, the aerial video is segmented according to the flight strip features to obtain multiple video frames; the multiple video frames are feature-weighted to obtain a mutually optimized training set; the mutually optimized training set includes multiple samples and the conditional probability of each sample; the conditional probability in the mutually optimized training set is iteratively adjusted to obtain an initial conditional probability, and when the initial conditional probability meets the preset iteration condition, the optimal conditional probability is obtained, and the multiple video frames are optimized according to the optimal conditional probability to obtain an optimization result, which can improve the accuracy of the video frame; the vehicle is identified according to the multiple video frames and the optimization results to obtain a recognition result, and the recognition result is coordinate-converted to obtain a precise vehicle tracking result; the present invention improves the accuracy of vehicle identification by performing vehicle identification on the video frames after the accuracy is improved.

[0064] In some embodiments of the present invention, step S101 includes:

[0065] Read the aerial video to obtain multiple initial video frames;

[0066] Set the number of tracking point detections and motion detections to determine whether the current initial video frame is the last video frame;

[0067] When the current initial video frame is not the last video frame, tracking point detection is performed on the current initial video frame according to the flight path feature to obtain a detection result;

[0068] When the detection result shows that there is a tracking point, the number of tracking point detections is updated. When the updated number of tracking point detections meets the preset detection conditions, the motion direction is obtained, the number of tracking point detections is cleared, and the number of motion detections is updated.

[0069] When the updated motion detection times meet the preset judgment conditions, the judgment result is obtained according to the motion direction, and the motion detection times are cleared;

[0070] When the result of the determination is that the motion direction has not changed, determining the frame number of the current initial video frame, determining the next initial video frame as the current initial video frame, and determining whether it is the last video frame;

[0071] When the current initial video frame is the last video frame, the aerial video is segmented according to all frame numbers to obtain multiple video frames.

[0072] In a specific embodiment of the present invention, the cruising posture often adopted by the drone during flight is a zigzag shape. Where I represents a frame of image, then I ( x, y ) represents the yth image frame of the xth flight strip. For drone aerial videos, the adjacent image frames of each flight strip have spatial motion relationships and overlaps. ,There are corresponding motion relationships and overlaps between the pictures of adjacent flight strips. It should be noted that for adjacent flight strips, due to the inconsistency of flight strip lengths, two frames of images with corresponding flight strip frame numbers may not necessarily have a motion relationship and overlap. In order to maximize the preservation of the image information of the UAV aerial video, it is necessary to fully utilize the extreme position of each flight strip, and it is necessary to meet the requirements that the UAV flight posture between adjacent flight strips is approximately perpendicular to the flight strip movement direction and the overall overlap between adjacent flight strips is greater than 50%. The aerial video can be read and tracked based on the Horn-Schunck algorithm and flight strip features. Figure 3As shown, the aerial video can be read to obtain multiple initial video frames, and then tracking point detection can be performed on the multiple initial video frames one by one. Specifically, the number of tracking point detections and the number of motion detections can be set, the current initial video frame can be determined, and it can be determined whether the current initial video frame is the last video frame. If it is, it means that all the multiple initial video frames have been processed. If not, the next process can be carried out. The current initial video frame can be subjected to tracking point detection according to the flight strip features, the position of the tracking point is detected, and the detection result is obtained. It can then be determined whether there is a tracking point in the detection result. If not, the next initial video frame is detected. Detection, determine the next initial video frame as the current initial video frame, return to the step of "determining whether the current initial video frame is the last video frame", if so, the number of tracking point detections can be updated, for example, if the number of tracking point detections is 1, the number of tracking point detections can be added by 1, and it can be determined whether the number of tracking point detections of the tracking point meets the preset detection conditions, if not, the next initial video frame is detected, the next initial video frame is determined as the current initial video frame, and return to the step of "determining whether the current initial video frame is the last video frame"; the position of the tracking point in the current initial video frame can be calculated in each tracking process. , indicating the current tracking point P exist t The position at the moment is ( x,y ), wherein the preset detection condition may be detecting the position of the same tracking point for a preset number of consecutive times, the preset number may be 5, and if the updated tracking point detection number is 5, it is determined that the preset detection condition is met, for example, the same tracking point detects position A in the first frame, position B in the second frame, position C in the third frame, position D in the fourth frame, and position E in the fifth frame, and obtains these 5 consecutive positions, then the number of tracking point detections of the tracking point meets the preset detection condition, in order to be able to calculate the direction, the least squares method can be used to fit multiple consecutive positions to calculate the direction of motion of the tracking point, and then the number of tracking point detections can be cleared, and subsequent detections of other tracking points can be performed, and then the number of motion detections can be updated, for example, if the number of motion detections is 1, the number of motion detections can be added by 1, and the direction of motion can be expressed by the slope of the straight line. kTo characterize, in order to be able to compare and calculate the motion direction, at least 2 motion directions are required, so it is necessary to determine whether the calculated motion direction has been updated with the number of motion detections that meets the preset judgment condition. When the updated number of motion detections is greater than the preset number, it can be determined that the preset judgment condition is met, and then the number of motion detections is cleared, and then subsequent iterative calculations are performed. The preset number can be 2. If not, the next initial video frame is detected, and the next initial video frame is determined as the current initial video frame, and the step of "determining whether the current initial video frame is the last video frame" is returned. If so, the multiple motion directions are compared and calculated to determine whether the motion direction has changed. Specifically, when the motion direction of the feature point calculated twice is , , when the relationship of formula (1) is satisfied, it is considered that the direction of movement has not changed:

[0073] (1)

[0074] Otherwise, the tracking point's direction of motion is considered to have changed. The next initial video frame is then tested and determined as the current initial video frame, and the process returns to the step of "determining whether the current initial video frame is the last video frame." If the direction of motion is considered unchanged, the frame number of the current initial video frame can be recorded. The next initial video frame is then tested and determined as the current initial video frame, and the process returns to the step of "determining whether the current initial video frame is the last video frame." This process continues until the result of the step of "determining whether the current initial video frame is the last video frame" is yes, indicating that all multiple initial video frames have been processed. The aerial video can then be segmented based on all recorded frame numbers to obtain multiple video frames. Based on the zigzag characteristics of the drone's cruising trajectory and the above two assumptions, the above basic algorithm process maximizes video segmentation and key information retention.

[0075] In some embodiments of the present invention, step S102 includes:

[0076] Set the video stabilization factor, shooting distance factor, and image quality factor;

[0077] Perform feature weighting on multiple video frames based on a preset algorithm and a video stabilization factor, a shooting distance factor, and an image quality factor to obtain a weight for each video frame;

[0078] Video frames whose weights are not within a preset threshold are removed from multiple video frames to obtain a mutual optimization training set.

[0079] In a specific embodiment of the present invention, in order to achieve the effect of weakening the independence of feature conditions, different weights are assigned to video frames to achieve this, and a video stabilization factor (VSF), an image quality factor (IQF), and a shooting distance factor (SDF) are set. Among them, the video stabilization factor uses a motion model based on affine transformation to solve the problem of selecting stable video clips; the image quality factor mainly uses the characteristic that blurred images easily lose high-frequency information to process and select high-quality images; the shooting distance factor selects image frames with similar content ratios by simply calculating the average shooting height. These three factors work together to solve the problem of extracting efficient key frames. The preset algorithm can be the Horn-Schunck algorithm, and the video stabilization factor, the shooting distance factor, and the image quality factor can be brought into the Horn-Schunck algorithm. The formula is shown in (2):

[0080] (2)

[0081] Where, is the video frame, is the total number of features, d, c, b Indicates category, It is called the prior probability, that is, the mutual optimization of the categories in the training set The frequency of occurrence, is the conditional probability, that is, the category The next video frame The frequency value of the feature corresponding to the feature value, , They are video stabilization factor, shooting distance factor and image quality factor respectively.

[0082] By performing feature weighting processing on multiple video frames through the Horn-Schunck algorithm, the weight of each video frame can be obtained. The output weight can be 1 or 0. Then, the video frames with weight 0 can be removed, and the video frames with weight 1 are left. The remaining video frames can be stored in the mutual optimization training set.

[0083] In some embodiments of the present invention, the mutual optimization training set further includes the posterior probability of each sample; step S103 includes:

[0084] Predict the mutual optimization training set and obtain the optimization accuracy of each sample;

[0085] According to the optimization accuracy and posterior probability of each sample, the most effective sample in the mutual optimization training set is determined, and the conditional probability of the most effective sample is determined as the initial conditional probability.

[0086] In a specific embodiment of the present invention, the Horn-Schunck algorithm can also calculate the posterior probability of each sample when calculating each video frame, so the mutual optimization training set also includes the posterior probability of each sample, and then the mutual optimization training set can be Make predictions, and then calculate the optimization accuracy of each sample The specific prediction process can be set according to actual conditions and is not limited in the embodiments of the present invention. The most effective sample in the mutual optimization training set can be determined based on the optimization accuracy and posterior probability of each sample. Specifically, in some embodiments of the present invention, the most effective sample in the mutual optimization training set is determined based on the optimization accuracy and posterior probability of each sample, including:

[0087] Calculate the optimized accuracy and posterior probability of each sample to get the difference;

[0088] The sample with the maximum absolute value of the difference is determined as the most effective sample in the mutual optimization training set.

[0089] In a specific embodiment of the present invention, the difference between the optimization accuracy and the posterior probability of each sample is calculated, and then the sample with the maximum absolute value of the difference can be determined as the most effective sample in the mutual optimization training set. . You can also use the conditional probability of the most effective sample Determine the initial conditional probability.

[0090] In some embodiments of the present invention, after iteratively adjusting the conditional probabilities in the mutual optimization training set to obtain the initial conditional probabilities, the method further includes:

[0091] Determine whether the initial conditional probability is greater than the initial conditional probability of the previous round;

[0092] If so, the most effective sample in the mutual optimization training set is deleted to obtain the modified mutual optimization training set, and the next round of iterative adjustment is carried out;

[0093] If not, the initial conditional probability of the previous round is determined to be the optimal conditional probability.

[0094] In a specific embodiment of the present invention, after obtaining the initial conditional probability, it can be determined whether the initial conditional probability is greater than the initial conditional probability of the previous round. If there is no initial conditional probability of the previous round, it means it is the first iteration and the next iteration process is carried out directly. If it is, it means the modification is effective and the most effective sample is From the mutual optimization training set Remove it, return to the step of "calculating the optimized accuracy and posterior probability of each sample to obtain the difference", and carry out the next round of iterative modification until If the initial condition probability is no longer improved and is greater than the initial condition probability of the previous round, it means that the current iterative modification process is invalid, and the initial condition probability of the previous round is determined as the optimal condition probability. , , .

[0095] In some embodiments of the present invention, optimizing multiple video frames according to the optimal conditional probability to obtain an optimization result includes:

[0096] Optimize the optimizer according to the optimal conditional probability to obtain the target optimizer;

[0097] Optimize multiple video frames according to the target optimizer to obtain optimization results.

[0098] In a specific embodiment of the present invention, the optimizer can be enhanced according to the optimal conditional probability to obtain an optimizer of a two-stage enhanced model, as shown in formula (3):

[0099] (3)

[0100] Where, is the total number of features, d, c, b Indicates category, It is called the prior probability, that is, the mutual optimization of the categories in the training set The frequency value of occurrence.

[0101] Then, multiple video frames can be optimized by the optimizer of the two-stage enhancement model to obtain d, c, b Optimization results for each category in .

[0102] In some embodiments of the present invention, Figure 4 As shown, step S104 includes:

[0103] S401, processing the optimization result based on the binocular parallax principle to obtain a depth map;

[0104] S402, iteratively calculating the optimization results and flight path characteristics based on a preset algorithm to obtain an optimal keyframe orientation result;

[0105] S403, performing target detection on multiple video frames to obtain the two-dimensional coordinates of the vehicle in the image plane;

[0106] S404: Perform coordinate transformation based on the depth map, the optimal key frame orientation result, and the two-dimensional coordinates of the image plane to obtain an accurate vehicle tracking result.

[0107] In a specific embodiment of the present invention, the video frames in the optimization result may be processed based on the binocular parallax principle to obtain a depth map. Specifically, in some embodiments of the present invention, step S401 includes:

[0108] The optimization results are processed based on the binocular parallax principle to obtain the parallax;

[0109] The parallax is calculated based on the triangulation principle and camera calibration parameters to obtain the depth value corresponding to each pixel and generate a depth map.

[0110] In a specific embodiment of the present invention, the video frames in the optimization result are processed based on the binocular parallax principle, the parallax of the corresponding points in the left and right images is calculated, and then the depth value corresponding to each pixel in each video frame can be obtained according to the triangulation principle. Then, a depth map can be generated based on all the depth values. The baseline distance between the two cameras is known. B ,focal length f , and the pixel coordinate difference of the corresponding point in the left and right images d (parallax), then the depth value Z=Bf / d .

[0111] After obtaining the optimized keyframes, the Beidou positioning system can be used to obtain the drone's flight path characteristics, which include position coordinate information. Weighted block adjustment incorporates absolute position information as a constraint to orient the drone video keyframes. The least-squares principle is used for adjustment to determine the keyframe's exterior orientation elements. The Levenberg-Marquardt algorithm and flight path characteristics are then iterated to determine the optimal keyframe orientation, which includes accurate values ​​for camera position and attitude.

[0112] It is also possible to perform target detection on multiple video frames, obtain the vehicle bounding box, and determine the two-dimensional coordinates of the vehicle in the image plane. The vehicle depth value Z can then be obtained in combination with the depth map. Then, based on the depth value Z, the accurate values ​​of the camera position (latitude and longitude, altitude) and posture (including pitch angle, roll angle, and yaw angle) in the optimal key frame orientation result, the two-dimensional image coordinates can be converted into three-dimensional world coordinates through the coordinate conversion formula to obtain the vehicle's accurate tracking result in the three-dimensional world coordinate system.

[0113] The embodiment of the present invention can obtain the precise location of passing vehicles through the above process, thereby realizing real-time monitoring of all vehicles on the highway, understanding the driving status and distribution of vehicles, and providing comprehensive data support for traffic management.

[0114] In order to better implement the vehicle precise tracking method in the embodiment of the present invention, based on the vehicle precise tracking method, the embodiment of the present invention also provides a vehicle precise tracking system, such as Figure 5 As shown, the vehicle precision tracking system 500 includes:

[0115] The video segmentation module 501 is used to obtain the aerial video and flight strip features of the UAV, and segment the aerial video according to the flight strip features to obtain multiple video frames;

[0116] A feature weighting module 502 is configured to perform feature weighting on multiple video frames to obtain a mutually optimized training set; the mutually optimized training set includes multiple samples and a conditional probability of each sample;

[0117] Iterative adjustment module 503 is used to iteratively adjust the conditional probabilities in the mutual optimization training set to obtain initial conditional probabilities. When the initial conditional probabilities meet the preset iteration conditions, the optimal conditional probabilities are obtained, and multiple video frames are optimized according to the optimal conditional probabilities to obtain optimization results.

[0118] The result determination module 504 is used to identify the vehicle based on multiple video frames, flight path features and optimization results to obtain a recognition result, and perform coordinate conversion on the recognition result to obtain a precise vehicle tracking result.

[0119] The vehicle precision tracking system 500 provided in the above embodiment can implement the technical solution described in the above embodiment of the vehicle precision tracking method. The specific implementation principles of the above modules or units can be found in the corresponding contents in the above embodiment of the vehicle precision tracking method, which will not be repeated here.

[0120] like Figure 6 As shown, the present invention also provides an electronic device 600. The electronic device 600 includes a processor 601, a memory 602 and a display 603. Figure 6 Only some of the components of the electronic device 600 are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0121] In some embodiments, the memory 602 may be an internal storage unit of the electronic device 600, such as a hard disk or memory of the electronic device 600. In other embodiments, the memory 602 may also be an external storage device of the electronic device 600, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 600.

[0122] Furthermore, the memory 602 may include both an internal storage unit of the electronic device 600 and an external storage device. The memory 602 is used to store application software installed in the electronic device 600 and various data.

[0123] In some embodiments, the processor 601 may be a central processing unit (CPU), a microprocessor, or other data processing chip, configured to execute program codes or process data stored in the memory 602 , such as the vehicle precision tracking method of the present invention.

[0124] In some embodiments, the display 603 can be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 603 is used to display information about the electronic device 600 and to display a visual user interface. Components 601-603 of the electronic device 600 communicate with each other via a system bus.

[0125] In some embodiments of the present invention, when the processor 601 executes the drone-based vehicle tracking program in the memory 602, the following steps may be implemented:

[0126] Obtain the aerial video and flight path features of the UAV, segment the aerial video according to the flight path features, and obtain multiple video frames;

[0127] Perform feature weighting on multiple video frames to obtain a mutually optimized training set; the mutually optimized training set includes multiple samples and the conditional probability of each sample;

[0128] Iteratively adjust the conditional probability in the mutual optimization training set to obtain the initial conditional probability. When the initial conditional probability meets the preset iteration condition, the optimal conditional probability is obtained, and multiple video frames are optimized according to the optimal conditional probability to obtain the optimization result;

[0129] The vehicle is identified based on multiple video frames, flight path features and optimization results to obtain the recognition result, and the recognition result is coordinate transformed to obtain the vehicle accurate tracking result.

[0130] It should be understood that, when the processor 601 executes the drone-based vehicle tracking program in the memory 602 , in addition to the above functions, it can also implement other functions. For details, please refer to the description of the corresponding method embodiment above.

[0131] Furthermore, the embodiments of the present invention do not specifically limit the type of electronic device 600 mentioned. The electronic device 600 may be a portable electronic device such as a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, or laptop computer. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The portable electronic devices mentioned above may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 600 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0132] Accordingly, an embodiment of the present application also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, the vehicle precise tracking method steps or functions provided in the above-mentioned method embodiments can be implemented.

[0133] Those skilled in the art will appreciate that all or part of the process flow of the above-described method embodiment can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0134] The above is a detailed introduction to the vehicle precision tracking method and system provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A vehicle precision tracking method, characterized in that: include: Acquire an aerial video and flight strip features of the UAV, and segment the aerial video according to the flight strip features to obtain multiple video frames; Performing feature weighting on the multiple video frames to obtain a mutually optimized training set; the mutually optimized training set includes multiple samples and a conditional probability of each sample; Iteratively adjusting the conditional probability in the mutual optimization training set to obtain an initial conditional probability, obtaining an optimal conditional probability when the initial conditional probability meets a preset iteration condition, and optimizing the multiple video frames according to the optimal conditional probability to obtain an optimization result; Identify the vehicle according to the multiple video frames, the flight path features, and the optimization result to obtain an identification result, and perform coordinate conversion on the identification result to obtain an accurate vehicle tracking result; The mutually optimized training set also includes the posterior probability of each sample; the iterative adjustment of the conditional probability in the mutually optimized training set to obtain the initial conditional probability, and when the initial conditional probability satisfies a preset iteration condition, obtaining the optimal conditional probability includes: Predicting the mutual optimization training set to obtain the optimization accuracy of each sample; Determining the most effective sample in the mutual optimization training set according to the optimization accuracy and the posterior probability of each sample, and determining the conditional probability of the most effective sample as the initial conditional probability; After iteratively adjusting the conditional probabilities in the mutual optimization training set to obtain initial conditional probabilities, the method further includes: Determine whether the initial condition probability is greater than the initial condition probability of the previous round; If yes, the most effective sample in the mutual optimization training set is deleted to obtain a modified mutual optimization training set, and the next round of iterative adjustment is performed; If not, then determine the initial conditional probability of the previous round as the optimal conditional probability; Determining the most effective sample in the mutual optimization training set according to the optimization accuracy and the posterior probability of each sample includes: Calculating the optimized accuracy and the posterior probability of each sample to obtain a difference; The sample with the maximum absolute value of the difference is determined as the most effective sample in the mutual optimization training set.

2. The vehicle precise tracking method according to claim 1, characterized in that: The aerial video is segmented according to the flight strip features to obtain multiple video frames, including: Reading the aerial video to obtain multiple initial video frames; Set the number of tracking point detections and motion detections to determine whether the current initial video frame is the last video frame; When the current initial video frame is not the last video frame, performing tracking point detection on the current initial video frame according to the flight strip feature to obtain a detection result; When the detection result indicates that a tracking point exists, the number of tracking point detections is updated; when the updated number of tracking point detections satisfies a preset detection condition, the motion direction is obtained, the number of tracking point detections is cleared, and the number of motion detections is updated; When the updated number of motion detections meets a preset judgment condition, a judgment result is obtained according to the motion direction, and the number of motion detections is cleared; When the judgment result is that the motion direction has not changed, determining the frame number of the current initial video frame, determining the next initial video frame as the current initial video frame, and judging whether it is the last video frame; When the current initial video frame is the last video frame, the aerial video is segmented according to all frame numbers to obtain multiple video frames.

3. The vehicle precise tracking method according to claim 1, characterized in that: The step of weighting the features of the plurality of video frames to obtain a mutually optimized training set includes: Set the video stabilization factor, shooting distance factor, and image quality factor; Performing feature weighting on the multiple video frames based on a preset algorithm and the video stabilization factor, the shooting distance factor, and the image quality factor to obtain a weight for each video frame; The video frames whose weights are not within a preset threshold value are removed from the multiple video frames to obtain a mutual optimization training set.

4. The vehicle precise tracking method according to claim 3, characterized in that: The optimizing the plurality of video frames according to the optimal conditional probability to obtain an optimization result includes: Optimizing the optimizer according to the optimal conditional probability to obtain a target optimizer; The multiple video frames are optimized according to the target optimizer to obtain an optimization result.

5. The vehicle precise tracking method according to claim 4, characterized in that: The identifying the vehicle according to the multiple video frames, the flight path features and the optimization result to obtain an identification result, and performing coordinate conversion on the identification result to obtain an accurate vehicle tracking result, including: Processing the optimization result based on the binocular parallax principle to obtain a depth map; Iteratively calculating the optimization result and the flight path characteristics based on the preset algorithm to obtain an optimal key frame orientation result; Performing target detection on the plurality of video frames to obtain two-dimensional coordinates of the vehicle in an image plane; Coordinate transformation is performed according to the depth map, the optimal key frame orientation result and the two-dimensional coordinates of the image plane to obtain a precise vehicle tracking result.

6. The vehicle precise tracking method according to claim 5, characterized in that: The step of processing the optimization result based on the binocular parallax principle to obtain a depth map includes: Processing the optimization result based on the binocular parallax principle to obtain parallax; The parallax is calculated based on the triangulation principle and camera calibration parameters to obtain the depth value corresponding to each pixel and generate a depth map.

7. A vehicle precision tracking system, characterized in that: include: A video segmentation module is used to obtain the aerial video and flight strip features of the UAV, and segment the aerial video according to the flight strip features to obtain multiple video frames; A feature weighting module is used to perform feature weighting on the multiple video frames to obtain a mutually optimized training set; the mutually optimized training set includes multiple samples and a conditional probability of each sample; an iterative adjustment module, configured to iteratively adjust the conditional probabilities in the mutual optimization training set to obtain initial conditional probabilities, obtain optimal conditional probabilities when the initial conditional probabilities meet preset iteration conditions, and optimize the multiple video frames according to the optimal conditional probabilities to obtain optimization results; a result determination module, configured to identify the vehicle based on the plurality of video frames, the flight path features, and the optimization result, obtain an identification result, and perform coordinate conversion on the identification result to obtain an accurate vehicle tracking result; The mutually optimized training set also includes the posterior probability of each sample; the iterative adjustment of the conditional probability in the mutually optimized training set to obtain the initial conditional probability, and when the initial conditional probability satisfies a preset iteration condition, obtaining the optimal conditional probability includes: Predicting the mutual optimization training set to obtain the optimization accuracy of each sample; Determining the most effective sample in the mutual optimization training set according to the optimization accuracy and the posterior probability of each sample, and determining the conditional probability of the most effective sample as the initial conditional probability; After iteratively adjusting the conditional probabilities in the mutual optimization training set to obtain initial conditional probabilities, the method further includes: Determine whether the initial condition probability is greater than the initial condition probability of the previous round; If yes, the most effective sample in the mutual optimization training set is deleted to obtain a modified mutual optimization training set, and the next round of iterative adjustment is performed; If not, then determine the initial conditional probability of the previous round as the optimal conditional probability; Determining the most effective sample in the mutual optimization training set according to the optimization accuracy and the posterior probability of each sample includes: Calculating the optimized accuracy and the posterior probability of each sample to obtain a difference; The sample with the maximum absolute value of the difference is determined as the most effective sample in the mutual optimization training set.

Images