Target tracking method and apparatus based on image space positioning, and device

Abstract

Provided in the present application are a target tracking method and apparatus based on image space positioning, and a device. The method comprises: acquiring first position coordinates of a plurality of target detection boxes at the current moment; on the basis of the first position coordinates, predicting second position coordinates of a candidate box of each target detection box at the next moment; on the basis of the second position coordinates, and third position coordinates of each target detection box at the next moment, calculating multi-dimensional similarities between each target detection box and a plurality of candidate boxes, wherein each multi-dimensional similarity comprises a multi-dimensional similarity determined on the basis of a dynamic threshold value of a centroid distance; and using a preset matching algorithm to output a target tracking result at the next moment according to the multi-dimensional similarities. The positions of targets in a world space are taken into consideration, thereby effectively solving the problem of positioning being inaccurate caused by single position information of a two-dimensional image; and a difference in the centroid distance between a distant target and a nearby target during consecutive imaging can be used, thereby improving the target positioning accuracy of a plurality of targets that overlap with each other.

Description

Target tracking method, device and equipment based on image space positioning Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a target tracking method, device, equipment and storage medium. Background Art

[0002] Target tracking algorithms help identify abnormal behaviors and potential dangers in vehicle processes, assisting autonomous vehicles in making more accurate decisions. Currently, the main steps of the target tracking algorithm include taking the position of the target detection box in the two-dimensional image as input, and outputting the predicted detection box position of the current target in the next frame through deep learning target detection, Kalman prediction, and hungary matching. However, the position change information of the two-dimensional image is single, and it cannot cover the tracking of distant and near targets in the real world. Moreover, in complex road test scenarios, such as parking lots or crosswalks, the number of targets is very large, and there is a lot of overlap between targets on the two-dimensional imaging plane of the image. Matching errors are very likely to occur during matching, and the accuracy of the target tracking results cannot be guaranteed. Summary of the Invention

[0003] In order to solve the above technical problems, the present application provides a target tracking method, device and equipment based on image space positioning.

[0004] In a first aspect, the present application provides a target tracking method based on image space positioning, comprising:

[0005] Obtaining first position coordinates of multiple target detection frames at a current moment, where the first position coordinates include first pixel coordinates and first world coordinates;

[0006] Based on the first position coordinates, predicting the second position coordinates of the candidate frame of each target detection frame at the next moment;

[0007] Based on the second position coordinate and the third position coordinate of the target detection frame at the next moment, calculating the multi-dimensional similarity between each target detection frame and multiple candidate frames, the multi-dimensional similarity including the multi-dimensional similarity determined based on the dynamic threshold of the centroid distance, and the third position coordinate is the actually measured position coordinate;

[0008] Utilize a preset matching algorithm and output the target tracking result at the next moment according to the multi-dimensional similarity.

[0009] In some implementations of the first aspect, calculating a multi-dimensional similarity between each of the object detection boxes and multiple candidate boxes based on the second position coordinates and a third position coordinate of the object detection box at a next moment includes:

[0010] For each target detection frame, using a preset centroid distance dynamic threshold, screening out at least one target candidate frame corresponding to the target detection frame from the plurality of candidate frames;

[0011] Based on the second position coordinates and the third position coordinates, a multi-dimensional similarity between each of the object detection boxes and the object candidate box is calculated.

[0012] In some implementations of the first aspect, for each target detection frame, using a preset dynamic threshold of centroid distance to select at least one target candidate frame corresponding to the target detection frame from the multiple candidate frames includes:

[0013] Calculating a first centroid distance between each target detection frame and a target tracking device, and calculating a second centroid distance between each target detection frame and a plurality of candidate frames based on the first centroid world coordinates of the target detection frame and the second centroid pixel coordinates of the candidate frames;

[0014] Based on a preset centroid distance dynamic threshold table, a dynamic threshold corresponding to the first centroid distance is determined.

[0015] The preset centroid distance dynamic threshold table also includes:

[0016] Label the historical training data and calculate the centroid distance for each pair of labeled target detection frames;

[0017] Counting the distribution of the centroid distances, dividing similarity thresholds, and generating a centroid distance dynamic threshold table;

[0018] The centroid distance dynamic threshold table is dynamically verified and optimized according to the target detection frame obtained each time.

[0019] For each target detection frame, target candidate frames whose distance from the second centroid of the target detection frame is greater than the dynamic threshold are screened out, and each target detection frame corresponds to at least one target candidate frame.

[0020] In some implementations of the first aspect,

[0021] The calculating, based on the second position coordinates and the third position coordinates, the multi-dimensional similarity between each of the target detection boxes and the target candidate box includes:

[0022] Calculating a detection frame centroid offset similarity between each target detection frame and the target candidate frame based on the third centroid pixel coordinates of the target detection frame and the second centroid pixel coordinates of the candidate frame;

[0023] Calculating a detection frame shape change similarity and a detection frame area similarity between each target detection frame and the target candidate frame based on the third frame pixel coordinates of the target detection frame and the second frame pixel coordinates of the candidate frame;

[0024] Calculating a world coordinate offset similarity between each target detection frame and the target candidate frame based on the third world coordinate of the target detection frame and the second world coordinate of the candidate frame;

[0025] The calculating of the multi-dimensional similarity between each target detection frame and the target candidate frame further includes:

[0026] A multi-dimensional similarity weighted model is used for calculation; wherein, the multi-dimensional similarity weighted model at least includes a distance measurement algorithm, an edge feature matching algorithm and a structural similarity index algorithm.

[0027] For each target detection frame, the detection frame centroid offset similarity, the detection frame shape change similarity, the detection frame area similarity, and the world coordinate offset similarity are weighted to obtain the multi-dimensional similarity.

[0028] In some implementations of the first aspect, obtaining first position coordinates of the plurality of target detection frames at a current moment includes:

[0029] Obtaining first pixel coordinates of multiple target detection frames at the current moment;

[0030] For each target detection frame, generating target ground point coordinates of the target detection frame based on the first pixel coordinates;

[0031] Based on the calibration internal parameters and calibration external parameters of the target tracking device, the spatial coordinates of the target ground point coordinates in the world space are calculated, and the spatial coordinates are the first world coordinates of the target detection box at the current moment.

[0032] Detecting the multiple target detection frames includes:

[0033] Performing distortion correction on the image captured by the target tracking device, performing target detection on the corrected image, generating a target detection frame, and performing correction processing on the target detection frame;

[0034] Among them, deep learning algorithm is used to perform target detection on the corrected image.

[0035] In some implementations of the first aspect, predicting, based on the first position coordinates, a second position coordinate of the candidate box of each object detection box at a next moment includes:

[0036] For each target detection frame, a Kalman filter algorithm is used to predict the second position coordinates of the candidate frame of the target detection frame at the next moment based on the first position coordinates of the target detection frame at the current moment and the fourth position coordinates at the previous moment. The Kalman filter algorithm further includes: performing parameter tuning on the state transition matrix and process noise based on the motion data of the target detection frame; inputting the first position coordinates and the fourth position coordinates into a preset motion model based on the tuned parameters to predict the candidate frame at the next moment; wherein the preset motion model adopts any one of a uniform velocity model, a random acceleration model, a steering model, or a nonlinear model.

[0037] In some implementations of the first aspect, outputting the target tracking result at the next moment based on the multi-dimensional similarity using a preset matching algorithm includes:

[0038] Using the Hungarian matching algorithm, according to the multi-dimensional similarity between each target detection frame and the multiple candidate frames, each target detection frame is matched with the final target candidate frame at the next moment;

[0039] The final target candidate box is output as the target tracking result.

[0040] In a second aspect, the present application further provides a target tracking device based on image space positioning, comprising:

[0041] An acquisition module, configured to acquire first position coordinates of multiple target detection frames at a current moment, wherein the first position coordinates include first pixel coordinates and first world coordinates;

[0042] A prediction module, configured to predict, based on the first position coordinates, a second position coordinate of the candidate frame of each target detection frame at a next moment;

[0043] a calculation module, configured to calculate a multi-dimensional similarity between each target detection frame and a plurality of candidate frames based on the second position coordinates and a third position coordinate of the target detection frame at a next moment, wherein the multi-dimensional similarity includes a multi-dimensional similarity determined based on a dynamic threshold of a centroid distance, and the third position coordinate is an actually measured position coordinate;

[0044] The output module is used to output the target tracking result at the next moment according to the multi-dimensional similarity using a preset matching algorithm.

[0045] In a third aspect, the present application further provides a computer device comprising a processor and a memory, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the target tracking method based on image space positioning as described in the first aspect is implemented.

[0046] In a fourth aspect, the present application further provides a computer-readable storage medium, characterized in that it stores a computer program, which, when executed by a processor, implements the target tracking method based on image space positioning as described in the first aspect.

[0047] Compared with the prior art, this application has at least the following beneficial effects:

[0048] By obtaining first position coordinates of multiple target detection frames at the current moment, where the first position coordinates include first pixel coordinates and first world coordinates, the position of the target in world space is comprehensively considered, effectively solving the problem of a single position change feature of the two-dimensional image, thereby effectively tracking distant and near targets in the real world. At the same time, the target tracking accuracy can be effectively improved for the problem of multiple target overlap in two-dimensional images in complex scenes. Based on the first position coordinates, the second position coordinates of the candidate frame of each target detection frame at the next moment are predicted. In combination with the target position prediction, the accuracy of target positioning is improved, solving the problem of inaccurate positioning caused by single position information based on the two-dimensional image. Based on the second position coordinates and the third position coordinates of the target detection frame at the next moment, multi-dimensional similarity is calculated between each target detection frame and the multiple candidate frames. The multi-dimensional similarity includes a multi-dimensional similarity determined based on a dynamic threshold of the center of mass distance. A preset matching algorithm is used to output the target tracking result at the next moment based on the multi-dimensional similarity. This can utilize the difference in center of mass distance between distant and near targets, as well as between small and large targets in two-dimensional continuous imaging, to improve the target positioning accuracy of multiple overlapping targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] FIG1 is a flow chart of a target tracking method based on image spatial positioning according to an embodiment of the present application;

[0050] FIG2 is a schematic diagram of a dynamic threshold relationship of centroid distance according to an embodiment of the present application;

[0051] FIG3 is a schematic diagram of the structure of a target tracking device based on image space positioning according to an embodiment of the present application;

[0052] FIG4 is a schematic diagram of the structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0054] Please refer to Figure 1, which is a flow chart of a target tracking method based on image space positioning provided by an embodiment of the present application. The target tracking method based on image space positioning in the embodiment of the present application can be applied to computer devices, including but not limited to vehicle-mounted terminals, smart phones, laptops, tablet computers, desktop computers, physical servers, and cloud servers. The computer device is provided with or externally connected to a target tracking device (such as a camera). As shown in Figure 1, the target tracking method based on image space positioning in this embodiment includes steps S101 to S104, which are detailed as follows:

[0055] Step S101 : obtaining first position coordinates of multiple target detection frames at a current moment, where the first position coordinates include first pixel coordinates and first world coordinates.

[0056] In this step, the target detection frame is the target bounding box obtained by performing bounding box detection on the 2D image collected by the target tracking device. Optionally, bounding box detection can be implemented based on an instance segmentation algorithm, which is not described here. Preferably, the target tracking device is a surround-view fisheye camera.

[0057] The first position coordinates are the position coordinates of the target detection frame, which are the actual measurements at the current moment. They include, but are not limited to, the centroid pixel coordinates describing the center of the target detection frame, the bounding box pixel coordinates describing the boundary of the target detection frame, the world coordinates of the centroid of the target detection frame center in world space, and the world coordinates of the bounding box describing the boundary of the target detection frame. Optionally, the pixel coordinates can be obtained during bounding box detection; the world coordinates can be obtained by transforming the pixel coordinates based on the coordinate relationship between the image and world space (such as camera intrinsic and extrinsic parameters).

[0058] In some embodiments, step S101 includes:

[0059] Obtaining first pixel coordinates of multiple target detection frames at the current moment;

[0060] For each target detection frame, generating target ground point coordinates of the target detection frame based on the first pixel coordinates;

[0061] Based on the calibration internal parameters and calibration external parameters of the target tracking device, the spatial coordinates of the target ground point coordinates in the world space are calculated, and the spatial coordinates are the first world coordinates of the target detection box at the current moment.

[0062] The detecting of the plurality of target detection frames includes:

[0063] Distortion correction is performed on the image captured by the target tracking device to eliminate any distortion that may exist in the image and ensure the accuracy of subsequent target detection. Distortion correction can use interpolation methods such as bilinear interpolation or bicubic interpolation to resample the pixels. For radial distortion, the pixel coordinates can be adjusted using a distortion correction formula. For tangential distortion, additional parameters can be used for correction.

[0064] Perform target detection on the corrected image and generate a target detection frame, and perform correction processing on the target detection frame. The correction processing includes but is not limited to the following aspects:

[0065] Multi-dimensional similarity matching: Using the multi-dimensional similarity evaluation method mentioned above, the target detection frames between adjacent frames are matched to correct the position and shape of the detection frames.

[0066] Post-processing technology: Post-processing technology can be used to further optimize the target detection box, such as non-maximum suppression (NMS) to eliminate overlapping boxes, bounding box regression to fine-tune the position of the box, etc.

[0067] Target tracking: Combined with the target tracking algorithm, the target's motion information between consecutive frames is used to optimize the position of the detection frame, thereby improving the stability and continuity of detection.

[0068] Shape adjustment: Based on the shape characteristics of the target, the detection frame is further adjusted to more accurately outline the target.

[0069] The corrected object detection results can be evaluated using metrics such as Intersection over Union (IoU) to assess the accuracy and coverage of the detection boxes. Based on the evaluation results, the entire detection system can be optimized, including adjusting the parameters of the object detection model, optimizing the post-processing algorithm, and adjusting the weights of multi-dimensional similarity.

[0070] Among them, when performing target detection on the corrected image, a deep learning algorithm is used for processing, such as Faster R-CNN, YOLO, SSD, but not limited to these.

[0071] In this embodiment, the target ground point represents the target's position in world space. Exemplarily, a deep learning algorithm is used to identify the target detection box name and pixel coordinates (including the x and y coordinates of the top left corner and the length w and width h of the target detection box) in the current frame image. Based on the target detection box pixel coordinates (x, y, w, h), the target ground point P (Px, Py) is equivalently generated. Optionally, Px = (x + w × 1 / 2) and Py = y + h, meaning the bottom midpoint of the target detection box is used as the target ground point.

[0072] After calibrating the internal and external parameters of the target tracking device, the world coordinate value of point P in world space is calculated to represent the position of the target in world space. Optionally, if the target tracking device is a surround-view fisheye camera, the target ground point P is the fisheye image coordinate. Therefore, the fisheye image is converted into a dedistorted image, and the corresponding pixel coordinate point of the target ground point P in the dedistorted image is found. The dedistorted image is actually a pinhole camera model. Based on the pinhole imaging principle, the world coordinate of point P can be obtained. The calculation formula can be:

[0073] ;

[0074] in, is the pixel coordinate of the target ground point, Is a constant, the first matrix and the second matrix on the right side of the equation can be combined into the camera intrinsic parameter matrix, and the third matrix is ​​the camera extrinsic parameter matrix. The world coordinates of the target ground point.

[0075] Step S102: predicting the second position coordinates of the candidate frame of each target detection frame at the next moment based on the first position coordinates.

[0076] In this step, the second position coordinates are the estimated values ​​for the next moment. Because the position of the target detection frame obtained through bounding box detection may differ from the actual position, the cumulative error will continue to increase if bounding box detection is performed on every frame. Therefore, this embodiment uses a motion model to predict the second position coordinates of the target detection frame in the next candidate frame. This position prediction method corrects the detection error and reduces the cumulative error.

[0077] In some embodiments, step S102 includes: for each target detection frame, using a Kalman filter algorithm, based on the first position coordinates of the target detection frame at the current moment and the fourth position coordinates at the previous moment, predicting the second position coordinates of the candidate frame of the target detection frame at the next moment. The Kalman filter algorithm further includes: performing parameter tuning on the state transition matrix and process noise based on the motion data of the target detection frame; based on the tuned parameters, inputting the first position coordinates and the fourth position coordinates into a preset motion model to predict the candidate frame at the next moment; wherein the preset motion model adopts any one of a uniform speed model, a random acceleration model, a steering model, or a nonlinear model.

[0078] In this embodiment, the Kalman filter algorithm can make predictions based on actual measurements (i.e., the position of the target detection box obtained by the bounding box detection algorithm) and estimated values ​​(i.e., the position of the target detection box estimated by the computational model), effectively reducing the cumulative error in the target tracking process and the estimation error of the motion model. It should be noted that the Kalman filter algorithm can adopt known algorithms, which will not be described in detail here.

[0079] Step S103, based on the second position coordinates and the third position coordinates of the target detection frame at the next moment, calculate the multidimensional similarity between each of the target detection frames and multiple candidate frames, wherein the multidimensional similarity includes a multidimensional similarity determined based on a dynamic threshold of the center of mass distance.

[0080] In this step, the third position coordinate is the actual measurement value at the next moment. Multi-dimensional similarity is the weighted result of the similarities calculated in multiple dimensions. The similarities in multiple dimensions include but are not limited to detection frame centroid offset similarity, detection frame shape change similarity, detection frame area similarity, and world coordinate offset similarity. Detection frame centroid offset similarity is the centroid offset similarity between the target detection frame and the candidate frame, detection frame shape change similarity is the shape change similarity between the target detection frame and the candidate frame, detection frame area similarity is the area similarity between the target detection frame and the candidate frame, and world coordinate offset similarity is the world coordinate offset similarity between the target detection frame and the candidate frame. Optionally, the second position coordinate and the third position coordinate are both coordinate sets, and the corresponding similarity can be calculated based on the corresponding coordinate values ​​representing the centroid, shape, area, and world coordinates in the second position coordinate and the third position coordinate. The similarity calculation formula can use cosine distance similarity, Euclidean distance similarity, or Manhattan distance similarity, etc., without limitation.

[0081] The dynamic threshold value of the centroid distance is determined by using the dynamic threshold value of the centroid distance to screen the target candidate frame, and the target candidate frame is used for the multi-dimensional similarity calculation process. It should be noted that, due to the obvious difference between the centroid distance of the far target and the centroid distance of the near target or the centroid distance of the large target and the centroid distance of the small target in two consecutive frames of images, for example, the centroid distance D1 between the far target a in the current frame image A and the far target a in the next frame image B, the centroid distance D2 between the near target b in the current frame image A and the near target b in the next frame image B, although the far target a and the near target b are on the same image, the centroid distance D1 of the far target a is significantly smaller than the centroid distance D2 of the near target b. Therefore, this application sets dynamic thresholds for different target types to screen target candidate frames, thereby reducing the matching operation amount of the candidate frames and improving the matching accuracy.

[0082] In some embodiments, step S103 includes:

[0083] For each target detection frame, using a preset centroid distance dynamic threshold, screening out at least one target candidate frame corresponding to the target detection frame from the plurality of candidate frames;

[0084] Based on the second position coordinates and the third position coordinates, a multi-dimensional similarity between each of the object detection boxes and the object candidate box is calculated.

[0085] In this embodiment, optionally, the step of screening target candidate boxes includes:

[0086] Calculating a first centroid distance between each target detection frame and a target tracking device, and calculating a second centroid distance between each target detection frame and a plurality of candidate frames based on the first centroid world coordinates of the target detection frame and the second centroid pixel coordinates of the candidate frames;

[0087] Determining a dynamic threshold corresponding to the first centroid distance based on a preset centroid distance dynamic threshold table;

[0088] The preset centroid distance dynamic threshold table further includes:

[0089] Label historical training data with information, including the location information and category information of the target detection frame;

[0090] For each pair of labeled object detection boxes, the centroid distance is calculated; preferably, the centroid distance can be calculated using Euclidean distance or other distance metrics.

[0091] The distribution of the centroid distance is counted, the similarity threshold is divided, and a centroid distance dynamic threshold table is generated; when dividing the similarity threshold, it can also be adjusted according to actual needs to ensure that different targets can be accurately distinguished during the target detection process.

[0092] The centroid distance dynamic threshold table is dynamically verified and tuned according to the target detection frame obtained each time. For example, during the dynamic verification process, the misjudgment situation is recorded, that is, the situation where different targets are regarded as the same target or the same target is regarded as different targets. According to the misjudgment situation, the threshold in the centroid distance dynamic threshold table is dynamically adjusted so that the system can distinguish targets more accurately in subsequent detections. The dynamic tuning results of each time are fed back to the target detection system to update the system parameters or algorithms, thereby improving the robustness and accuracy of the system. By continuously monitoring the performance of the target detection system and regularly evaluating the system, the update strategy of the centroid distance dynamic threshold table is further optimized according to the evaluation results to ensure that the system can work effectively in different scenarios and data. The centroid distance dynamic threshold table is dynamically verified and tuned according to the target detection frame obtained each time, thereby improving the performance and effect of the target detection system.

[0093] For each target detection frame, target candidate frames whose distance from the second centroid of the target detection frame is greater than the dynamic threshold are screened out, and each target detection frame corresponds to at least one target candidate frame.

[0094] In this optional embodiment, the first center of mass distance between the target detection frame and the target tracking device is calculated by the first center of mass world coordinate and the world coordinate of the target tracking device to determine whether the target detection frame belongs to a far target or a near target, and the corresponding dynamic threshold is determined by the first center of mass distance. The dynamic threshold is used for candidate frame screening of the target detection frame, thereby screening at least one target candidate frame corresponding to the target detection frame, effectively excluding other candidate frames that obviously do not meet the candidate frame conditions of the target detection frame, thereby reducing the amount of computation in the subsequent similarity calculation process and the matching process, and reducing the interference of invalid candidate frames in the matching process, thereby improving the target tracking accuracy. For example, for the center of mass distance dynamic threshold table, as shown in Figure 2, the center of mass distance dynamic threshold relationship diagram is used to represent the center of mass distance dynamic threshold table. When the first center of mass distance (target world distance) is 3 meters, the dynamic threshold is 3; when the first center of mass distance (target world distance) is 15 meters, the dynamic threshold is 3.

[0095] Optionally, the step of calculating the multi-dimensional similarity includes:

[0096] Calculating a box_IOU similarity between each target detection box and the target candidate box based on the third centroid pixel coordinates of the target detection box and the second centroid pixel coordinates of the candidate box;

[0097] Based on the third border pixel coordinates of the target detection box and the second border pixel coordinates of the candidate box, calculating the detection box shape change (box_SHAPE) similarity and the detection box area (box_AREA) similarity between each target detection box and the target candidate box;

[0098] Calculating a world coordinate offset similarity between each target detection frame and the target candidate frame based on the third world coordinate of the target detection frame and the second world coordinate of the candidate frame;

[0099] Specifically, the calculating the multi-dimensional similarity between each target detection frame and the target candidate frame further includes:

[0100] The calculation is performed using a multidimensional similarity weighted model, wherein the multidimensional similarity weighted model at least includes:

[0101] Distance measurement algorithms, such as Euclidean distance and Manhattan distance, are used to calculate the spatial distance between the target detection box and the target candidate box.

[0102] Edge feature matching algorithms, such as Canny edge detection, extract edge features from the target detection frame and the target candidate frame, and use appropriate matching algorithms, such as feature point matching and edge line matching, to calculate the edge feature similarity between them.

[0103] and the structural similarity index algorithm, which evaluates the overall similarity by comparing the brightness, contrast, and structural similarity of two images.

[0104] For each target detection frame, the detection frame centroid offset similarity, the detection frame shape change similarity, the detection frame area similarity, and the world coordinate offset similarity are weighted to obtain the multi-dimensional similarity.

[0105] In this optional embodiment, optionally, the coordinate parameters of the candidate box are divided by the coordinate parameters of the target detection box to obtain a similarity parameter.

[0106] Step S104: using a preset matching algorithm and based on the multi-dimensional similarity, outputting the target tracking result at the next moment.

[0107] In this step, since there may be multiple target detection frames in two consecutive frames of images, it is difficult to determine the target detection frame in the next frame of image corresponding to the current target detection frame. For example, there are 20 target detection frames on consecutive images A and B. For the second position coordinate predicted based on a target detection frame a on image A, it is difficult to directly determine which target detection frame in image B at the next moment the second position coordinate corresponds to. Therefore, this application uses a preset matching algorithm to match the correspondence between multiple target detection frames of the current frame image and multiple candidate frames of the next frame image.

[0108] In some embodiments, step S104 includes: using the Hungarian matching algorithm to match each target detection frame with the final target candidate frame at the next moment based on the multi-dimensional similarity between each target detection frame and the multiple candidate frames; and outputting the final target candidate frame as the target tracking result.

[0109] In this embodiment, all multi-dimensional similarities obtained in step S103 are input into the Hungarian matching algorithm, and each target detection frame and each candidate frame are traversed and cycled to obtain a candidate frame corresponding to each target detection frame.

[0110] In order to execute the target tracking method based on image space positioning corresponding to the above method embodiment to achieve the corresponding functions and technical effects. Referring to FIG3, FIG3 shows a structural block diagram of a target tracking device based on image space positioning provided by an embodiment of the present application. For ease of explanation, only the parts related to this embodiment are shown. The target tracking device based on image space positioning provided by an embodiment of the present application includes:

[0111] An acquisition module 301 is configured to acquire first position coordinates of multiple target detection frames at a current moment, where the first position coordinates include first pixel coordinates and first world coordinates.

[0112] A prediction module 302 is configured to predict a second position coordinate of a candidate frame of each target detection frame at a next moment based on the first position coordinate;

[0113] a calculation module 303, configured to calculate a multi-dimensional similarity between each target detection frame and a plurality of candidate frames based on the second position coordinates and a third position coordinate of the target detection frame at a next moment, wherein the multi-dimensional similarity includes a multi-dimensional similarity determined based on a dynamic threshold of a centroid distance, and the third position coordinate is an actually measured position coordinate;

[0114] The output module 304 is configured to output the target tracking result at the next moment according to the multi-dimensional similarity using a preset matching algorithm.

[0115] In some embodiments, the calculation module 303 includes:

[0116] a screening module, configured to screen out, for each target detection frame, at least one target candidate frame corresponding to the target detection frame from the plurality of candidate frames using a preset dynamic threshold of centroid distance;

[0117] A calculation module is used to calculate the multi-dimensional similarity between each target detection box and the target candidate box based on the second position coordinates and the third position coordinates.

[0118] In some embodiments, the screening module is specifically configured to:

[0119] Calculating a first centroid distance between each target detection frame and a target tracking device, and calculating a second centroid distance between each target detection frame and a plurality of candidate frames based on the first centroid world coordinates of the target detection frame and the second centroid pixel coordinates of the candidate frames;

[0120] Determining a dynamic threshold corresponding to the first centroid distance based on a preset centroid distance dynamic threshold table;

[0121] For each target detection frame, target candidate frames whose distance from the second centroid of the target detection frame is greater than the dynamic threshold are screened out, and each target detection frame corresponds to at least one target candidate frame.

[0122] In some embodiments, the computing module is specifically configured to:

[0123] Calculating a detection frame centroid offset similarity between each target detection frame and the target candidate frame based on the third centroid pixel coordinates of the target detection frame and the second centroid pixel coordinates of the candidate frame;

[0124] Calculating a detection frame shape change similarity and a detection frame area similarity between each target detection frame and the target candidate frame based on the third frame pixel coordinates of the target detection frame and the second frame pixel coordinates of the candidate frame;

[0125] Calculating a world coordinate offset similarity between each target detection frame and the target candidate frame based on the third world coordinate of the target detection frame and the second world coordinate of the candidate frame;

[0126] For each target detection frame, the detection frame centroid offset similarity, the detection frame shape change similarity, the detection frame area similarity, and the world coordinate offset similarity are weighted to obtain the multi-dimensional similarity.

[0127] In some embodiments, the acquisition module 301 is specifically configured to:

[0128] Obtaining first pixel coordinates of multiple target detection frames at the current moment;

[0129] For each target detection frame, generating target ground point coordinates of the target detection frame based on the first pixel coordinates;

[0130] Based on the calibration internal parameters and calibration external parameters of the target tracking device, the spatial coordinates of the target ground point coordinates in the world space are calculated, and the spatial coordinates are the first world coordinates of the target detection box at the current moment.

[0131] In some embodiments, the prediction module 302 is specifically configured to:

[0132] For each of the target detection frames, a Kalman filter algorithm is used to predict the second position coordinates of the candidate frame of the target detection frame at the next moment based on the first position coordinates of the target detection frame at the current moment and the fourth position coordinates at the previous moment.

[0133] In some embodiments, the output module 304 is specifically configured to:

[0134] Using the Hungarian matching algorithm, according to the multi-dimensional similarity between each target detection frame and the multiple candidate frames, each target detection frame is matched with the final target candidate frame at the next moment;

[0135] The final target candidate box is output as the target tracking result.

[0136] The above-mentioned target tracking device based on image space positioning can implement the target tracking method based on image space positioning of the above-mentioned method embodiment. The optional options in the above-mentioned method embodiment are also applicable to this embodiment and will not be described in detail here. The remaining contents of the embodiment of this application can refer to the contents of the above-mentioned method embodiment and will not be repeated in this embodiment.

[0137] Figure 4 is a schematic diagram of the structure of a computer device provided in one embodiment of the present application. As shown in Figure 4, the computer device 4 of this embodiment includes: at least one processor 40 (only one is shown in Figure 4), a memory 41, and a computer program 42 stored in the memory 41 and executable on the at least one processor 40. When the processor 40 executes the computer program 42, the steps of any of the above-described method embodiments are implemented.

[0138] The computer device 4 may be a vehicle-mounted terminal, a smartphone, a tablet computer, a desktop computer, a cloud server, or other computing device. The computer device may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will appreciate that FIG4 is merely an example of a computer device 4 and does not limit the computer device 4. The computer device 4 may include more or fewer components than shown, or a combination of certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0139] The processor 40 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0140] In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as a hard drive or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the computer device 4. Furthermore, the memory 41 may include both an internal storage unit of the computer device 4 and an external storage device. The memory 41 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory 41 may also be used to temporarily store data that has been output or is about to be output.

[0141] In addition, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0142] An embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device implements the steps in the above-mentioned various method embodiments when executing the computer program product.

[0143] In several embodiments provided in the present application, it is understood that each box in the flow chart or block diagram can represent a part of a module, program segment or code, and the part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which depends on the functions involved.

[0144] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0145] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application by those skilled in the art should be included within the scope of protection of this application.

Claims

1. A target tracking method based on image space positioning, characterized in that: include: Obtaining first position coordinates of multiple target detection frames at a current moment, where the first position coordinates include first pixel coordinates and first world coordinates; Based on the first position coordinates, predicting the second position coordinates of the candidate box of each of the target detection boxes at the next moment; Based on the second position coordinates and the third position coordinates of the target detection frame at the next moment, calculating the multi-dimensional similarity between each of the target detection frames and multiple candidate frames, the multi-dimensional similarity including the multi-dimensional similarity determined based on the dynamic threshold of the centroid distance, and the third position coordinates are the actually measured position coordinates; Utilize a preset matching algorithm and output the target tracking result at the next moment according to the multi-dimensional similarity.

2. The target tracking method based on image space positioning according to claim 1, characterized in that: The calculating the multi-dimensional similarity between each of the target detection frames and a plurality of candidate frames based on the second position coordinates and the third position coordinates of the target detection frame at the next moment includes: For each of the target detection frames, at least one target candidate frame corresponding to the target detection frame is selected from the plurality of candidate frames using a preset centroid distance dynamic threshold; Based on the second position coordinates and the third position coordinates, a multi-dimensional similarity between each of the object detection boxes and the object candidate box is calculated.

3. The target tracking method based on image space positioning according to claim 2, characterized in that: The method of selecting, for each target detection frame, at least one target candidate frame corresponding to the target detection frame from a plurality of candidate frames by using a preset centroid distance dynamic threshold comprises: Based on the first centroid world coordinates of the target detection frame and the second centroid pixel coordinates of the candidate frame, calculating a first centroid distance between each target detection frame and a target tracking device, and calculating a second centroid distance between each target detection frame and a plurality of candidate frames; Determine the dynamic threshold corresponding to the first centroid distance based on a preset centroid distance dynamic threshold table; For each of the target detection frames, target candidate frames whose second centroid distance with the target detection frame is greater than the dynamic threshold are screened out, and each of the target detection frames corresponds to at least one of the target candidate frames.

4. The target tracking method based on image space positioning according to claim 3, characterized in that: The preset centroid distance dynamic threshold table also includes: Label the historical training data and calculate the centroid distance for each pair of labeled target detection frames; Counting the distribution of the centroid distances, dividing similarity thresholds, and generating a centroid distance dynamic threshold table; The centroid distance dynamic threshold table is dynamically verified and tuned according to the target detection frame obtained each time.

5. The target tracking method based on image space positioning according to claim 2, characterized in that: The calculating, based on the second position coordinates and the third position coordinates, the multi-dimensional similarity between each of the target detection boxes and the target candidate box comprises: Based on the third centroid pixel coordinates of the target detection box and the second centroid pixel coordinates of the candidate box, the detection box centroid offset similarity between each of the target detection boxes and the target candidate box is calculated.

6. The target tracking method based on image space positioning according to claim 5, characterized in that: The calculating the multi-dimensional similarity between each of the target detection frames and the target candidate frames based on the second position coordinates and the third position coordinates also includes: Based on the third border pixel coordinates of the target detection frame and the second border pixel coordinates of the candidate frame, the detection frame shape change similarity and the detection frame area similarity between each target detection frame and the target candidate frame are calculated.

7. The target tracking method based on image space positioning according to claim 6, characterized in that: The calculating the multi-dimensional similarity between each of the target detection frames and the target candidate frames based on the second position coordinates and the third position coordinates also includes: Based on the third world coordinate of the target detection frame and the second world coordinate of the candidate frame, the world coordinate offset similarity between each target detection frame and the target candidate frame is calculated.

8. The target tracking method based on image space positioning according to claim 7, characterized in that: The calculating of the multi-dimensional similarity between each of the target detection frames and the target candidate frames also includes: A multidimensional similarity weighted model is used for calculation; wherein the multidimensional similarity weighted model at least includes a distance measurement algorithm, an edge feature matching algorithm and a structural similarity index algorithm; For each target detection frame, the detection frame centroid offset similarity, the detection frame shape change similarity, the detection frame area similarity and the world coordinate offset similarity are weighted to obtain the multi-dimensional similarity.

9. The target tracking method based on image space positioning according to claim 1, characterized in that: The obtaining of first position coordinates of multiple target detection frames at the current moment includes: Obtaining first pixel coordinates of multiple target detection frames at the current moment; For each of the target detection frames, generating target ground point coordinates of the target detection frame based on the first pixel coordinates; Based on the calibrated internal parameters and the calibrated external parameters of the target tracking device, the spatial coordinates of the target ground point coordinates in the world space are calculated, and the spatial coordinates are the first world coordinates of the target detection frame at the current moment.

10. The target tracking method based on image space positioning according to claim 9, characterized in that: The method further includes detecting the multiple target detection frames, including: Performing distortion correction on the image collected by the target tracking device, performing target detection on the corrected image, generating a target detection frame, and performing correction processing on the target detection frame; Among them, deep learning algorithm is used to perform target detection on the corrected image.

11. The target tracking method based on image space positioning according to claim 1, characterized in that: The predicting, based on the first position coordinates, the second position coordinates of the candidate frame of each target detection frame at the next moment includes: For each of the target detection frames, a Kalman filter algorithm is used to predict the second position coordinates of the candidate frame of the target detection frame at the next moment based on the first position coordinates of the target detection frame at the current moment and the fourth position coordinates at the previous moment.

12. The target tracking method based on image space positioning according to claim 11, characterized in that: The Kalman filter algorithm further includes: According to the motion data of the target detection frame, the state transfer matrix and the process noise are tuned. According to the tuned parameters, the first position coordinates and the fourth position coordinates are input into the preset motion model to predict the candidate frame at the next moment. The preset motion model adopts any one of a uniform speed model, a random acceleration model, a steering model or a nonlinear module.

13. The target tracking method based on image space positioning according to claim 1, characterized in that: The using of a preset matching algorithm and outputting the target tracking result at the next moment according to the multi-dimensional similarity includes: Using the Hungarian matching algorithm, according to the multi-dimensional similarity between each target detection frame and the multiple candidate frames, matching each target detection frame with the final target candidate frame at the next moment; The final target candidate box is output as the target tracking result.

14. A target tracking device based on image space positioning, characterized in that: include: An acquisition module, used to acquire first position coordinates of multiple target detection frames at a current moment, wherein the first position coordinates include first pixel coordinates and first world coordinates; A prediction module, configured to predict, based on the first position coordinates, a second position coordinate of a candidate frame of each of the target detection frames at a next moment; a calculation module, configured to calculate a multi-dimensional similarity between each of the target detection frames and a plurality of candidate frames based on the second position coordinates and a third position coordinate of the target detection frame at a next moment, wherein the multi-dimensional similarity includes a multi-dimensional similarity determined based on a dynamic threshold of a centroid distance, and the third position coordinate is an actually measured position coordinate; The output module is used to output the target tracking result at the next moment according to the multi-dimensional similarity by using a preset matching algorithm.

15. A computer device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the method for target tracking based on image space positioning according to any one of claims 1 to 7 is implemented.

Images