Target tracking method and device based on image space positioning, equipment and storage medium

Abstract

The application provides a target tracking method and device based on image space positioning, a storage medium, and equipment. A first position coordinate of a plurality of target detection boxes at a current time is obtained. Based on the first position coordinate, a second position coordinate of a candidate box of each target detection box at a next time is predicted. Based on the second position coordinate and a third position coordinate of the target detection box at the next time, a multi-dimensional similarity between each target detection box and the plurality of candidate boxes is calculated. The multi-dimensional similarity includes multi-dimensional similarity determined based on a centroid distance dynamic threshold. A preset matching algorithm is used to output a target tracking result at the next time according to the multi-dimensional similarity. The position of the target in the world space is considered, the problem of inaccurate positioning caused by single position information of a two-dimensional image is effectively solved, and the difference in centroid distance between a far target and a near target in continuous imaging is used to improve the positioning accuracy of multiple overlapping targets.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular to a target tracking method, apparatus, device, and storage medium. Background Technology

[0002] Target tracking algorithms help identify abnormal behaviors and potential hazards in vehicle processes, assisting autonomous vehicles in making more accurate decisions. Currently, the main steps of target tracking algorithms involve taking the target detection box position in a 2D image as input, using deep learning for target detection, Kalman prediction, and Hungary matching, and outputting the predicted detection box position of the current target in the next frame. However, 2D images provide limited information on positional changes and cannot cover both distant and near target tracking in the real world. Furthermore, in complex road test scenarios, such as parking lots or pedestrian crossings, the number of targets is very large, and there is significant overlap between targets on the 2D imaging plane, making matching errors highly likely and compromising the accuracy of target tracking results. Summary of the Invention

[0003] To address the aforementioned technical problems, this application provides a target tracking method, apparatus, device, and storage medium based on image spatial positioning.

[0004] In a first aspect, this application provides a target tracking method based on image spatial localization, including: Obtain the first position coordinates of multiple target detection boxes at the current time, where the first position coordinates include the first pixel coordinates and the first world coordinates; Based on the first position coordinates, predict the second position coordinates of the candidate box for each target detection box at the next time step; Based on the second position coordinates and the third position coordinates of the target detection box at the next time step, calculate the multi-dimensional similarity between each target detection box and multiple candidate boxes. The multi-dimensional similarity includes multi-dimensional similarity determined based on the centroid distance dynamic threshold. The third position coordinates are the actual measured position coordinates. Using a preset matching algorithm, the target tracking result for the next moment is output based on the multi-dimensional similarity.

[0005] In some implementations of the first aspect, based on the second position coordinates and the third position coordinates of the target detection box at the next time step, a multi-dimensional similarity between each target detection box and multiple candidate boxes is calculated, including: For each of the target detection boxes, at least one target candidate box corresponding to the target detection box is selected from multiple candidate boxes using a preset centroid distance dynamic threshold. Based on the second and third position coordinates, the multi-dimensional similarity between each target detection box and the target candidate box is calculated.

[0006] In some implementations of the first aspect, the step of filtering at least one target candidate box corresponding to the target detection box from multiple candidate boxes using a preset centroid distance dynamic threshold for each target detection box includes: Based on the first centroid world coordinates of the target detection box and the second centroid pixel coordinates of the candidate boxes, calculate the first centroid distance between each target detection box and the target tracking device, and calculate the second centroid distance between each target detection box and the multiple candidate boxes; Based on a preset dynamic threshold table for centroid distance, the dynamic threshold corresponding to the first centroid distance is determined. For each target detection box, target candidate boxes with a second centroid distance greater than the dynamic threshold are selected, and each target detection box corresponds to at least one target candidate box.

[0007] In some implementations of the first aspect, The step of calculating the multi-dimensional similarity between each target detection box and the target candidate box based on the second and third position coordinates includes: Based on the third centroid pixel coordinates of the target detection box and the second centroid pixel coordinates of the candidate box, calculate the detection box centroid offset similarity between each target detection box and the target candidate box; Based on the pixel coordinates of the third border of the target detection box and the pixel coordinates of the second border of the candidate box, calculate the similarity of the detection box shape change and the similarity of the detection box area between each target detection box and the target candidate box; Based on the third world coordinates of the target detection box and the second world coordinates of the candidate box, calculate the world coordinate offset similarity between each target detection box and the target candidate box; For each target detection box, the similarity of the detection box centroid offset, the similarity of the detection box shape change, the similarity of the detection box area, and the similarity of the world coordinate offset are weighted to obtain the multi-dimensional similarity.

[0008] In some implementations of the first aspect, obtaining the first position coordinates of multiple target detection boxes at the current time includes: Obtain the first pixel coordinates of the multiple target detection boxes at the current time; For each target detection box, the target ground point coordinates of the target detection box are generated based on the first pixel coordinates; Based on the calibration intrinsic and extrinsic parameters of the target tracking device, the spatial coordinates of the target ground point coordinates in world space are calculated, and the spatial coordinates are the first world coordinates of the target detection box at the current moment.

[0009] In some implementations of the first aspect, predicting the second position coordinates of the candidate boxes for each target detection box at the next time step based on the first position coordinates includes: For each target detection box, the Kalman filter algorithm is used to predict the second position coordinates of the candidate box at the next time step based on the first position coordinates of the target detection box at the current time step and the fourth position coordinates at the previous time step.

[0010] In some implementations of the first aspect, the step of using a preset matching algorithm to output the target tracking result at the next time step based on the multi-dimensional similarity includes: Using the Hungarian matching algorithm, based on the multi-dimensional similarity between each target detection box and multiple candidate boxes, the final target candidate box corresponding to each target detection box at the next time step is matched; The final target candidate box is output as the target tracking result.

[0011] Secondly, this application also provides a target tracking device based on image spatial localization, comprising: The acquisition module is used to acquire the first position coordinates of multiple target detection boxes at the current time, wherein the first position coordinates include the first pixel coordinates and the first world coordinates; The prediction module is used to predict the second position coordinates of the candidate box of each target detection box at the next time step based on the first position coordinates; The calculation module is used to calculate the multi-dimensional similarity between each target detection box and multiple candidate boxes based on the second position coordinates and the third position coordinates of the target detection box at the next time step. The multi-dimensional similarity includes multi-dimensional similarity determined based on a dynamic threshold of centroid distance, and the third position coordinates are the actual measured position coordinates. The output module is used to output the target tracking result at the next moment based on the multi-dimensional similarity using a preset matching algorithm.

[0012] Thirdly, this application also provides a computer device, including a processor and a memory, the memory being used to store a computer program, which, when executed by the processor, implements the target tracking method based on image spatial positioning as described in the first aspect.

[0013] Fourthly, this application also provides a computer-readable storage medium, characterized in that it stores a computer program, which, when executed by a processor, implements the target tracking method based on image spatial positioning as described in the first aspect.

[0014] Compared with the prior art, this application has at least the following beneficial effects: By acquiring the first position coordinates of multiple target detection boxes at the current moment, including the first pixel coordinates and the first world coordinates, the position of the target in world space is comprehensively considered, effectively solving the problem of single position change features in two-dimensional images. This allows for effective tracking of both distant and near targets in the real world, and also effectively improves target tracking accuracy by addressing the problem of multiple target overlap in two-dimensional images in complex scenes. Based on the first position coordinates, the second position coordinates of the candidate boxes for each target detection box at the next moment are predicted. This, combined with the target position prediction, improves the accuracy of target localization and solves the problem of inaccurate localization caused by single position information based on two-dimensional images. Based on the second position coordinates and the third position coordinates of the target detection boxes at the next moment, a multi-dimensional similarity between each target detection box and multiple candidate boxes is calculated. This multi-dimensional similarity includes a multi-dimensional similarity determined by a dynamic threshold based on centroid distance. Using a preset matching algorithm, the target tracking result at the next moment is output based on the multi-dimensional similarity. This allows for the utilization of the centroid distance differences between distant and near targets, as well as between small and large targets, in two-dimensional continuous imaging, thereby improving the target localization accuracy for multiple overlapping targets. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a target tracking method based on image spatial localization, as shown in an embodiment of this application. Figure 2 This is a schematic diagram illustrating the dynamic threshold relationship of centroid distance in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a target tracking device based on image spatial positioning, as shown in an embodiment of this application. Figure 4 This is a schematic diagram of the structure of a computer device shown in an embodiment of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0017] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a target tracking method based on image spatial localization provided in an embodiment of this application. The target tracking method based on image spatial localization in this embodiment can be applied to computer devices, including but not limited to in-vehicle terminals, smartphones, laptops, tablets, desktop computers, physical servers, and cloud servers, etc., and the computer device is equipped with or connected to a target tracking device (such as a camera). Figure 1 As shown, the target tracking method based on image spatial localization in this embodiment includes steps S101 to S104, which are described in detail below: Step S101: Obtain the first position coordinates of multiple target detection boxes at the current time. The first position coordinates include the first pixel coordinates and the first world coordinates.

[0018] In this step, the target detection box is the target bounding box obtained by performing bounding box detection on the 2D image acquired by the target tracking device. Optionally, the bounding box detection can be implemented based on an instance segmentation algorithm, which will not be elaborated here. Preferably, the target tracking device is a surround-view fisheye camera.

[0019] The first set of position coordinates is the set of position coordinates of the target detection box, which are the actual measured values ​​at the current moment. It includes, but is not limited to, the centroid pixel coordinates describing the center of the target detection box, the bounding box pixel coordinates describing the boundary of the target detection box, the centroid world coordinates describing the center of the target detection box in world space, and the bounding box world coordinates describing the boundary of the target detection box. Optionally, the pixel coordinates can be obtained during bounding box detection; the world coordinates can be obtained by transforming the pixel coordinates according to the coordinate relationship between the image and world space (such as camera intrinsic and extrinsic parameters).

[0020] In some embodiments, step S101 includes: Obtain the first pixel coordinates of the multiple target detection boxes at the current time; For each target detection box, the target ground point coordinates of the target detection box are generated based on the first pixel coordinates; Based on the calibration intrinsic and extrinsic parameters of the target tracking device, the spatial coordinates of the target ground point coordinates in world space are calculated, and the spatial coordinates are the first world coordinates of the target detection box at the current moment.

[0021] In this embodiment, the target grounding point is used to represent the position of the target in world space. For example, a deep learning algorithm is used to identify the name of the target detection box and its pixel coordinates (including the coordinates x and y of the top-left corner vertex and the length w and width h of the target detection box) in the current frame image; based on the pixel coordinates (x, y, w, h), an equivalent target grounding point P(Px, Py) is generated. Optionally, Px = (x + w × 1 / 2), Py = y + h, that is, the bottom midpoint of the target detection box is used as the target grounding point.

[0022] After calibrating the intrinsic and extrinsic parameters of the target tracking device, the world coordinates of point P in world space are calculated to represent the position of the target in world space. Optionally, if the target tracking device is a panoramic fisheye camera, then the target ground point P is represented by fisheye image coordinates. Therefore, the fisheye image is converted into a distortion-free image, and the corresponding pixel coordinates of the target ground point P in the distortion-free image are found. The distortion-free image is actually a pinhole camera model. Based on the pinhole imaging principle, the world coordinates of point P can be obtained. The calculation formula is as follows: ; in,( , () represents the pixel coordinates of the target grounding point. The first and second matrices on the right side of the equation can be combined to form the camera intrinsic parameter matrix, and the third matrix is ​​the camera extrinsic parameter matrix. , () represents the world coordinates of the target ground point.

[0023] Step S102: Based on the first position coordinates, predict the second position coordinates of the candidate box for each target detection box at the next time step.

[0024] In this step, the second position coordinates are the estimated values ​​for the next time step. Since there may be errors between the target detection box position obtained by bounding box detection and the actual position, if bounding box detection is performed on every frame of the image, the cumulative error will continue to increase. Therefore, this embodiment uses a motion model to predict the second position coordinates of the target detection box in the next candidate box, so as to correct the detection error and reduce the cumulative error through position prediction.

[0025] In some embodiments, step S102 includes: for each target detection box, using a Kalman filter algorithm, predicting the second position coordinates of the candidate box of the target detection box at the next time step based on the first position coordinates of the target detection box at the current time step and the fourth position coordinates at the previous time step.

[0026] In this embodiment, the Kalman filter algorithm can make predictions based on actual measured values ​​(i.e., the target detection box positions obtained through the bounding box detection algorithm) and estimated values ​​(i.e., the target detection box positions estimated through the computational model), effectively reducing the cumulative error and the estimation error of the motion model during target tracking. It should be noted that known algorithms can be used for the Kalman filter algorithm, which will not be elaborated upon here.

[0027] Step S103: Based on the second position coordinates and the third position coordinates of the target detection box at the next time step, calculate the multi-dimensional similarity between each target detection box and multiple candidate boxes. The multi-dimensional similarity includes multi-dimensional similarity determined based on the centroid distance dynamic threshold.

[0028] In this step, the third position coordinates are the actual measured values ​​at the next time step. Multi-dimensional similarity is a weighted result of similarities calculated across multiple dimensions. These multiple dimensions include, but are not limited to, detection box centroid offset similarity, detection box shape change similarity, detection box area similarity, and world coordinate offset similarity. Detection box centroid offset similarity is the centroid offset similarity between the target detection box and the candidate box; detection box shape change similarity is the shape change similarity between the target detection box and the candidate box; detection box area similarity is the area similarity between the target detection box and the candidate box; and world coordinate offset similarity is the world coordinate offset similarity between the target detection box and the candidate box. Optionally, both the second and third position coordinates are coordinate sets. The corresponding similarity can be calculated based on the coordinate values ​​representing centroid, shape, area, and world coordinates in the second and third position coordinates. The similarity calculation formula can use cosine distance similarity, Euclidean distance similarity, or Manhattan distance similarity, etc., without limitation.

[0029] The centroid distance dynamic threshold determination is a process of filtering target candidate boxes using a dynamic centroid distance threshold and then using these candidate boxes for multi-dimensional similarity calculation. It should be noted that there are significant differences in the centroid distance between distant and near targets, or between large and small targets, in two consecutive frames. For example, the centroid distance D1 between distant target a in the current frame image A and distant target a in the next frame image B, and the centroid distance D2 between near target b in the current frame image A and near target b in the next frame image B, even though distant target a and near target b are in the same image, the centroid distance D1 of distant target a is significantly smaller than the centroid distance D2 of near target b. Therefore, this application sets dynamic thresholds for different target types to filter target candidate boxes, thereby reducing the amount of matching computation and improving matching accuracy.

[0030] In some embodiments, step S103 includes: For each of the target detection boxes, at least one target candidate box corresponding to the target detection box is selected from multiple candidate boxes using a preset centroid distance dynamic threshold. Based on the second and third position coordinates, the multi-dimensional similarity between each target detection box and the target candidate box is calculated.

[0031] In this embodiment, optionally, the filtering step of the target candidate box includes: Based on the first centroid world coordinates of the target detection box and the second centroid pixel coordinates of the candidate boxes, calculate the first centroid distance between each target detection box and the target tracking device, and calculate the second centroid distance between each target detection box and the multiple candidate boxes; Based on a preset dynamic threshold table for centroid distance, the dynamic threshold corresponding to the first centroid distance is determined. For each target detection box, target candidate boxes with a second centroid distance greater than the dynamic threshold are selected, and each target detection box corresponds to at least one target candidate box.

[0032] In this optional embodiment, a first centroid distance between the target detection box and the target tracking device is calculated using the world coordinates of the first centroid and the world coordinates of the target tracking device. This determines whether the target detection box belongs to a distant or near target. A corresponding dynamic threshold is then determined using this first centroid distance. This dynamic threshold is used to filter candidate boxes for the target detection box, thereby filtering at least one candidate box corresponding to the target detection box. This effectively eliminates other candidate boxes that clearly do not meet the candidate box conditions of the target detection box, thus reducing the computational load of subsequent similarity calculation and matching processes, reducing interference from invalid candidate boxes in the matching process, and improving target tracking accuracy. For example, the centroid distance dynamic threshold table is as follows: Figure 2 The diagram shown illustrates the dynamic threshold relationship of centroid distance. This diagram represents the dynamic threshold table for centroid distance. When the first centroid distance (target world distance) is 3 meters, the dynamic threshold is 3; when the first centroid distance (target world distance) is 15 meters, the dynamic threshold is 3.

[0033] Optionally, the steps for calculating multi-dimensional similarity include: Based on the third centroid pixel coordinates of the target detection box and the second centroid pixel coordinates of the candidate box, calculate the detection box centroid offset (box_IOU) similarity between each target detection box and the target candidate box; Based on the pixel coordinates of the third border of the target detection box and the pixel coordinates of the second border of the candidate box, calculate the similarity of the detection box shape change (box_SHAPE) and the detection box area (box_AREA) between each target detection box and the target candidate box; Based on the third world coordinates of the target detection box and the second world coordinates of the candidate box, calculate the world coordinate offset similarity between each target detection box and the target candidate box; For each target detection box, the similarity of the detection box centroid offset, the similarity of the detection box shape change, the similarity of the detection box area, and the similarity of the world coordinate offset are weighted to obtain the multi-dimensional similarity.

[0034] In this optional embodiment, the coordinate parameters of the candidate box are divided by the coordinate parameters of the target detection box to obtain the similarity parameter.

[0035] Step S104: Using a preset matching algorithm, output the target tracking result for the next moment based on the multi-dimensional similarity.

[0036] In this step, since multiple target detection boxes may exist in two consecutive frames, it is difficult to determine which target detection box in the next frame corresponds to the current target detection box. For example, there are 20 target detection boxes in consecutive images A and B. Based on a target detection box a in image A, the second position coordinate is predicted, but it is difficult to directly determine which target detection box in image B at the next moment corresponds to the second position coordinate. Therefore, this application uses a preset matching algorithm to match the correspondence between multiple target detection boxes in the current frame and multiple candidate boxes in the next frame.

[0037] In some embodiments, step S104 includes: using the Hungarian matching algorithm to match the final target candidate box corresponding to each target detection box at the next time step based on the multi-dimensional similarity between each target detection box and the plurality of candidate boxes; and outputting the final target candidate box as the target tracking result.

[0038] In this embodiment, all the multi-dimensional similarities obtained in step S103 are input into the Hungarian matching algorithm, and each target detection box and each candidate box are traversed and looped to obtain the candidate box corresponding to each target detection box.

[0039] To implement the image spatial localization-based target tracking method corresponding to the above method embodiments, and to achieve the corresponding functions and technical effects. See also Figure 3 , Figure 3This diagram illustrates a structural block diagram of a target tracking device based on image spatial localization according to an embodiment of this application. For ease of explanation, only the parts relevant to this embodiment are shown. The target tracking device based on image spatial localization provided in this embodiment includes: The acquisition module 301 is used to acquire the first position coordinates of multiple target detection boxes at the current time, wherein the first position coordinates include the first pixel coordinates and the first world coordinates; Prediction module 302 is used to predict the second position coordinates of the candidate box of each target detection box at the next time step based on the first position coordinates; The calculation module 303 is used to calculate the multi-dimensional similarity between each target detection box and multiple candidate boxes based on the second position coordinates and the third position coordinates of the target detection box at the next time step. The multi-dimensional similarity includes multi-dimensional similarity determined based on a dynamic threshold of centroid distance. The third position coordinates are the actual measured position coordinates. The output module 304 is used to output the target tracking result at the next moment based on the multi-dimensional similarity using a preset matching algorithm.

[0040] In some embodiments, the computing module 303 includes: The filtering module is used to filter at least one target candidate box corresponding to the target detection box from multiple candidate boxes for each target detection box using a preset centroid distance dynamic threshold; The calculation module is used to calculate the multi-dimensional similarity between each target detection box and the target candidate box based on the second position coordinates and the third position coordinates.

[0041] In some embodiments, the filtering module is specifically used for: Based on the first centroid world coordinates of the target detection box and the second centroid pixel coordinates of the candidate boxes, calculate the first centroid distance between each target detection box and the target tracking device, and calculate the second centroid distance between each target detection box and the multiple candidate boxes; Based on a preset dynamic threshold table for centroid distance, the dynamic threshold corresponding to the first centroid distance is determined. For each target detection box, target candidate boxes with a second centroid distance greater than the dynamic threshold are selected, and each target detection box corresponds to at least one target candidate box.

[0042] In some embodiments, the computing module is specifically used for: Based on the third centroid pixel coordinates of the target detection box and the second centroid pixel coordinates of the candidate box, calculate the detection box centroid offset similarity between each target detection box and the target candidate box; Based on the pixel coordinates of the third border of the target detection box and the pixel coordinates of the second border of the candidate box, calculate the similarity of the detection box shape change and the similarity of the detection box area between each target detection box and the target candidate box; Based on the third world coordinates of the target detection box and the second world coordinates of the candidate box, calculate the world coordinate offset similarity between each target detection box and the target candidate box; For each target detection box, the similarity of the detection box centroid offset, the similarity of the detection box shape change, the similarity of the detection box area, and the similarity of the world coordinate offset are weighted to obtain the multi-dimensional similarity.

[0043] In some embodiments, the acquisition module 301 is specifically used for: Obtain the first pixel coordinates of the multiple target detection boxes at the current time; For each target detection box, the target ground point coordinates of the target detection box are generated based on the first pixel coordinates; Based on the calibration intrinsic and extrinsic parameters of the target tracking device, the spatial coordinates of the target ground point coordinates in world space are calculated, and the spatial coordinates are the first world coordinates of the target detection box at the current moment.

[0044] In some embodiments, the prediction module 302 is specifically used for: For each target detection box, the Kalman filter algorithm is used to predict the second position coordinates of the candidate box at the next time step based on the first position coordinates of the target detection box at the current time step and the fourth position coordinates at the previous time step.

[0045] In some embodiments, the output module 304 is specifically used for: Using the Hungarian matching algorithm, based on the multi-dimensional similarity between each target detection box and multiple candidate boxes, the final target candidate box corresponding to each target detection box at the next time step is matched; The final target candidate box is output as the target tracking result.

[0046] The aforementioned image spatial positioning-based target tracking device can implement the image spatial positioning-based target tracking method of the above method embodiments. The options in the above method embodiments are also applicable to this embodiment, and will not be detailed here. The remaining content of this application's embodiments can be referred to the content of the above method embodiments, and will not be repeated in this embodiment.

[0047] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 4 As shown, the computer device 4 in this embodiment includes: at least one processor 40 ( Figure 4 (Only one is shown in the diagram), memory 41, and computer program 42 stored in the memory 41 and executable on the at least one processor 40, wherein the processor 40 executes the computer program 42 to implement the steps in any of the above method embodiments.

[0048] The computer device 4 can be a vehicle-mounted terminal, smartphone, tablet computer, desktop computer, cloud server, or other computing device. This computer device may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will understand that... Figure 4 The computer device 4 is merely an example and does not constitute a limitation on the computer device 4. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0049] The processor 40 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0050] In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 may be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Furthermore, the memory 41 may include both internal and external storage units of the computer device 4. The memory 41 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 41 can also be used to temporarily store data that has been output or will be output.

[0051] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0052] This application provides a computer program product that, when run on a computer device, enables the computer device to execute the steps described in the various method embodiments above.

[0053] In the several embodiments provided in this application, it will be understood that each block in the flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0054] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0055] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application for those skilled in the art.

Claims

1. A target tracking method based on image spatial localization, characterized in that, include: Obtain the first position coordinates of multiple target detection boxes at the current time, where the first position coordinates include the first pixel coordinates and the first world coordinates; Based on the first position coordinates, predict the second position coordinates of the candidate box for each target detection box at the next time step; Based on the second position coordinates and the third position coordinates of the target detection box at the next time step, calculate the multi-dimensional similarity between each target detection box and multiple candidate boxes. The multi-dimensional similarity includes multi-dimensional similarity determined based on the centroid distance dynamic threshold. The third position coordinates are the actual measured position coordinates. Using a preset matching algorithm, the target tracking result at the next moment is output based on the multi-dimensional similarity. The step of calculating the multi-dimensional similarity between each target detection box and multiple candidate boxes based on the second position coordinates and the third position coordinates of the target detection box at the next time step includes: For each of the target detection boxes, at least one target candidate box corresponding to the target detection box is selected from multiple candidate boxes using a preset centroid distance dynamic threshold. Based on the second and third position coordinates, calculate the multi-dimensional similarity between each target detection box and the target candidate box; For each of the target detection boxes, at least one target candidate box corresponding to the target detection box is selected from multiple candidate boxes using a preset dynamic threshold for centroid distance, including: Based on the first centroid world coordinates of the target detection box and the second centroid pixel coordinates of the candidate boxes, calculate the first centroid distance between each target detection box and the target tracking device, and calculate the second centroid distance between each target detection box and the multiple candidate boxes; Based on a preset dynamic threshold table for centroid distance, the dynamic threshold corresponding to the first centroid distance is determined. For each target detection box, target candidate boxes with a second centroid distance less than the dynamic threshold are selected, and each target detection box corresponds to at least one target candidate box; The step of calculating the multi-dimensional similarity between each target detection box and the target candidate box based on the second and third position coordinates includes: Based on the third centroid pixel coordinates of the target detection box and the second centroid pixel coordinates of the candidate box, calculate the detection box centroid offset similarity between each target detection box and the target candidate box; Based on the third border pixel coordinates of the target detection box and the second border pixel coordinates of the candidate box, calculate the detection box shape change similarity and detection box area similarity between each target detection box and the target candidate box; based on the third world coordinates of the target detection box and the second world coordinates of the candidate box, calculate the world coordinate offset similarity between each target detection box and the target candidate box. For each target detection box, the similarity of the detection box centroid offset, the similarity of the detection box shape change, the similarity of the detection box area, and the similarity of the world coordinate offset are weighted to obtain the multi-dimensional similarity.

2. The target tracking method based on image spatial localization according to claim 1, characterized in that, The step of obtaining the first position coordinates of multiple target detection boxes at the current time includes: Obtain the first pixel coordinates of the multiple target detection boxes at the current time; For each target detection box, the target grounding point coordinates of the target detection box are generated based on the first pixel coordinates; the spatial coordinates of the target grounding point coordinates in world space are calculated based on the calibration intrinsic and calibration extrinsic parameters of the target tracking device, and the spatial coordinates are the first world coordinates of the target detection box at the current moment.

3. The target tracking method based on image spatial localization according to claim 1, characterized in that, The step of predicting the second position coordinates of the candidate box for each target detection box at the next time step based on the first position coordinates includes: For each target detection box, the Kalman filter algorithm is used to predict the second position coordinates of the candidate box at the next time step based on the first position coordinates of the target detection box at the current time step and the fourth position coordinates at the previous time step.

4. The target tracking method based on image spatial localization according to claim 1, characterized in that, The step of using a preset matching algorithm to output the target tracking result at the next time step based on the multi-dimensional similarity includes: Using the Hungarian matching algorithm, based on the multi-dimensional similarity between each target detection box and multiple candidate boxes, the final target candidate box corresponding to each target detection box at the next time step is matched; The final target candidate box is output as the target tracking result.

5. A target tracking device based on image spatial positioning according to any one of claims 1 to 4, characterized in that, include: The acquisition module is used to acquire the first position coordinates of multiple target detection boxes at the current time, wherein the first position coordinates include the first pixel coordinates and the first world coordinates; The prediction module is used to predict the second position coordinates of the candidate box of each target detection box at the next time step based on the first position coordinates; The calculation module is used to calculate the multi-dimensional similarity between each target detection box and multiple candidate boxes based on the second position coordinates and the third position coordinates of the target detection box at the next time step. The multi-dimensional similarity includes multi-dimensional similarity determined based on a dynamic threshold of centroid distance, and the third position coordinates are the actual measured position coordinates. The output module is used to output the target tracking result at the next moment based on the multi-dimensional similarity using a preset matching algorithm.

6. A computer device, characterized in that, It includes a processor and a memory, the memory being used to store a computer program, which, when executed by the processor, implements the image spatial localization-based target tracking method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the target tracking method based on image spatial positioning as described in any one of claims 1 to 4.

Images