A target tracking method and apparatus

Abstract

Embodiments of the present application provide a target tracking method and device, comprising: obtaining image position information of a target to be tracked in an image coordinate system of a first camera; converting the image position information from the image coordinate system to a world coordinate system to obtain world position information; determining a driving-out direction of the target to be tracked when the target drives out of a collection range of the first camera according to the world position information; obtaining a plurality of to-be-matched cameras adjacent to the first camera, and determining a relative direction of each to-be-matched camera relative to the first camera; selecting at least one second camera from each to-be-matched camera whose relative direction matches the driving-out direction; and performing cross-camera tracking on the target to be tracked according to image data of the first camera and image data of the second camera. Through the above method, the robustness of cross-camera tracking is improved.

Description

Technical Field

[0001] This application relates to the field of smart city technology, and in particular to a target tracking method and apparatus. Background Technology

[0002] In existing technologies, when matching targets to be tracked across multiple cameras, it is necessary to rely on manually drawn region information to identify the direction of the target and the correspondence between upstream and downstream cameras. This method relies too heavily on human experience, which greatly reduces the robustness of cross-camera tracking. Summary of the Invention

[0003] The purpose of this application is to provide a target tracking method and apparatus to improve the robustness of cross-camera tracking. The specific technical solution is as follows:

[0004] In a first aspect, embodiments of this application provide a target tracking method, the method comprising:

[0005] Obtain the image position information of the target to be tracked in the image coordinate system of the first camera;

[0006] The image position information is transformed from the image coordinate system to the world coordinate system to obtain the world position information;

[0007] Based on the world location information, determine the direction in which the target to be tracked leaves the range of the first camera;

[0008] Obtain multiple cameras adjacent to the first camera to be matched, and determine the relative direction of each camera to be matched relative to the first camera;

[0009] Select at least one second camera from among the cameras to be matched whose relative direction matches the direction of departure;

[0010] Based on the image data from the first camera and the image data from the second camera, the target to be tracked is tracked across cameras.

[0011] In one possible implementation, the image location information includes the image coordinates of at least two target trajectory points of the target to be tracked in the image coordinate system;

[0012] The step of transforming the image position information from the image coordinate system to the world coordinate system to obtain world position information includes:

[0013] By using a pre-determined homography matrix between the image coordinate system and the world coordinate system, the image coordinates of each of the at least two target trajectory points are mapped and calculated to obtain the world coordinates of each target trajectory point, wherein the world position information includes the world coordinates of each target trajectory point.

[0014] In one possible implementation, determining the direction of departure of the target being tracked when it leaves the range of the first camera, based on the world location information, includes:

[0015] Based on the world coordinates of at least two target trajectory points in the world location information, the departure direction vector of the target to be tracked when it leaves the acquisition range of the first camera is calculated.

[0016] Based on the departure direction vector, the departure direction of the target to be tracked when it leaves the acquisition range of the first camera is determined.

[0017] In one possible implementation, calculating the exit direction vector of the target when it leaves the acquisition range of the first camera based on the world coordinates of at least two target trajectory points in the world location information includes:

[0018] Obtain the first world coordinates and the Nth world coordinates from the world location information; wherein, the first world coordinates are the world coordinates of the last target trajectory point collected among all target trajectory points, or the first world coordinates are the average of the world coordinates of the last i+1 target trajectory points collected among all target trajectory points; the Nth world coordinates are the world coordinates of the Nth target trajectory point collected from the end among all target trajectory points, or the Nth world coordinates are the average of the world coordinates of the Nth to N+ith target trajectory points collected from the end among all target trajectory points; N is an integer greater than 1, and i is a positive integer;

[0019] Calculate the vector pointing from the first world coordinate to the Nth world coordinate to obtain the departure direction vector when the target to be tracked leaves the acquisition range of the first camera.

[0020] In one possible implementation, determining the exit direction of the target when it leaves the acquisition range of the first camera based on the exit direction vector includes:

[0021] The departure direction vector is converted into a unit direction vector; wherein the horizontal coordinate of the unit direction vector is x and the vertical coordinate is y.

[0022] When x is greater than 0, y is greater than 0, and x is greater than y, the direction in which the target to be tracked leaves the collection range of the first camera is determined to be east.

[0023] When x is greater than 0, y is greater than 0, and x is less than y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be north;

[0024] When x is less than 0, y is greater than 0, and -x is greater than y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be west.

[0025] When x is less than 0, y is greater than 0, and -x is less than y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be north;

[0026] When x is less than 0, y is less than 0, and -x is greater than -y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be west.

[0027] When x is less than 0, y is less than 0, and -x is less than -y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be south.

[0028] When x is greater than 0, y is less than 0, and x is greater than -y, the direction in which the target to be tracked leaves the collection range of the first camera is determined to be east.

[0029] When x is greater than 0, y is less than 0, and x is less than -y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be south.

[0030] In one possible implementation, acquiring a plurality of cameras to be matched adjacent to the first camera includes:

[0031] Obtain the world coordinates of each camera in the world coordinate system;

[0032] For each camera other than the first camera, calculate the distance between the camera and the first camera based on the world coordinates of the camera and the world coordinates of the first camera;

[0033] Select the M cameras closest to the first camera to obtain multiple cameras to be matched that are adjacent to the first camera.

[0034] In one possible implementation, determining the relative orientation of each camera to be matched relative to the first camera includes:

[0035] For each camera to be matched, the relative direction vector of the camera to be matched relative to the first camera is calculated;

[0036] Based on the relative direction vectors, the relative direction of each of the cameras to be matched relative to the first camera is determined.

[0037] In one possible implementation, selecting at least one second camera from among the cameras to be matched whose relative direction matches the departure direction includes:

[0038] Among the cameras to be matched whose relative direction matches the departure direction, select the K cameras closest to the first camera to obtain K second cameras, where K is a preset positive integer.

[0039] In one possible implementation, the method further includes:

[0040] For each second camera, determine the direction of entry of the target to be tracked when it enters the acquisition range of that second camera;

[0041] The driving direction is matched with the driving direction of the target to be tracked when it enters the collection range of the second camera to obtain the direction matching result of the second camera.

[0042] Based on the orientation matching result of the second camera, determine whether the cross-camera tracking result between the first camera and the second camera is reliable.

[0043] Secondly, embodiments of this application provide a target tracking device, the device comprising:

[0044] The first acquisition module is used to acquire the image position information of the target to be tracked in the image coordinate system of the first camera;

[0045] The second acquisition module is used to transform the image position information from the image coordinate system to the world coordinate system to obtain world position information;

[0046] The first determining module is used to determine the direction of departure of the target to be tracked when it leaves the acquisition range of the first camera, based on the world location information.

[0047] The third acquisition module is used to acquire multiple cameras to be matched that are adjacent to the first camera;

[0048] The second determining module is used to determine the relative direction of each camera to be matched relative to the first camera;

[0049] The selection module is used to select at least one second camera from among the cameras to be matched whose relative direction matches the driving direction;

[0050] The tracking module is used to perform cross-camera tracking of the target to be tracked based on image data from the first camera and image data from the second camera.

[0051] In one possible implementation, the image location information includes the image coordinates of at least two target trajectory points of the target to be tracked in the image coordinate system;

[0052] The second acquisition module includes:

[0053] The first acquisition submodule is used to perform mapping calculations on the image coordinates of each of the at least two target trajectory points by using a pre-determined homography matrix between the image coordinate system and the world coordinate system, so as to obtain the world coordinates of each of the target trajectory points, wherein the world position information includes the world coordinates of each of the target trajectory points.

[0054] In one possible implementation, the first determining module includes:

[0055] The second acquisition submodule is used to calculate the departure direction vector of the target to be tracked when it leaves the acquisition range of the first camera, based on the world coordinates of at least two target trajectory points in the world location information.

[0056] The first determining submodule is used to determine the direction of departure of the target to be tracked when it leaves the acquisition range of the first camera, based on the departure direction vector.

[0057] In one possible implementation, the second acquisition submodule is specifically used for:

[0058] Obtain the first world coordinates and the Nth world coordinates from the world location information; wherein, the first world coordinates are the world coordinates of the last target trajectory point collected among all target trajectory points, or the first world coordinates are the average of the world coordinates of the last i+1 target trajectory points collected among all target trajectory points; the Nth world coordinates are the world coordinates of the Nth target trajectory point collected from the end among all target trajectory points, or the Nth world coordinates are the average of the world coordinates of the Nth to N+ith target trajectory points collected from the end among all target trajectory points; N is an integer greater than 1, and i is a positive integer;

[0059] Calculate the vector pointing from the first world coordinate to the Nth world coordinate to obtain the departure direction vector when the target to be tracked leaves the acquisition range of the first camera.

[0060] In one possible implementation, the first determining submodule is specifically used for:

[0061] The departure direction vector is converted into a unit direction vector; wherein the horizontal coordinate of the unit direction vector is x and the vertical coordinate is y.

[0062] When x is greater than 0, y is greater than 0, and x is greater than y, the direction in which the target to be tracked leaves the collection range of the first camera is determined to be east.

[0063] When x is greater than 0, y is greater than 0, and x is less than y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be north;

[0064] When x is less than 0, y is greater than 0, and -x is greater than y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be west.

[0065] When x is less than 0, y is greater than 0, and -x is less than y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be north;

[0066] When x is less than 0, y is less than 0, and -x is greater than -y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be west.

[0067] When x is less than 0, y is less than 0, and -x is less than -y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be south.

[0068] When x is greater than 0, y is less than 0, and x is greater than -y, the direction in which the target to be tracked leaves the collection range of the first camera is determined to be east.

[0069] When x is greater than 0, y is less than 0, and x is less than -y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be south.

[0070] In one possible implementation, the third acquisition module includes:

[0071] The third acquisition submodule is used to acquire the world coordinates of each camera in the world coordinate system.

[0072] The calculation submodule is used to calculate the distance between each camera other than the first camera, based on the world coordinates of the camera and the world coordinates of the first camera.

[0073] The fourth acquisition submodule is used to select the M cameras closest to the first camera to obtain multiple cameras to be matched that are adjacent to the first camera.

[0074] In one possible implementation, the second determining module includes:

[0075] The fifth acquisition submodule is used to calculate the relative direction vector of each camera to be matched relative to the first camera for each camera to be matched.

[0076] The second determining submodule is used to determine the relative direction of each of the cameras to be matched relative to the first camera based on the relative direction vectors.

[0077] In one possible implementation, the selection module includes:

[0078] The sixth acquisition submodule is used to select the K cameras closest to the first camera from among the cameras to be matched whose relative direction matches the driving direction, to obtain K second cameras, where K is a preset positive integer.

[0079] In one possible implementation, the device further includes:

[0080] The third determining module is used to determine the direction of entry of the target to be tracked when it enters the acquisition range of each second camera.

[0081] The fourth acquisition module is used to match the driving direction with the driving direction when the target to be tracked enters the acquisition range of the second camera, and obtain the direction matching result of the second camera;

[0082] The fourth determining module is used to determine whether the cross-camera tracking result between the first camera and the second camera is reliable based on the orientation matching result of the second camera.

[0083] Thirdly, embodiments of this application provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0084] Memory, used to store computer programs;

[0085] When a processor executes a program stored in memory, it implements any of the steps described in the first aspect above.

[0086] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the steps described in the first aspect above.

[0087] Beneficial effects of the embodiments in this application:

[0088] This application provides a target tracking method and apparatus, comprising: acquiring image position information of a target to be tracked in the image coordinate system of a first camera; transforming the image position information from the image coordinate system to the world coordinate system to obtain world position information; determining the direction of departure of the target when it leaves the acquisition range of the first camera based on the world position information; acquiring a plurality of cameras adjacent to the first camera to be matched, and determining the relative direction of each camera to be matched relative to the first camera; selecting at least one second camera among the cameras to be matched whose relative direction matches the departure direction; and performing cross-camera tracking of the target based on the image data of the first camera and the image data of the second camera. By automatically determining the second camera to which the target is heading when it leaves the acquisition range of the first camera through the departure direction of the target, cross-camera tracking of the target is achieved, realizing automatic cross-camera target tracking. Compared with the prior art, which relies too much on manually drawn area information, this improves the robustness of cross-camera tracking.

[0089] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0090] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0091] Figure 1 This is a specific scenario diagram of cross-camera vehicle tracking in existing technologies;

[0092] Figure 2 This is a schematic diagram illustrating the movement of the same vehicle in different camera images in the existing technology;

[0093] Figure 3 This is a schematic diagram of the basic process of multi-vehicle tracking across cameras in existing technologies;

[0094] Figure 4a This is a schematic diagram of a region division method in existing cross-camera vehicle tracking techniques.

[0095] Figure 4b This is a schematic diagram of another region division method in existing cross-camera vehicle tracking techniques.

[0096] Figure 5 This is a schematic diagram of a first flowchart of the target tracking method provided in the embodiments of this application;

[0097] Figure 6a This is a second flowchart illustrating the target tracking method provided in the embodiments of this application;

[0098] Figure 6b A schematic diagram of image coordinates for the target tracking method provided in the embodiments of this application;

[0099] Figure 6c A world coordinate diagram illustrating the target tracking method provided in this application embodiment;

[0100] Figure 6d This is a schematic diagram of the first type of projective transformation;

[0101] Figure 6e This is a second schematic diagram of projective transformation;

[0102] Figure 7 This is a third flowchart illustrating the target tracking method provided in the embodiments of this application;

[0103] Figure 8a This is a fourth flowchart illustrating the target tracking method provided in the embodiments of this application;

[0104] Figure 8b A schematic diagram of the direction vector of the target moving out of the range of the first camera;

[0105] Figure 9a A fifth flowchart illustrating the target tracking method provided in this application embodiment;

[0106] Figure 9b A schematic diagram of a unit direction vector;

[0107] Figure 10 A sixth flowchart illustrating the target tracking method provided in this application embodiment;

[0108] Figure 11a A seventh flowchart illustrating the target tracking method provided in this application embodiment;

[0109] Figure 11b This is a schematic diagram showing the relative orientation of each camera to be matched relative to the first camera;

[0110] Figure 12 An eighth flowchart illustrating the target tracking method provided in this application embodiment;

[0111] Figure 13 A ninth flowchart illustrating the target tracking method provided in this application embodiment;

[0112] Figure 14 A schematic diagram of the target tracking device provided in the embodiments of this application;

[0113] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0114] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0115] First, let's briefly explain cross-camera target tracking in existing technologies, taking cross-camera vehicle tracking as an example.

[0116] The purpose of cross-camera vehicle tracking is to identify the same vehicle passing through multiple intersections, in specific scenarios such as... Figure 1 As shown in the figure, C041-C045 are the locations of 5 cameras at different intersections. If a car passes through C045-C041 from west to east, it is necessary to identify that the car is the same car in the images of the 5 cameras.

[0117] In situations with heavy traffic, vehicles congested together, multiple cameras at different angles, and varying lighting conditions, identifying a vehicle as the same car in images from five cameras is very difficult. Figure 2 As shown in the image, the two cars shown in the box are the same car from different cameras, but due to lighting and other reasons, it is not easy to identify them as the same car.

[0118] The basic process of multi-vehicle tracking across cameras is as follows: Figure 3 As shown, it includes vehicle detection, vehicle feature extraction, single-camera vehicle tracking, and cross-camera vehicle matching.

[0119] The purpose of object detection is to identify vehicles in an image, detecting as many targets as possible. Vehicle feature extraction uses a Re-identification (REI) model to extract features from detected vehicles, which are then used for subsequent single-camera tracking and multi-camera matching to compare and match vehicle features. Single-camera vehicle tracking tracks the same vehicle in an image by using its features and trajectory. Cross-camera vehicle matching, after obtaining the vehicle trajectory from each single camera, matches the trajectories of the same vehicle from multiple cameras based on vehicle features, achieving cross-camera tracking. Cross-camera vehicle matching includes the following steps: 1. Converting vehicle image coordinates to world coordinates (GPS coordinates); 2. Calculating the vehicle's entry and exit directions based on the vehicle trajectory; 3. Searching for the vehicle's entry and exit cameras based on its direction; 4. Matching vehicles from adjacent cameras.

[0120] Cross-camera vehicle matching is a crucial step in cross-camera vehicle tracking. After obtaining the single-camera vehicle tracking trajectories of all vehicles from each camera, the key to cross-camera vehicle matching is matching the trajectories of the same vehicle across multiple cameras. The main steps and challenges include:

[0121] 1. First, determine the adjacent cameras. The difficulty lies in how to determine the previous camera and the next camera in the current camera range when a vehicle enters or exits, given N cameras.

[0122] 2. Matching vehicle trajectories between adjacent cameras is challenging due to the large number of matching samples and the poor matching results.

[0123] Regarding step 1, traditional cross-camera vehicle tracking methods rely on prior knowledge. Each camera image is manually divided into multiple regions, and the vehicle's entry and exit directions are determined by identifying the starting and ending points of its trajectory within these regions. Then, prior knowledge is used to determine adjacent cameras; for example, if a vehicle exits region 1 of camera A, it will enter region 3 of camera C. The traditional method of region division based on prior knowledge is as follows: Figure 4a and Figure 4b As shown, Figure 4a The corresponding division method is to divide the image into 5 regions, where "Zone" represents a region, "Leave Main Road" means leaving the main street, "Enter Main Road" means entering the main street, "Go to Next Camera" means going to the next camera, "From Next Camera" means coming from the next camera, "Go to Previous Camera" means going to the previous camera, and "From Previous Camera" means coming from the previous camera. Figure 4b The corresponding division method is to divide the image into 9 regions. Figure 4a and Figure 4b The corresponding division methods are all designed to easily determine the entry and exit directions of a vehicle's trajectory, thereby matching the vehicle in the next or previous camera. However, this manual division method relies too heavily on human experience. The regions drawn by each person for the same scene are not exactly the same, and even small differences between regions can have a significant impact on the final matching result, severely reducing the robustness of cross-camera tracking.

[0124] To improve the robustness of cross-camera tracking, embodiments of this application provide a target tracking method and apparatus.

[0125] Next, a target tracking method provided in the embodiments of this application will be described in detail. See [link to relevant documentation]. Figure 5 This includes the following steps:

[0126] Step S501: Obtain the image position information of the target to be tracked in the image coordinate system of the first camera.

[0127] The target to be tracked can be customized according to actual tracking needs. In one example, the target to be tracked can be a motor vehicle or a non-motor vehicle. This application does not make specific limitations on this.

[0128] The image position information can be the trajectory position coordinates of the target to be tracked in the image coordinate system of the first camera. Target tracking methods in a single camera can refer to target tracking methods in related technologies, and are not specifically limited in this application.

[0129] Step S502: Transform the image position information from the image coordinate system to the world coordinate system to obtain the world position information.

[0130] In this application's embodiments, the world coordinate system refers to the coordinate system of the actual monitoring scene. For example, the world coordinate system can be a latitude and longitude coordinate system, such as the BeiDou coordinate system or the GPS (Global Positioning System) coordinate system; another example is a coordinate system custom-established for the actual monitoring scene. Transforming the trajectory coordinate information of all targets to be tracked to the world coordinate system has the advantage of ensuring that the trajectory coordinates of the targets and the camera's world coordinates (GPS coordinate information) are in the same coordinate system, allowing the direction of the targets to be tracked to be determined using latitude and longitude directions. World position information refers to the trajectory coordinate information of the targets to be tracked in the world coordinate system, corresponding to the image position information.

[0131] Step S503: Based on the world location information, determine the direction in which the target to be tracked leaves the acquisition range of the first camera.

[0132] In one example, after obtaining the world location information of the target to be tracked, the exit direction of the target to be tracked is determined based on the world location information of the target to be tracked, such as the world coordinates at the end of the target's trajectory, so as to subsequently determine which camera the target to be tracked might go to after leaving the first camera.

[0133] Step S504: Obtain multiple cameras to be matched that are adjacent to the first camera, and determine the relative direction of each camera to be matched relative to the first camera.

[0134] The multiple cameras adjacent to the first camera that need to be matched can be pre-defined manually, or they can be calculated based on the distance between each camera and the first camera. After obtaining the multiple cameras adjacent to the first camera that need to be matched, the relative orientation of the cameras to be matched with respect to the first camera can be determined based on the world coordinates of the cameras to be matched and the world coordinates of the first camera.

[0135] Step S505: Select at least one second camera from the cameras to be matched whose relative direction matches the driving direction;

[0136] In one example, all cameras to be matched that match the relative direction and the direction of departure can be selected as each second camera; in another example, among the cameras to be matched that match the relative direction and the direction of departure, cameras whose distance from the first camera is less than a preset distance threshold can be selected to obtain each second camera; in yet another example, among the cameras to be matched that match the relative direction and the direction of departure, a preset number of cameras that are closest to the first camera can be selected to obtain each second camera.

[0137] Step S506: Based on the image data from the first camera and the image data from the second camera, perform cross-camera tracking on the target to be tracked.

[0138] The image data from the first camera and the second camera include information such as the features and trajectory of the target to be tracked. After obtaining the target trajectory of each single camera of the first and second cameras, the trajectories of the target to be tracked by the first and second cameras can be matched according to the features of the target to be tracked, with a high degree of matching.

[0139] In the above embodiments, the direction of the target to be tracked when it leaves the range of the first camera is automatically determined to be the direction of the target to be tracked to the second camera, and the target to be tracked is tracked across cameras. This realizes automatic cross-camera target tracking, which improves the robustness of cross-camera tracking compared with the prior art which relies too much on manually drawn area information.

[0140] In one possible implementation, the image location information includes the image coordinates of at least two target trajectory points of the target to be tracked in the image coordinate system;

[0141] See Figure 6a This is a schematic diagram of a second flowchart of the target tracking method provided in the embodiments of this application, based on Figure 5 Step S502 in the document has been refined to include the following steps:

[0142] Step S601: Using a pre-determined homography matrix between the image coordinate system and the world coordinate system, the image coordinates of each of the at least two target trajectory points are mapped and calculated to obtain the world coordinates of each target trajectory point, wherein the world position information includes the world coordinates of each target trajectory point.

[0143] In one example, the specific conversion process of the image location information of the target to be tracked into world location information is as follows:

[0144] 1. See Figure 6b Mark the coordinates of more than 4 points in the image coordinate system of the first camera to obtain the image coordinates corresponding to the 4 points;

[0145] 2. See Figure 6c Find the points on the image of the first camera in the world coordinate system to obtain the world coordinates of the four points;

[0146] 3. Robust methods based on RANSAC (Random Sample Consensus, an algorithm that calculates mathematical model parameters of data from a set of sample datasets containing outliers to obtain valid sample data) can be used, or the minimum median robust method can be used to calculate the transformation matrix between the image coordinate system and the world coordinate system. After obtaining the transformation matrix, the trajectory coordinates of all targets to be tracked in the image of the first camera can be converted into world coordinates, that is, the image position information of the targets to be tracked is converted into world position information.

[0147] First, assuming any point x in the image coordinate system, there exists a transformation relationship such that x' = H*x, where x' is the coordinate of the point in the world coordinate system, and H is the homography matrix for the transformation from the image coordinate system to the world coordinate system.

[0148] In one example, the non-homogeneous coordinates on a 2D (two-dimensional plane) image are (x, y), while the homogeneous coordinates are (x, y, 1), which can also be written as (x / z, y / z, 1) or (x, y, z). Homogeneous coordinates have many advantages, such as clearly determining whether a point lies on a straight line.

[0149] like Figure 6dAs shown, a projective transformation can be an operation performed on a two-dimensional plane. A and A', B and B', C and C', and D and D' are different pairs of points on the same plane. The non-homogeneous coordinates of each pair of points can be represented by a projective transformation. In one example, the non-homogeneous coordinates of point A can be (x1, y1), and the non-homogeneous coordinates of A' can be (x2, y2). The projective transformation from A to A' can be expressed as: (x2, y2) = H'*(x1, y1), where H' is the projective matrix in the two-dimensional operation.

[0150] like Figure 6e As shown, the homogeneous coordinates of a pair of points on two images from different viewpoints (the original image plane and the new image plane) can be expressed by a projective transformation, for example: x1 = H * x2. Projective transformation is also called homography, where the prefix "homography" means "same." Homography is the graphy produced from the same source. Therefore, the matrix H in the above formula is called the homography matrix. The homography matrix is ​​the projection from one plane to another (projection center as shown in the image). Figure 6e The homography matrix (shown in the diagram) can be seen as a transition from two-dimensional to three-dimensional operations of the projective matrix. If a unique solution is required, four point pairs are needed to solve H (corresponding to eight equations, solving for the eight unknowns in H).

[0151] The RANSAC algorithm assumes that the data contains both correct and outlier data (or noise). Correct data is denoted as inliers, and outliers as outliers. RANSAC also assumes that, given a set of correct data, there exists a method to compute model parameters that satisfy these data. The core idea of ​​this algorithm is randomness and assumption. Randomness involves randomly selecting sampled data based on the probability of correct data occurrences; according to the law of large numbers, random simulation can approximate the correct results. Assumption involves assuming that the selected sampled data are all correct, then using these correct data to compute other points through a model that satisfies the problem, and finally assigning a score to the result.

[0152] The RANSAC algorithm is widely used in computer vision and mathematics, for example, in line fitting, plane fitting, calculating transformation matrices between images or point clouds, and calculating fundamental matrices. Its algorithm process is as follows:

[0153] 1. There is a model that fits the assumed inliers, meaning that all unknown parameters can be calculated from the assumed inliers.

[0154] 2. Use the model obtained in step 1 to test all other data. If a point fits the estimated model, consider it an inlier.

[0155] 3. If a sufficient number of points are classified as hypothetical inliers, then the estimated model is reasonably reasonable.

[0156] 4. Then, re-estimate the model using all the assumed inliers (e.g., using least squares), since it has only been estimated by the initial assumed inliers.

[0157] 5. Finally, the model is evaluated by estimating the error rate of the in-place points and the model.

[0158] 6. The above process is repeated a fixed number of times, and each time the generated model is either discarded because it has too few inliers or selected because it is better than the existing models.

[0159] In the above embodiments, by utilizing the homography matrix, the image coordinates of each of the at least two target trajectory points are mapped and calculated, thereby obtaining the world coordinates of each of the at least two target trajectory points. The at least two target trajectory points are used to facilitate subsequent determination of the direction of departure when the target to be tracked leaves the acquisition range of the first camera.

[0160] See Figure 7 This is a schematic diagram of the third process of the target tracking method provided in the embodiments of this application, based on Figure 5 Step S503 in the document has been refined to include the following steps:

[0161] Step S701: Calculate the departure direction vector of the target to be tracked when it leaves the acquisition range of the first camera, based on the world coordinates of at least two target trajectory points in the world location information.

[0162] The target trajectory point is the trajectory point of the target to be tracked in world coordinates.

[0163] Step S702: Determine the direction of departure of the target to be tracked when it leaves the acquisition range of the first camera, based on the departure direction vector.

[0164] The direction of the aforementioned departure direction vector is the departure direction of the target being tracked when it leaves the acquisition range of the first camera.

[0165] In the above embodiment, the departure direction vector of the target to be tracked when it leaves the acquisition range of the first camera is obtained based on the world coordinates of at least two target trajectory points in the world location information; then, the departure direction of the target to be tracked when it leaves the acquisition range of the first camera is determined based on the departure direction vector.

[0166] See Figure 8a This is a schematic diagram of the fourth process of the target tracking method provided in the embodiments of this application, based on Figure 7 Step S701 in the document has been refined to include the following steps:

[0167] Step S801: Obtain the first world coordinates and the Nth world coordinates from the world location information; wherein, the first world coordinates are the world coordinates of the last target trajectory point collected among all target trajectory points, or the first world coordinates are the average of the world coordinates of the last i+1 target trajectory points collected among all target trajectory points; the Nth world coordinates are the world coordinates of the Nth target trajectory point collected from the end among all target trajectory points, or the Nth world coordinates are the average of the world coordinates of the Nth to N+ith target trajectory points collected from the end among all target trajectory points; N is an integer greater than 1, and i is a positive integer;

[0168] In one example, such as Figure 8b As shown, you can take the world coordinates of two target trajectory points of the target to be tracked (the last trajectory point p1 and the Nth trajectory point from the end) (N can be adjusted according to the actual situation), or you can take the average of the world coordinates of the last two trajectory points and the average of the world coordinates of the Nth trajectory point from the end and the (N+1)th trajectory point from the end. This can eliminate noise interference.

[0169] Step S802: Calculate the vector from the first world coordinates to the Nth world coordinates to obtain the departure direction vector when the target to be tracked leaves the acquisition range of the first camera.

[0170] In one example, see Figure 8b After obtaining the world coordinates of the two target trajectory points at the end of the trajectory within the range of the first camera, p1-pn is calculated to obtain a vector pointing from the target to the direction of departure of the target, i.e., the departure direction vector.

[0171] In the above embodiment, the vector pointing from the first world coordinate to the Nth world coordinate is calculated using the first world coordinate and the Nth world coordinate in the world location information, thereby realizing the acquisition of the driving direction vector when the target to be tracked drives out of the acquisition range of the first camera.

[0172] See Figure 9a This is a fifth flowchart illustrating the target tracking method provided in this application embodiment, based on... Figure 7 Step S702 in the document has been refined to include the following steps:

[0173] Step S901: Convert the driving direction vector into a unit direction vector; wherein the horizontal coordinate of the unit direction vector is x and the vertical coordinate is y.

[0174] In step S902, when x > 0, y > 0, and x > y, it is determined that the driving-out direction when the target to be tracked drives out of the acquisition range of the first camera is east.

[0175] In step S903, when x > 0, y > 0, and x < y, it is determined that the driving-out direction when the target to be tracked drives out of the acquisition range of the first camera is north.

[0176] In step S904, when x < 0, y > 0, and -x > y, it is determined that the driving-out direction when the target to be tracked drives out of the acquisition range of the first camera is west.

[0177] In step S905, when x < 0, y > 0, and -x < y, it is determined that the driving-out direction when the target to be tracked drives out of the acquisition range of the first camera is north.

[0178] In step S906, when x < 0, y < 0, and -x > -y, it is determined that the driving-out direction when the target to be tracked drives out of the acquisition range of the first camera is west.

[0179] In step S907, when x < 0, y < 0, and -x < -y, it is determined that the driving-out direction when the target to be tracked drives out of the acquisition range of the first camera is south.

[0180] In step S908, when x > 0, y < 0, and x > -y, it is determined that the driving-out direction when the target to be tracked drives out of the acquisition range of the first camera is east.

[0181] In step S909, when x > 0, y < 0, and x < -y, it is determined that the driving-out direction when the target to be tracked drives out of the acquisition range of the first camera is south.

[0182] The driving-out direction vector is unitized to obtain a unit direction vector, and then the direction of this unit direction vector is judged, which is the driving-out direction when the target to be tracked drives out of the acquisition range of the first camera.

[0183] In an example, referring to Figure 9b , the actual geographical directions are the four directions of east, west, south, and north. The abscissa x is the longitude, and the ordinate y is the latitude. Assume that the unit direction vector after normalization is (x, y):

[0184] If x > 0 and y > 0, the unit direction vector is in the first quadrant. If x > y, the unit direction vector is close to the positive semi-axis of the x-axis, and the driving-out direction when the target to be tracked drives out of the acquisition range of the first camera is east. If x < y, the unit direction vector is close to the positive semi-axis of the y-axis, and the driving-out direction when the target to be tracked drives out of the acquisition range of the first camera is north.

[0185] If x < 0 and y > 0, the unit direction vector is in the second quadrant. If -x > y, the unit direction vector is close to the negative x-axis. When the target to be tracked exits the acquisition range of the first camera, the exit direction is west. If -x < y, the unit direction vector is close to the positive y-axis. When the target to be tracked exits the acquisition range of the first camera, the exit direction is north.

[0186] If x < 0 and y < 0, the unit direction vector is in the third quadrant. If -x > -y, the unit direction vector is close to the negative x-axis. When the target to be tracked exits the acquisition range of the first camera, the exit direction is west. If -x < -y, the unit direction vector is close to the negative y-axis. When the target to be tracked exits the acquisition range of the first camera, the exit direction is south.

[0187] If x > 0 and y < 0, the unit direction vector is in the fourth quadrant. If x > -y, the unit direction vector is close to the positive x-axis. When the target to be tracked exits the acquisition range of the first camera, the exit direction is east. If x < -y, the unit direction vector is close to the negative y-axis. When the target to be tracked exits the acquisition range of the first camera, the exit direction is south.

[0188] Once the exit direction when the target to be tracked exits the acquisition range of the first camera is determined, it is convenient to subsequently judge the next possible camera to go to based on this exit direction.

[0189] In the above embodiment, by analyzing and judging the horizontal and vertical coordinates of the unit direction vector, the determination of the exit direction when the target to be tracked exits the acquisition range of the first camera is achieved.

[0190] See Figure 10 , which is the sixth process schematic diagram of the target tracking method provided by the embodiment of the present application. Based on Figure 5 the step S504 in is refined, including the following steps:

[0191] Step S1001, obtain the world coordinates of each camera in the world coordinate system;

[0192] Step S1002, for each camera except the first camera, calculate the distance between this camera and the first camera according to the world coordinates of this camera and the world coordinates of the first camera;

[0193] Step S1003, select M cameras closest to the first camera to obtain multiple cameras to be matched adjacent to the first camera.

[0194] In one example, select M cameras closest to the first camera. The value range of M can be 5 - 10, and the value range can be adjusted according to the actual situation.

[0195] Step S1004: For each camera to be matched, calculate the relative direction vector of the camera to be matched relative to the first camera.

[0196] Traverse the above M cameras and calculate the direction vector of each camera relative to the first camera. The calculation process is the same as in the above embodiment and will not be repeated here.

[0197] Step S1005: Determine the relative direction of each of the cameras to be matched relative to the first camera based on the relative direction vectors.

[0198] In the above embodiments, the relative direction of each camera to be matched relative to the first camera is obtained by calculating the relative direction vector of each camera to be matched relative to the first camera.

[0199] See Figure 11a This is a seventh flowchart illustrating the target tracking method provided in this application embodiment, based on... Figure 5 Step S505 in the document has been refined to include the following steps:

[0200] Step S1101: Among the cameras to be matched whose relative direction matches the driving direction, select the K cameras closest to the first camera to obtain K second cameras, where K is a preset positive integer.

[0201] In one example, a camera whose relative direction is the same as the direction of travel is selected as a candidate camera to which the target is to be tracked. If there are more than two candidate cameras, the distances between all candidate cameras and the first camera are calculated, and the two closest cameras are used for cross-camera trajectory matching of the target. This effectively narrows the trajectory matching range, reducing the matching range from all cameras to 1-2 cameras.

[0202] In one example, such as Figure 11b As shown, 3 is the first camera, 1, 2, 4, and 5 are cameras to be matched, and 5 is the camera to be matched with the relative direction and the direction of departure.

[0203] In the above embodiments, by acquiring data from the second camera, cross-camera matching of the target to be tracked between adjacent cameras can be achieved, effectively narrowing the range of cross-camera trajectory matching of the target to be tracked and improving the speed and accuracy of matching.

[0204] See Figure 12 This is an eighth flowchart illustrating the target tracking method provided in this application embodiment, based on... Figure 5 It also includes the following steps:

[0205] Step S1201: For each second camera, determine the direction of entry of the target to be tracked into the acquisition range of the second camera;

[0206] The process of determining the direction of entry when the target to be tracked enters the acquisition range of the second camera is the same as the process of determining the exit direction in the above embodiment, and will not be repeated here.

[0207] Step S1202: Match the exit direction with the entry direction of the target to be tracked when it enters the acquisition range of the second camera to obtain the direction matching result of the second camera;

[0208] For each second camera, the direction matching result of the second camera can be either a match between the direction in which the target moves out of the first camera's acquisition range and the direction in which it moves into the second camera's acquisition range, or a mismatch between the direction in which the target moves out of the first camera's acquisition range and the direction in which it moves into the second camera's acquisition range.

[0209] Step S1203: Based on the orientation matching result of the second camera, determine whether the cross-camera tracking result between the first camera and the second camera is reliable.

[0210] If the direction in which the target leaves the range of the first camera matches the direction in which it enters the range of the second camera, the cross-camera tracking result between the first and second cameras is reliable; if the direction in which the target leaves the range of the first camera does not match the direction in which it enters the range of the second camera, the cross-camera tracking result between the first and second cameras is unreliable.

[0211] In the above embodiments, for each second camera, the reliability of the cross-camera tracking result between the first camera and the second camera is determined by the direction matching result of the second camera, which adds strong data support to the target tracking method.

[0212] In one example, see Figure 13 This is a schematic diagram of the ninth type of target tracking method provided in the embodiments of this application, taking cross-camera vehicle tracking as an example. Cross-camera vehicle matching includes the following steps: 1. Converting vehicle image coordinates to world (GPS) coordinates; 2. Calculating the vehicle's entry and exit directions based on the vehicle trajectory; 3. Searching for vehicles entering and exiting cameras based on the vehicle's direction; 4. Matching vehicles from adjacent cameras.

[0213] In the above embodiments, based on the direction of the vehicle entering / leaving the camera's capture range, the system automatically searches for the camera preceding the vehicle or the camera following the vehicle, and performs cross-camera tracking of the vehicle.

[0214] Based on the same concept, embodiments of this application also provide a target tracking device. See also Figure 13 This is a schematic diagram of a target tracking device provided in an embodiment of this application. The device includes:

[0215] The first acquisition module 1410 is used to acquire the image position information of the target to be tracked in the image coordinate system of the first camera;

[0216] The second acquisition module 1420 is used to transform the image position information from the image coordinate system to the world coordinate system to obtain world position information;

[0217] The first determining module 1430 is used to determine the direction of departure of the target to be tracked when it leaves the collection range of the first camera, based on the world location information.

[0218] The third acquisition module 1440 is used to acquire multiple cameras to be matched that are adjacent to the first camera;

[0219] The second determining module 1450 is used to determine the relative direction of each camera to be matched relative to the first camera;

[0220] The selection module 1460 is used to select at least one second camera from among the cameras to be matched whose relative direction matches the driving direction;

[0221] The tracking module 1470 is used to perform cross-camera tracking of the target to be tracked based on the image data from the first camera and the image data from the second camera.

[0222] In the above embodiments, the direction of the target to be tracked when it leaves the range of the first camera is automatically determined to be the direction of the target to be tracked to the second camera, and the target to be tracked is tracked across cameras. This realizes automatic cross-camera target tracking, which improves the robustness of cross-camera tracking compared with the prior art which relies too much on manually drawn area information.

[0223] In one possible implementation, the image location information includes the image coordinates of at least two target trajectory points of the target to be tracked in the image coordinate system;

[0224] The second acquisition module 1420 includes:

[0225] The first acquisition submodule is used to perform mapping calculations on the image coordinates of each of the at least two target trajectory points by using a pre-determined homography matrix between the image coordinate system and the world coordinate system, so as to obtain the world coordinates of each of the target trajectory points, wherein the world position information includes the world coordinates of each of the target trajectory points.

[0226] In the above embodiments, by utilizing the homography matrix, the image coordinates of each of the at least two target trajectory points are mapped and calculated, thereby obtaining the world coordinates of each of the at least two target trajectory points. The at least two target trajectory points are used to facilitate subsequent determination of the direction of departure when the target to be tracked leaves the acquisition range of the first camera.

[0227] In one possible implementation, the first determining module 1430 includes:

[0228] The second acquisition submodule is used to calculate the departure direction vector of the target to be tracked when it leaves the acquisition range of the first camera, based on the world coordinates of at least two target trajectory points in the world location information.

[0229] The first determining submodule is used to determine the direction of departure of the target to be tracked when it leaves the acquisition range of the first camera, based on the departure direction vector.

[0230] In the above embodiment, the departure direction vector of the target to be tracked when it leaves the acquisition range of the first camera is obtained based on the world coordinates of at least two target trajectory points in the world location information; then, the departure direction of the target to be tracked when it leaves the acquisition range of the first camera is determined based on the departure direction vector.

[0231] In one possible implementation, the second acquisition submodule is specifically used for:

[0232] Obtain the first world coordinates and the Nth world coordinates from the world location information; wherein, the first world coordinates are the world coordinates of the last target trajectory point collected among all target trajectory points, or the first world coordinates are the average of the world coordinates of the last i+1 target trajectory points collected among all target trajectory points; the Nth world coordinates are the world coordinates of the Nth target trajectory point collected from the end among all target trajectory points, or the Nth world coordinates are the average of the world coordinates of the Nth to N+ith target trajectory points collected from the end among all target trajectory points; N is an integer greater than 1, and i is a positive integer;

[0233] Calculate the vector pointing from the first world coordinate to the Nth world coordinate to obtain the departure direction vector when the target to be tracked leaves the acquisition range of the first camera.

[0234] In the above embodiment, the vector pointing from the first world coordinate to the Nth world coordinate is calculated using the first world coordinate and the Nth world coordinate in the world location information, thereby realizing the acquisition of the driving direction vector when the target to be tracked drives out of the acquisition range of the first camera.

[0235] In one possible implementation, the first determining submodule is specifically used for:

[0236] The departure direction vector is converted into a unit direction vector; wherein the horizontal coordinate of the unit direction vector is x and the vertical coordinate is y.

[0237] When x is greater than 0, y is greater than 0, and x is greater than y, the direction in which the target to be tracked leaves the collection range of the first camera is determined to be east.

[0238] When x is greater than 0, y is greater than 0, and x is less than y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be north;

[0239] When x is less than 0, y is greater than 0, and -x is greater than y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be west.

[0240] When x is less than 0, y is greater than 0, and -x is less than y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be north;

[0241] When x is less than 0, y is less than 0, and -x is greater than -y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be west.

[0242] When x is less than 0, y is less than 0, and -x is less than -y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be south.

[0243] When x is greater than 0, y is less than 0, and x is greater than -y, the direction in which the target to be tracked leaves the collection range of the first camera is determined to be east.

[0244] When x is greater than 0, y is less than 0, and x is less than -y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be south.

[0245] In the above embodiments, by analyzing and judging the horizontal and vertical coordinates of the unit direction vector, the direction of departure when the target to be tracked leaves the acquisition range of the first camera is determined.

[0246] In one possible implementation, the third acquisition module 1440 includes:

[0247] The third acquisition submodule is used to acquire the world coordinates of each camera in the world coordinate system.

[0248] The calculation submodule is used to calculate the distance between each camera other than the first camera, based on the world coordinates of the camera and the world coordinates of the first camera.

[0249] The fourth acquisition submodule is used to select the M cameras closest to the first camera to obtain multiple cameras to be matched that are adjacent to the first camera.

[0250] In one possible implementation, the second determining module 1450 includes:

[0251] The fifth acquisition submodule is used to calculate the relative direction vector of each camera to be matched relative to the first camera for each camera to be matched.

[0252] The second determining submodule is used to determine the relative direction of each of the cameras to be matched relative to the first camera based on the relative direction vectors.

[0253] In the above embodiments, the relative direction of each camera to be matched relative to the first camera is obtained by calculating the relative direction vector of each camera to be matched relative to the first camera.

[0254] In one possible implementation, the selection module 1460 includes:

[0255] The sixth acquisition submodule is used to select the K cameras closest to the first camera from among the cameras to be matched whose relative direction matches the driving direction, to obtain K second cameras, where K is a preset positive integer.

[0256] In the above embodiments, by acquiring data from the second camera, cross-camera matching of the target to be tracked between adjacent cameras can be achieved, effectively narrowing the range of cross-camera trajectory matching of the target to be tracked and improving the speed and accuracy of matching.

[0257] In one possible implementation, the device further includes:

[0258] The third determining module is used to determine the direction of entry of the target to be tracked when it enters the acquisition range of each second camera.

[0259] The fourth acquisition module is used to match the driving direction with the driving direction when the target to be tracked enters the acquisition range of the second camera, and obtain the direction matching result of the second camera;

[0260] The fourth determining module is used to determine whether the cross-camera tracking result between the first camera and the second camera is reliable based on the orientation matching result of the second camera.

[0261] In the above embodiments, for each second camera, the reliability of the cross-camera tracking result between the first camera and the second camera is determined by the direction matching result of the second camera, which adds strong data support to the target tracking method.

[0262] This application also provides an electronic device, such as... Figure 15 As shown, it includes a processor 1501, a communication interface 1502, a memory 1503, and a communication bus 1504. The processor 1501, communication interface 1502, and memory 1503 communicate with each other via the communication bus 1504.

[0263] Memory 1503 is used to store computer programs;

[0264] When processor 1501 executes the program stored in memory 1503, it performs the following steps:

[0265] Obtain the image position information of the target to be tracked in the image coordinate system of the first camera;

[0266] The image position information is transformed from the image coordinate system to the world coordinate system to obtain the world position information;

[0267] Based on the world location information, determine the direction in which the target to be tracked leaves the range of the first camera;

[0268] Obtain multiple cameras adjacent to the first camera to be matched, and determine the relative direction of each camera to be matched relative to the first camera;

[0269] Select at least one second camera from among the cameras to be matched whose relative direction matches the direction of departure;

[0270] Based on the image data from the first camera and the image data from the second camera, the target to be tracked is tracked across cameras.

[0271] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0272] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0273] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0274] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0275] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the target tracking methods described above.

[0276] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the target tracking methods described above.

[0277] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0278] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0279] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the embodiments for apparatus, electronic devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0280] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A target tracking method, characterized in that, The method includes: The image position information of the target to be tracked in the image coordinate system of the first camera is obtained, and the image position information includes the image coordinates of at least two target trajectory points of the target to be tracked in the image coordinate system; The image position information is transformed from the image coordinate system to the world coordinate system to obtain world position information, which includes the world coordinates of each target trajectory point. Obtain the first world coordinates and the Nth world coordinates from the world location information; wherein, the first world coordinates are the world coordinates of the last target trajectory point collected among all target trajectory points, or the first world coordinates are the average of the world coordinates of the last i+1 target trajectory points collected among all target trajectory points; the Nth world coordinates are the world coordinates of the Nth target trajectory point collected from the end among all target trajectory points, or the Nth world coordinates are the average of the world coordinates of the Nth to N+ith target trajectory points collected from the end among all target trajectory points; N is an integer greater than 1, and i is a positive integer; calculate the vector from the first world coordinates to the Nth world coordinates to obtain the departure direction vector when the target to be tracked leaves the collection range of the first camera; Based on the exit direction vector, determine the exit direction of the target when it leaves the acquisition range of the first camera; Obtain multiple cameras adjacent to the first camera to be matched, and determine the relative direction of each camera to be matched relative to the first camera; Select at least one second camera from among the cameras to be matched whose relative direction matches the direction of departure; Based on the image data from the first camera and the image data from the second camera, the target to be tracked is tracked across cameras.

2. The method according to claim 1, characterized in that, The step of transforming the image position information from the image coordinate system to the world coordinate system to obtain world position information includes: By using a pre-determined homography matrix between the image coordinate system and the world coordinate system, the image coordinates of each of the at least two target trajectory points are mapped and calculated to obtain the world coordinates of each target trajectory point.

3. The method according to claim 1, characterized in that, Determining the exit direction of the target when it leaves the capture range of the first camera based on the exit direction vector includes: The departure direction vector is converted into a unit direction vector; wherein the horizontal coordinate of the unit direction vector is x and the vertical coordinate is y. When x is greater than 0, y is greater than 0, and x is greater than y, the direction in which the target to be tracked leaves the collection range of the first camera is determined to be east. When x is greater than 0, y is greater than 0, and x is less than y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be north; When x is less than 0, y is greater than 0, and -x is greater than y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be west. When x is less than 0, y is greater than 0, and -x is less than y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be north; When x is less than 0, y is less than 0, and -x is greater than -y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be west. When x is less than 0, y is less than 0, and -x is less than -y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be south. When x is greater than 0, y is less than 0, and x is greater than -y, the direction in which the target to be tracked leaves the collection range of the first camera is determined to be east. When x is greater than 0, y is less than 0, and x is less than -y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be south.

4. The method according to claim 1, characterized in that, The step of acquiring multiple cameras to be matched adjacent to the first camera includes: Obtain the world coordinates of each camera in the world coordinate system; For each camera other than the first camera, calculate the distance between the camera and the first camera based on the world coordinates of the camera and the world coordinates of the first camera; Select the M cameras closest to the first camera to obtain multiple cameras to be matched that are adjacent to the first camera.

5. The method according to claim 1, characterized in that, Determining the relative orientation of each camera to be matched relative to the first camera includes: For each camera to be matched, the relative direction vector of the camera to be matched relative to the first camera is calculated; Based on the relative direction vectors, the relative direction of each of the cameras to be matched relative to the first camera is determined.

6. The method according to claim 1, characterized in that, The step of selecting at least one second camera from among the cameras to be matched whose relative direction matches the departure direction includes: Among the cameras to be matched whose relative direction matches the departure direction, select the K cameras closest to the first camera to obtain K second cameras, where K is a preset positive integer.

7. The method according to claim 1, characterized in that, The method further includes: For each second camera, determine the direction of entry of the target to be tracked when it enters the acquisition range of that second camera; The driving direction is matched with the driving direction of the target to be tracked when it enters the collection range of the second camera to obtain the direction matching result of the second camera. Based on the orientation matching result of the second camera, determine whether the cross-camera tracking result between the first camera and the second camera is reliable.

8. A target tracking device, characterized in that, The device includes: The first acquisition module is used to acquire image position information of the target to be tracked in the image coordinate system of the first camera, wherein the image position information includes the image coordinates of at least two target trajectory points of the target to be tracked in the image coordinate system; The second acquisition module is used to transform the image position information from the image coordinate system to the world coordinate system to obtain world position information, wherein the world position information includes the world coordinates of each of the target trajectory points; The first determining module includes: The second acquisition submodule is used to acquire the first world coordinates and the Nth world coordinates from the world location information; wherein, the first world coordinates are the world coordinates of the last acquired target trajectory point among all the target trajectory points, or the first world coordinates are the average of the world coordinates of the last i+1 acquired target trajectory points among all the target trajectory points; the Nth world coordinates are the world coordinates of the Nth acquired target trajectory point from the end among all the target trajectory points, or the Nth world coordinates are the average of the world coordinates of the Nth to N+i acquired target trajectory points from the end among all the target trajectory points; N is an integer greater than 1, and i is a positive integer; calculate the vector from the first world coordinates to the Nth world coordinates to obtain the exit direction vector when the target to be tracked leaves the acquisition range of the first camera; the first determination submodule is used to determine the exit direction when the target to be tracked leaves the acquisition range of the first camera based on the exit direction vector. The third acquisition module is used to acquire multiple cameras to be matched that are adjacent to the first camera; The second determining module is used to determine the relative direction of each camera to be matched relative to the first camera; The selection module is used to select at least one second camera from among the cameras to be matched whose relative direction matches the driving direction; The tracking module is used to perform cross-camera tracking of the target to be tracked based on image data from the first camera and image data from the second camera.

9. The apparatus according to claim 8, characterized in that, The second acquisition module includes: The first acquisition submodule is used to perform mapping calculations on the image coordinates of each of the at least two target trajectory points by using a pre-determined homography matrix between the image coordinate system and the world coordinate system, so as to obtain the world coordinates of each of the target trajectory points.

10. The apparatus according to claim 8, characterized in that, The first determining submodule is specifically used for: The departure direction vector is converted into a unit direction vector; wherein the horizontal coordinate of the unit direction vector is x and the vertical coordinate is y. When x is greater than 0, y is greater than 0, and x is greater than y, the direction in which the target to be tracked leaves the collection range of the first camera is determined to be east. When x is greater than 0, y is greater than 0, and x is less than y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be north; When x is less than 0, y is greater than 0, and -x is greater than y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be west. When x is less than 0, y is greater than 0, and -x is less than y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be north; When x is less than 0, y is less than 0, and -x is greater than -y, the direction of departure of the target being tracked when it leaves the acquisition range of the first camera is determined to be west. When x is less than 0, y is less than 0, and -x is less than -y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be south. When x is greater than 0, y is less than 0, and x is greater than -y, the direction in which the target to be tracked leaves the collection range of the first camera is determined to be east. When x is greater than 0, y is less than 0, and x is less than -y, the direction in which the target to be tracked leaves the acquisition range of the first camera is determined to be south.

11. The apparatus according to claim 8, characterized in that, The third acquisition module includes: The third acquisition submodule is used to acquire the world coordinates of each camera in the world coordinate system. The calculation submodule is used to calculate the distance between each camera other than the first camera, based on the world coordinates of the camera and the world coordinates of the first camera. The fourth acquisition submodule is used to select the M cameras closest to the first camera to obtain multiple cameras to be matched that are adjacent to the first camera.

12. The apparatus according to claim 8, characterized in that, The second determining module includes: The fifth acquisition submodule is used to calculate the relative direction vector of each camera to be matched relative to the first camera for each camera to be matched. The second determining submodule is used to determine the relative direction of each of the cameras to be matched relative to the first camera based on the relative direction vectors.

13. The apparatus according to claim 8, characterized in that, The selection module includes: The sixth acquisition submodule is used to select the K cameras closest to the first camera from among the cameras to be matched whose relative direction matches the driving direction, to obtain K second cameras, where K is a preset positive integer.

14. The apparatus according to claim 8, characterized in that, The device further includes: The third determining module is used to determine the direction of entry of the target to be tracked when it enters the acquisition range of each second camera. The fourth acquisition module is used to match the driving direction with the driving direction when the target to be tracked enters the acquisition range of the second camera, and obtain the direction matching result of the second camera; The fourth determining module is used to determine whether the cross-camera tracking result between the first camera and the second camera is reliable based on the orientation matching result of the second camera.

Images