Target tracking method, device, equipment and vehicle

Abstract

The present invention provides a method for obtaining a relationship matrix between a single-camera vehicle tracking sequence set at time T and a cross-camera vehicle tracking sequence set at time T-1, calculating the value of each element in the relationship matrix based on a camera field of view coefficient, a tracking identification coefficient, an image feature coefficient, a target category coefficient, and a world coordinate coefficient, and then updating the cross-camera vehicle tracking sequence corresponding to time T-1 and time T based on the relationship matrix and creating a new cross-camera vehicle tracking sequence, so that the same target vehicle can still keep the same tracking identifier when it appears in different cameras at different times.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a vehicle-mounted cross-camera vehicle target tracking method, device, equipment and vehicle. Background Art

[0002] Autonomous driving has become the mainstream development direction of future cars. In autonomous driving technology, multiple cameras are often installed at different positions on the vehicle body to achieve 360-degree visual perception of the vehicle's surroundings. At present, there are mature single-camera vehicle tracking methods that can keep the same identifier of the same vehicle at different times in the same camera. However, the existing single-camera vehicle tracking methods cannot distinguish whether the vehicles detected in different cameras are the same vehicle. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a target tracking method to solve the problem in the prior art that the single-camera vehicle tracking method cannot distinguish whether the vehicles detected by different cameras are the same vehicle.

[0004] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0005] A target tracking method, comprising:

[0006] Based on the single-camera vehicle tracking sequence set at time T-1 and the vehicle detection target set corresponding to time T, a single-camera vehicle tracking sequence set at time T is obtained;

[0007] Get the cross-camera vehicle tracking sequence set at time T-1;

[0008] Obtain a relationship matrix between a single-camera vehicle tracking sequence set at time T and a cross-camera vehicle tracking sequence set at time T-1;

[0009] Determine whether the value of the element in the relationship matrix meets the preset condition. When the preset condition is not met, add the single-camera vehicle tracking sequence corresponding to the element that meets the preset condition to the unmatched single-camera vehicle tracking sequence set at time T. When the preset condition is met, add the single-camera vehicle tracking sequence corresponding to the element that meets the preset condition to the associated single-camera tracking sequence set of the cross-camera vehicle tracking sequence at time T-1.

[0010] Update the cross-camera vehicle tracking sequence corresponding to time T-1 and time T based on the associated single-camera tracking sequence set;

[0011] A new cross-camera vehicle tracking sequence is created based on the unmatched single-camera vehicle tracking sequence set.

[0012] Optionally, in the target tracking method, before obtaining the single-camera vehicle tracking sequence set at time T based on the single-camera vehicle tracking sequence set at time T-1 and the vehicle detection target set corresponding to time T, the method further includes:

[0013] Obtaining a vehicle detection target set corresponding to time T, wherein the vehicle detection target set includes a set of all vehicle targets detected at that time;

[0014] A single-camera vehicle tracking sequence set at time T-1 is obtained, wherein the single-camera vehicle tracking sequence set includes a sequence set of tracking targets detected within a time period from a time when the target is initially detected to a specific time.

[0015] Optionally, the target tracking method further includes:

[0016] Obtain the image information captured by each camera at time T;

[0017] Calculating the three-dimensional information of the target vehicle contained in the image information based on the image information captured at time T;

[0018] Calculating the world coordinate position of the target vehicle based on the camera parameters of each camera and the three-dimensional information;

[0019] Each element in each set includes at least the three-dimensional information and world coordinate information of the target vehicle corresponding to the element.

[0020] Optionally, in the above target tracking method, the three-dimensional information at least includes:

[0021] The object's 2D bounding box, object type, object image feature vector, and capturing camera identifier.

[0022] Optionally, in the target tracking method, based on the single-camera vehicle tracking sequence set at time T-1 and the vehicle detection target set corresponding to time T, a single-camera vehicle tracking sequence set at time T is obtained, including:

[0023] Based on the single-camera vehicle tracking sequence set at time T-1 and the vehicle detection target set corresponding to time T, a single-camera vehicle tracking sequence set at time T is obtained by using a SORT target tracking method.

[0024] Optionally, in the target tracking method, determining whether the value of the element in the relationship matrix satisfies a preset condition includes:

[0025] Obtaining a relationship matrix corresponding to each single-camera vehicle tracking sequence and the cross-camera vehicle tracking sequence;

[0026] Obtaining the minimum element of each row in the relationship matrix;

[0027] Determine whether the value of the minimum element is less than 0. If it is greater than 0, it indicates that the single-camera vehicle tracking sequence corresponding to the minimum element does not match all cross-camera vehicle tracking sequences, and the value of the element does not meet the preset condition. If it is not greater than 0, it indicates that the preset condition is met, the single-camera vehicle tracking sequence corresponding to the minimum element matches the cross-camera vehicle tracking sequence corresponding to the minimum element, and the single-camera vehicle tracking sequence corresponding to the minimum element is added to the associated single-camera tracking sequence set of the cross-camera vehicle tracking sequence corresponding to the minimum element.

[0028] Optionally, in the above target tracking method, updating the cross-camera vehicle tracking sequence corresponding to time T-1 and time T based on the associated single-camera tracking sequence set includes:

[0029] A single-camera tracking sequence set matched by a cross-camera vehicle tracking sequence that successfully matches a single-camera vehicle tracking sequence is obtained; according to the capture camera identifier of the tracking target at time T in the single-camera vehicle tracking sequence and the camera selection priority table, the tracking target at time T in the single-camera vehicle tracking sequence with the highest priority in the single-camera tracking sequence set is obtained; and the tracking target at time T is added to the end of the cross-camera vehicle tracking sequence at time T-1.

[0030] Optionally, the target tracking method further includes:

[0031] If it is found according to the capture camera identifier of the tracking target at time T in the single-camera vehicle tracking sequence and the camera selection priority table that the single-camera tracking identifier of the single-camera tracking sequence does not exist in the associated single-camera tracking identifier set of the cross-camera vehicle tracking sequence, the single-camera tracking identifier is added to the associated single-camera tracking identifier set of the cross-camera vehicle tracking sequence to generate the cross-camera tracking sequence at time T.

[0032] Optionally, the target tracking method further includes:

[0033] The cross-camera vehicle tracking sequence that is not matched to the single-camera vehicle tracking sequence is added to the unmatched cross-camera vehicle tracking sequence set, and each sequence in the unmatched cross-camera vehicle tracking sequence set in each stored set is deleted.

[0034] A target tracking device comprises a memory and a processor; when the computer program is executed by the processor, each step of any one of the target tracking methods described above is implemented.

[0035] Based on the above technical solution, the above solution provided by the embodiment of the present invention obtains the relationship matrix between the single-camera vehicle tracking sequence set at time T and the cross-camera vehicle tracking sequence set at time T-1, calculates the value of each element in the relationship matrix based on the camera field of view coefficient, tracking identification coefficient, image feature coefficient, target category coefficient, and world coordinate coefficient, and then updates the cross-camera vehicle tracking sequence corresponding to time T-1 and time T based on the relationship matrix, and creates a new cross-camera vehicle tracking sequence, so that the same target vehicle can still keep the same tracking identifier when it appears in different cameras at different times. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0037] Figure 1 A flowchart of a target tracking method disclosed in an embodiment of the present application;

[0038] Figure 2 A schematic diagram of the configuration of each camera disclosed in the embodiment of the present application;

[0039] Figure 3 A schematic diagram of the structure of a target tracking device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The applicant has studied and found that in the cross-camera vehicle tracking solution, the historical frame information of the vehicle in different cameras is obtained through the cross-camera tracking identifier of the vehicle. The processing algorithm of the corresponding attributes can effectively solve the above problems that are difficult to handle in single-camera detection and tracking, and provide more accurate and stable vehicle attributes. Therefore, we propose a vehicle-mounted cross-camera vehicle tracking method based on single-camera vehicle tracking.

[0042] See also Figure 1 , the present application discloses a target tracking method, which may include:

[0043] Step S101: Based on the single-camera vehicle tracking sequence set at time T-1 and the vehicle detection target set corresponding to time T, a single-camera vehicle tracking sequence set at time T is obtained.

[0044] This step first obtains the vehicle detection target set corresponding to time T and the single-camera vehicle tracking sequence set at time T-1, wherein the vehicle detection target set includes the set of all vehicle targets detected at that time, and the single-camera vehicle tracking sequence set includes the sequence set of tracking targets detected in the time period from the initial detection of the target to the specific time.

[0045] In this step, the vehicle detection target set J corresponding to time T is obtained T , the vehicle detection target set J corresponding to the time T T , refers to the set of all vehicle targets detected at time T, including multiple different vehicle targets at the same time. In this scheme, each element in the set includes at least the three-dimensional information and world coordinate system information of the detected vehicle target.

[0046] The process of obtaining the three-dimensional information and the world coordinate system information is as follows:

[0047] The images captured by each camera at time T are input into the trained target detection neural network. After the target detection neural network processes the images, the three-dimensional information of the vehicle is obtained. The three-dimensional information includes the two-dimensional bounding box of the target vehicle, the target type, the target image feature vector and the capture camera identifier. The two-dimensional bounding box is a rectangular box whose boundaries are parallel to the edge of the image and can just frame the target vehicle; the target types are car, bus, and truck; the target image feature is a 256-dimensional feature vector; the camera identifier is the unique identifier of each camera. In this solution, 7 cameras are used, and the identifiers are FC60 (Front Center 60), FC120 (Front Center 120), FL (Front Left), FR (Front Right), RL (Rear Left), RR (Rear Right), and RC (Rear Center). Among them, the specific generation scheme of the three-dimensional information can refer to the existing technical scheme, for example, the patent "Vehicle three-dimensional information detection method, device, equipment and readable storage medium" with publication number CN112036389A.

[0048] The configuration of each camera is as follows Figure 2 As shown, Figure 2The center position of the sector corresponds to the installation position of the camera. The forward standard camera angle corresponding to FC60 is 60°, the forward wide-angle camera angle corresponding to FC120 is 120°, the side rear periphery corresponding to FL and FR is considered to be 120°, the side front periphery corresponding to RR and RL is considered to be 120°, and the rear periphery corresponding to RC is considered to be 60°.

[0049] The world coordinate position of the vehicle is calculated based on the three-dimensional information of the vehicle and the camera parameters. The vehicle detection target set containing three-dimensional information and world coordinate information obtained at time T is recorded as J^T. The world coordinate position includes two values, the world horizontal coordinate and the world vertical coordinate. The horizontal coordinate is based on the center point of the front of the vehicle as the origin, the ground is the XOY plane, the right direction is the positive direction of the horizontal coordinate, and the forward direction is the positive direction of the vertical coordinate. The specific generation scheme of the world coordinate can refer to the existing technical scheme, for example, the patent "A vehicle ranging method and device" with publication number CN112099031A can be referred to.

[0050] The single-camera vehicle tracking sequence set includes a sequence set of tracking targets detected within a time period from the time when the target is initially detected to a specific time.

[0051] In this step, the single-camera vehicle tracking sequence set D at time T-1 is obtained. T-1 , the single-camera vehicle tracking sequence set D at time T-1 T-1 , refers to the set of all tracking targets detected in chronological order from the time when the target is initially detected to the time T-1.

[0052] In this step, when the single-camera vehicle tracking sequence set D at time T-1 is obtained T-1 And the vehicle detection target set J corresponding to time T T Later, for the single-camera vehicle tracking sequence set D at time T-1 T-1 And the vehicle detection target set J at time T T , use SORT target tracking method (Simple Online and Realtime Tracking) or other tracking methods to obtain the single camera vehicle tracking sequence set D at time T T .

[0053] In this scheme, a single-camera vehicle tracking sequence includes a sequence of all the tracking targets of the vehicle from the moment when the vehicle first detects the target to the current moment, as well as the single-camera tracking identifier of the sequence. References: Bewley A, Ge Z, Ott L, et al. Simple Online and Realtime Tracking[C] / / 2016IEEE International Conference on Image Processing(ICIP).IEEE, 2016.

[0054] Step S102: obtaining a cross-camera vehicle tracking sequence set at time T-1, wherein the cross-camera vehicle tracking sequence set includes a sequence consisting of all cross-camera tracking targets in a time period from the time when the target is initially detected to a specific time.

[0055] In this step, the cross-camera vehicle tracking sequence set G at time T-1 is obtained. T-1 , the cross-camera vehicle tracking sequence set G T-1 It includes a sequence of all cross-camera tracking targets of the vehicle from the moment the target is first detected to the current moment (T-1), the cross-camera tracking identifier of the sequence, and the set of single-camera tracking identifiers associated with the sequence.

[0056] Step S103: Obtain a relationship matrix between the single-camera vehicle tracking sequence set at time T and the cross-camera vehicle tracking sequence set at time T-1.

[0057] In this step, we first obtain the single-camera vehicle tracking sequence set D at time T T The cross-camera vehicle tracking sequence set G at time T-1 T-1 The initial relationship matrix R T .

[0058] Assume that the single-camera vehicle tracking sequence set D at time T T The number of tracking sequences in is D, and the set of cross-camera vehicle tracking sequences at time T-1 is G T-1 The number of tracking sequences in is G. Establish a single-camera vehicle tracking sequence set D at time T T The set of cross-camera vehicle tracking sequences G with T-1 time T-1 The initial relationship matrix R T , where the initial relationship matrix R T is a two-dimensional matrix of D×G, and express and The relationship coefficients, where i∈{1,2,…,D},j∈{1,2,…,G}, are the initial relationship matrix R T All relationship coefficients are initialized to kMax=999.

[0059] Then, the camera field of view coefficient, tracking identification coefficient, image feature coefficient, target category coefficient, and world coordinate coefficient of the single-camera vehicle tracking sequence at time T and the cross-camera vehicle tracking sequence at time T-1 are obtained.

[0060] In this step, the specific calculation process of the camera field of view coefficient, tracking identification coefficient, image feature coefficient, target category coefficient, and world coordinate coefficient is as follows:

[0061] About camera field of view Compute a single-camera vehicle tracking sequence one by one With cross-camera vehicle tracking sequence The camera field of view The value of is checked in the camera field of view table (Table 1), that is, the tracking sequence The row where the camera identifier of the vehicle target at time T is located and the tracking sequence The coefficient of the column position of the camera identifier of the vehicle target at time T-1.

[0062] FC60 FC120 FL FR RL RR RC FC60 0 0 0 0 0 0 1 FC120 0 0 0 0 0 0 1 FL 0 0 0 1 0 1 0 FR 0 0 1 0 1 0 0 RL 0 0 0 1 0 1 0 RR 0 0 1 0 1 0 0 RC 1 1 0 0 0 0 0

[0063] Table 1

[0064] In Table 1, 0 represents that the fields of view have an intersection, and 1 represents that the fields of view have no intersection. FC60 (FrontCenter60), FC120 (FrontCenter120), FL (FrontLeft), FR (FrontRight), RL (RearLeft), RR (RearRight), and RC (RearCenter) are camera identifiers, respectively.

[0065] About Tracking Identification Factors Compute single-camera vehicle tracking sequence one by one With cross-camera vehicle tracking sequence Tracking coefficient If the tracking sequence The single camera tracking identifier is If it appears in the associated single camera tracking identifier, then the tracking identifier coefficient Otherwise, the tracking indicator coefficient

[0066] About image feature coefficients Compute single-camera vehicle tracking sequence one by one With cross-camera vehicle tracking sequence Image feature coefficient Remember the tracking sequence The image feature vector of the vehicle target at time T is F(D), and the tracking sequence is The image feature vector of the vehicle target at time T-1 is F(G), according to the formula Calculate the image feature coefficient

[0067] About vehicle category coefficient Compute single-camera vehicle tracking sequence one by one With cross-camera vehicle tracking sequence Vehicle category coefficient If the tracking sequence The category and tracking sequence of vehicle targets at time T The category of the vehicle target at time T-1 is the same, so the category coefficient otherwise

[0068] About vehicle world coordinate coefficients Compute single-camera vehicle tracking sequence one by one With cross-camera vehicle tracking sequence The vehicle world coordinate coefficients Remember the tracking sequence The world coordinates of the vehicle target at time T are (x D ,y D ), remember the tracking sequence The world coordinates of the vehicle target at time T-1 are (x G ,y G ), based on the formula Calculate the world coordinate coefficients Among them, λ1=0.75, λ2=0.5, x thr =2,y thr =5.

[0069] Finally, the initial relationship matrix R is calculated based on the camera field of view coefficient, tracking identification coefficient, image feature coefficient, target category coefficient, and world coordinate coefficient. T The values ​​of each element in .

[0070] After obtaining the camera field of view coefficient Tracking ID coefficient Image feature coefficient Category coefficient World Coordinate Coefficient Later, based on the formula Calculations are tracked With tracking sequence The relationship coefficient The relationship coefficient is recorded as the value of the element in the relationship matrix.

[0071] Step S104: Determine whether the value of the element in the relationship matrix meets the preset condition. When the preset condition is not met, the single-camera vehicle tracking sequence corresponding to the element that meets the preset condition is added to the unmatched single-camera vehicle tracking sequence set at time T. When the preset condition is met, the single-camera vehicle tracking sequence corresponding to the element that meets the preset condition is added to the associated single-camera tracking sequence set corresponding to the cross-camera vehicle tracking sequence at time T-1.

[0072] After obtaining each far value in the relationship matrix, the value of the element in the relationship matrix is ​​judged using a preset condition, and the cross-camera vehicle tracking sequence is updated based on the judgment result.

[0073] Specifically, in this step, the preset condition is that the value of the smallest element in a row of the relationship matrix is ​​greater than 0. In this step, the relationship matrix R is traversed row by row. T , assuming the relationship matrix R T In the example, the column where the smallest element in the i-th row is located is k i ,if Then the single camera vehicle tracking sequence does not match any cross-camera vehicle tracking sequences, Represents the i-th single-camera vehicle tracking sequence, and the tracking sequence Add to the unmatched single-camera vehicle tracking sequence set UD at time T; if The tracking sequence With cross-camera vehicle tracking sequence Successful matching will track the sequence Added cross-camera vehicle tracking sequence A collection of associated single-camera tracking sequences In is the Ki-th cross-camera vehicle tracking sequence and is marked To successfully match the cross-camera vehicle tracking sequence, traverse the relationship matrix R row by row T Finally, all unmarked successfully matched cross-camera vehicle tracking sequences are added to the unmatched cross-camera vehicle tracking sequence set UG at time T. Unmarked successful matching means that the tracked target is lost at time T and needs to be deleted later.

[0074] Step S105: updating the cross-camera vehicle tracking sequences corresponding to time T-1 and time T based on the associated single-camera tracking sequence set.

[0075] In this step, based on the relationship between the element values ​​in the relationship matrix and the preset conditions, the single-camera vehicle tracking sequence is divided into an unmatched single-camera vehicle tracking sequence set UD and an associated single-camera vehicle tracking sequence set UD. And after the unmatched cross-camera vehicle tracking sequence set UG, according to the set UD, the set And the cross-camera vehicle tracking sequence of the set UG for time T-1 Cross-camera tracking sequence at time T Make updates;

[0076] Specifically, when performing the above operations on each sequence, for each cross-camera vehicle tracking sequence marked as a successful match Its associated single camera tracking sequence set MD j Contains at least one single-camera vehicle tracking sequence. According to the capture camera identifier of the tracking target at time T in the single-camera vehicle tracking sequence and the camera selection priority table (Table 2), select the associated single-camera tracking sequence set MD j The single-camera vehicle tracking sequence with the highest priority is recorded as the tracking sequence Copy Tracking Sequence The tracking target at time T (the 3D information and world coordinate system of the tracking target) is added to the cross-camera vehicle tracking sequence at time T-1 At the end. Traverse the collection MD j , if the set MD j The single-camera tracking identifier of any single-camera tracking sequence in does not exist in the cross-camera vehicle tracking sequence Add the single camera tracking identifier to the associated single camera tracking identifier set of From the associated single camera tracking identifier set, we get the cross-camera tracking sequence at time T

[0077] Camera Identifier Priority FC 1 FC120 3 FL 6 FR 7 RL 4 RR 5 RC 2

[0078] Table 2

[0079] Step S106: creating a new cross-camera vehicle tracking sequence based on the unmatched single-camera vehicle tracking sequence set.

[0080] In this step, a new cross-camera vehicle tracking sequence is created based on the unmatched single-camera vehicle tracking sequence set UD. In this process, the unmatched single-camera vehicle tracking sequence set UD is traversed. For each single-camera vehicle tracking sequence in the unmatched single-camera vehicle tracking sequence set UD, Generate a new cross-camera tracking identifier G_ID k , new cross-camera tracking identifier G_ID kThe value can be the maximum value of the existing cross-camera tracking identifier plus 1. Create a new empty cross-camera vehicle tracking sequence Will The tracking target at time T is added to the cross-camera vehicle tracking sequence At the end, we will go to the cross-camera vehicle tracking sequence Single camera tracking identifier added In the set of associated single camera tracking identifiers, let The cross-camera tracking identifier is equal to G_ID k , completing the new cross-camera vehicle tracking sequence The cross-camera vehicle tracking sequence set G at time T is created T , and finally Add to the cross-camera vehicle tracking sequence set G T-1 middle.

[0081] According to the unmatched cross-camera vehicle tracking sequence set UG, the lost cross-camera vehicle tracking sequence is deleted. The unmatched cross-camera vehicle tracking sequence set UG is traversed. For each cross-camera vehicle tracking sequence From G T-1 delete

[0082] In the technical solution disclosed in the above-mentioned embodiment of the present application, cross-camera vehicle tracking is realized on the basis of single-camera vehicle tracking, by obtaining the relationship matrix between the single-camera vehicle tracking sequence set at time T and the cross-camera vehicle tracking sequence set at time T-1, and calculating the value of each element in the relationship matrix based on the camera field of view coefficient, tracking identification coefficient, image feature coefficient, target category coefficient, and world coordinate coefficient, and then updating the cross-camera vehicle tracking sequence corresponding to time T-1 and time T based on the relationship matrix, and creating a new cross-camera vehicle tracking sequence, so that the same target vehicle can still keep the same tracking identifier when it appears in different cameras at different times.

[0083] Corresponding to the above method, the present application also discloses a target tracking device, Figure 3 For a hardware structure diagram of a target tracking device provided by an embodiment of the present invention, see Figure 3 As shown, it includes: it may include: at least one processor 100, at least one communication interface 200, at least one memory 300 and at least one communication bus 400;

[0084] In the embodiment of the present invention, the number of the processor 100, the communication interface 200, the memory 300, and the communication bus 400 is at least one, and the processor 100, the communication interface 200, and the memory 300 communicate with each other through the communication bus 400; obviously, Figure 3The communication connections shown for the processor 100, the communication interface 200, the memory 300, and the communication bus 400 are merely optional;

[0085] Optionally, the communication interface 200 may be an interface of a communication module, such as an interface of a GSM module;

[0086] The processor 100 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0087] The memory 300 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0088] The processor 100 is specifically configured to:

[0089] Based on the single-camera vehicle tracking sequence set at time T-1 and the vehicle detection target set corresponding to time T, a single-camera vehicle tracking sequence set at time T is obtained;

[0090] Get the cross-camera vehicle tracking sequence set at time T-1;

[0091] Obtain a relationship matrix between a single-camera vehicle tracking sequence set at time T and a cross-camera vehicle tracking sequence set at time T-1;

[0092] Obtain the camera field of view coefficient, tracking identification coefficient, image feature coefficient, target category coefficient, and world coordinate coefficient of the single-camera vehicle tracking sequence at time T and the cross-camera vehicle tracking sequence at time T-1;

[0093] The value of each element in the relationship matrix is ​​calculated based on the camera field of view coefficient, tracking identification coefficient, image feature coefficient, target category coefficient, and world coordinate coefficient;

[0094] Determine whether the value of the element in the relationship matrix meets the preset condition. When the preset condition is not met, add the single-camera vehicle tracking sequence corresponding to the element that meets the preset condition to the unmatched single-camera vehicle tracking sequence set at time T. When the preset condition is met, add the single-camera vehicle tracking sequence corresponding to the element that meets the preset condition to the associated single-camera tracking sequence set of the cross-camera vehicle tracking sequence at time T-1.

[0095] Update the cross-camera vehicle tracking sequence corresponding to time T-1 and time T based on the associated single-camera tracking sequence set;

[0096] A new cross-camera vehicle tracking sequence is created based on the unmatched single-camera vehicle tracking sequence set.

[0097] For the convenience of description, the above system is described as being divided into various modules according to their functions. Of course, when implementing the present invention, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0098] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0099] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0100] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0101] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0102] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A target tracking method, characterized in that: include: Based on the single-camera vehicle tracking sequence set at time T-1 and the vehicle detection target set corresponding to time T, a single-camera vehicle tracking sequence set at time T is obtained; Get the cross-camera vehicle tracking sequence set at time T-1; Obtain a relationship matrix between a single-camera vehicle tracking sequence set at time T and a cross-camera vehicle tracking sequence set at time T-1; Obtaining a relationship matrix corresponding to each single-camera vehicle tracking sequence and the cross-camera vehicle tracking sequence; Obtaining the minimum element of each row in the relationship matrix; Determine whether the value of the minimum element is less than 0. If it is greater than 0, it indicates that the single-camera vehicle tracking sequence corresponding to the minimum element does not match any cross-camera vehicle tracking sequence. The single-camera vehicle tracking sequence corresponding to the minimum element is added to the unmatched single-camera vehicle tracking sequence set at time T. If it is not greater than 0, it indicates that the preset condition is met, the single-camera vehicle tracking sequence corresponding to the minimum element matches the cross-camera vehicle tracking sequence corresponding to the minimum element, and the single-camera vehicle tracking sequence corresponding to the minimum element is added to the associated single-camera tracking sequence set of the cross-camera vehicle tracking sequence corresponding to the minimum element at time T-1; Obtain a single-camera tracking sequence set that matches the cross-camera vehicle tracking sequence that successfully matches the single-camera vehicle tracking sequence, obtain the tracking target at time T in the single-camera vehicle tracking sequence with the highest priority in the single-camera tracking sequence set according to the capture camera identifier of the tracking target at time T in the single-camera vehicle tracking sequence and the camera selection priority table, and add the tracking target at time T to the end of the cross-camera vehicle tracking sequence at time T-1; For each single-camera vehicle tracking sequence in the unmatched single-camera vehicle tracking sequence set Do the following: Generate a new cross-camera tracking identifier G_ID k ; Create an empty cross-camera vehicle tracking sequence Will The tracking target at time T is added to the cross-camera vehicle tracking sequence At the end, it will be added to the cross-camera vehicle tracking sequence Single camera tracking identifier added in In the set of associated single camera tracking identifiers, let The cross-camera tracking identifier is equal to G_ID k ; Finished creating a new cross-camera vehicle tracking sequence 2. The target tracking method according to claim 1, characterized in that: Based on the single-camera vehicle tracking sequence set at time T-1 and the vehicle detection target set corresponding to time T, before obtaining the single-camera vehicle tracking sequence set at time T, it also includes: Obtaining a vehicle detection target set corresponding to time T, wherein the vehicle detection target set includes a set of all vehicle targets detected at that time; A single-camera vehicle tracking sequence set at time T-1 is obtained, wherein the single-camera vehicle tracking sequence set includes a sequence set of tracking targets detected within a time period from a time when the target is initially detected to a specific time.

3. The target tracking method according to claim 2, characterized in that: Also includes: Obtain the image information captured by each camera at time T; Calculating the three-dimensional information of the target vehicle contained in the image information based on the image information captured at time T; Calculating the world coordinate position of the target vehicle based on the camera parameters of each camera and the three-dimensional information; Each element in each set includes at least the three-dimensional information and world coordinate information of the target vehicle corresponding to the element.

4. The target tracking method according to claim 3, characterized in that: The three-dimensional information at least includes: The object's 2D bounding box, object type, object image feature vector, and capturing camera identifier.

5. The target tracking method according to claim 1, characterized in that: Based on the single-camera vehicle tracking sequence set at time T-1 and the vehicle detection target set corresponding to time T, a single-camera vehicle tracking sequence set at time T is obtained, including: Based on the single-camera vehicle tracking sequence set at time T-1 and the vehicle detection target set corresponding to time T, the SORT target tracking method is used to obtain the single-camera vehicle tracking sequence set at time T.

6. The target tracking method according to claim 1, characterized in that: Also includes: If it is found according to the capture camera identifier of the tracking target at time T in the single-camera vehicle tracking sequence and the camera selection priority table that the single-camera tracking identifier of the single-camera tracking sequence does not exist in the associated single-camera tracking identifier set of the cross-camera vehicle tracking sequence, the single-camera tracking identifier is added to the associated single-camera tracking identifier set of the cross-camera vehicle tracking sequence to generate the cross-camera tracking sequence at time T.

7. The target tracking method according to claim 1, characterized in that: Also includes: The cross-camera vehicle tracking sequence that is not matched to the single-camera vehicle tracking sequence is added to the unmatched cross-camera vehicle tracking sequence set, and each sequence in the unmatched cross-camera vehicle tracking sequence set in each stored set is deleted.

8. A target tracking device, comprising a processor; characterized in that: The processor executes each step of the target tracking method according to any one of claims 1 to 7.

Images