Multi-target tracking method, device, equipment and computer readable storage medium

Abstract

A multi-target tracking method, device, equipment and computer readable storage medium. The method comprises the following steps: first, extracting feature points by using a gradient difference method; then, determining a to-be-detected region of a front-view road image at a t-1 moment and a t moment based on the feature points; extracting features in the to-be-detected region of the front-view road image at the t-1 moment and the t moment by using a convolutional neural network respectively to obtain first feature information and second feature information; and calculating information of a target corresponding to the feature points in the front-view road image at the t-1 moment in the to-be-detected region of the front-view road image at the t moment. When the features in the image are extracted by using the convolutional neural network, the tracking of the target can be completed without extracting features from the entire overhead view, the computational amount of the convolutional neural network is greatly reduced, and the technical problem that the computational amount is large when the target is tracked in the related art and the manufacturing cost of the vehicle is high is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to a multi-target tracking method and device, equipment and a computer readable storage medium. BACKGROUND

[0002] At present, as an indispensable technology in the field of intelligent driving, multi-target tracking plays an important role in maintaining the state attributes of target objects, keeping and monitoring the time sequence information of target objects.

[0003] In related technologies, when tracking a target, in order to obtain more accurate tracking results, a neural network model is often relied on to perform convolution operation on the entire image. However, these neural network models usually involve a large number of complex mathematical calculations, so a chip with high computing power is required. The higher the computing power of the chip, the higher the price, which leads to an increase in the manufacturing cost of the vehicle. Therefore, when tracking a target, how to reduce the calculation amount of the neural network and thus reduce the manufacturing cost of the vehicle is a technical problem to be solved at present. SUMMARY

[0004] The present application provides a multi-target tracking method, device, equipment and computer readable storage medium, which can solve the technical problem of high manufacturing cost of the vehicle caused by large calculation amount when tracking a target in the prior art.

[0005] In a first aspect, the embodiments of the present application provide a multi-target tracking method, which comprises:

[0006] At t-1, a feature point in a front road image is determined by a gradient difference method;

[0007] A to-be-detected region in the front road image is determined based on the front road image and the feature point;

[0008] Features in the to-be-detected region are extracted by a convolutional neural network to obtain first feature information;

[0009] At t, the step of determining the feature point in the front road image by the gradient difference method is returned to execute to obtain second feature information, wherein t-1 and t are adjacent time points at which adjacent two frames of front road images are acquired;

[0010] Information of a target object corresponding to the feature point in the to-be-detected region of the front road image at t-1, the first feature information and the second feature information are used to obtain information of the target object corresponding to the feature point in the front road image at t-1 in the to-be-detected region of the front road image at t.

[0011] In combination with the first aspect, in an implementation, the step of determining a region to be detected in the front-view road image based on the front-view road image and the feature points comprises:

[0012] transforming the front-view road image into a top-view image, and performing grid division on the top-view image;

[0013] mapping the feature points onto the top-view image, and determining coordinates of the feature points on the top-view image after mapping;

[0014] taking a grid in which the coordinates of the mapped feature points are located as a target grid;

[0015] determining a region in which the target grid is located as the region to be detected in the front-view road image.

[0016] In combination with the first aspect, in an implementation, the step of determining the coordinates of the feature points on the top-view image after mapping comprises:

[0017] obtaining coordinates (u, v) of the feature points in the front-view road image;

[0018] substituting the coordinates (u, v) of the feature points in the front-view road image into a preset formula to calculate the coordinates of the mapped feature points, the preset formula being as follows:

[0019]

[0020] wherein, represents the coordinates of the feature points in the front-view road image, represents the coordinates of the mapped feature points on the top-view image, and Z cam represents a height of the camera from the ground, P represents a product of a camera intrinsic parameter matrix and a camera extrinsic parameter matrix, and Res represents a resolution of the top-view image in a physical space.

[0021] In combination with the first aspect, in an implementation, the first feature information comprises a feature descriptor and coordinates of a feature point in a region to be detected in a front-view road image at time t-1, the second feature information comprises a feature descriptor and coordinates of a feature point in a region to be detected in a front-view road image at time t, and when there is only one coordinate of a feature point in the region to be detected in the front-view road image at time t-1 and the region to be detected in the front-view road image at time t, the step of obtaining information of a target object corresponding to the feature point in the region to be detected in the front-view road image at time t-1 based on information of the target object corresponding to the feature point, the first feature information and the second feature information comprises:

[0022] calculate a first Euclidean distance between the feature descriptor in the to-be-detected region of the front-view road image at the time t-1 and the feature descriptor in the to-be-detected region of the front-view road image at the time t;

[0023] when the first Euclidean distance is less than a first preset value, determine that the target object corresponding to the feature point in the front-view road image at the time t-1 and the target object corresponding to the feature point in the front-view road image at the time t are the same target object;

[0024] based on the information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at the time t-1, correct the information of the target object in the to-be-detected region of the front-view road image at the time t by using a Kalman filtering algorithm, to obtain the information of the target object in the to-be-detected region of the front-view road image at the time t.

[0025] In combination with the first aspect, in an implementation manner, the first feature information includes the coordinates of the feature descriptor and the feature point in the to-be-detected region of the front-view road image at the time t-1, and the second feature information includes the coordinates of the feature descriptor and the feature point in the to-be-detected region of the front-view road image at the time t, when there are at least two coordinates of the feature point in the to-be-detected region of the front-view road image at the time t-1 and / or in the to-be-detected region of the front-view road image at the time t, the step of obtaining the information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at the time t-1 in the front-view road image at the time t-1 based on the information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at the time t-1, the first feature information and the second feature information includes:

[0026] calculate a first Euclidean distance between the feature descriptor in the to-be-detected region of the front-view road image at the time t-1 and the feature descriptor in the to-be-detected region of the front-view road image at the time t;

[0027] when the first Euclidean distance is less than a first preset value, if the number of the coordinates of the feature point in the to-be-detected region of the front-view road image at the time t-1 is less than or equal to the number of the coordinates of the feature point in the to-be-detected region of the front-view road image at the time t, select any feature point in the to-be-detected region of the front-view road image at the time t-1 as a first target feature point;

[0028] calculate a second Euclidean distance between the first target feature point and each feature point in the to-be-detected region of the front-view road image at the time t;

[0029] take the feature point in the to-be-detected region of the front-view road image at the time t corresponding to the minimum second Euclidean distance as a second target feature point;

[0030] determine that the target object corresponding to the first target feature point and the target object corresponding to the second target feature point are the same target object;

[0031] based on information of a target corresponding to the first target feature point in the to-be-detected region of the front-view road image at the t-1 moment, correcting information of the target in the to-be-detected region of the front-view road image at the t moment by a Kalman filtering algorithm to obtain information of the target in the to-be-detected region of the front-view road image at the t moment;

[0032] deleting the first target feature point from the to-be-detected region of the front-view road image at the t-1 moment, deleting the second target feature point from the to-be-detected region of the front-view road image at the t moment, and returning to execute the step of selecting any feature point in the to-be-detected region of the front-view road image at the t-1 moment as the first target feature point until all feature points in the to-be-detected region of the front-view road image at the t-1 moment are selected to obtain information of targets corresponding to feature points in the to-be-detected region of the front-view road image at the t-1 moment in the to-be-detected region of the front-view road image at the t moment.

[0033] In combination with the first aspect, in an implementation manner, after the first Euclidean distance is less than the first preset value, the method further includes:

[0034] if the number of coordinates of the feature points in the to-be-detected region of the front-view road image at the t-1 moment is greater than the number of coordinates of the feature points in the to-be-detected region of the front-view road image at the t moment, selecting any feature point in the to-be-detected region of the front-view road image at the t-1 moment as the first target feature point;

[0035] calculating second Euclidean distances between the first target feature point and each feature point in the to-be-detected region of the front-view road image at the t moment;

[0036] taking a feature point in the to-be-detected region of the front-view road image at the t moment corresponding to the minimum second Euclidean distance as the second target feature point;

[0037] determining that the target corresponding to the first target feature point and the target corresponding to the second target feature point are the same target;

[0038] based on information of a target corresponding to the first target feature point in the to-be-detected region of the front-view road image at the t-1 moment, correcting information of the target in the to-be-detected region of the front-view road image at the t moment by a Kalman filtering algorithm to obtain information of the target in the to-be-detected region of the front-view road image at the t moment;

[0039] delete the first target feature point from the to-be-detected region of the front-looking road image at the t-1 moment, delete the second target feature point from the to-be-detected region of the front-looking road image at the t moment, return to execute the step of selecting any feature point in the to-be-detected region of the front-looking road image at the t-1 moment as the first target feature point until the number of coordinates of the feature points in the to-be-detected region of the front-looking road image at the t moment is zero;

[0040] based on the information of the target object corresponding to the feature point in the to-be-detected region of the front-looking road image at the t-1 moment, the information of the target object in the to-be-detected region of the front-looking road image at the t moment is predicted by a Kalman filtering algorithm, and the information of the target object in the to-be-detected region of the front-looking road image at the t moment is obtained.

[0041] In a second aspect, the embodiments of the present application provide a multi-target tracking device, the multi-target tracking device comprises:

[0042] The first determination module is configured to determine the feature points in the front-looking road image by using a gradient difference method at the t-1 moment.

[0043] The second determination module is configured to determine the to-be-detected region in the front-looking road image based on the front-looking road image and the feature points.

[0044] The feature extraction module is configured to extract the features in the to-be-detected region by using a convolutional neural network to obtain first feature information.

[0045] The cycle module is configured to return to execute the step of determining the feature points in the front-looking road image by using the gradient difference method at the t moment to obtain second feature information, wherein the t-1 moment and the t moment are adjacent moments at which adjacent two front-looking road images are acquired.

[0046] The information tracking module is configured to obtain the information of the target object corresponding to the feature point in the to-be-detected region of the front-looking road image at the t-1 moment in the to-be-detected region of the front-looking road image at the t moment based on the information of the target object corresponding to the feature point in the to-be-detected region of the front-looking road image at the t-1 moment, the first feature information and the second feature information.

[0047] In combination with the second aspect, in an implementation manner, the second determination module is configured to:

[0048] convert the front-looking road image into a top view, and perform grid division on the top view;

[0049] map the feature points onto the top view, and determine the coordinates of the feature points on the top view after mapping;

[0050] take the grid where the coordinates of the mapped feature points are located as a target grid;

[0051] The area where the target grid is located is determined as a to-be-detected area in the front-view road image.

[0052] In a third aspect, an embodiment of the present application provides a multi-target tracking device, which comprises a processor, a memory, and a multi-target tracking program stored in the memory and executable by the processor, wherein the multi-target tracking program, when executed by the processor, implements the steps of the multi-target tracking method.

[0053] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a multi-target tracking program, wherein the multi-target tracking program, when executed by a processor, implements the steps of the multi-target tracking method.

[0054] The technical scheme provided by the embodiment of the present application has the following beneficial effects:

[0055] The feature points in the front-view road image are determined by the gradient difference method at t-1; the to-be-detected area in the front-view road image is determined based on the front-view road image and the feature points; the features in the to-be-detected area are extracted by the convolutional neural network to obtain first feature information; at t, the step of determining the feature points in the front-view road image by the gradient difference method is returned to execute, to obtain second feature information, wherein t-1 and t are adjacent time instants at which adjacent two frames of front-view road images are acquired; the information of the target corresponding to the feature points in the to-be-detected area of the front-view road image at t-1, the first feature information, and the second feature information are used to obtain the information of the target corresponding to the feature points in the front-view road image at t-1 in the to-be-detected area of the front-view road image at t, so that when the features in the image are extracted by the convolutional neural network, the tracking of the target can be completed without feature extraction on the entire top view, the computational amount of the convolutional neural network is greatly reduced, and therefore a chip with a higher calculation amount is not required, the manufacturing cost of the vehicle is reduced, and the technical problem that the calculation amount is large when the target is tracked in the related art and the manufacturing cost of the vehicle is high is solved. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 The flowchart of an embodiment of the multi-target tracking method of the present application is shown in the figure.

[0057] Figure 2 The detailed flowchart of step S20 in the embodiment of the multi-target tracking method of the present application is shown in the figure. Figure 1

[0058] Figure 3 The flowchart of determining the to-be-detected area in the multi-target tracking method of the present application is shown in the figure.

[0059] Figure 4 ​A feature point distribution diagram of a multi-target tracking method of the present application;

[0060] Figure 5 A functional module diagram of an embodiment of a multi-target tracking device of the present application;

[0061] Figure 6 A hardware structure diagram of a multi-target tracking device involved in an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor fall within the scope of protection of the present application.

[0063] The terms “include” and “have” and any variations thereof in the specification and claims of the present application and the above-described drawings are intended to cover the non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally further include steps or units not listed, or can optionally further include other steps or units inherent to the process, method, product or device. The terms “first”, “second” and “third” and the like descriptions are used to distinguish different objects, and do not represent the order or limit the types of “first”, “second” and “third”.

[0064] In the description of the embodiments of the present application, “exemplary”, “for example” or “for instance” is used to represent as an example, illustration or description. Any embodiment or design scheme described as “exemplary”, “for example” or “for instance” in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the words “exemplary”, “for example” or “for instance” are intended to present the relevant concept in a specific manner.

[0065] In the description of the embodiments of the present application, unless otherwise specified, “ / ” represents the meaning of or, for example, A / B can represent A or B; “and / or” in the text only represents a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A alone, A and B together, and B alone, in addition, in the description of the embodiments of the present application, “multiple” means two or more than two.

[0066] In some of the processes described in this application, there will be more, fewer or different steps as compared to those shown in this application, and the order in which the steps are performed can be different from that which is shown. Furthermore, the processes described in this application can be performed in parallel or sequentially.

[0067] For the purpose of making the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings.

[0068] In a first aspect, the embodiments of the present application provide a multi-target tracking method.

[0069] In an embodiment, the multi-target tracking method comprises the following steps: Figure 1 , Figure 1 The flowchart of an embodiment of the multi-target tracking method of the present application is shown in FIG. 1. As shown in FIG. 1, the multi-target tracking method comprises the following steps: Figure 1

[0070] Step S10, determining a feature point in a front-view road image at t-1 time point by using a gradient difference method;

[0071] In this embodiment, a front-view road image is obtained by using a front-view camera of a vehicle, wherein the front-view road image corresponds to a t-1 time point. The feature point in the front-view road image is calculated by using the gradient difference method, which mainly judges the pixel difference between the neighboring areas around the point. For example, the FAST corner detection algorithm is used in the gradient difference method, and the difference between the neighboring pixels around the center point is judged. If the absolute value of the pixel difference between the N continuous neighboring pixels and the center point is greater than or less than a threshold t, the center point is a feature point.

[0072] Step S20, determining a to-be-detected region in the front-view road image based on the front-view road image and the feature point;

[0073] In this embodiment, the front-view road image is converted into a top view, and the region where the feature point is located in the top view is determined as the to-be-detected region.

[0074] Step S30, extracting the features in the to-be-detected region by using a convolutional neural network to obtain first feature information;

[0075] ​In the embodiment, after the to-be-detected region in the front-view road image is determined, only the features in the to-be-detected region are extracted by the convolutional neural network to obtain the first feature information. Since the to-be-detected region in the front-view road image is the region in which the feature points are located in the bird's-eye view after the front-view road image is converted into the bird's-eye view, the convolutional neural network does not need to extract features from the entire bird's-eye view by determining the to-be-detected region in the front-view road image, thereby greatly reducing the calculation amount of the convolutional neural network.

[0076] In step S40, at time t, the step of determining the feature points in the front-view road image by the gradient difference method is executed again to obtain the second feature information, where time t-1 and time t are adjacent time points at which adjacent two front-view road images are acquired.

[0077] In the embodiment, when the next front-view road image is acquired, i.e., at time t, the step of determining the feature points in the front-view road image by the gradient difference method is executed again to obtain the second feature information, i.e., the features in the to-be-detected region in the bird's-eye view converted from the next front-view road image. Time t-1 and time t are adjacent time points at which adjacent two front-view road images are acquired.

[0078] In step S50, the information of the target object corresponding to the feature points in the to-be-detected region in the front-view road image at time t-1, the first feature information, and the second feature information are used to obtain the information of the target object corresponding to the feature points in the to-be-detected region in the front-view road image at time t.

[0079] In the embodiment, the information of the target object corresponding to the feature points in the adjacent two images is tracked, i.e., the information of the target object corresponding to the feature points in the to-be-detected region in the front-view road image at time t-1, the first feature information, and the second feature information are used to obtain the information of the target object corresponding to the feature points in the to-be-detected region in the front-view road image at time t.

[0080] In this embodiment, feature points in the forward-looking road image are determined at time t-1 using the gradient difference method; a region to be detected in the forward-looking road image is determined based on the forward-looking road image and the feature points; features in the region to be detected are extracted using a convolutional neural network to obtain first feature information; at time t, the step of determining feature points in the forward-looking road image using the gradient difference method is returned to obtain second feature information, where time t-1 and time t are adjacent times for acquiring two adjacent frames of forward-looking road images; based on the information of the target object corresponding to the feature points in the region to be detected in the forward-looking road image at time t-1, the first feature information, and the second feature information, the information of the target object corresponding to the feature points in the forward-looking road image at time t-1 in the region to be detected in the forward-looking road image at time t-1 is obtained. This allows target tracking to be completed without extracting features from the entire top view when using a convolutional neural network to extract features from the image, greatly reducing the computational load of the convolutional neural network. Consequently, a high-performance chip is not required, reducing the manufacturing cost of the vehicle. This solves the technical problem in related technologies where the large computational load during target tracking leads to high vehicle manufacturing costs.

[0081] Furthermore, in one embodiment, reference is made to Figure 2 , Figure 2 For this application Figure 1 A detailed flowchart of step S20. (See attached diagram.) Figure 2 As shown, step S20 includes:

[0082] Step S201: Convert the forward-looking road image into a top view and divide the top view into a grid.

[0083] Step S202: Map the feature points onto the top view and determine the coordinates of the mapped feature points on the top view;

[0084] Step S203: Use the grid where the coordinates of the mapped feature points are located as the target grid;

[0085] Step S204: Determine that the area where the target grid is located is the area to be detected in the forward-looking road image.

[0086] In this embodiment, refer to Figure 3 , Figure 3 This is a flowchart illustrating the process of determining the region to be detected in the multi-target tracking method of this application. Figure 3 As shown, a forward-looking road image is acquired, and then the coordinates of each pixel in the forward-looking road image are transformed based on the camera's intrinsic and extrinsic parameter matrices to obtain a top-down view. The top-down view is then divided into multiple grids. It should be noted that the size of each grid can be adjusted to obtain the optimal parameters.

[0087] The coordinates of the feature points in the front-view road image are also transformed, the feature points in the front-view road image are mapped to the top-view image, and the coordinates of the mapped feature points in the top-view image are determined.

[0088] The grid where the coordinates of the mapped feature points are located is taken as a target grid, and the area where the target grid is located is determined as the to-be-detected area in the front-view road image. Specifically, it is assumed that the left upper corner point of a grid is A(x top , y top ), the right lower corner point is D(x down , y down ), and the coordinates of the mapped feature point C are C(x c , y c ). If x top < x c < x down and y top < y c < y down , the feature point C belongs to the grid, and then it is determined that the grid is a to-be-detected area, and the flag of the grid is set to True. Otherwise, the feature point C does not belong to the grid, and the flag of the grid is set to False. By detecting the flags in multiple grids, the to-be-detected area in the front-view road image can be determined, that is, the area where the grid with the True flag is located is the to-be-detected area in the front-view road image.

[0089] Further, in an embodiment, the step of determining the coordinates of the feature points in the top-view image after mapping includes:

[0090] obtaining the coordinates (u, v) of the feature points in the front-view road image;

[0091] substituting the coordinates (u, v) of the feature points in the front-view road image into a preset formula to calculate the coordinates of the mapped feature points, and the preset formula is as follows:

[0092]

[0093] wherein, represents the coordinates of the feature points in the front-view road image, represents the coordinates of the feature points in the top-view image after mapping, Z cam represents the height of the camera from the ground, P represents the product of the camera intrinsic matrix and the camera extrinsic matrix, and Res represents the resolution of the top-view image in the physical space.

[0094] In this embodiment, the camera intrinsic matrix Kcam and the camera extrinsic matrix The coordinates (u, v) of the feature points in the front-view road image are obtained. The coordinates (u, v) of the feature points in the front-view road image are substituted into a preset formula to obtain the coordinates of the mapped feature points, and the preset formula is as follows:

[0095]

[0096] wherein, is used to represent the coordinates of the feature points in the front-view road image, is used to represent the coordinates of the feature points on the top-view image after mapping, and Z cam is used to represent the height of the camera from the ground, P is used to represent the product of the camera intrinsic matrix and the camera extrinsic matrix, and Res is used to represent the resolution of the top-view image in the physical space.

[0097] Further, in an embodiment, the first feature information includes the feature descriptors and the coordinates of the feature points in the to-be-detected region of the front-view road image at t-1, and the second feature information includes the feature descriptors and the coordinates of the feature points in the to-be-detected region of the front-view road image at t. When there is only one coordinate of the feature point in the to-be-detected region of the front-view road image at t-1 and in the to-be-detected region of the front-view road image at t, step S50 includes:

[0098] calculating a first Euclidean distance between the feature descriptors in the to-be-detected region of the front-view road image at t-1 and the feature descriptors in the to-be-detected region of the front-view road image at t;

[0099] When the first Euclidean distance is less than a first preset value, it is determined that the target object corresponding to the feature point in the front-view road image at t-1 is the same target object as the target object corresponding to the feature point in the front-view road image at t;

[0100] Based on the information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at t-1, the information of the target object in the to-be-detected region of the front-view road image at t is corrected through a Kalman filtering algorithm to obtain the information of the target object in the to-be-detected region of the front-view road image at t.

[0101] In this embodiment, the first feature information includes the feature descriptors and the coordinates of the feature points in the to-be-detected region of the front-view road image at t-1, and the second feature information includes the feature descriptors and the coordinates of the feature points in the to-be-detected region of the front-view road image at t. The feature descriptors are obtained by extracting high-discrimination information from the neighborhood region of the feature points on the image and discarding irrelevant information to simplify the representation of the image. For reference Figure 4 , Figure 4 is a schematic diagram of the distribution of feature points of the multi-target tracking method of the present application. As Figure 4As shown, Figure 4 In the grid 1, there is only one coordinate of the feature point in the to-be-detected region of the front-view road image at the time t-1 and in the to-be-detected region of the front-view road image at the time t. The first Euclidean distance between the feature descriptor in the to-be-detected region of the front-view road image at the time t-1 and the feature descriptor in the to-be-detected region of the front-view road image at the time t is calculated. When the first Euclidean distance between the feature point in the grid 1 at the time t-1 and the feature point in the grid 1 at the time t is less than a first preset value, it is determined that the target object corresponding to the feature point in the front-view road image at the time t-1 is the same target object as the target object corresponding to the feature point in the front-view road image at the time t.

[0102] Based on the information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at the time t-1, the information of the target object in the to-be-detected region of the front-view road image at the time t is corrected by a Kalman filtering algorithm α = α' + Kβ, to obtain the information of the target object in the to-be-detected region of the front-view road image at the time t. Wherein, α represents the information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at the time t-1 in the to-be-detected region of the front-view road image at the time t, α' is used to represent the detection information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at the time t-1 in the to-be-detected region of the front-view road image at the time t extracted by the convolutional neural network, K represents the Kalman gain, and β represents the information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at the time t-1, and the prediction information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at the time t-1 obtained by the Kalman filtering algorithm in the to-be-detected region of the front-view road image at the time t.

[0103] Further, when the first Euclidean distance between the feature point in the grid 1 at the time t-1 and the feature point in the grid 1 at the time t is greater than or equal to the first preset value, it is determined that the target object corresponding to the feature point in the front-view road image at the time t-1 is not the same target object as the target object corresponding to the feature point in the front-view road image at the time t.

[0104] Further, in an embodiment, the first feature information includes the coordinates of the feature descriptor and the feature point in the to-be-detected region of the front-view road image at the time t-1, and the second feature information includes the coordinates of the feature descriptor and the feature point in the to-be-detected region of the front-view road image at the time t. When there are at least two coordinates of the feature point in the to-be-detected region of the front-view road image at the time t-1 and / or in the to-be-detected region of the front-view road image at the time t, the step S50 includes:

[0105] calculate a first Euclidean distance between the feature descriptor in the to-be-detected region of the front-view road image at the time t-1 and the feature descriptor in the to-be-detected region of the front-view road image at the time t;

[0106] When the first Euclidean distance is less than a first preset value, if the number of coordinates of the feature point in the to-be-detected region of the front-view road image at the time t-1 is less than or equal to the number of coordinates of the feature point in the to-be-detected region of the front-view road image at the time t, any feature point in the to-be-detected region of the front-view road image at the time t-1 is selected as a first target feature point;

[0107] calculate a second Euclidean distance between the first target feature point and each feature point in the to-be-detected region of the front-view road image at the time t;

[0108] the feature point in the to-be-detected region of the front-view road image at the time t corresponding to the minimum second Euclidean distance is taken as a second target feature point;

[0109] determine that the target object corresponding to the first target feature point and the target object corresponding to the second target feature point are the same target object;

[0110] based on the information of the target object corresponding to the first target feature point in the to-be-detected region of the front-view road image at the time t-1, the information of the target object in the to-be-detected region of the front-view road image at the time t is corrected through a Kalman filtering algorithm, to obtain the information of the target object in the to-be-detected region of the front-view road image at the time t;

[0111] the first target feature point is deleted from the to-be-detected region of the front-view road image at the time t-1, the second target feature point is deleted from the to-be-detected region of the front-view road image at the time t, and the step of selecting any feature point in the to-be-detected region of the front-view road image at the time t-1 as the first target feature point is returned to be executed until all the feature points in the to-be-detected region of the front-view road image at the time t-1 are selected, to obtain the information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at the time t-1 in the to-be-detected region of the front-view road image at the time t.

[0112] In this embodiment, the first feature information includes the feature descriptor and the coordinates of the feature point in the to-be-detected region of the front-view road image at the time t-1, and the second feature information includes the feature descriptor and the coordinates of the feature point in the to-be-detected region of the front-view road image at the time t. The first Euclidean distance between the feature descriptor in the to-be-detected region of the front-view road image at the time t-1 and the feature descriptor in the to-be-detected region of the front-view road image at the time t is calculated.

[0113] When the first Euclidean distance is greater than or equal to the first preset value, it is determined that the target corresponding to the feature point in the front-view road image at the time t-1 and the targets corresponding to the two feature points in the front-view road image at the time t are not the same target.

[0114] When the first Euclidean distance is less than the first preset value, the method further comprises: Figure 4 Figure 4 In the grid 2, there are two feature points in the to-be-detected region (the grid 2) of the front-view road image at the time t-1, and there are three feature points in the to-be-detected region (the grid 2) of the front-view road image at the time t. Any feature point in the to-be-detected region of the front-view road image at the time t-1 is selected as a first target feature point, and a second Euclidean distance between the first target feature point and each feature point in the to-be-detected region (the grid 2) of the front-view road image at the time t is calculated. That is, three second Euclidean distances are obtained, and the minimum second Euclidean distance is selected from the three second Euclidean distances. The feature point in the to-be-detected region of the front-view road image at the time t corresponding to the minimum second Euclidean distance is taken as a second target feature point. It is determined that the target corresponding to the first target feature point and the targets corresponding to the second target feature point are the same target.

[0115] Then, based on the information of the target corresponding to the first target feature point in the to-be-detected region of the front-view road image at the time t-1, the information of the target in the to-be-detected region of the front-view road image at the time t is corrected through a Kalman filtering algorithm, and the information of the target corresponding to the first target feature point in the to-be-detected region of the front-view road image at the time t is obtained.

[0116] The first target feature point is deleted from the to-be-detected region of the front-view road image at the time t-1, and the second target feature point is deleted from the to-be-detected region of the front-view road image at the time t. The step of selecting any feature point in the to-be-detected region of the front-view road image at the time t-1 as a first target feature point is returned to be executed until all the feature points in the to-be-detected region of the front-view road image at the time t-1 are selected, and the information of the target corresponding to each feature point in the to-be-detected region of the front-view road image at the time t-1 in the to-be-detected region of the front-view road image at the time t is obtained. In the prior art, the feature point closest to the feature point in the grid 2 at the time t-1 needs to be searched in the entire image corresponding to the time t, the search range is large, and the efficiency is slow. In the embodiment, the search is only needed to be performed in the to-be-detected region of the front-view road image at the time t, the search range is greatly reduced, and the efficiency of target tracking is improved.

[0117] Further, in an embodiment, after the first Euclidean distance is less than the first preset value, the method further comprises:

[0118] ​If the number of coordinates of the feature points in the to-be-detected region of the front-view road image at the t-1 moment is greater than the number of coordinates of the feature points in the to-be-detected region of the front-view road image at the t moment, any feature point in the to-be-detected region of the front-view road image at the t-1 moment is selected as the first target feature point;

[0119] The second Euclidean distance between the first target feature point and each feature point in the to-be-detected region of the front-view road image at the t moment is calculated;

[0120] The feature point in the to-be-detected region of the front-view road image at the t moment corresponding to the minimum second Euclidean distance is taken as the second target feature point;

[0121] It is determined that the target object corresponding to the first target feature point and the target object corresponding to the second target feature point are the same target object;

[0122] Based on the information of the target object corresponding to the first target feature point in the to-be-detected region of the front-view road image at the t-1 moment, the information of the target object in the to-be-detected region of the front-view road image at the t moment is corrected through a Kalman filtering algorithm, to obtain the information of the target object in the to-be-detected region of the front-view road image at the t moment;

[0123] The first target feature point is deleted from the to-be-detected region of the front-view road image at the t-1 moment, and the second target feature point is deleted from the to-be-detected region of the front-view road image at the t moment, and the step of selecting any feature point in the to-be-detected region of the front-view road image at the t-1 moment as the first target feature point is returned to be executed until the number of coordinates of the feature points in the to-be-detected region of the front-view road image at the t moment is zero;

[0124] Based on the information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at the t-1 moment, the information of the target object in the to-be-detected region of the front-view road image at the t moment is predicted through a Kalman filtering algorithm, to obtain the information of the target object in the to-be-detected region of the front-view road image at the t moment.

[0125] In this embodiment, if the number of coordinates of the feature points in the to-be-detected region of the front-view road image at the t-1 moment is greater than the number of coordinates of the feature points in the to-be-detected region of the front-view road image at the t moment, the step of selecting any feature point in the to-be-detected region of the front-view road image at the t-1 moment as the first target feature point is continued to be referred to in the grid 3 of Figure 4 , Figure 5 In the grid 3 of the to-be-detected region of the front-view road image at the t-1 moment (grid 3), there are 3 feature points, and in the to-be-detected region of the front-view road image at the t moment (grid 3), there are 2 feature points.

[0126] select any feature point in the to-be-detected region of the front-view road image at t-1 as a first target feature point, and calculate second Euclidean distances between the first target feature point and each feature point in the to-be-detected region of the front-view road image at t. Two second Euclidean distances are obtained, and the minimum second Euclidean distance is selected from the two second Euclidean distances. The feature point in the to-be-detected region of the front-view road image at t corresponding to the minimum second Euclidean distance is taken as a second target feature point. The target object corresponding to the first target feature point and the target object corresponding to the second target feature point are determined to be the same target object.

[0127] Based on the information of the target object corresponding to the first target feature point in the to-be-detected region of the front-view road image at t-1, the information of the target object in the to-be-detected region of the front-view road image at t is corrected by using the Kalman filtering algorithm, to obtain the information of the target object in the to-be-detected region of the front-view road image at t.

[0128] The first target feature point is deleted from the to-be-detected region of the front-view road image at t-1, and the second target feature point is deleted from the to-be-detected region of the front-view road image at t. The step of selecting any feature point in the to-be-detected region of the front-view road image at t-1 as a first target feature point is returned to be executed until the number of coordinates of the feature points in the to-be-detected region of the front-view road image at t is zero.

[0129] Since the number of coordinates of the feature points in the to-be-detected region of the front-view road image at t-1 is greater than the number of coordinates of the feature points in the to-be-detected region of the front-view road image at t, that is, the feature points in the to-be-detected region of the front-view road image at t-1 are more than the feature points in the to-be-detected region of the front-view road image at t, when the number of coordinates of the feature points in the to-be-detected region of the front-view road image at t is zero, there are still unselected feature points in the to-be-detected region of the front-view road image at t-1. Based on the information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at t-1, the information of the target object in the to-be-detected region of the front-view road image at t is predicted by using the Kalman filtering algorithm, to obtain the information of the target object in the to-be-detected region of the front-view road image at t. Starting from the frame of the front-view road image at t-1, if the unselected feature point is still not selected after three frames of prediction, the feature point is deleted. It should be noted that the number of frames of prediction is only for reference in the embodiment, and can also be two frames, four frames or five frames of prediction. The actual scene application is selected as the priority, and is not limited herein.

[0130] In a second aspect, the embodiment of the present application further provides a multi-target tracking device.

[0131] In an embodiment, with reference to Figure 5 ,Figure 5 is a functional module schematic diagram of an embodiment of the multi-target tracking device. As shown in the figure, the multi-target tracking device comprises: Figure 6

[0132] a first determining module 10, configured to determine feature points in a front road image at t-1 by using a gradient difference method;

[0133] a second determining module 20, configured to determine a to-be-detected region in the front road image based on the front road image and the feature points;

[0134] a feature extraction module 30, configured to extract features in the to-be-detected region by using a convolutional neural network to obtain first feature information;

[0135] a loop module 40, configured to return to execute the step of determining the feature points in the front road image by using the gradient difference method at t to obtain second feature information, wherein t-1 and t are adjacent time instants at which adjacent two frames of the front road image are acquired;

[0136] an information tracking module 50, configured to obtain information of a target corresponding to the feature points in the to-be-detected region in the front road image at t-1 based on information of the target, the first feature information and the second feature information to obtain information of the target corresponding to the feature points in the front road image at t-1 in the to-be-detected region in the front road image at t.

[0137] Further, in an embodiment, the second determining module 20 is configured to:

[0138] convert the front road image into a top view and perform grid division on the top view;

[0139] map the feature points onto the top view and determine coordinates of the feature points on the top view after mapping;

[0140] take a grid in which the coordinates of the mapped feature points are located as a target grid;

[0141] determine a region in which the target grid is located as the to-be-detected region in the front road image.

[0142] Further, in an embodiment, the second determining module 20 is further configured to:

[0143] obtain coordinates (u, v) of the feature points in the front road image;

[0144] substitute the coordinates (u, v) of the feature points in the front road image into a preset formula to calculate the coordinates of the mapped feature points, and the preset formula is as follows:

[0145] ​

[0146] wherein, is used to represent the coordinate of the feature point in the front-view road image, is used to represent the coordinate of the feature point after mapping on the top-view image, Z cam is used to represent the height of the camera from the ground, P is used to represent the product of the camera intrinsic matrix and the camera extrinsic matrix, and Res is used to represent the resolution of the top-view image in the physical space.

[0147] Further, in an embodiment, the first feature information includes the feature descriptor and the coordinate of the feature point in the to-be-detected region of the front-view road image at the time t-1, and the second feature information includes the feature descriptor and the coordinate of the feature point in the to-be-detected region of the front-view road image at the time t. When there is only one coordinate of the feature point in the to-be-detected region of the front-view road image at the time t-1 and in the to-be-detected region of the front-view road image at the time t, the information tracking module 50 is configured to:

[0148] calculate a first Euclidean distance between the feature descriptor in the to-be-detected region of the front-view road image at the time t-1 and the feature descriptor in the to-be-detected region of the front-view road image at the time t;

[0149] when the first Euclidean distance is less than a first preset value, determine that the target object corresponding to the feature point in the front-view road image at the time t-1 and the target object corresponding to the feature point in the front-view road image at the time t are the same target object;

[0150] based on the information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at the time t-1, correct the information of the target object in the to-be-detected region of the front-view road image at the time t by using a Kalman filtering algorithm, to obtain the information of the target object in the to-be-detected region of the front-view road image at the time t.

[0151] Further, in an embodiment, the first feature information includes the feature descriptor and the coordinate of the feature point in the to-be-detected region of the front-view road image at the time t-1, and the second feature information includes the feature descriptor and the coordinate of the feature point in the to-be-detected region of the front-view road image at the time t. When there is at least one coordinate of the feature point in the to-be-detected region of the front-view road image at the time t-1 and / or in the to-be-detected region of the front-view road image at the time t, the information tracking module 50 is further configured to:

[0152] calculate a first Euclidean distance between the feature descriptor in the to-be-detected region of the front-view road image at the time t-1 and the feature descriptor in the to-be-detected region of the front-view road image at the time t;

[0153] if the number of coordinates of the feature points in the to-be-detected region of the front-view road image at the t-1 time is less than or equal to the number of coordinates of the feature points in the to-be-detected region of the front-view road image at the t time, selecting any feature point in the to-be-detected region of the front-view road image at the t-1 time as the first target feature point when the first Euclidean distance is less than a first preset value;

[0154] calculating second Euclidean distances between the first target feature point and each feature point in the to-be-detected region of the front-view road image at the t time;

[0155] taking, as the second target feature point, a feature point in the to-be-detected region of the front-view road image at the t time corresponding to the minimum second Euclidean distance;

[0156] determining that a target object corresponding to the first target feature point and a target object corresponding to the second target feature point are the same target object;

[0157] based on information of the target object corresponding to the first target feature point in the to-be-detected region of the front-view road image at the t-1 time, correcting information of the target object in the to-be-detected region of the front-view road image at the t time by using a Kalman filtering algorithm, to obtain the information of the target object in the to-be-detected region of the front-view road image at the t time;

[0158] deleting the first target feature point from the to-be-detected region of the front-view road image at the t-1 time, deleting the second target feature point from the to-be-detected region of the front-view road image at the t time, and returning to the step of selecting any feature point in the to-be-detected region of the front-view road image at the t-1 time as the first target feature point, until all the feature points in the to-be-detected region of the front-view road image at the t-1 time are selected, to obtain information of target objects corresponding to the feature points in the to-be-detected region of the front-view road image at the t-1 time in the to-be-detected region of the front-view road image at the t time.

[0159] Further, in an embodiment, the information tracking module 50 is further configured to:

[0160] if the number of coordinates of the feature points in the to-be-detected region of the front-view road image at the t-1 time is greater than the number of coordinates of the feature points in the to-be-detected region of the front-view road image at the t time, selecting any feature point in the to-be-detected region of the front-view road image at the t-1 time as the first target feature point when the first Euclidean distance is less than a first preset value;

[0161] calculating second Euclidean distances between the first target feature point and each feature point in the to-be-detected region of the front-view road image at the t time;

[0162] taking, as the second target feature point, a feature point in the to-be-detected region of the front-view road image at the t time corresponding to the minimum second Euclidean distance;

[0163] determine that the target object corresponding to the first target feature point and the target object corresponding to the second target feature point are the same target object;

[0164] correct, based on the information of the target object corresponding to the first target feature point in the to-be-detected region of the front-view road image at the t-1 moment, information of the target object in the to-be-detected region of the front-view road image at the t moment by using a Kalman filtering algorithm, to obtain the information of the target object in the to-be-detected region of the front-view road image at the t moment;

[0165] delete the first target feature point from the to-be-detected region of the front-view road image at the t-1 moment, delete the second target feature point from the to-be-detected region of the front-view road image at the t moment, and return to execute the step of selecting any feature point in the to-be-detected region of the front-view road image at the t-1 moment as the first target feature point until the number of coordinates of the feature points in the to-be-detected region of the front-view road image at the t moment is zero;

[0166] predict, based on the information of the target object corresponding to the feature point in the to-be-detected region of the front-view road image at the t-1 moment, information of the target object in the to-be-detected region of the front-view road image at the t moment by using a Kalman filtering algorithm, to obtain the information of the target object in the to-be-detected region of the front-view road image at the t moment.

[0167] Correspondingly, the functions of each module in the multi-target tracking device are the same as the steps in the multi-target tracking method, and the functions and implementation processes are not repeated here.

[0168] In a third aspect, an embodiment of the present application provides a multi-target tracking device.

[0169] Reference Figure 6 , Figure 6 FIG. 1 is a schematic diagram of a hardware structure of a multi-target tracking device according to an embodiment of the present application. In the embodiment of the present application, the multi-target tracking device can include a processor, a memory, a communication interface, and a communication bus.

[0170] The communication bus can be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0171] The communication interface includes an input / output (I / O) interface, a physical interface, and a logical interface, which are interfaces for interconnecting devices inside the AAAA device, and interfaces for interconnecting the AAAA device with other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber interface, an ATM interface, etc.; the user device can be a display (Display), a keyboard (Keyboard), etc.

[0172] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), and the like.

[0173] The processor can be a general-purpose processor, which can invoke a multi-target tracking program stored in the memory and execute the multi-target tracking method provided by the embodiments of the present application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the multi-target tracking program is invoked can refer to various embodiments of the multi-target tracking method of the present application, which will not be described here.

[0174] Those skilled in the art can understand that the hardware structure shown in the above-mentioned embodiments is not a limitation of the present application, and can include more or less components than the illustrated components, or combine certain components, or different component arrangements. ​

[0175] In a fourth aspect, the embodiments of the present application further provide a readable storage medium.

[0176] The readable storage medium of the present application stores a multi-target tracking program, wherein the multi-target tracking program is executed by the processor to implement the steps of the multi-target tracking method as described above.

[0177] The method implemented when the multi-target tracking program is executed can refer to various embodiments of the multi-target tracking method of the present application, which will not be described here.

[0178] It should be noted that the above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0179] ​Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and the necessary general hardware platform, of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a plurality of instructions for making a terminal device execute the method described in each embodiment of the present application.

[0180] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A multi-target tracking method, characterized in that, The multi-target tracking method includes: At time t-1, feature points in the forward-looking road image are determined using the gradient difference method; The region to be detected in the forward-looking road image is determined based on the forward-looking road image and the feature points; The features in the region to be detected are extracted by a convolutional neural network to obtain the first feature information; At time t, return to the step of determining the feature points in the forward-looking road image by gradient difference method to obtain the second feature information, where time t-1 and time t are adjacent times for obtaining two adjacent frames of forward-looking road images; Based on the information of the target object corresponding to the feature point in the detection area of ​​the forward-looking road image at time t-1, the first feature information, and the second feature information, the information of the target object corresponding to the feature point in the forward-looking road image at time t-1 in the detection area of ​​the forward-looking road image at time t-1 is obtained. The step of determining the region to be detected in the forward-looking road image based on the forward-looking road image and the feature points includes: The forward-looking road image is converted into a top view, and the top view is then divided into grids. Map the feature points onto the top view and determine the coordinates of the mapped feature points on the top view; The grid containing the coordinates of the mapped feature points is used as the target grid. The region where the target grid is located is determined to be the region to be detected in the forward-looking road image; The first feature information includes feature descriptors and coordinates of feature points in the region to be detected in the forward-looking road image at time t-1, and the second feature information includes feature descriptors and coordinates of feature points in the region to be detected in the forward-looking road image at time t.

2. The multi-target tracking method as described in claim 1, characterized in that, The step of determining the coordinates of the feature points on the top view after mapping includes: Obtain the coordinates of feature points in the forward-looking road image. ; The coordinates of feature points in the forward-looking road image Substituting the values ​​into the preset formula, the coordinates of the mapped feature points are calculated. The preset formula is as follows: in, Used to represent the coordinates of feature points in the forward-looking road image. Used to represent the coordinates of feature points on the top view after mapping. The value is used to represent the height of the camera above the ground, P represents the product of the camera intrinsic matrix and the camera extrinsic matrix, and Res represents the resolution of the top view in physical space.

3. The multi-target tracking method as described in claim 1, characterized in that, When both the detection region of the forward-looking road image at time t-1 and the detection region of the forward-looking road image at time t have the coordinates of only one feature point, the step of obtaining the information of the target object corresponding to the feature point in the detection region of the forward-looking road image at time t-1 in the detection region of the forward-looking road image at time t-1 based on the target object information, the first feature information, and the second feature information includes: Calculate the first Euclidean distance between the feature descriptors in the region to be detected in the forward-looking road image at time t-1 and the feature descriptors in the region to be detected in the forward-looking road image at time t; When the first Euclidean distance is less than the first preset value, it is determined that the target object corresponding to the feature point in the forward-looking road image at time t-1 is the same target object as the target object corresponding to the feature point in the forward-looking road image at time t. Based on the information of the target object corresponding to the feature points in the detection region of the forward-looking road image at time t-1, the information of the target object in the detection region of the forward-looking road image at time t is corrected by the Kalman filter algorithm, thus obtaining the information of the target object in the detection region of the forward-looking road image at time t.

4. The multi-target tracking method as described in claim 1, characterized in that, When there are coordinates of at least two feature points in the region to be detected in the forward-looking road image at time t-1 and / or in the region to be detected in the forward-looking road image at time t, the step of obtaining the information of the target object corresponding to the feature points in the region to be detected in the forward-looking road image at time t-1 in the region to be detected based on the information of the target object corresponding to the feature points in the region to be detected in the forward-looking road image at time t-1, the first feature information, and the second feature information includes: Calculate the first Euclidean distance between the feature descriptors in the region to be detected in the forward-looking road image at time t-1 and the feature descriptors in the region to be detected in the forward-looking road image at time t; When the first Euclidean distance is less than the first preset value, if the number of coordinates of feature points in the detection area of ​​the forward-looking road image at time t-1 is less than or equal to the number of coordinates of feature points in the detection area of ​​the forward-looking road image at time t, then any feature point is selected from the detection area of ​​the forward-looking road image at time t-1 as the first target feature point. Calculate the second Euclidean distance between the first target feature point and each feature point in the detection region of the forward-looking road image at time t; The feature points in the detection region of the forward-looking road image at time t corresponding to the minimum second Euclidean distance are used as the second target feature points; The target object corresponding to the first target feature point and the target object corresponding to the second target feature point are determined to be the same target object; Based on the information of the target object corresponding to the first target feature point in the detection area of ​​the forward-looking road image at time t-1, the information of the target object in the detection area of ​​the forward-looking road image at time t is corrected by the Kalman filter algorithm to obtain the information of the target object in the detection area of ​​the forward-looking road image at time t. The first target feature point is deleted from the detection area of ​​the forward-looking road image at time t-1, and the second target feature point is deleted from the detection area of ​​the forward-looking road image at time t. The process returns to the step of selecting any feature point from the detection area of ​​the forward-looking road image at time t-1 as the first target feature point, until all feature points in the detection area of ​​the forward-looking road image at time t-1 have been selected, thus obtaining the information of the target object corresponding to the feature point in the detection area of ​​the forward-looking road image at time t in the detection area of ​​the forward-looking road image at time t.

5. The multi-target tracking method as described in claim 4, characterized in that, After the first Euclidean distance is less than a first preset value, the following is included: If the number of coordinates of feature points in the region to be detected in the forward-looking road image at time t-1 is greater than the number of coordinates of feature points in the region to be detected in the forward-looking road image at time t, then any feature point in the region to be detected in the forward-looking road image at time t-1 is selected as the first target feature point. Calculate the second Euclidean distance between the first target feature point and each feature point in the detection region of the forward-looking road image at time t; The feature points in the detection region of the forward-looking road image at time t corresponding to the minimum second Euclidean distance are used as the second target feature points; The target object corresponding to the first target feature point and the target object corresponding to the second target feature point are determined to be the same target object; Based on the information of the target object corresponding to the first target feature point in the detection area of ​​the forward-looking road image at time t-1, the information of the target object in the detection area of ​​the forward-looking road image at time t is corrected by the Kalman filter algorithm to obtain the information of the target object in the detection area of ​​the forward-looking road image at time t. The first target feature point is deleted from the detection area of ​​the forward-looking road image at time t-1, the second target feature point is deleted from the detection area of ​​the forward-looking road image at time t, and the process returns to the step of selecting any feature point from the detection area of ​​the forward-looking road image at time t-1 as the first target feature point, until the number of coordinates of feature points in the detection area of ​​the forward-looking road image at time t is zero. Based on the information of the target object corresponding to the feature points in the detection region of the forward-looking road image at time t-1, the information of the target object in the detection region of the forward-looking road image at time t is predicted by the Kalman filter algorithm, thus obtaining the information of the target object in the detection region of the forward-looking road image at time t.

6. A multi-target tracking device, characterized in that, The multi-target tracking device includes: The first determining module is used to determine the feature points in the forward-looking road image at time t-1 using the gradient difference method. The second determining module is used to determine the region to be detected in the forward-looking road image based on the forward-looking road image and the feature points; The feature extraction module is used to extract features from the region to be detected using a convolutional neural network to obtain first feature information; The loop module is used to return to the step of determining the feature points in the forward-looking road image by gradient difference method at time t, and obtain the second feature information, where time t-1 and time t are the adjacent times of obtaining two adjacent frames of forward-looking road images; The information tracking module is used to obtain the information of the target object corresponding to the feature point in the front-view road image at time t-1 in the front-view road image at time t-1 in the front-view road image at time t-1 based on the information of the target object corresponding to the feature point in the front-view road image at time t-1, the first feature information, and the second feature information. The second determining module is used for: The forward-looking road image is converted into a top view, and the top view is then divided into grids. Map the feature points onto the top view and determine the coordinates of the mapped feature points on the top view; The grid containing the coordinates of the mapped feature points is used as the target grid. The region where the target grid is located is determined to be the region to be detected in the forward-looking road image; The first feature information includes feature descriptors and coordinates of feature points in the region to be detected in the forward-looking road image at time t-1, and the second feature information includes feature descriptors and coordinates of feature points in the region to be detected in the forward-looking road image at time t.

7. A multi-target tracking device, characterized in that, The multi-target tracking device includes a processor, a memory, and a multi-target tracking program stored in the memory and executable by the processor, wherein when the multi-target tracking program is executed by the processor, it implements the steps of the multi-target tracking method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a multi-target tracking program, wherein when the multi-target tracking program is executed by a processor, it implements the steps of the multi-target tracking method as described in any one of claims 1 to 5.

Images