Method and device for multi-target tracking, storage medium and electronic device

Abstract

The present disclosure provides a method, device, electronic device and storage medium for multi-target tracking, and relates to the field of image processing technology. The method includes: acquiring a target frame image and a second image of a frame after the target frame image; comparing the target frame image and the second image to determine the target object that appears in the target frame image and does not appear in the second image; predicting whether the target object is blocked in the second image according to the target frame image and the second image; in the case where it is predicted that the target object is blocked in the second image, determining the predicted position of the target object in the second image through a Kalman filter motion model. This method can further determine whether an object that does not match the target (i.e., the target object) is blocked, instead of directly discarding it, thereby reducing tracking errors caused by missed detection or short-term occlusion, thereby improving the accuracy of target tracking.

Description

Background Art

[0002] In the application of smart transportation, target tracking algorithms play an important role in many scenarios. Among them, the accuracy and efficiency of multi-target tracking algorithms directly or indirectly affect the implementation of related applications in many fields and have become a research hotspot. Target tracking algorithms in related technologies include: multi-target tracking based on detection algorithms, end-to-end multi-target tracking based on deep learning networks, etc.

[0003] In target tracking algorithms in related technologies, accurate tracking is often not possible for target tracking scenarios with obstructions.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present disclosure is to provide a method, device, electronic device and storage medium for multi-target tracking to solve the problem of inaccurate tracking in a target tracking scenario with obstructions.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a method for multi-target tracking is provided, including: acquiring a target frame image and a second image one frame after the target frame image; comparing the target frame image and the second image to determine a target object that appears in the target frame image and does not appear in the second image; predicting whether the target object is occluded in the second image based on the target frame image and the second image; and determining a predicted position of the target object in the second image by using a Kalman filter motion model when it is predicted that the target object is occluded in the second image.

[0008] In one embodiment of the present disclosure, the step of comparing a target frame image with a second image to determine a target object that appears in the target frame image and does not appear in the second image includes: performing target detection on the target frame image and the second image to obtain a first object set detected for the target frame image and a second object set detected for the second image; comparing objects in the first object set and the second object set to determine a target object that appears in the target frame image and does not appear in the second image.

[0009] In one embodiment of the present disclosure, the step of predicting whether a target object is occluded in the second image based on a target frame image and a second image includes: determining a first prediction value of the target object in a spatial feature dimension based on the target frame image; determining a second prediction value of the target object in a response graph feature dimension based on the target frame image and the second image; fusing the first prediction value and the second prediction value to obtain an occlusion prediction quantization value of the target object in the second image; obtaining a preset occlusion threshold, and determining whether the target object is occluded in the second image based on the occlusion threshold and the occlusion prediction quantization value.

[0010] In one embodiment of the present disclosure, the step of determining a first prediction value of a target object in a spatial feature dimension based on a target frame image includes: intercepting a first area block containing the target object in the target frame image; dividing the first area block into a central block and multiple edge blocks; wherein the central block corresponds to the target object; respectively determining a local HOG feature distance between each edge block and the central block, and taking the minimum value of the multiple local HOG feature distances as the first prediction value.

[0011] In one embodiment of the present disclosure, the step of determining a second prediction value of a target object in a response map feature dimension based on a target frame image and a second image includes: determining an area in the second image that is located at the same position as a first area block as a second area block; obtaining a first response map of the first area block, obtaining a second response map of the second area block, and obtaining target frame information of the target object; and determining a second prediction value based on the first response map, the second response map, and the target frame information.

[0012] In one embodiment of the present disclosure, the step of determining the predicted position of the target object in the second image through a Kalman filter motion model includes: obtaining the current motion state vector of the Kalman filter motion model; predicting the position of the target object in the second image through the current motion state vector as the predicted position; and adjusting the weighting coefficient of each motion state change rate in the motion vector, thereby updating the current motion state vector to an updated motion state vector.

[0013] In one embodiment of the present disclosure, it also includes: acquiring a third image one frame after the second image; when no target object is obtained by target detection on the third image and it is predicted that the target object is occluded in the third image, acquiring an updated motion state vector; predicting the position of the target object in the third image by using the updated motion state vector as the predicted position of the target object in the third image.

[0014] According to another aspect of the present disclosure, there is provided an apparatus for multi-target tracking, comprising: an acquisition module for acquiring a target frame image and a second image one frame after the target frame image; a target object determination module for comparing the target frame image and the second image to determine a target object that appears in the target frame image and does not appear in the second image; an occlusion prediction module for predicting whether the target object is occluded in the second image based on the target frame image and the second image; and a position prediction module for determining a predicted position of the target object in the second image by using a Kalman filter motion model when it is predicted that the target object is occluded in the second image.

[0015] According to yet another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method for multi-target tracking is implemented.

[0016] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-mentioned method for multi-target tracking by executing the executable instructions.

[0017] The method for multi-target tracking provided by the embodiment of the present disclosure can determine the target object that appears in the target frame image and does not appear in the second image by comparing the target frame image and the second image of the frame after the target frame image, and then predict whether the target object is occluded in the second image by image processing based on the target frame image and the second image, and when it is predicted that the target object is occluded in the second image, the predicted position of the target object in the second image is determined by the Kalman filter motion model for multi-target tracking. In the method provided by the present disclosure, for the object (i.e., the target object) that is not matched to the target, it can be further determined whether it is occluded, instead of being directly discarded, thereby reducing the tracking error caused by missed detection or short-term occlusion, thereby improving the accuracy of target tracking.

[0018] Furthermore, in the method for multi-target tracking provided by the embodiment of the present disclosure, when performing position prediction through the Kalman filter motion model, a method for adjusting the weighted coefficient of the motion state change rate is also provided, so as to obtain a Kalman motion state that is closer to the actual situation and make the prediction effect more accurate.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0021] Figure 1 A schematic diagram showing an exemplary system architecture to which the method for multi-target tracking according to an embodiment of the present disclosure can be applied;

[0022] Figure 2 A flowchart of a method for multi-target tracking according to an embodiment of the present disclosure is shown;

[0023] Figure 3 A flowchart of predicting whether a target object is occluded in a second image in a method for multi-target tracking according to an embodiment of the present disclosure is shown;

[0024] Figure 4 A flowchart of a method for multi-target tracking according to another embodiment of the present disclosure is shown;

[0025] Figure 5 A flowchart showing whether a target object is occluded in a second image in a method for multi-target tracking according to another embodiment of the present disclosure is shown;

[0026] Figure 6 A flowchart of updating a motion state vector in a method for multi-target tracking according to an embodiment of the present disclosure is shown;

[0027] Figure 7 A block diagram showing an apparatus for multi-target tracking according to an embodiment of the present disclosure; and

[0028] Figure 8 A structural block diagram of a computer device for multi-target tracking in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0030] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0031] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present disclosure, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0032] In view of the technical problems existing in the above-mentioned related technologies, the embodiments of the present disclosure provide a method for multi-target tracking, so as to solve at least one or all of the above-mentioned technical problems.

[0033] Figure 1 A schematic diagram showing an exemplary system architecture of a method for multi-target tracking in accordance with an embodiment of the present disclosure is shown; Figure 1 As shown:

[0034] The system architecture may include a server 101, a network 102, and a client 103. The network 102 is used to provide a medium for a communication link between the client 103 and the server 101. The network 102 may include various connection types, such as wired, wireless communication links, or optical fiber cables.

[0035] The server 101 may be a server that provides various services, such as a background management server that provides processing and analysis services for images collected by the image acquisition device. The background management server may receive and process the images collected by the image acquisition device, and feed back the processing results to the client 103.

[0036] The client 103 may be a mobile terminal such as a mobile phone, a game console, a tablet computer, an e-book reader, smart glasses, smart home devices, AR (Augmented Reality) devices, VR (Virtual Reality) devices, etc., or the client 103 may also be a personal computer, such as a laptop computer and a desktop computer.

[0037] In some optional embodiments, the server 101 may obtain the target frame image and a second image one frame after the target frame image; the server 101 may compare the target frame image and the second image to determine the target object that appears in the target frame image and does not appear in the second image; the server 101 may predict whether the target object is obscured in the second image based on the target frame image and the second image; the server 101 may determine the predicted position of the target object in the second image through a Kalman filter motion model when it is predicted that the target object is obscured in the second image.

[0038] The client 103 can provide the operator with functions such as data display, data query, and data export. For example, after the server 101 determines the predicted position of the target object, the predicted position can be displayed in the interface of the client 103, and the trajectory of the target object can also be displayed based on the predicted position for the operator to use.

[0039] It should be understood that Figure 1 The number of clients, networks and servers is merely illustrative, and the server 101 may be a physical server, a server cluster consisting of multiple servers, or a cloud server. It may have any number of clients, networks and servers according to actual needs.

[0040] Below, each step of the method for multi-target tracking in the exemplary embodiment of the present disclosure will be described in more detail with reference to the accompanying drawings and embodiments.

[0041] Figure 2 A flowchart of a method for multi-target tracking according to an embodiment of the present disclosure is shown. The method provided by the embodiment of the present disclosure can be performed as follows: Figure 1 The process is executed in the server or client shown in the figure, but the present disclosure is not limited to this.

[0042] In the following examples, the server cluster 101 is used as the execution subject for example description.

[0043] like Figure 2 As shown, the method for multi-target tracking provided by the embodiment of the present disclosure may include the following steps:

[0044] Step S201, acquiring a target frame image and a second image one frame after the target frame image;

[0045] Step S203, comparing the target frame image and the second image to determine a target object that appears in the target frame image but does not appear in the second image;

[0046] Step S205, predicting whether the target object is blocked in the second image according to the target frame image and the second image;

[0047] Step S207: when it is predicted that the target object is blocked in the second image, a predicted position of the target object in the second image is determined by using a Kalman filter motion model.

[0048] In the method for multi-target tracking provided by the present disclosure, the target object that appears in the target frame image and does not appear in the second image can be determined by comparing the target frame image and the second image of the frame after the target frame image, and then, based on the target frame image and the second image, it is predicted whether the target object is occluded in the second image by image processing, and when it is predicted that the target object is occluded in the second image, the predicted position of the target object in the second image is determined by the Kalman filter motion model for multi-target tracking. In the method provided by the present disclosure, for an object that is not matched to the target (i.e., the target object), it can be further determined whether it is occluded, rather than being directly discarded, thereby reducing tracking errors caused by missed detection or short-term occlusion, thereby improving the accuracy of target tracking.

[0049] Next, we will combine Figure 2 The steps S201 to S207 of the method for multi-target tracking in the exemplary embodiment are described in more detail.

[0050] Step S201, acquiring a target frame image and a second image one frame after the target frame image.

[0051] The image acquisition device may first acquire an image, and then obtain the target frame image and the second image therefrom. For example, a video may be shot by a camera device, and then the target frame image and the second image of a frame after the target frame image are acquired from the video.

[0052] Step S203 , comparing the target frame image and the second image to determine a target object that appears in the target frame image but does not appear in the second image.

[0053] In some embodiments, step S203 may further include: performing target detection on the target frame image and the second image to obtain a first object set detected for the target frame image and a second object set detected for the second image; comparing the objects in the first object set and the second object set to determine the target object that appears in the target frame image and does not appear in the second image.

[0054] The target frame image and the second image can be processed respectively by a target detection algorithm to obtain the object in the image. The target detection algorithm can be, for example, a network model such as YOLOv1, YOLOv2, or YOLOv3, which is not limited in the present disclosure.

[0055] Step S205 , predicting whether the target object is blocked in the second image according to the target frame image and the second image.

[0056] Figure 3 A flowchart of predicting whether a target object is occluded in a second image in a method for multi-target tracking according to an embodiment of the present disclosure is shown. Figure 3 As shown, Figure 2 Step S205 in the embodiment may further include the following steps S301 to S307:

[0057] Step S301, determining a first prediction value of a target object in a spatial feature dimension according to a target frame image.

[0058] In some embodiments, step S301 may further include: intercepting a first area block containing a target object in the target frame image; dividing the first area block into a central block and multiple edge blocks; wherein the central block corresponds to the target object; respectively determining the local HOG feature distance between each edge block and the central block, and taking the minimum value of the multiple local HOG feature distances as the first prediction value.

[0059] Step S303: determining a second prediction value of the target object in the feature dimension of the response graph according to the target frame image and the second image.

[0060] In some embodiments, step S303 may further include: determining the area in the second image that is located at the same position as the first area block as the second area block; obtaining a first response map of the first area block, obtaining a second response map of the second area block, and obtaining target frame information of the target object; determining a second prediction value based on the first response map, the second response map and the target frame information.

[0061] Step S305: fusing the first prediction value and the second prediction value to obtain an occlusion prediction quantized value of the target object in the second image.

[0062] Step S307 , obtaining a preset occlusion threshold, and determining whether the target object is occluded in the second image according to the occlusion threshold and the occlusion prediction quantized value.

[0063] It can be seen that through the implementation Figure 3 The method shown can determine whether occlusion occurs from the spatial feature dimension and the response map feature dimension by means of image processing, thereby ensuring the accuracy of the judgment result.

[0064] Step S207: when it is predicted that the target object is blocked in the second image, a predicted position of the target object in the second image is determined by using a Kalman filter motion model.

[0065] In some embodiments, step S207 may further include: obtaining a current motion state vector of the Kalman filter motion model; predicting the position of the target object in the second image using the current motion state vector as a predicted position; and adjusting the weighting coefficients of the rates of change of each motion state in the motion vector, thereby updating the current motion state vector to an updated motion state vector.

[0066] Among them, the weighted change rate calculation formula can be used to determine the weighted coefficient of the change rate of each motion state in the motion vector to obtain a Kalman motion state that is closer to the actual situation, thereby further obtaining a more accurate prediction result and reducing the prediction error caused by occlusion.

[0067] In some embodiments, the method for multi-target tracking provided by the present disclosure also includes: acquiring a third image one frame after the second image; when no target object is obtained by performing target detection on the third image and it is predicted that the target object is occluded in the third image, acquiring an updated motion state vector; predicting the position of the target object in the third image through the updated motion state vector as the predicted position of the target object in the third image.

[0068] It can be seen that the Kalman filter motion model used in the present disclosure, in which the motion state vector can be updated according to whether the target object is detected in the image frame, is more suitable for real-time tracking scenarios and improves tracking accuracy.

[0069] Figure 4 A flowchart of a method for multi-target tracking according to another embodiment of the present disclosure is shown; Figure 4 As shown, the following steps S401 to S410 are included:

[0070] S401, obtain a video, determine the first frame image to be detected in the video (i.e., the target frame image), use the yolo network to perform target detection on the first frame image, and detect that the target set of objects in the first frame image is P. Use the targets in the set P to initialize the Kalman filter in the Kalman filter motion model.

[0071] S402, target detection is performed on the next frame image (i.e., the second image) of the first frame image, and the detected target set is recorded as Q. The intersection-over-union (IoU) of the targets in set Q and the targets in set P is calculated, and the targets of adjacent frames are matched, thereby determining the matched objects that appear in both the target frame image and the second image, and the target objects that appear in the target frame image but not in the second image. The calculation formula of the intersection-over-union can be:

[0072]

[0073] S403: For the matched object, the position detected in the second image may be used as the position information of the matched object in the second image.

[0074] S404, for the target object, in the present disclosure, an occlusion discrimination module can be set to predict whether the target object is occluded in the second image, that is, to determine whether the target object is occluded in the second image by image processing technology. The occlusion discrimination module can implement an occlusion adaptive evaluation algorithm for predicting whether the target object is occluded in the second image.

[0075] Figure 5 A flowchart of predicting whether a target object is occluded in a second image in a method for multi-target tracking according to another embodiment of the present disclosure is shown. Figure 5 As shown, including:

[0076] Determine a response map 501 and a first area block 502 of the target object according to a first frame image (ie, a target frame image) and a next frame image (ie, a second image);

[0077] Then, the local HOG distance between the unmatched target (i.e., the target object) and its adjacent image blocks can be calculated to obtain the spatial occlusion score 504D of the target. i .

[0078] Specifically, in some practical applications, the idea of ​​scoring the occlusion score in the spatial feature dimension can be: considering that occlusion is always caused by the environment near the target and the objects due to movement changes, and occlusion will destroy the edge and gradient information of the original detected target. The HOG feature is an operator that uses the directional density of the gradient to describe the local features of the image, so the size of the HOG feature change is used to reflect the occlusion situation. The local HOG distance (LHD, Local HOG Distance) is used as the spatial feature function to measure the size of the feature change. The calculation method of LHD can be: take the unmatched target as the center block and intercept a large picture block. That is, this picture block is divided into nine patches, the target location is the center patch (center patch), and there are eight edge patches (surrouding patches) around it. For each patch in the picture block, the patch is further divided into n local A small block of size S*S is used to extract HOG features in 9 directions. Then the features of each patch can be expressed as d hog ×n local The matrix of is used to represent the LHD of the corresponding center block and edge block, then the LHD of the corresponding center block and edge block can be expressed as:

[0079]

[0080] Among them, V center and Vsur are the HOG feature matrices of the target block and edge block respectively, and Tr represents the rank of the matrix. For an occluded target, 8 LHDs can be calculated, and then the minimum distance D can be taken i As a spatial feature, the smaller the distance, the greater the possibility of occlusion and the more serious the occlusion.

[0081] The response map score 503L of the target object at the same position in the second image can also be calculated i .

[0082] Specifically, in some practical applications, the spatial feature function can measure the occlusion situation from a spatial perspective. In order to increase the robustness of the judgment result, an occlusion judgment based on the response graph is added. The idea is: the more similar the two signals are, the higher their correlation is. In a video sequence, when the response graphs of two regions in consecutive frames form a one-dimensional Gaussian distribution, it can be determined that the two targets are the same target or the similarity is higher. And the steeper the peak of the Gaussian distribution, the more accurate the positioning, and the more dispersed the edges, the higher the confidence of the prediction. When occlusion occurs, it will affect the peak and edge distribution of the target response graph. Based on this, the standard defined by the following formula can be used, that is, according to the response graph score 503L i To measure tracking performance:

[0083]

[0084] Among them, M, N are the length and width of the target box, I(x,y) max and I(x,y) min can be the maximum and minimum values ​​of the response graph, respectively. It can be defined as a weighted distance function, which can be expressed as:

[0085]

[0086] When the peak of the response graph is steeper and the edges are smaller, The larger the value, the less likely occlusion will occur. When the peak of the response graph is gentle, the edges are scattered, and the value is large, When the value is small, the target may be occluded.

[0087] Then the fusion process can be performed through the following formula to obtain the occlusion prediction quantization value B i :

[0088]

[0089] Among them, B i represents the final occlusion score, They can represent the occlusion scores obtained by spatial features and response map dimensions respectively. Can represent fusion operations.

[0090] In getting B i Afterwards, a preset occlusion threshold may be obtained, and the occlusion prediction quantization value may be compared with the preset occlusion threshold to implement fusion judgment 505 .

[0091] For example, when the occlusion prediction quantization value is less than the occlusion threshold, it is determined that the target object is occluded in the second image, that is, the result of 506 is determined;

[0092] When the occlusion prediction quantized value is greater than or equal to the occlusion threshold, it is determined that the target object is not occluded in the second image, that is, the result of 507 is determined.

[0093] S405, when it is determined that the target object is not blocked in the second image, it can be determined as missed detection, and then the response map information of the unmatched target can be used to predict the position of the unmatched target in the current frame.

[0094] S406, determining whether the number of times the target object is not detected exceeds a loss threshold, if so, executing S407, if not, executing S408.

[0095] S407: When the number of times that the target object is not detected exceeds the loss threshold, the target object may be considered lost and no longer tracked.

[0096] S408, when the number of times the target object is not detected does not exceed the loss threshold, the Kalman filter in the present disclosure may be used to predict the position of the unmatched target in the current frame.

[0097] Figure 6 A flowchart of updating the motion state vector in a method for multi-target tracking according to an embodiment of the present disclosure, and then predicting the position of the unmatched target in the current frame is shown; Figure 6 As shown, including:

[0098] S601, predicting the position of the occluded target in the current frame.

[0099] S602, correcting the weight of the rate of change of the motion state of the blocked target.

[0100] S603, obtaining a motion vector that is closer to the actual situation.

[0101] In some practical applications, when the target is first occluded, the motion vector of the traditional Kalman filter is still used in the current frame to predict the position of the target in the current frame, and the weighted rate of change calculation formula is used to update the rate of change, obtain a motion vector closer to the actual situation and save it. When the next frame of video arrives, it is determined whether the occluded target is still occluded. If the target is detected, the traditional Kalman filter method is used for prediction and tracking. If the target is still not detected, when it does not exceed the set loss threshold, the corrected motion vector is used for prediction and tracking, and the weight is continued to be corrected to obtain the latest motion state vector.

[0102] Specifically, this embodiment can use the following method to update the Kalman motion state and obtain subsequent target prediction results:

[0103] Use X = {x t ,y t ,Δx t ,Δy t} represents the Kalman filter motion state vector at time t. In adjacent frames, the update of the motion state can be achieved using the following formula:

[0104]

[0105] Considering that when updating the rate of change, the closer the distance to the occlusion occurs, the greater its impact on the rate of change, so the rate of change at the nearest moment in the above formula should be given a greater weight. In summary, in order to obtain a rate of change that is closer to reality, consider adjusting the weights of each item in the rate of change and use the weight w n Substituting the coefficients in the sum, the rate of change calculation formula is updated to:

[0106]

[0107] The weight can satisfy the following formula (the range of δ value is (0,1)):

[0108]

[0109] In the method for updating the motion state vector provided by the present disclosure, (Δx t ,Δy t ) can represent the rate of change of motion state, w n It can represent the weight of the change rate of the neighboring frames, which is used to characterize the impact of the neighboring moments before the occlusion on the prediction of the current frame: the closer to the occlusion moment, the greater the impact on the prediction.

[0110] S409, after obtaining the position (including detected and predicted) of the target object in the current frame image (ie, the second image) by the above method, update the position information of the target object in the current frame image according to the obtained position.

[0111] S410, acquiring the next frame of image, continuing target detection, and implementing a method for multi-target tracking.

[0112] It can be seen that the present disclosure implements a multi-target tracking algorithm through an optimized yolo detection network and an improved Kalman filter. Tracking and counting vehicles based on this model can improve the accuracy of recognition to achieve better multi-target tracking effects. Specifically, the present disclosure has the following advantages:

[0113] (1) When updating the motion state, the impact of each moment before the occlusion on the prediction result is taken into account, which improves the statistical accuracy and reduces the tracking error introduced by the occlusion. The target position predicted in this way is more accurate and can better reflect the real situation.

[0114] (2) The optimized algorithm has a high processing rate, combining the advantages of accuracy and speed, avoiding the problem of sacrificing rate for accuracy. It has production application value in real-time application fields such as vehicle real-time tracking and unmanned driving.

[0115] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0116] Figure 7 A block diagram of a device 700 for multi-target tracking according to an embodiment of the present disclosure is shown. Figure 7 As shown, including:

[0117] An acquisition module 701 is used to acquire a target frame image and a second image one frame after the target frame image; a target object determination module 702 is used to compare the target frame image and the second image to determine a target object that appears in the target frame image and does not appear in the second image; an occlusion prediction module 703 is used to predict whether the target object is occluded in the second image based on the target frame image and the second image; a position prediction module 704 is used to determine the predicted position of the target object in the second image through a Kalman filter motion model when it is predicted that the target object is occluded in the second image.

[0118] Figure 7 For other contents of the embodiment, reference can be made to the other embodiments mentioned above.

[0119] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0120] Figure 8 The block diagram of a computer device for multi-target tracking in an embodiment of the present disclosure is shown. It should be noted that the electronic device shown in the figure is only an example and should not bring any limitation to the function and scope of use of the embodiment of the present invention.

[0121] Refer to the following Figure 8 An electronic device 800 according to this embodiment of the present invention will be described. Figure 8 The electronic device 800 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0122] like Figure 8 As shown, the electronic device 800 is in the form of a general computing device. The components of the electronic device 800 may include but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810).

[0123] The storage unit stores program codes, which can be executed by the processing unit 810, so that the processing unit 810 performs the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 810 can perform the following steps: Figure 2 The method shown in .

[0124] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .

[0125] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0126] Bus 830 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0127] The electronic device 800 may also communicate with one or more external devices 900 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 850. Furthermore, the electronic device 800 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0128] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0129] The program product for implementing the above method according to the embodiment of the present invention may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0130] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0131] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0132] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0133] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0134] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0135] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0136] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0137] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A method for multi-target tracking, characterized in that: include: Acquire a target frame image and a second image one frame after the target frame image; Comparing the target frame image and the second image, determining a target object that appears in the target frame image and does not appear in the second image; Predicting whether the target object is blocked in the second image according to the target frame image and the second image; In the case where it is predicted that the target object is blocked in the second image, determining a predicted position of the target object in the second image by using a Kalman filter motion model; Wherein, the step of predicting whether the target object is blocked in the second image according to the target frame image and the second image comprises: Determine a first prediction value of the target object in a spatial feature dimension according to the target frame image; determine a second prediction value of the target object in a response image feature dimension according to the target frame image and the second image; perform fusion processing on the first prediction value and the second prediction value to obtain an occlusion prediction quantization value of the target object in the second image; obtain a preset occlusion threshold, and determine whether the target object is occluded in the second image according to the occlusion threshold and the occlusion prediction quantization value; The step of determining a first prediction value of the target object in a spatial feature dimension according to the target frame image comprises: A first area block including the target object is intercepted in the target frame image; the first area block is divided into a central block and a plurality of edge blocks; wherein the central block corresponds to the target object; a local HOG feature distance between each edge block and the central block is determined respectively, and a minimum value among the plurality of local HOG feature distances is used as the first prediction value; The step of determining the second predicted value of the target object in the feature dimension of the response graph according to the target frame image and the second image comprises: Determine the area in the second image that is in the same position as the first area block as the second area block; obtain a first response map of the first area block, obtain a second response map of the second area block, and obtain target frame information of the target object; determine the second prediction value according to the first response map, the second response map and the target frame information.

2. The method according to claim 1, characterized in that The step of comparing the target frame image and the second image to determine the target object that appears in the target frame image and does not appear in the second image includes: Performing target detection on the target frame image and the second image to obtain a first object set detected on the target frame image and a second object set detected on the second image; The objects in the first object set and the second object set are compared to determine the target objects that appear in the target frame image and do not appear in the second image.

3. The method according to claim 2, characterized in that The step of determining the predicted position of the target object in the second image by using a Kalman filter motion model comprises: Obtaining a current motion state vector of the Kalman filter motion model; Predicting the position of the target object in the second image by using the current motion state vector as the predicted position; and The weighted coefficients of the change rates of the motion states in the motion state vector are adjusted, thereby updating the current motion state vector to an updated motion state vector.

4. The method according to claim 3, characterized in that Also includes: Acquire a third image one frame after the second image; When the target object is not obtained by performing target detection on the third image and it is predicted that the target object is blocked in the third image, acquiring the updated motion state vector; The position of the target object in the third image is predicted by using the updated motion state vector as the predicted position of the target object in the third image.

5. A device for multi-target tracking, characterized in that: include: An acquisition module, used for acquiring a target frame image and a second image one frame after the target frame image; A target object determination module, used for comparing the target frame image and the second image, and determining a target object that appears in the target frame image and does not appear in the second image; An occlusion prediction module, configured to predict whether the target object is occluded in the second image according to the target frame image and the second image; A predicted position module, configured to determine a predicted position of the target object in the second image by using a Kalman filter motion model when it is predicted that the target object is blocked in the second image; The step of predicting whether the target object is occluded in the second image by the occlusion prediction module according to the target frame image and the second image comprises: Determine a first prediction value of the target object in a spatial feature dimension according to the target frame image; determine a second prediction value of the target object in a response image feature dimension according to the target frame image and the second image; perform fusion processing on the first prediction value and the second prediction value to obtain an occlusion prediction quantization value of the target object in the second image; obtain a preset occlusion threshold, and determine whether the target object is occluded in the second image according to the occlusion threshold and the occlusion prediction quantization value; The step of determining a first prediction value of the target object in a spatial feature dimension according to the target frame image comprises: A first area block including the target object is intercepted in the target frame image; the first area block is divided into a central block and a plurality of edge blocks; wherein the central block corresponds to the target object; a local HOG feature distance between each edge block and the central block is determined respectively, and a minimum value among the plurality of local HOG feature distances is used as the first prediction value; The step of determining the second predicted value of the target object in the feature dimension of the response graph according to the target frame image and the second image comprises: Determine the area in the second image that is in the same position as the first area block as the second area block; obtain a first response map of the first area block, obtain a second response map of the second area block, and obtain target frame information of the target object; determine the second prediction value according to the first response map, the second response map and the target frame information.

6. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method for multi-target tracking according to any one of claims 1 to 4 is implemented.

7. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the method for multi-target tracking as described in any one of claims 1 to 4.

Images