A target tracking method, device, system, terminal device and storage medium

Abstract

The present application provides a target tracking method, device, system, terminal device and storage medium, which are applicable to the field of computer technology. The method includes: obtaining historical position information of each object to be tracked; screening out target objects that do not match the initial trajectory from the objects to be tracked according to the matching relationship between the first position information in the historical position information of each object to be tracked and the initial trajectory obtained by fitting the second position information; obtaining the current position information of the target object; determining the target trajectory of the target object based on the initial trajectory obtained by fitting the second position information of the target object and the current position information of the target object, so as to perform target tracking according to the target trajectory, thereby enabling the object with tracking failure to establish a connection with the corresponding initial trajectory, enabling the object with tracking failure to be successfully tracked again, and thus improving the recognition accuracy of tracking.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and particularly relates to a target tracking method, device, system, terminal device, and storage medium. Background Art

[0002] Moving target tracking belongs to the content of computer image analysis and processing, which includes processing an image sequence to study the laws of moving targets, or providing semantic and non-semantic information support for the decision-making and alarm of the system, including motion detection, target classification, target tracking, behavior understanding, event detection, etc. As an important branch in the field of computer image processing, the research and application of target tracking methods are increasingly widely applied to various fields of science and technology, national defense construction, aerospace, medicine and health, and the national economy. Therefore, researching target tracking technology has great practical value and broad development prospects.

[0003] In order to achieve the tracking of moving targets, when the existing tracking algorithms perform tracking, they will regard the tracking object as moving at a constant speed, calculate the speed and forward direction of the tracking object based on the historical motion trajectory of the tracking object, predict the current position of the tracking object based on the above information, and then match and detect the target near the predicted current position of the predicted tracking object, so as to achieve the tracking of the target.

[0004] However, during a robot competition, since there are multiple robots on both the enemy and friendly sides, when the existing tracking algorithms perform tracking, they may identify the same tracking object as multiple different tracking objects, thus introducing a new tracking target, and at this time, it is impossible to determine the camp of the tracking target, and manual secondary screening of the tracking target is required to determine its specific information. Therefore, the recognition accuracy during tracking is not high and manual assistance is needed. Summary of the Invention

[0005] In view of this, the embodiments of this application provide a target tracking method, which can solve the problem of low recognition accuracy caused by identifying the same tracking object as multiple different tracking objects during tracking in the prior art.

[0006] The first aspect of the embodiments of this application provides a target tracking method, including:

[0007] Obtain the historical position information of each object to be tracked;

[0008] According to the matching relationship between the first position information in the historical position information of each object to be tracked and the initial trajectory fitted from the second position information, screen out the target objects that do not match the initial trajectory from the objects to be tracked;

[0009] Obtain the current position information of the target object;

[0010] Determine the target trajectory of the target object based on the initial trajectory fitted from the second position information of the target object and the current position information of the target object, so as to perform target tracking according to the target trajectory.

[0011] The second aspect of the embodiments of the present application provides a target tracking device, including:

[0012] A historical position information acquisition module, configured to acquire the historical position information of each object to be tracked;

[0013] A target object screening module, configured to screen out target objects that do not match the initial trajectory from the objects to be tracked according to the matching relationship between the first position information in the historical position information of each object to be tracked and the initial trajectory fitted from the second position information;

[0014] A current position information acquisition module, configured to acquire the current position information of the target object; and

[0015] A target trajectory determination module, configured to determine the target trajectory of the target object based on the initial trajectory fitted from the second position information of the target object and the current position information of the target object, so as to perform target tracking according to the target trajectory.

[0016] The third aspect of the embodiments of the present application provides a target tracking system, including:

[0017] The target tracking device as described in the second aspect above; and an information acquisition device that communicates with the target tracking device, where the information acquisition device is configured to acquire the position information of each object to be tracked and send the position information of each object to be tracked to the target tracking device.

[0018] The fourth aspect of the embodiments of the present application provides a terminal device, where the terminal device includes a memory and a processor, and a computer program that can run on the processor is stored on the memory, and when the processor executes the computer program, the steps of the target tracking method as described in any one of the first aspects above are implemented.

[0019] The fifth aspect of the embodiments of the present application provides a computer-readable storage medium, including: a computer program is stored, and it is characterized in that when the computer program is executed by a processor, the steps of the target tracking method as described in any one of the first aspects above are implemented.

[0020] The beneficial effects of the embodiments of the present application compared with the prior art are:

[0021] Based on the matching relationship between the first position information in the historical position information of each object to be tracked and the initial trajectory obtained by fitting the second position information, determine the objects for which tracking fails and the corresponding initial trajectories. Then, obtain the current position information of the objects for which tracking fails, and fit the current position information of the objects for which tracking fails into the corresponding initial trajectories to obtain the current trajectories of the objects for which tracking fails, so as to establish a connection between the objects for which tracking fails and the corresponding initial trajectories, enabling the objects for which tracking fails to be successfully re-tracked at the current moment and not being recognized as a new tracking target, thereby improving the recognition accuracy during tracking. Description of the Drawings

[0022] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following-described drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0023] Figure 1 is an application environment diagram of a target tracking method provided by an embodiment of the present application;

[0024] Figure 2 is a flowchart of the steps of a target tracking method provided by an embodiment of the present application;

[0025] Figure 3 is a flowchart of the steps of a method for screening target objects provided by an embodiment of the present application;

[0026] Figure 4 is a flowchart of the steps of a method for determining the first trajectory and the second trajectory provided by an embodiment of the present application;

[0027] Figure 5 is a flowchart of the steps of a method for obtaining the current objects to be tracked provided by an embodiment of the present application;

[0028] Figure 6 is a flowchart of the steps of a method for determining target objects provided by an embodiment of the present application;

[0029] Figure 7 is a flowchart of the steps of a method for determining target trajectories provided by an embodiment of the present application;

[0030] Figure 8 is a structural block diagram of a target tracking device provided by an embodiment of the present application;

[0031] Figure 9 is an internal structural block diagram of a terminal device provided by an embodiment of the present application. Detailed Embodiments

[0032] In the following description, specific details such as specific system architectures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0033] As Figure 1 shown, in one embodiment, a target tracking system is proposed, which includes a target tracking device 110 and an information collection device 120 that communicates with the target tracking device. The information collection device is used to collect the position information of each object to be tracked and send the position information of each object to be tracked to the target tracking device 110.

[0034] Among them, the target tracking device 110 can be an independent physical server or terminal, or a server cluster composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud servers, cloud databases, cloud storage, and CDN. The information collection device 120 can be several cameras installed outside the venue to monitor the competition venue. The target tracking device 110 and the information collection device can be wirelessly connected through a network, or directly connected through a data cable. The present application does not limit this here.

[0035] For example, in a robot competition, the target tracking device is a computer set near the competition venue, and the information collection device is two cameras respectively set on both sides of the competition venue. The cameras are used to monitor the competition venue to obtain venue images and are connected to the computer set in the competition venue through a network. The computer analyzes the venue images obtained by the cameras, so as to obtain the running trajectories of the robots on the venue, and then tracks the robots according to the obtained running trajectories.

[0036] As Figure 2 shown, in one embodiment, a target tracking method is proposed. In this embodiment, it is mainly illustrated by applying this method to the above-mentioned target tracking device 110. The target tracking method includes:

[0037] Step S202: Obtain the historical position information of each object to be tracked.

[0038] Among them, the historical position information of each object to be tracked can be manually input by the staff or automatically obtained by the computer. This application does not limit the acquisition method of the historical position information. For example, in a robot competition, the object to be tracked is the robot on the field. The computer set near the field parses the field images provided by the camera frame by frame to obtain the positions of each robot in each frame, which are the historical position information of each robot.

[0039] Step S204: According to the matching relationship between the first position information in the historical position information of each object to be tracked and the initial trajectory obtained by fitting the second position information, screen out the target objects that do not match the initial trajectory from the objects to be tracked.

[0040] Among them, the first position information of the historical position information of the object to be tracked is the position information of the object to be tracked at the moment of tracking failure, and the second position information is the position information of the object to be tracked before the moment of tracking failure. The moment of tracking failure is the moment when there is a tracking failure for one tracking object. Then the initial trajectory is the movement trajectory of the object to be tracked from the start moment of tracking to the moment before the moment of tracking failure. For example: in this embodiment, the historical position information of each robot is obtained by parsing the field images provided by the camera frame by frame. Suppose the tracking of a robot fails in the k-th frame, then the k-th frame is the moment of tracking failure. The first position information is the position information at the k-th frame, and the second position information is the position information from the 1st frame to the k-1th frame. The initial trajectory is the movement trajectory of the robot from the 1st frame to the k-1th frame.

[0041] Therefore, at the moment of tracking failure, for a target object to be tracked, if its current position matches the position predicted from its previous trajectory, it is considered that the tracking of the target object to be tracked is successful; if its current position does not match the position predicted from its previous trajectory, that is, the current position of the target object to be tracked is not obtained and the current position information of the target object to be tracked cannot be fitted to the corresponding trajectory to track the position at the next moment. Therefore, it is considered that the tracking of the target object to be tracked fails, and the target object to be tracked is the object with tracking failure. Since the purpose of the method proposed in this application is to track the object with tracking failure again, the object with tracking failure is the target object of this application. Taking a robot competition as an example, in the k-th frame, the predicted position obtained according to the movement trajectory of the robot from the 1st frame to the k-1th frame is x. If the robot is detected at the position x of the field image in the k-th frame, or among the position information of the robot obtained by performing target detection on the field image in the k-th frame, there is a robot whose degree of coincidence with the position x is higher than the preset threshold, it is considered that at the k-th moment, the tracking of the robot is successful; if the robot is not detected at the position x of the field image in the k-th frame and among the detected position information of the robot, there is no robot whose degree of coincidence with the position x is higher than the threshold, at this moment, it is considered that the tracking of the robot fails, and the robot is the target object.

[0042] Step S206, obtain the current position information of the target object.

[0043] Among them, the current position information of the target object, that is, the current position information of the object with tracking failure, can be manually input by the staff or the object with tracking failure can actively upload its own positioning information. For example, in a robot competition, since the cameras set outside the field can capture the positions of all robots on the field at the current moment, the position information of all robots on the field can be obtained by performing target detection on the field image of the current frame, and then these position information can be screened to find the position information of the robot most likely to have tracking failure.

[0044] Step S208, determine the target trajectory of the target object based on the initial trajectory fitted from the second position information of the target object and the current position information of the target object, so as to perform target tracking according to the target trajectory.

[0045] Among them, after determining the current position of the object with tracking failure, adding the current position of the object with tracking failure to the corresponding initial trajectory can obtain the motion trajectory of the object with tracking failure from the start of tracking to this moment. Therefore, the object with tracking failure will not be recognized as a different tracking object, thereby improving the recognition accuracy of tracking and enabling subsequent tracking of the object with tracking failure based on its motion trajectory from the start of tracking to this moment. In a robot competition, after corresponding the robot with tracking failure to its corresponding initial trajectory, its camp can be known. After adding the current position information of the robot with tracking failure to the corresponding initial trajectory, its motion trajectory can be obtained, which is also beneficial for our robot to respond.

[0046] In one embodiment, as Figure 3 shown, step S204 includes:

[0047] Step S302, determine the first trajectory and the second trajectory according to the matching relationship between the first position information in the historical position information of each object to be tracked and the initial trajectory fitted from the second position information.

[0048] Among them, the first trajectory is the initial trajectory of the object with successful tracking, that is, the position information of the object at the moment of tracking failure is consistent with the position information predicted according to the trajectory of the object before the moment of tracking failure. Therefore, in the following text, the first trajectory will be referred to as the trajectory of successful tracking. The second trajectory is the initial trajectory of the object with tracking failure, that is, the position information of the object at the moment of tracking failure is inconsistent with the position information predicted according to the trajectory of the object before the moment of tracking failure. Therefore, in the following text, the second trajectory will be referred to as the trajectory of tracking failure. Classifying the trajectories into two categories of successful tracking and tracking failure is convenient for using the trajectory information of the trajectory of successful tracking to screen the detected objects obtained from the current field image, so as to obtain potential candidate objects.

[0049] Step S304, determine the current position information of the tracking object corresponding to the first trajectory,

[0050] Among them, in this embodiment, the trajectory is obtained after analyzing the site images provided by the camera. Since the frame rate of the camera is constant, that is, the time interval between the front and rear frame site images is constant, therefore, the movement direction and movement speed of the corresponding tracking object can be obtained from the trajectory, and then the possible position of the tracking object in the next frame of the site image can be predicted. In a robot competition, due to the site limitations and obstacles on the site, the movement speed of the robot will not be very fast, so it can be regarded as a uniform motion. After analyzing the front and rear frame site images to obtain the forward speed and forward direction of the robot, combined with the trajectory of the robot, the current position information of the robot can be predicted.

[0051] Step S306: Obtain the current position information of each object to be tracked.

[0052] Among them, the current position information of each object to be tracked can be manually input by the staff or obtained after the object to be tracked uploads its own positioning information. In this embodiment, since the camera set outside the site can capture all the robots on the site, after performing object detection on the current frame of the site image, the robots in the image can be recognized, and then their actual position information can be determined according to their position information in the site image.

[0053] Step S308: According to the current position information of each object to be tracked and the current position information of the tracking object corresponding to the first trajectory, screen out the objects to be tracked that do not match the first trajectory;

[0054] Among them, as described above, obtain the movement direction and movement speed of the corresponding tracking object from the successfully tracked trajectory, and then regard the tracking object as a uniformly advancing object, and the position of the tracking object at the next moment can be predicted according to the above information. The position at the next moment can be predicted in the following way.

[0055] The state of the object to be tracked is: x=(u, v, s, r, u', v', s'), where u and v represent the coordinates of the object to be tracked in the image; s represents the area of the object to be tracked in the image; r represents the aspect ratio of the object to be tracked box in the image; u', v', s' represent the speeds of the above parameters.

[0056] The spatial state equation of the object to be tracked is: x' k＝ Ax k-1 +W k where x' k is the predicted position at time k; x k-1 is the corrected position at time k-1, that is, the corrected value after prediction; A is the state transition matrix; W kis the process noise. In this embodiment, the process noise does not change over time.

[0057] The observation method of the object to be tracked is: y k = Cx' k + V k , where y k is the detection position of the object to be tracked obtained by target detection at time k; x' k is the predicted position at time k; C is the conversion matrix for converting the state to the output; V k is the observation noise. In this embodiment, the process noise does not change over time.

[0058] A state space model is established, and the position of the object to be tracked at time k can be predicted based on the corrected position at time k-1:

[0059] x' k = Ax k-1 ;

[0060] Based on the error covariance P k-1 at time k-1 and the variance Q of the process noise, the new error P' at time k is predicted k :

[0061] P' k = AP k-1 A T + Q;

[0062] Based on the predicted new error P' k and the observation noise V k the gain K k is calculated:

[0063] K k = P' k C T (CP k C T + V k ) -1 ;

[0064] Based on the predicted position x' at time k k and the detection position y at the current time k , the position correction value x at time k can be obtained k ,

[0065] y k = Cx k + V k ;

[0066] x k = x' k + K k (yk -C x' k );

[0067] Update the error covariance to obtain P k value, to prepare for predicting the new error covariance at time k+1:

[0068] P k =(I-K k C)P' k ;

[0069] I is the identity matrix.

[0070] Then, update the iteration number k = k + 1, and repeat the above steps to start the calculation at time k+1.

[0071] In the above processing, since the possible noise in the processing is considered and the adjustment is made according to the actual position at the previous moment, the predicted position is more accurate.

[0072] In this embodiment, since the position information of the detected object obtained from the site image is a rectangular detection frame with an angle, that is, the position information at the previous moment is a rectangular frame with an angle, the predicted position information is also a rectangular prediction frame with an angle. Therefore, the intersection over union (IoU) between the rectangular detection frame with an angle of each detected object and the rectangular prediction frame with an angle predicted according to the successfully tracked trajectory can be calculated, that is, the ratio of the intersection of the two to the union of the two, to determine the matching degree between the two. Specifically, calculate the IoU between the current position information of each detected object and the predicted position information of each trajectory based on the successfully tracked trajectory as the value matrix, and assign the detected object to the successfully tracked trajectory according to the following steps, that is, determine the successfully tracked objects among the detected objects.

[0073] Set the predicted positions obtained according to each successfully tracked trajectory as set X, and set the positions of each detected object as set Y. Then, assigning the detected object to the successfully tracked trajectory is essentially assigning the objects in set Y to the objects in set X, and making as many objects in set X have matching results as possible. Among them, the objects in set Y can only be matched with one object in set X. If the IoU between the object x1 in set X and y1 in set Y is greater than the preset threshold (this IoU is recorded in the above value matrix), that is, y1 can be assigned to x1, otherwise, y1 cannot be assigned to x1. Then, perform the assignment according to the following steps:

[0074] 1. Find the object y1 that the current object x1 can match. If the object y1 has been matched, go to step 3; otherwise, go to step 2.

[0075] 2. Denote the matching object of the object y1 as the matching object of the current object x1, and go to step 6.

[0076] 3. Find the object x2 that the object y1 has already matched, and check whether x2 can match another object y2. If it can, go to step 4; otherwise, go to step 5.

[0077] 4. Update the matching object of x2 to y2, and determine the matching object of y1 as x1, then go to step 6.

[0078] 5. Find the next object that can be matched under the current object x1. If there is such an object, go to step 1; otherwise, it means that there is no object that can be matched under the current object x1, and go to step 6.

[0079] 6. Go to the next object and then go to step 1.

[0080] If the predicted position successfully matches the detected object, then determine the detected object as the tracking object of the successfully tracked trajectory corresponding to the predicted position. At this time, use the initial tracking ID carried by the successfully tracked trajectory, rather than the detection ID obtained in the target detection stage, as the ID of the corresponding detected object. The initial tracking ID is the tracking ID input by the staff when starting target tracking. If the predicted position does not match the detected object, then the successfully tracked trajectory corresponding to the predicted position is not assigned a detected object. Set the trajectory status of the successfully tracked trajectory to tracking failure, and wait for the next moment to match a detected object for it.

[0081] After determining the objects with successful tracking among the detected objects, use the objects with successful tracking to detect and screen the remaining detected objects, including: deleting the detected objects whose degree of coincidence between the current position information and the predicted position information obtained based on the first trajectory is higher than the preset threshold; deleting the detected objects whose degree of coincidence between the current position information and the current position information of the detected object corresponding to the first trajectory is higher than the preset threshold.

[0082] Among them, calculate the intersection over union (IoU) between the current position information of each remaining detected object and the predicted position obtained based on the successfully tracked trajectory, that is, the IoU between the rectangular detection box of each remaining detected object and the rectangular prediction box obtained based on the successfully tracked trajectory, and delete the detected objects whose IoU is higher than the preset threshold. This threshold can be obtained according to the experience of the staff in this field, and this application does not limit this threshold here. Preferably, this threshold is 0.3. If the IoU is higher than the threshold, it means that the corresponding detected object overlaps too much with the successfully tracked trajectory, and it may be that the object to be tracked is recognized as two detected objects in the target detection stage.

[0083] Then, calculate the intersection over union (IoU) between the current position information of each remaining detection object and the current position information of the tracking objects that have been successfully tracked, that is, the IoU between the rectangular detection boxes of each remaining detection object and the rectangular detection boxes of each successfully tracked object, and delete the detection objects with an IoU higher than a preset threshold. This threshold can be obtained based on the experience of those skilled in the art, and this application does not limit this threshold. Preferably, this threshold is 0.2. If the IoU is higher than the threshold, it indicates that the corresponding detection object overlaps too much with the successfully tracked object, and it may be that during the object detection stage, the object to be tracked is recognized as two detection objects.

[0084] Through the above steps, the detection objects that overlap significantly with the successfully tracked objects among the detection objects are deleted, so that there is no connection between the remaining detection objects and the successfully tracked trajectories, which is convenient for determining the tracking objects corresponding to the trajectories with tracking failures from the remaining detection objects.

[0085] Step S310: Screen out target objects from the objects to be tracked that do not match the first trajectory according to the current position information of the objects to be tracked that do not match the first trajectory and the second trajectory.

[0086] Among them, according to the distance between the current position information of the remaining detection objects and the trajectories with tracking failures, the tracking objects corresponding to the trajectories with tracking failures are determined among the remaining detection objects, so as to complete the re - tracking of the objects with tracking failures.

[0087] In one embodiment, as Figure 4 shown, step S302 includes:

[0088] Step S402: Obtain the motion direction information and motion speed information of each object to be tracked.

[0089] Among them, since the trajectory is obtained after analyzing the site images provided by the camera, and the frame rate of the camera is constant, thus, the motion direction information and motion speed information of the corresponding tracking object can be directly obtained from the trajectory.

[0090] Step S404: Determine the predicted position information corresponding to each object to be tracked according to the motion direction information and motion speed information of each object to be tracked and the initial trajectory fitted from the second position information in the historical position information of each object to be tracked.

[0091] After obtaining the motion direction information and motion speed information of the object to be tracked, combined with the corresponding trajectory of the object to be tracked, and since the time interval for the camera to acquire the site image is short, the object to be tracked can be regarded as moving at a constant speed. Therefore, the predicted positions of each object to be tracked at the next moment can be calculated to classify the objects to be tracked into successfully tracked objects and failed tracked objects.

[0092] Step S406: Determine whether the degree of coincidence between the predicted position information corresponding to each object to be tracked and the first position information in the historical position information of each object to be tracked is higher than a preset coincidence threshold. If so, determine the initial trajectory of the object to be tracked with the degree of coincidence between the predicted position information and the first position information higher than the preset coincidence threshold as the first trajectory; if not, determine the initial trajectory of the object to be tracked with the degree of coincidence between the predicted position information and the first position information not higher than the preset coincidence threshold as the second trajectory.

[0093] Among them, the degree of coincidence between the predicted position of the object to be tracked and the position at the moment of tracking failure can be determined by calculating the intersection over union between them. This coincidence threshold can be obtained through the relevant experience of those skilled in the art, and this application does not limit this coincidence threshold.

[0094] When the degree of coincidence between the position information of the object to be tracked at the moment of tracking failure and the corresponding predicted position information is high, it can be regarded that the object to be tracked is successfully tracked at this moment. Therefore, the object to be tracked is regarded as a successfully tracked object, and the trajectory corresponding to the object to be tracked is regarded as a successfully tracked trajectory; when the degree of coincidence between the position information of the object to be tracked at the moment of tracking failure and the corresponding predicted position information is lower than the preset coincidence threshold, it means that the object to be tracked is not successfully tracked at this moment. Therefore, the object to be tracked is regarded as a failed tracked object, and the trajectory corresponding to the object to be tracked is regarded as a failed tracked trajectory. The trajectories are divided into successfully tracked trajectories and failed tracked trajectories, which is convenient for screening the current detected objects using the successfully tracked trajectories, and thus convenient for determining the tracked objects corresponding to the failed tracked trajectories.

[0095] In one embodiment, as Figure 5 shown, step S306 includes:

[0096] Step S502: Obtain the current site information.

[0097] Among them, the venue information includes the venue image, the position coordinates of the obstacles set in the competition venue, and the size information of the competition venue. The above venue information can be obtained by other devices or manually input by the staff. Preferably, the venue image is provided by a camera set outside the competition venue, and the position coordinates of the obstacles in the competition venue and the size information of the competition venue are manually input by the staff.

[0098] Step S504: Convert the venue image into a bird's-eye view of the venue according to the coordinate information of the fixed objects in the venue.

[0099] Among them, according to the position coordinates of the obstacles in the competition venue, the venue image is converted into a bird's-eye view of the venue through homography transformation, which is convenient for target detection of the bird's-eye view of the venue to obtain the objects to be tracked on the venue.

[0100] Step S506: Adjust the resolution of the bird's-eye view of the venue according to the venue size information.

[0101] Among them, the bird's-eye view of the venue is adjusted according to the actual size of the competition venue, so that the length and width of the bird's-eye view of the venue correspond to the length of the competition venue. For example, if the venue is 808 cm * 448 cm, the picture size is set to 808 pixels * 448 pixels. In this way, the picture coordinates can directly correspond to the venue coordinates, and there is no need to convert the picture coordinates into venue coordinates, which simplifies the processing steps.

[0102] Step S508: Obtain the current position information of each object to be tracked from the adjusted bird's-eye view of the venue.

[0103] Among them, the bird's-eye view of the venue is input into the YOLO (You Only Look Once) V2 detection network for detection to identify the robots in the bird's-eye view of the venue. Then, the attributes of each detected robot are obtained, that is, the classification of the detection object, the current detection ID, the position information, and the rotation angle of the detection object, which are used to determine the angle of the detection frame. Preferably, the color of the detection object is also obtained to distinguish between friendly and enemy robots.

[0104] The YOLO V2 detection network includes four parts, namely: 1*1 convolution, 3*3 convolution, 5*5 convolution, 3*3 max pooling, and finally the operation results of the four parts are combined. Information of different scales of the image to be detected is extracted through multiple convolutional kernels and finally fused, so as to better represent the image to be detected.

[0105] Input the image to be detected into the YOLO V2 detection network, then perform convolution operations on it to obtain the feature maps of the last layer with dimensions of 20*20 and 40*40. Then, extract anchor boxes on each grid of 20*20 and 40*40, and output the anchor boxes with probabilities exceeding the threshold, which are the robot detection results of the image to be detected. In this embodiment, the size of the threshold is not limited, and those skilled in the art can determine it according to needs. The feature maps of 20*20 and 40*40 are obtained, which facilitates the object recognition of large and small targets and improves the recognition accuracy.

[0106] The object detection method based on the YOLO V2 neural network provided in this embodiment improves the object detection speed while ensuring the object detection rate. It can meet both the real-time requirement of object detection and the accuracy requirement of object detection.

[0107] The training method for this YOLO V2 neural network is as follows:

[0108] Obtain multiple images as training samples; preprocess the training samples, input the preprocessed training samples into the YOLO V2 detection network, and obtain the feature map output by the last pooling layer; in the feature map, extract the object detection bounding boxes according to the K-means dimensional clustering algorithm; determine the deviation value between the object detection bounding box and the calibrated position of the object to obtain the loss value; correct the weight sizes of the parameters of each convolutional layer through backpropagation until the loss value meets the preset conditions to obtain the YOLO V2 detection network.

[0109] Among them, the images used as training samples are all bird's-eye views and include robots. The preprocessing of the sample images is as follows: mark the position, size, color, and self-rotation angle of the robots in the training samples, and equalize and denoise the training samples. Then, input the preprocessed training samples into the YOLO V2 detection network, perform convolution and pooling operations to obtain the feature maps of the last layer with dimensions of 20*20 and 40*40, and then extract the object detection bounding boxes on each grid of 20*20 and 40*40 according to the K-means dimensional clustering algorithm. Compare the object detection bounding boxes with the pre-marked positions and calculate the loss value. The calculation method of the loss value can be:

[0110]

[0111] where y (t) represents the actual position of the robot in the training sample, f(x (t)) represents the target detection bounding box output by the YOLO V2 detection network, T represents the total number of training samples, and t represents the number of training samples. The specific calculation method of the loss in this embodiment is not limited and can be determined according to needs.

[0112] The preset conditions satisfied by the loss value in this embodiment are not limited, and those skilled in the art can determine them according to needs. Backpropagating the loss value aims to obtain the optimal global parameter matrix, and then applying the multi-layer neural network to classification or regression tasks. Forward-propagate the input signal until the loss value is generated at the output, backpropagate the loss value information to correct the weight matrix of each convolutional parameter, and continuously iterate to obtain the YOLO V2 detection network. The object detection method based on the YOLO V2 neural network provided in this embodiment performs clustering through the K-means dimensional clustering algorithm, extracts the target detection bounding box, and has a short calculation time and high speed.

[0113] In one embodiment, as Figure 6 shown, step S310 includes:

[0114] Step S602, according to the termination position information of the second trajectory and the current position information of the to-be-tracked object that does not match the first trajectory, screen out candidate objects.

[0115] Among them, the termination position information of the second trajectory is the last position information of the object that fails to be tracked in its corresponding trajectory, that is, the last position information of the trajectory that fails to be tracked. Although the last position information of the trajectory will change during the tracking process, at any given moment, the last position information of the trajectory can be uniquely determined. For example, if the k-th frame is the moment when the tracking fails, then the second trajectory is the movement trajectory of the robot that fails to be tracked from the 1st frame to the k-1th frame, and the last position is the position of the robot that fails to be tracked at the k-1th frame.

[0116] Calculate the distance between the current position of the detected object screened by the successfully tracked object and the last position of the trajectory that fails to be tracked. Since the object that fails to be tracked cannot be undetected by the object detection algorithm for a relatively long period of time, the detected object with the current position information closest to the last position of the trajectory that fails to be tracked is more likely to be the tracked object corresponding to this trajectory. Therefore, the detected object with the current position information closest to the last position information of the trajectory that fails to be tracked is determined as the candidate object.

[0117] Step S604, when it is determined that the distance between the current position information of the candidate object and the termination position information of the second trajectory is lower than the preset distance threshold, then determine the candidate object as the target object.

[0118] Among them, after screening out candidate objects, if the distance between the current distance of the candidate object and the last position of the trajectory with tracking failure is higher than a preset distance threshold, it indicates that the candidate object is far from the last position of the trajectory with tracking failure, and it cannot be the tracking target corresponding to the trajectory with tracking failure. It may be an incorrect result obtained during object detection. That is, at the current moment, the tracking target corresponding to the trajectory with tracking failure has not been obtained. If the distance between the current distance of the candidate object and the last position of the trajectory with tracking failure is lower than the preset distance threshold, it indicates that the candidate object is close to the last position of the trajectory with tracking failure, and it is very likely to be the tracking object corresponding to the trajectory with tracking failure. Therefore, it is regarded as the target object, and the trajectory status of the trajectory with tracking failure is set to tracking success.

[0119] In addition, if at the current moment, the trajectory with tracking failure has not been initialized, that is, the time when the tracking object corresponding to the trajectory with tracking failure appears in the camera is lower than the preset threshold, and the position information left by it is not enough to form a trajectory, the candidate object will not be regarded as the target object.

[0120] As Figure 7 shown, in one embodiment, step S208 includes:

[0121] Step S702, when it is determined that there are multiple second trajectories for the target object, obtain the termination position information of each second trajectory.

[0122] Among them, in some special cases, such as when robots collide, there will be multiple robots at the collision position. If the number of detected objects obtained in the target detection stage is lower than the actual number of robots, there will be multiple trajectories that regard the identified robots as their tracking targets.

[0123] Step S704, calculate the distance value between the termination position information of each second trajectory and the current position information of the target object.

[0124] Among them, calculate the distance between the last position of each trajectory with tracking failure and the target object to determine the trajectory corresponding to the target object.

[0125] Step S706, determine the second trajectory with the smallest distance value between the termination position information and the current position information of the target object as the initial trajectory of the target object.

[0126] Among them, the detection object closest to the last position of the trajectory where tracking fails at the current position is more likely to be the tracking target of the trajectory where tracking fails. Therefore, the trajectory is matched with the detection object, and the states of other trajectories are updated to tracking failure (the states of other trajectories have been updated to tracking success in step S604), thereby avoiding the phenomenon of misallocation and ensuring that the tracked objects are separated by a certain distance before re-tracking. By determining the initial trajectory and the current position of the target object, the trajectory where tracking fails is successfully matched with the corresponding tracking result, realizing the re-tracking of the tracked object without introducing new tracked objects, thereby improving the recognition rate during tracking.

[0127] Preferably, in order to improve the accuracy of tracking, the error checking and correction of the tracking result can also be performed according to the self-positioning information sent by our robot and based on the known partial positioning data. When the difference between the tracking result and the actual situation is too large, other tracking results will be checked to see if there is a correct result. If there is, it means the tracking is incorrect, and the two target tracks are swapped, and at this time, correction can be performed.

[0128] Step S708: Determine the target trajectory of the target object according to the initial trajectory of the target object and the current position information of the target object, and perform target tracking according to the target trajectory.

[0129] Among them, adding the current position information of the target object to the initial trajectory of the target object can obtain the current trajectory information of the target object, and the target object can be further tracked according to this trajectory information. In addition, the corresponding tracking target for the trajectory where tracking fails is re-determined, so that it will not be regarded as a new tracked object, thereby improving the recognition rate of tracking.

[0130] Before step S702, it includes:

[0131] When it is determined that there are multiple tracking objects corresponding to the first trajectory, the current position information of each tracking object is averaged.

[0132] Among them, since there are multiple cameras in the venue, the same trajectory may match detection objects in the venue images obtained by different cameras. Therefore, in terms of position information, the average value of the corresponding multiple position information is taken, and the type and rotation angle are based on the detection result of the nearby camera. Then, the position information after the averaging process is used as the current detection position information, so that the detection position information is more appropriate for the actual coordinates of the corresponding tracking object.

[0133] In one embodiment, a target tracking device is provided, as Figure 8 shown, the target tracking device includes:

[0134] A historical location information acquisition module 810 is configured to acquire the historical location information of each object to be tracked;

[0135] A target object screening module 820 is configured to screen out target objects that do not match the initial trajectory from the objects to be tracked according to the matching relationship between the first location information in the historical location information of each object to be tracked and the initial trajectory obtained by fitting the second location information;

[0136] A current location information acquisition module 830 is configured to acquire the current location information of the target object; and

[0137] A target trajectory determination module 840 is configured to determine the target trajectory of the target object based on the initial trajectory obtained by fitting the second location information of the target object and the current location information of the target object, so as to perform target tracking according to the target trajectory.

[0138] Wherein, for the process of the target tracking device module to implement its respective functions, reference may be specifically made to the description of the foregoing embodiments, which will not be elaborated herein.

[0139] In one embodiment, a terminal device is provided, as Figure 9 shown. The terminal device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0140] Acquire the historical location information of each object to be tracked;

[0141] According to the matching relationship between the first location information in the historical location information of each object to be tracked and the initial trajectory obtained by fitting the second location information, screen out target objects that do not match the initial trajectory from the objects to be tracked;

[0142] Acquire the current location information of the target object;

[0143] Based on the initial trajectory obtained by fitting the second location information of the target object and the current location information of the target object, determine the target trajectory of the target object, so as to perform target tracking according to the target trajectory.

[0144] In one embodiment, a computer-readable storage medium is provided. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the processor is caused to execute the following steps:

[0145] Acquire the historical location information of each object to be tracked;

[0146] Based on the matching relationship between the first position information in the historical position information of each object to be tracked and the initial trajectory obtained by fitting the second position information, screen out the target objects that do not match the initial trajectory from the objects to be tracked;

[0147] Obtain the current position information of the target object;

[0148] Based on the initial trajectory obtained by fitting the second position information of the target object and the current position information of the target object, determine the target trajectory of the target object, so as to perform target tracking according to the target trajectory.

[0149] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0150] It should be understood that when used in the specification and appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0151] It should also be understood that the term "and / or" used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0152] As used in the specification and appended claims of the present application, the term "if" can be interpreted as "when" or "once" or "in response to determining" or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined" or "in response to determining" or "once [the described condition or event] is detected" or "in response to detecting [the described condition or event]" according to the context.

[0153] In addition, in the description of the specification and appended claims of the present application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions, and cannot be understood as indicating or implying relative importance. It should also be understood that although the terms "first", "second", etc. are used in the text in some embodiments of the present application to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, the first table can be named the second table, and similarly, the second table can be named the first table, without departing from the scope of the various described embodiments. The first table and the second table are both tables, but they are not the same table.

[0154] Reference to "one embodiment" or "some embodiments" etc. described in the specification of this application means that a specific feature, structure or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprise", "include", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized.

[0155] All or part of the processes of the above-described embodiment methods of this application can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0156] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0157] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0158] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A target tracking method, characterized in that, Including: Obtaining historical position information of each object to be tracked; Screening out target objects that do not match the initial trajectory from the objects to be tracked according to the matching relationship between the first position information in the historical position information of each object to be tracked and the initial trajectory fitted from the second position information; Obtaining the current position information of the target object; Determining the target trajectory of the target object based on the initial trajectory fitted from the second position information of the target object and the current position information of the target object, so as to perform target tracking according to the target trajectory.

2. The target tracking method according to claim 1, characterized in that, Screening out target objects that do not match the initial trajectory from the objects to be tracked according to the matching relationship between the first position information in the historical position information of each object to be tracked and the initial trajectory fitted from the second position information, including: Determining a first trajectory and a second trajectory according to the matching relationship between the first position information in the historical position information of each object to be tracked and the initial trajectory fitted from the second position information; the first trajectory is the trajectory in the initial trajectory that matches the historical position information; the second trajectory is the trajectory in the initial trajectory that does not match the first position information; Determining the current position information of the tracking object corresponding to the first trajectory according to the first trajectory; Obtaining the current position information of each object to be tracked; Screening out the objects to be tracked that do not match the first trajectory according to the current position information of each object to be tracked and the current position information of the tracking object corresponding to the first trajectory; Screening out target objects from the objects to be tracked that do not match the first trajectory according to the current position information of the objects to be tracked that do not match the first trajectory and the second trajectory.

3. The target tracking method according to claim 2, wherein The determining of the first trajectory and the second trajectory according to the matching relationship between the first position information in the historical position information of each object to be tracked and the initial trajectory fitted from the second position information includes: Obtaining the motion direction information and motion speed information of each object to be tracked; Determining the predicted position information corresponding to each object to be tracked according to the motion direction information and motion speed information of each object to be tracked and the initial trajectory fitted from the second position information in the historical position information of each object to be tracked; Judging whether the degree of coincidence between the predicted position information corresponding to each object to be tracked and the first position information in the historical position information of each object to be tracked is higher than a preset coincidence threshold; If so, determining the initial trajectory of the object to be tracked whose degree of coincidence between the predicted position information and the first position information is higher than the preset coincidence threshold as the first trajectory; If not, determining the initial trajectory of the object to be tracked whose degree of coincidence between the predicted position information and the first position information is not higher than the preset coincidence threshold as the second trajectory.

4. The target tracking method according to claim 2, characterized in that The obtaining of the current position information of each object to be tracked includes: Obtaining the current site information; the site information includes the site image, the coordinate information of the fixed objects in the site, and the site size information; Converting the site image into a site bird's-eye view according to the coordinate information of the fixed objects in the site; Adjust the resolution of the bird's-eye view of the site according to the site size information; Obtain the current position information of each object to be tracked from the adjusted bird's-eye view of the site.

5. The target tracking method according to claim 2, wherein The screening of the target object from the objects to be tracked that do not match the first trajectory according to the current position information of the objects to be tracked that do not match the first trajectory and the second trajectory includes: Screen out candidate objects according to the termination position information of the second trajectory and the current position information of the objects to be tracked that do not match the first trajectory; the candidate object is the object to be tracked with the current position information closest to the termination position information of the second trajectory; When it is determined that the distance between the current position information of the candidate object and the termination position information of the second trajectory is lower than the preset distance threshold, the candidate object is determined as the target object.

6. The target tracking method according to claim 5, wherein The determining the target trajectory of the target object based on the initial trajectory obtained by fitting the second position information of the target object and the current position information of the target object for target tracking according to the target trajectory includes: When it is determined that there are multiple second trajectories of the target object, obtain the termination position information of each second trajectory; Calculate the distance values between the termination position information of each second trajectory and the current position information of the target object; Determine the second trajectory with the smallest distance value between the termination position information and the current position information of the target object as the initial trajectory of the target object; Determine the target trajectory of the target object according to the initial trajectory of the target object and the current position information of the target object, and perform target tracking according to the target trajectory.

7. A target tracking device, characterized in that, Including: A historical position information acquisition module, configured to acquire the historical position information of each object to be tracked; A target object screening module, configured to screen out target objects that do not match the initial trajectory from the objects to be tracked according to the matching relationship between the first position information in the historical position information of each object to be tracked and the initial trajectory obtained by fitting the second position information; A current position information acquisition module, configured to acquire the current position information of the target object; And A target trajectory determination module, configured to determine the target trajectory of the target object based on the initial trajectory obtained by fitting the second position information of the target object and the current position information of the target object for target tracking according to the target trajectory.

8. A target tracking system, characterized in that, Including: The target tracking device according to claim 7; And an information acquisition device that communicates with the target tracking device, where the information acquisition device is configured to acquire the position information of each object to be tracked and send the position information of each object to be tracked to the target tracking device.

9. A terminal device, characterized in that The terminal device includes a memory and a processor, and a computer program capable of running on the processor is stored on the memory, and when the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images