A target re-tracking method, device, equipment and computer-readable storage medium

Abstract

The present invention discloses a target re-tracking method, device, equipment and computer-readable storage medium, which are applied to the field of image processing and computer vision technology, including: using a target detection model to detect a queue to obtain an initial queue target detection result; using a tracking algorithm to obtain an identification number corresponding to each target in the queue according to the initial queue target detection result; using a target re-tracking model to perform a first-level tracking according to the identification number to obtain a target tracking result; the target re-tracking model is a model that predicts the position information of the target in the current frame based on the position information of the target in the previous frame and the Kalman filter, and corrects the position information of the target in the current frame and the Kalman filter parameters based on the observation information of the target in the current frame to achieve target tracking. The present application predicts and corrects the position information of the target in real time according to the identification number by correcting the Kalman filter parameters, so that the target will not be lost when occlusion or other interference factors occur.

Description

Technical Field

[0001] The present invention relates to the field of image processing and computer vision technology, and in particular to a target re-tracking method, device, equipment and computer-readable storage medium. Background Art

[0002] With the development of the economy and the increase in travel activities, queue management and tracking have become important issues in various scenarios. Whether it is a supermarket checkout counter, counter business processing or a scenic spot ticket entrance, it is necessary to effectively manage and track the information of queuing customers. Traditional queue management methods usually rely on manual operations, which not only wastes a lot of human resources, but also has low efficiency. Therefore, it is necessary to design intelligent queue management and tracking technology to improve the efficiency and accuracy of queues. Queue management relies on target tracking technology, which is mainly achieved through image processing and computer vision technology. Existing technologies only track individuals, and each person is independent, so it is impossible to achieve intelligent tracking of queues. Summary of the invention

[0003] In view of this, an object of the present invention is to provide a target re-tracking method, device, equipment and computer-readable storage medium, which solve the technical problem in the prior art that a queue cannot be tracked intelligently.

[0004] In order to solve the above technical problems, the present invention provides a target re-tracking method, comprising:

[0005] Use the target detection model to detect the queue and obtain the initial queue target detection result;

[0006] Using a tracking algorithm to obtain an identification number corresponding to each target in the queue according to the initial queue target detection result;

[0007] A target re-tracking model is used to perform primary tracking according to the identification number to obtain a target tracking result; wherein the target re-tracking model is a model that predicts the position information of the target in the current frame based on the position information of the target in the previous frame and the Kalman filter, and corrects the position information and Kalman filter parameters of the target in the current frame based on the observation information of the target in the current frame to achieve target tracking.

[0008] Optionally, the detecting the queue using the target detection model to obtain the initial queue target detection result includes:

[0009] The targets in the queue are detected using a head-shoulder target detection model to obtain the initial queue target detection result; wherein the head-shoulder target detection model is a model obtained by adjusting and training the parameters of the initial target detection model using head-shoulder training enhancement data and transfer learning.

[0010] Optionally, obtaining an identification number corresponding to each target in the queue according to the initial queue target detection result by using a tracking algorithm includes:

[0011] The DeepSORT model is used to determine the identification number corresponding to each target in each frame according to the initial queue target detection result.

[0012] Optionally, the using the target re-tracking model to perform primary tracking according to the identification number to obtain a target tracking result includes:

[0013] On the basis of obtaining the identifier corresponding to each target, adding speed and direction attributes; wherein the speed and direction attributes include the speed and direction attributes of each target and the speed and direction attributes of the team as a whole;

[0014] The Kalman filter correction algorithm is used to correct the Kalman filter prediction based on the speed and direction attributes to obtain real-time predicted and corrected target position information.

[0015] Optionally, the Kalman filter prediction is corrected based on the speed and direction attributes to obtain real-time predicted and corrected target position information, including:

[0016] Using the initialized Kalman filter to perform prediction, in the prediction process, using the state vector and the process noise covariance matrix to predict the next frame target position information of the detection target, and obtaining a prediction vector according to the next frame target position information;

[0017] Detect and track the target of the next frame using a target detection and tracking algorithm to obtain an observation vector; wherein the observation vector is a vector including the observed position, speed and direction angle;

[0018] Determine the Euclidean distance between the prediction vector and the observation vector, and when the Euclidean distance is not greater than a set distance threshold, determine that the observation vector is a target observation vector;

[0019] The target observation vector, observation matrix and observation noise covariance matrix are used to correct the predicted state vector and state covariance matrix to obtain the real-time predicted and corrected target position information.

[0020] Optionally, after determining the Euclidean distance between the prediction vector and the observation vector, the method further includes:

[0021] When it is determined that the Euclidean distance is greater than the set distance threshold, it is determined that the current observation vector is incorrect, and the observation vector is recalculated using the speed and direction attributes of the team as a whole.

[0022] Optionally, after performing primary tracking according to the identification number using the target re-tracking model to obtain a target tracking result, the method further includes:

[0023] Add the targets that were not tracked in the previous frame to the set to be deleted;

[0024] The new target given by the current frame tracking algorithm is added to the set to be determined, and the set to be deleted and the set to be determined are used for secondary tracking to obtain the identification rematching result.

[0025] The present application also provides a target re-tracking device, comprising:

[0026] An initial queue target detection result determination module is used to detect the queue using a target detection model to obtain an initial queue target detection result;

[0027] An identification number determination module, used to obtain an identification number corresponding to each target in the queue according to the initial queue target detection result using a tracking algorithm;

[0028] A re-tracking module is used to perform primary tracking according to the identification number using a target re-tracking model to obtain a target tracking result; wherein the target re-tracking model is a model that predicts the position information of the target in the current frame based on the position information of the target in the previous frame and the Kalman filter, and corrects the position information and Kalman filter parameters of the target in the current frame based on the observation information of the target in the current frame to achieve target tracking.

[0029] The present application also provides a target re-tracking device, comprising:

[0030] Memory for storing computer programs;

[0031] A processor is used to implement the steps of the above-mentioned target re-tracking method when executing the computer program.

[0032] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the target re-tracking method as described above are implemented.

[0033] It can be seen that the present invention detects the queue by using the target detection model to obtain the initial queue target detection result; uses the tracking algorithm to obtain the identification number corresponding to each target in the queue according to the initial queue target detection result; uses the target re-tracking model to perform first-level tracking according to the identification number to obtain the target tracking result; wherein, the target re-tracking model is a model that predicts the position information of the current frame target based on the position information of the previous frame target and the Kalman filter, and corrects the position information and Kalman filter parameters of the current frame target based on the observation information of the current frame target to achieve target tracking. Compared with the current inability to track the entire queue, the present application corrects the Kalman filter parameters to predict and correct the position, speed and direction of the target in real time, so that the target will not be lost when occlusion or other interference factors occur, and realizes continuous tracking of the target. By accurately tracking the position and motion information of each person, the number of people in the queue and the queue time of each person can be counted in real time. This is of great significance for optimizing service processes, improving service efficiency and providing better user experience.

[0034] In addition, the present invention also provides a target re-tracking device, equipment and computer-readable storage medium, which also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0036] Figure 1 A flow chart of a target re-tracking method provided by an embodiment of the present invention;

[0037] Figure 2 A flowchart of a head-shoulder model training method provided by an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of head and shoulders data provided by an embodiment of the present invention;

[0039] Figure 4 A flowchart of a re-tracking method provided by an embodiment of the present invention;

[0040] Figure 5 A structural diagram of a re-tracking framework provided by an embodiment of the present invention;

[0041] Figure 6 A flowchart of a method for performing queue tracking using a target re-tracking cascade matching model provided by an embodiment of the present invention;

[0042] Figure 7 A schematic diagram of a team direction provided by an embodiment of the present invention;

[0043] Figure 8 A flowchart of a method for correcting Kalman filter prediction based on speed and direction attributes provided by an embodiment of the present invention;

[0044] Fig. 9 A flowchart of a re-tracking and matching method provided by an embodiment of the present invention;

[0045] Fig.10 A schematic diagram of the structure of a target re-tracking device provided by an embodiment of the present invention;

[0046] Fig.11 A schematic diagram of the structure of a target re-tracking device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Please refer to Figure 1 , Figure 1 A flow chart of a target re-tracking method provided by an embodiment of the present invention. The method may include:

[0049] S101, using the target detection model to detect the queue and obtain the initial queue target detection result.

[0050] This embodiment can select a suitable detection model according to actual needs and scene characteristics, and this embodiment does not limit the specific target detection model. For example, the target detection model in this embodiment can be a single detection model; or the target detection model in this embodiment can be a two-stage detection model. It should be noted that if a higher real-time detection is required, you can select the YOLO (You Only Look Once, an object recognition and positioning algorithm based on a deep neural network) series of models, such as YOLOv5 or YOLOv8. If higher accuracy is required, the latest target detection model can be selected. The initial queue detection result in this embodiment refers to all targets in the queue, for example, it can be all target persons in the queue; or it can also be all target pets in the queue.

[0051] It should be further explained that, in order to improve the accuracy of target detection, the above-mentioned use of the target detection model to detect the queue and obtain the initial queue target detection result can include: using the head and shoulder target detection model to detect the target in the queue to obtain the initial queue target detection result; wherein the head and shoulder target detection model is a model obtained by adjusting and training the parameters of the initial target detection model using head and shoulder training enhancement data and transfer learning. The head and shoulder data in this embodiment is the head and shoulder data of a person. This embodiment uses the head and shoulder target detection model for inspection. Compared with the traditional complete human body detection model, it avoids the problem of body occlusion that often occurs in the queue and improves the accuracy of detection.

[0052] S102, using a tracking algorithm to obtain an identification number corresponding to each target in the queue according to the initial queue target detection result.

[0053] This embodiment does not limit the specific tracking algorithm. For example, the tracking algorithm in this embodiment may be a DeepSORT model (multi-target tracking algorithm); or the tracking algorithm in this embodiment may be a SORT model (classic target tracking algorithm). This embodiment may use a tracking algorithm to obtain an identification number (ID) corresponding to each target in the queue based on the initial queue target detection result. The queue in this embodiment refers to lines of position classes of 3 or more people.

[0054] It should be further explained that in order to improve the efficiency of identification detection and the applicability of the model, the above-mentioned use of the tracking algorithm to obtain the identification number corresponding to each target in the queue according to the initial queue target detection result can include: using the DeepSORT model to determine the identification number corresponding to each target in each frame according to the initial queue target detection result. DeepSORT in this embodiment is a very common multi-target tracking algorithm at present, with a fast speed, and can be connected to any detector, such as YOLO V3, YOLO V4, YOLO V5, YOLO V4, etc. Therefore, the applicability of the DeepSORT model is relatively strong.

[0055] S103, using the target re-tracking model to perform primary tracking according to the identification number to obtain the target tracking result; wherein the target re-tracking model is a model that predicts the position information of the target in the current frame based on the position information of the target in the previous frame and the Kalman filter, and corrects the position information of the target in the current frame and the Kalman filter parameters based on the observation information of the target in the current frame to achieve target tracking.

[0056] In this embodiment, the target re-tracking model is used to perform primary tracking according to the identification number to obtain the target tracking result. In this embodiment, the Kalman filter parameters are corrected to predict and correct the position information of the target. The position information of the current frame target may include position, speed and direction. Since the position of the current frame target can be corrected in time, the parameter error and offset caused by occlusion can be solved, and the tracking accuracy can be improved.

[0057] It should be further explained that, in order to improve the correction effect, the above-mentioned target re-tracking model is used to perform primary tracking according to the identification number to obtain the target tracking result, which may include: adding speed and direction attributes on the basis of obtaining the corresponding identification of each target; wherein the speed and direction attributes include the speed and direction attributes of each target, and the speed and direction attributes of the team as a whole; using the Kalman filter correction algorithm, the Kalman filter prediction is corrected based on the speed and direction attributes to obtain real-time predicted and corrected target position information.

[0058] The target tracking result in this embodiment includes target position information. This embodiment does not limit the specific position information. For example, the target position information in this embodiment may be position, speed and direction; or the target position information in this embodiment may also be position coordinates and direction. Among them, the process of determining the speed and direction in this embodiment may be to determine the position of the target tracking point (center point or non-center point) between consecutive frames, and calculate the speed components of the target tracking point in the horizontal and vertical directions. Thus, the direction of the target tracking point is determined according to the speed components of the target tracking point in the horizontal and vertical directions. According to the direction and speed of each target tracking point, the direction and speed of the current overall team can be calculated. This embodiment will predict and correct the target position information in real time according to the speed and direction attributes of each target and the speed and direction attributes of the team as a whole, thereby improving the correction effect.

[0059] For ease of understanding, the embodiment of the present invention provides a specific process for determining the speed and direction attributes. This embodiment can combine the results of target detection and tracking, and calculate the speed of the target according to the position change of the center point of the same target between consecutive frames. Assuming that the coordinates of the center point of the target in the current frame are (x, y), the coordinates of the center point of the target in the previous frame are (x_prev, y_prev), and the time interval is t, the speed of the target can be calculated by the following formula:

[0060] V=(vx,vy)= ((x - x_prev) / t, (y - y_prev) / t)) (Formula 1)

[0061] In this way, the velocity components of the target in the horizontal and vertical directions can be obtained, where vx represents the velocity component in the horizontal direction and vy represents the velocity component in the vertical direction.

[0062] Calculate the target direction: Based on the horizontal and vertical components of the target velocity, the target direction can be calculated. Assuming the velocity component of the target is (vx, vy), the target direction can be calculated by the following formula:

[0063] theta = atan2(vy, vx) (Formula 2)

[0064] This way you can get the direction angle of the target.

[0065] Calculate the direction and speed of the entire team. With the direction and speed of each target, we can calculate the queue direction and queue speed of the current queue: Assuming that there are n people in the current queue, with speeds and directions (vx1, vy1, theta1), (vx2, vy2, theta2), ..., (vxn, vyn, thetan), then the average speed and direction of the entire team can be calculated as follows:

[0066] Average speed:

[0067]

[0068]

[0069] Average Direction:

[0070] avg_theta = atan2(avg_vy, avg_vx) (Equation 5)

[0071] This embodiment adds speed and direction attributes on the basis of identification attributes, thereby using the team's unique attributes (speed, direction) to calculate the state distance and matching degree between the front and rear frame personnel in the team, solving the problem of ID mutual cutting and ID loss in the tracking process, achieving accurate tracking of personnel, and thus ensuring the accuracy of queue time calculation. It can be understood that the speed and direction attributes of each person are used in the state vector, X = [px, py, vx, vy, theta], which is used throughout the entire process of Kalman filtering, and the speed and direction attributes of the team are only used during correction.

[0072] It should be further explained that in order to improve the accuracy of the real-time predicted and corrected target position information, the above-mentioned correction of the Kalman filter prediction based on the speed and direction attributes to obtain the real-time predicted and corrected target position information can include: using the initialized Kalman filter to make predictions, and using the state vector and the process noise covariance matrix to predict the next frame target position information of the detected target during the prediction process, and obtaining the prediction vector based on the next frame target position information; using the target detection and tracking algorithm to detect and track the next frame target to obtain the observation vector; wherein the observation vector is a vector including the observed position, speed and direction angle; determining the Euclidean distance between the prediction vector and the observation vector, and when the Euclidean distance is not greater than the set distance threshold, determining the observation vector as the target observation vector; using the target observation vector, the observation matrix and the observation noise covariance matrix to correct the predicted state vector and the state covariance matrix to obtain the real-time predicted and corrected target position information. It can be understood that this embodiment uses an initialized Kalman filter for prediction, and in the prediction process uses the state vector and the process noise covariance matrix to predict the next frame, speed and direction angle of the detection target, and obtains the prediction vector according to the next frame position, speed and direction angle; wherein the state vector is a vector including the position of the target on the x-axis and the position on the y-axis, the speed on the x-axis, the speed on the y-axis and the direction angle, and the process noise covariance matrix represents the process noise variance of each vector; the target detection and tracking algorithm is used to detect and track the next frame target to obtain the observation vector; wherein the observation vector is a vector including the observation The vector of the position, speed and direction angle of the predicted vector; determine the Euclidean distance between the predicted vector and the observed vector, and when the Euclidean distance is not greater than the set distance threshold, determine that the observed vector is the target observation vector; use the target observation vector, the observation matrix and the observation noise covariance matrix to correct the predicted state vector and the state covariance matrix to obtain the real-time predicted and corrected target position, target speed and target direction; wherein the observation matrix represents the vector required for the state vector to be mapped to the observation vector; the observation noise covariance matrix represents the observation noise variance of each vector, and the state covariance matrix represents the difference between the observed state vector and the predicted state vector. This embodiment does not limit the specific state vector. For example, the state vector can be the target position, target speed and target direction predicted and corrected in real time by this embodiment, which will be corrected according to the predicted vector and the observed vector, and the observation noise will be used for calculation in the determination process, thereby improving the accuracy of the real-time predicted and corrected target position, target speed and target direction.

[0073] It should be further explained that, in order to improve the accuracy of the observation vector, after the Euclidean distance between the predicted vector and the observation vector is determined as above, it may also include: when it is determined that the Euclidean distance is greater than the set distance threshold, it is determined that the current observation vector is incorrect, and the observation vector is recalculated using the overall speed and direction attributes of the team. In this embodiment, when the Euclidean distance is greater than the distance threshold, it is determined that the current observation vector is incorrect, and the observation vector is recalculated. This embodiment takes into account the situation where the observation vector may be calculated inaccurately, and recalculates the observation vector, thereby improving the accuracy of the observation vector determination.

[0074] It should be further explained that, in order to improve the accuracy of the tracking result determination, after the target tracking result is obtained by performing the first-level tracking according to the identification number using the target re-tracking model, it can also include: adding the target that was not tracked in the previous frame to the set to be deleted; adding the new target given by the current frame tracking algorithm to the set to be determined, and performing the second-level tracking using the set to be deleted and the set to be determined to obtain the identification rematching result. This embodiment will perform the second-level tracking, and perform the identification rematching using the set to be deleted and the set to be determined, thereby improving the accuracy of the target tracking result.

[0075] It should be further explained that in order to improve the accuracy of secondary tracking, the new target given by the current frame tracking algorithm is added to the to-be-determined set, and the to-be-determined set and the to-be-determined set are used for secondary tracking to obtain the identification re-matching result, which may include: for each target in the to-be-determined set, the matching degree is calculated with the to-be-deleted set in turn to obtain the matching degree; determining whether the matching degree is greater than or equal to the matching degree threshold; if the matching degree is greater than or equal to the matching degree threshold, the match is successful, and the to-be-deleted target with the highest matching degree is selected as the matching item, and the identification information of the matching item is assigned to the corresponding to the to-be-determined target; if the matching degree is lower than the set threshold, the current to-be-determined target is determined to be a new target. The matching degree formula in this embodiment is M=w1*exp(-L^2 / (2*variance(L)))+w2*exp(-S^2 / (2*variance(S)))+w3*exp(-T^2 / (2*variance(T)), exp represents the exponent e, L represents the distance between the target in the set to be determined and the center point of the target frame in the set to be deleted, S represents the number of occurrences of the target in the set to be deleted, T represents the angle between the two direction vectors of the target in the set to be determined and the target in the set to be deleted, variance (L), variance (S), variance (T) are adjustment parameters for controlling the decay speed of the matching degree; w1, w2, w3 are weights for adjusting the importance of each factor. In this embodiment, when determining each target in the set to be determined and matching it with the target in the set to be deleted in turn, multiple factors such as distance, angle, decay speed adjustment parameters and weights of each factor are considered, so the accuracy of matching degree calculation can be improved, thereby improving the accuracy of tracking result determination.

[0076] It should be further explained that, in order to improve the efficiency of queuing, after the target tracking result is obtained by performing identification rematching according to the identification number using the target retracking cascade matching model, the method may further include: determining the number of people in the queue and the queuing time of each target according to the target tracking result. This embodiment can count the number of people in the queue and the queuing time of each person in real time by determining the number of people in the queue and the queuing time of each person according to the target tracking result, thereby more efficiently managing and optimizing the queue and improving the user experience.

[0077] The target re-tracking method provided by the embodiment of the present invention may include: S101, using the target detection model to detect the queue and obtain the initial queue target detection result; S102, using the tracking algorithm to obtain the identification number corresponding to each target in the queue according to the initial queue target detection result; S103, using the target re-tracking model to perform primary tracking according to the identification number to obtain the target tracking result; wherein the target re-tracking model is a model that predicts the position information of the target in the current frame based on the position information of the target in the previous frame and the Kalman filter, and corrects the position information and Kalman filter parameters of the target in the current frame based on the observation information of the target in the current frame to achieve target tracking. It can be seen that compared with the current tracking of only a single person, the present application can intelligently track the entire queue, and by correcting the Kalman filter parameters, the position, speed and direction of the target are predicted and corrected in real time, so that the target will not be lost when occlusion or other interference factors appear, thereby achieving continuous tracking of the target. In addition, this embodiment uses a head-and-shoulder target detection model for inspection. Compared with the traditional complete human body detection model, it avoids the problem of body occlusion that often occurs in the queue and improves the accuracy of detection; and, the DeepSORT model is used to determine the representation of the target, which improves the applicability of the model; and, on the basis of the identification attribute, this embodiment adds speed and direction attributes, thereby using the team's unique attributes (speed, direction) to calculate the state distance and matching degree between the front and rear frame personnel in the team, solve the problem of ID mutual cutting and ID loss in the tracking process, realize accurate tracking of personnel, and thus ensure the accuracy of queuing time calculation; and, in the determination process, observation noise is used for calculation, which improves real-time The accuracy of the predicted and corrected target position information; and, this embodiment takes into account the situation that the observation vector may be calculated inaccurately, and will recalculate the observation vector, thereby improving the accuracy of the observation vector determination; and, in this embodiment, when determining each target in the set to be determined and matching it with the targets in the set to be deleted in turn, multiple factors will be considered, such as the adjustment parameters of the distance, angle, attenuation speed and the weights of each factor, so the accuracy of the matching degree calculation can be improved, thereby improving the accuracy of the tracking result determination; and, by determining the number of people in the team and the queuing time of each target according to the target tracking results, this embodiment can count the number of people in the team and the queuing time of each person in real time, thereby more efficiently managing and optimizing the team and improving the user experience.

[0078] In order to make the present invention easier to understand, please refer to Figure 2 , Figure 2 A flowchart of a head-shoulder model training method provided by an embodiment of the present invention may include:

[0079] S201, obtaining head-shoulder training data; wherein the head-shoulder training data includes pedestrian images in various angles, sizes, lighting conditions, and occlusion conditions.

[0080] The head and shoulders data in this embodiment are as follows Figure 3 As shown, Figure 3 A schematic diagram of head and shoulder data provided for an embodiment of the present invention. The head and shoulder data are collected and labeled offline, and the target detection model is trained. In the actual queue management task, the current frame target is detected by the trained head and shoulder detection model. Compared with the traditional complete human body model, the head and shoulder model can better represent the characteristics of a person, and also avoids the problem of body occlusion that often occurs in the queue. In order to improve the accuracy of personnel detection, this embodiment can collect more personnel image data and annotate them. Ensure that the data set contains pedestrian images of various angles, sizes, lighting conditions and occlusion conditions.

[0081] S202, performing data enhancement processing on the head and shoulder training data using a data enhancement method to obtain head and shoulder training enhanced data.

[0082] This embodiment expands the diversity of the data set through data enhancement techniques, such as rotation, scaling, translation, flipping, etc. This helps to improve the generalization ability of the model, so that it can better adapt to detection tasks in different scenarios.

[0083] S203, training the target detection model using the head and shoulder training enhancement data to obtain a pre-trained target detection model.

[0084] This embodiment can select a suitable detection model according to actual needs and scene characteristics. For example, if a high real-time detection is required, a YOLO series model such as YOLO V5 or YOLO V8 can be selected. If higher accuracy is required, the latest target detection model can be selected.

[0085] S204, using transfer learning to adjust the parameters of the pre-trained target detection model according to the weight of the target detection model in the general data set and the general pedestrian data set, to obtain a target head and shoulder model.

[0086] This embodiment uses the weights of the pre-trained model on large-scale general data sets, such as ImageNet (the name of the computer vision system recognition project), COCO (a large and rich object detection, segmentation and captioning data set), Market-1501 (a fixed number of training sets and test sets), CityPersons (pedestrian detection data set) and other pedestrian data sets for transfer learning. Combining the specific data set of the personnel detection task with the pre-trained model can speed up the convergence of the model and improve the accuracy of the detection. At the hardware level: For scenarios with high real-time requirements, you can consider using high-performance hardware devices such as GPU (Graphics Processing Unit) or TPU (TensorProcessing Unit) to accelerate the reasoning process of the model. In addition, you can also use technologies such as model compression and quantization to reduce the computing and storage resource consumption of the model.

[0087] In order to make the present invention easier to understand, please refer to Figure 4 , Figure 4 A flowchart of a re-tracking method provided by an embodiment of the present invention may specifically include:

[0088] Step 401: Detect the target using the head and shoulder target detection model to obtain the initial team member detection result.

[0089] The overall framework diagram corresponding to this embodiment is as follows Figure 5 As shown, Figure 5 A schematic diagram of the structure of a re-tracking framework provided by an embodiment of the present invention is provided. Kalman filtering is used to correct the problem of structural ID (id) mutual cutting, and matching degree is used to solve the problem of switching to a new ID (id).

[0090] Step 402: Use the DeepSORT model to obtain the ID corresponding to each target based on the initial team member detection results.

[0091] The DeepSORT model in this embodiment can be other target tracking models. In this embodiment, the same target is assigned the same ID in different frames to achieve target tracking. The process of processing using DeepSORT in this embodiment includes: 1) Target feature extraction: Extract the feature representation of the target from the target position detected in the first frame. Use a deep learning model (such as CNN) to extract the feature vector of the target. Generally, a pre-trained image classification model (such as ResNet, VGG, etc.) or a feature extraction network dedicated to target tracking can be used to extract the feature vector of the target. 2) Target matching: In subsequent video frames, the target feature vector is matched with the target feature vector in the previous frame to determine the position of the target object in the current frame. The Hungarian algorithm is used to solve the matching problem between multiple targets. Matching can use various distance metrics, such as Euclidean distance, cosine similarity, etc. (Target and identification) Three people are numbered 123 to the next one to find again 3) Target trajectory management: According to the result of target matching, the target trajectory is managed and updated. A Kalman filter is used to predict and update the position of the target.

[0092] Step 403: Use the target re-tracking cascade matching model to perform identification re-matching according to the ID of each target to obtain the target tracking result; wherein the target re-tracking cascade matching model is a model that measures and corrects the position, speed and direction of the target based on the overall queue attributes to prevent ID switching and ID loss during the tracking process.

[0093] The target re-tracking cascade matching model in this embodiment includes the above-mentioned primary tracking and secondary tracking. This embodiment takes into account the situation that the previous tracking algorithm cannot effectively handle the ID swap (two / multiple target ID swap) or lost tracking (ID lost, switched to a new ID) due to factors such as occlusion, motion, and small target shape. In this case, the target re-tracking cascade matching method is used to re-match the ID.

[0094] Step 404, determining the number of people in the team and the queuing time for each target based on the target tracking result.

[0095] The technical effects of the embodiments of the present invention are as follows:

[0096] First, the method of the present invention is based on target detection. The detection model changes the traditional full-body model to a head-shoulder model. The head-shoulder model can better represent the characteristics of a person than the traditional complete human body model. The head and shoulders of a person are important parts of human movement and behavior. The head-shoulder model can avoid the problem of body occlusion: in queue management tasks, human body parts are often occluded. The head-shoulder model focuses on the head and shoulders, avoiding the problem of body occlusion, thereby improving the robustness of human detection.

[0097] Second, the speed and direction attributes in target tracking: By analyzing the position change of the target between consecutive frames, the target's movement direction and speed are calculated. This traditional target tracking method with the addition of speed and direction attributes can provide more accurate target positioning and prediction capabilities.

[0098] Third, a correction method based on team attributes: In response to the occlusion and ID switching problems in the target tracking algorithm, a method for correcting the Kalman filter prediction based on the team's speed and direction attributes is proposed. By correcting the filter parameters, the parameter errors and offsets caused by occlusion can be solved, and the tracking accuracy can be improved.

[0099] Fourth, cascade matching: The algorithm combines the Kalman filter correction and re-tracking matching methods to solve the ID swap and target loss problems caused by long-term occlusion in target tracking. The first stage optimizes the tracking accuracy by correcting the Kalman filter, effectively solving the problem of ID swap; while the re-tracking matching handles the situation of new targets by calculating the matching degree and matching the set to be deleted.

[0100] Fifth, the system of the present invention is implemented in a mobile terminal, which is responsible for the collection of image data, the reasoning of the detection and recognition model, and the calculation of the optimization of the recognition results. The present invention can accurately track the location and movement information of each person, and your algorithm realizes the intelligent management of the queue. This can be used to count the number of people in the queue and the queue time of each person in real time, thereby providing more efficient queue management and optimization.

[0101] In order to make the present invention easier to understand, please refer to Figure 6 , Figure 6 An example flow chart of a method for performing queue tracking using a target re-tracking cascade matching model provided by an embodiment of the present invention may specifically include:

[0102] S601, based on the ID corresponding to each target, add speed and direction attributes, wherein the speed and direction attributes include the speed and direction attributes of each ID and the speed and direction attributes of the team as a whole.

[0103] For easier understanding, please refer to Figure 7 , Figure 7 A schematic diagram of a team direction provided by an embodiment of the present invention.

[0104] S602, based on the speed and direction attributes, the Kalman filter prediction is corrected to obtain the real-time predicted and corrected target position, target speed and target direction; wherein, the correction process is to determine the state vector, state transfer matrix, observation vector, observation matrix, process noise covariance matrix and observation noise covariance matrix, initialize the Kalman filter to obtain the initialized state vector and the initialized covariance matrix, use the state transfer matrix and the process noise covariance matrix to predict the next frame position and speed of the target, detect and track the next frame to obtain the observation vector, and use the observation vector, observation matrix and observation noise covariance matrix to correct the predicted state vector and state covariance matrix.

[0105] This embodiment can utilize the above target re-tracking model to correct the Kalman filter prediction based on the speed and direction attributes to obtain a real-time predicted and corrected target position, target speed, and target direction.

[0106] S603, using the re-tracking matching method to re-match the predicted and corrected target position, target speed and target direction to obtain the target queue tracking result; wherein, the re-tracking matching method is to use the target that was not tracked in the previous frame as the set to be deleted, and the new target given by the current frame tracking algorithm as the set to be determined, and determine the target ID with a matching relationship according to the matching degree between the set to be deleted and the set to be determined.

[0107] Due to occlusion, the prediction model in the target tracking algorithm is still based on the position information of the target in the last frame before it is occluded, but in fact the target position has moved significantly. Therefore, the prediction based on target detection alone will result in a large difference between the target position in the next frame and the actual position. This will cause the tracking algorithm to lose the target and confuse the reappearing target with other target IDs, resulting in the problem of ID cutting. Due to the particularity of the queue scene, the queue generally moves in one direction. Therefore, this patent proposes a method for correcting the Kalman filter prediction based on the speed and direction attributes of the queue, correcting the Kalman filter parameter errors and offsets caused by occlusion in the traditional target algorithm, solving the problem of ID cutting and improving the tracking accuracy.

[0108] In order to make the present invention easier to understand, please refer to Figure 8 , Figure 8 An example flow chart of a method for correcting Kalman filter prediction based on speed and direction attributes provided by an embodiment of the present invention may specifically include:

[0109] S801, define the state vector X and the state transfer matrix A, the state vector includes the position of the target on the x-axis and y-axis, the speed and direction angle on the x-axis and y-axis; where X = [px, py, vx, vy, theta]; A = [10dt 00; 010dt 0; 00100; 00010; 00001], dt is the time interval between two frames.

[0110] In this embodiment, we first define the state vector and the state transfer matrix. The state vector includes the position, speed and direction of the target and can be expressed as:

[0111] X=[px,py,vx,vy,theta] (Formula 6)

[0112] Among them, px and py are the positions of the target in the x and y directions, vx and vy are the speeds of the target in the x and y directions, and theta is the direction angle of the target. It should be noted that the positions of the five values ​​in the state vector can be swapped. For example, the state vector can also be X = [vx, vy, px, py, theta], or the state vector X = [theta, vx, vy, px, py]

[0113] The state transition matrix A describes how the state vector evolves over time. In the queue tracking problem, the evolution of the state vector can be expressed as:

[0114] X(k+1)=A*X(k)+w(k) (Formula 7)

[0115] Among them, w(k) is the process noise, which represents the uncertainty in the state vector. In the queue problem, it is a zero-mean Gaussian white noise. X(k+1) represents the state vector at the next moment, and X(k) represents the state vector at the current moment.

[0116] The state transfer matrix A can be expressed as:

[0117] A=[10 dt 00; 010 dt 0; 00100; 00010; 00001] (Formula 8)

[0118] Wherein, dt is the time interval between two frames. It should be noted that the state transfer matrix A is a matrix representing state transfer, so A can be a matrix representing transfer with time as the interval; or it can also be a matrix representing transfer with distance as the interval.

[0119] S802, during the queue tracking process, the evolution process of the state transfer vector is, X(k+1)=A*X(k)+w(k); wherein w(k) is the process noise.

[0120] S803, define the observation vector Z and the observation matrix H, the observation vector contains the position information of the target; wherein, Z = [px', py', vx', vy', theta'], px and py are the positions of the target in the x and y directions, vx', vy', theta' are calculated by px', py' and px, py, the mapping of the observation vector and the state vector can be expressed as, Z(k) = H*X(k) + v(k), v(k) is the observation noise, indicating the uncertainty in the observation vector, H = [10000; 01000].

[0121] In this embodiment, the observation vector contains the position information of the target and can be expressed as:

[0122] Z=[px',py',vx',vy',theta'] (Formula 9)

[0123] Z represents the observation vector, px and py are the positions of the target in the x and y directions, and vx', vy', and theta' are calculated by px', py', and px, py.

[0124] The observation matrix H describes how the state vector is mapped to the observation vector. The mapping between the observation vector and the state vector can be expressed as:

[0125] Z(k)=H*X(k)+v(k) (Formula 10)

[0126] Where v(k) is the observation noise, which represents the uncertainty in the observation vector and is also defined as a zero-mean Gaussian white noise.

[0127] The observation matrix H can be expressed as:

[0128] H = [10000; 01000] (Formula 11)

[0129] S804, define the process noise covariance matrix Q and the observation noise covariance matrix R; wherein, Q = diag([q1 q2q3 q4 q5]), R = diag([r1 r2]); q1 to q5 are the variances of the process noise, r1 and r2 are the variances of the observation noise, and diag refers to a diagonal matrix.

[0130] This embodiment defines a process noise covariance matrix Q and an observation noise covariance matrix R. It is simply assumed that they are both diagonal matrices, and the variances of the process noise and the observation noise are respectively:

[0131] Q=diag([q1 q2 q3 q4 q5]) (Formula 12)

[0132] R=diag([r1 r2 r3 r4 r5]) (Formula 13)

[0133] Among them, q1 to q5 are the five process noises corresponding to the state vector, and r1 to r5 are the five observation noises corresponding to the state vector. diag means diagonal matrix, which is a special square matrix in which all elements except the elements on the diagonal are zero.

[0134] S805, initializing the Kalman filter, initializing the state vector X and the state covariance matrix P.

[0135] Before starting prediction and correction in this embodiment, the Kalman filter needs to be initialized. The state vector x and the state covariance matrix P (the difference between the ideal value of the state vector and the observed value of the state vector) need to be initialized. It can be assumed that their initial values ​​are:

[0136] X(0)=[px0,py0,vx0,vy0,theta0] (Equation 14)

[0137] P(0) = diag([p1 p2 p3 p4p5]) (Formula 15)

[0138] Q=diag([q1 q2 q3 q4 q5])

[0139] Among them, px0, py0, vx0, vy0 and theta0 are the initial position, speed and direction of the target, px0 and py0 are given by the target tracking results, vx0, vy0 and theta0 are the expanded target tracking attributes calculated above, and p1 to p5 are the initial state vector differences of each element in the state vector.

[0140] S806, using the state transfer matrix A and the process noise covariance matrix Q to predict the next frame position and speed of the target, to obtain a prediction vector.

[0141] This embodiment uses a Kalman filter for prediction. In the prediction stage, the state transfer matrix A and the process noise covariance matrix Q are used to predict the next frame position and speed of the target. The prediction process can be expressed as:

[0142] X(k+1|k)=A*X(k|k) (Formula 16)

[0143] P(k+1|k)=A*P(k|k)*A'+Q (Equation 17)

[0144] Among them, X(k|k) and P(k|k) are the state vector and state covariance matrix of the previous frame, and X(k+1|k) and P(k+1|k) are the predicted state vector and state covariance matrix.

[0145] S807, using the target detection and tracking algorithm to detect and track the next frame to obtain an observation vector Z(k+1).

[0146] This embodiment uses the target detection algorithm to detect and track the next frame and obtain the observation vector Z(k+1). With the observation vector, the Kalman filter parameters can be corrected. However, if the ID is easily switched due to occlusion, the observation vector at this time deviates from the true value. Correcting the filter parameters based on the wrong observation vector will cause the parameters to deviate from the correct value, resulting in the problem of target loss in the subsequent tracking process.

[0147] S808, calculate the Euclidean distance between the predicted vector and the observed vector. If the Euclidean distance is greater than the set distance threshold, it is determined that the current observed vector Z(k+1) is incorrect, and then the observation vector is recalculated using the team attributes.

[0148] Before the modification of this embodiment, this patent proposed a parameter correction method to correct the parameters with large deviations. The specific method is:

[0149] For each target in the current frame, calculate the Euclidean distance between its prediction vector and the observation vector, formula:

[0150] d=sqrt(X(k+1|k)-Z(k+1))2 (Formula 18)

[0151] If the state distance is greater than the set distance threshold, the current observation vector Z(k+1) is judged to be incorrect, and then the observation vector is recalculated using the team attributes:

[0152] Z(k+1)[px,py]=X(k)[px,py]+dt*F (Equation 19)

[0153] F is the team attribute, that is, the average speed and direction of the entire team mentioned above, and dt is the time interval between two frames.

[0154] S809, using the observation vector Z, the observation matrix H and the observation noise covariance matrix R to correct the predicted state vector and state covariance matrix, so as to achieve real-time prediction and correction of the position, speed and direction of the target.

[0155] In the correction stage, this embodiment uses the observation vector z, the observation matrix H and the observation noise covariance matrix R to correct the predicted state vector and state covariance matrix. The correction process can be expressed as:

[0156] K(k+1)=P(k+1|k)*H'*inv(H*P(k+1|k)*H'+R) (Formula 20)

[0157] X(k+1|k+1)=X(k+1|k)+K(k+1)*(Z(k+1)-H*X(k+1|k)) (Equation 21)

[0158] P(k+1|k+1)=(IK(k+1)*H)*P(k+1|k) (Equation 22)

[0159] Where K is the Kalman gain, which is used to measure the uncertainty between predictions and observations, I is the identity matrix, and inv represents the inverse of the matrix.

[0160] In each frame, the state vector and state covariance matrix of the previous frame are used to predict the state vector and state covariance matrix of the next frame, and the observation vector of the current frame is used to correct the predicted state vector and state covariance matrix. In this way, the position, speed and direction of the target can be predicted and corrected in real time.

[0161] In order to make the present invention easier to understand, please refer to Fig. 9 , Fig. 9 A flowchart of a re-tracking and matching method provided by an embodiment of the present invention may specifically include:

[0162] S901, adding the target that was not tracked in the previous frame to the to-be-deleted set, and adding the new target given by the tracking algorithm of the current frame to the to-be-determined set.

[0163] S902, for each target in the set to be determined, calculate the matching degree with the set to be deleted in turn to determine the matching degree. The matching degree formula is M=w1*exp(-L^2 / (2*variance(L)))+w2*exp(-S^2 / (2*variance(S)))+w3*exp(-T^2 / (2*variance(T)), L represents the distance between the target in the set to be determined and the center point of the target frame in the set to be deleted, S represents the number of occurrences of the target in the set to be deleted, T represents the directional consistency of the target in the set to be determined and the target in the set to be deleted (calculate the angle between the two directional vectors), variance L, variance S, variance T are adjustment parameters used to control the decay speed of the matching degree. w1, w2, w3 are weights used to adjust the importance of each factor.

[0164] S903: If the matching degree is higher than or equal to the set threshold, the match is successful, and the target to be deleted with the highest matching degree is selected as the matching item, and the ID information of the target to be deleted is assigned to the corresponding target to be determined.

[0165] S904: If the matching degree is lower than the set threshold, the current target to be determined is determined to be a new target.

[0166] A target re-tracking device provided by an embodiment of the present invention is introduced below. The target re-tracking device described below and the target re-tracking method described above can be referred to each other.

[0167] Please refer to Fig.10 , Fig.10 A schematic diagram of the structure of a target re-tracking device provided by an embodiment of the present invention may include:

[0168] The initial queue target detection result determination module 100 is used to detect the queue using the target detection model to obtain the initial queue target detection result;

[0169] The identification number determination module 200 is used to obtain the identification number corresponding to each target in the queue according to the initial queue target detection result by using a tracking algorithm;

[0170] The re-tracking module 300 is used to perform primary tracking according to the identification number using a target re-tracking model to obtain a target tracking result; wherein the target re-tracking model is a model that predicts the position information of the target in the current frame based on the position information of the target in the previous frame and the Kalman filter, and corrects the position information and Kalman filter parameters of the target in the current frame based on the observation information of the target in the current frame to achieve target tracking.

[0171] Further, based on the above embodiment, the above initial queue target detection result determination module 100 may include:

[0172] A head-shoulder target detection model detection unit is used to detect targets in a queue using the head-shoulder target detection model to obtain the initial queue target detection result; wherein the head-shoulder target detection model is a model obtained by adjusting and training the parameters of the initial target detection model using head-shoulder training enhancement data and transfer learning.

[0173] Further, based on any of the above embodiments, the identification number determination module 200 may include:

[0174] The identification number determination unit is used to determine the identification number corresponding to each target in each frame according to the initial queue target detection result by using the DeepSORT model.

[0175] Further, based on the above embodiment, the above re-tracking module 300 may include:

[0176] A speed and direction attribute determination unit, configured to add speed and direction attributes based on the identification corresponding to each target; wherein the speed and direction attributes include the speed and direction attributes of each target and the speed and direction attributes of the team as a whole;

[0177] The prediction and correction unit is used to use a Kalman filter correction algorithm to correct the Kalman filter prediction based on the speed and direction attributes to obtain real-time predicted and corrected target position information.

[0178] Further, based on the above embodiment, the above prediction and correction unit may include:

[0179] A prediction vector determination subunit is used to make predictions using an initialized Kalman filter, and in the prediction process, the state vector and the process noise covariance matrix are used to predict the next frame target position information of the detection target, and a prediction vector is obtained according to the next frame target position information;

[0180] An observation vector determination subunit is used to detect and track the target of the next frame using a target detection and tracking algorithm to obtain an observation vector; wherein the observation vector is a vector including the observed position, speed and direction angle;

[0181] a target observation vector determination subunit, used to determine the Euclidean distance between the prediction vector and the observation vector, and when the Euclidean distance is not greater than a set distance threshold, determine that the observation vector is a target observation vector;

[0182] The prediction and correction subunit is used to correct the predicted state vector and state covariance matrix using the target observation vector, observation matrix and observation noise covariance matrix to obtain the real-time predicted and corrected target position information.

[0183] Further, based on the above embodiment, the above target re-tracking device may further include:

[0184] The observation vector recalculation subunit is used to determine that the current observation vector is wrong when it is determined that the Euclidean distance is greater than the set distance threshold, and recalculate the observation vector using the speed and direction attributes of the team as a whole.

[0185] Further, based on any of the above embodiments, the target re-tracking device may further include:

[0186] A to-be-deleted set determination module is used to add targets that were not tracked in the previous frame into the to-be-deleted set;

[0187] The secondary tracking module is used to add the new target given by the current frame tracking algorithm to the set to be determined, and perform secondary tracking using the set to be deleted and the set to be determined to obtain the identification rematching result.

[0188] It should be noted that the order of the modules and units in the above target re-tracking device can be changed without affecting the logic.

[0189] A target re-tracking device provided by an embodiment of the present invention may include: an initial queue target detection result determination module 100, which is used to detect the queue using a target detection model to obtain an initial queue target detection result; an identification number determination module 200, which is used to obtain an identification number corresponding to each target in the queue according to the initial queue target detection result using a tracking algorithm; a re-tracking module 300, which is used to perform a first-level tracking according to the identification number using a target re-tracking model to obtain a target tracking result; wherein the target re-tracking model is a model that predicts the position information of the current frame target based on the position information of the previous frame target and the Kalman filter, and corrects the position information and Kalman filter parameters of the current frame target based on the observation information of the current frame target to achieve target tracking. It can be seen that compared with the current tracking of only a single person, the present application can intelligently track the entire queue, and by correcting the Kalman filter parameters, the position, speed and direction of the target are predicted and corrected in real time, so that the target will not be lost when occlusion or other interference factors occur, thereby achieving continuous tracking of the target. Moreover, this embodiment uses the head and shoulder target detection model for inspection, which avoids the problem of body occlusion that often occurs in the queue compared to the traditional complete human body detection model, and provides detection accuracy; and, the DeepSORT model is used to determine the representation of the target, which improves the applicability of the model; and, on the basis of the identification attribute, this embodiment adds speed and direction attributes, thereby using the team's unique attributes (speed, direction) to calculate the state distance and matching degree between the front and rear frame personnel in the team, solve the problems of ID mutual cutting and ID loss that occur during the tracking process, and achieve accurate tracking of personnel, thereby ensuring the accuracy of queuing time calculation; and, in the determination process, observation noise is used for calculation, which improves the accuracy of real-time prediction and correction of target position information; and, in this embodiment, considering the situation that the observation vector may be calculated inaccurately, the observation vector is recalculated, thereby improving the accuracy of the observation vector determination; and, in this embodiment, when determining each target in the set to be determined and matching it with the target in the set to be deleted in turn, multiple factors such as distance, angle, adjustment parameters of attenuation speed and weights of each factor are considered, so that the accuracy of matching degree calculation can be improved, thereby improving the accuracy of tracking result determination.

[0190] A target re-tracking device provided by an embodiment of the present invention is introduced below. The target re-tracking device described below and the target re-tracking method described above can refer to each other.

[0191] Please refer to Fig.11 , Fig.11 A schematic diagram of the structure of a target re-tracking device provided by an embodiment of the present invention may include:

[0192] A memory 10, used for storing computer programs;

[0193] The processor 20 is used to execute a computer program to implement the above-mentioned target re-tracking method.

[0194] The memory 10 , the processor 20 , and the communication interface 30 all communicate with each other via a communication bus 40 .

[0195] In the embodiment of the present invention, the memory 10 is used to store one or more programs, and the program may include program code, and the program code includes computer operation instructions. In the embodiment of the present invention, the memory 10 may store programs for implementing the following functions:

[0196] Use the target detection model to detect the queue and obtain the initial queue target detection result;

[0197] Using the tracking algorithm, the identification number corresponding to each target in the queue is obtained according to the initial queue target detection result;

[0198] The target re-tracking model is used to perform primary tracking according to the identification number to obtain the target tracking result; wherein, the target re-tracking model is a model that predicts the position information of the target in the current frame based on the position information of the target in the previous frame and the Kalman filter, and corrects the position information of the target in the current frame and the Kalman filter parameters based on the observation information of the target in the current frame to achieve target tracking.

[0199] In a possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0200] In addition, the memory 10 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include an NVRAM. The memory stores an operating system and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and processing hardware-based tasks.

[0201] The processor 20 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic device, a microprocessor or any conventional processor, etc. The processor 20 may call a program stored in the memory 10 .

[0202] The communication interface 30 may be an interface of a communication module, and is used to connect to other devices or systems.

[0203] Of course, it should be noted that Fig.11 The structure shown does not constitute a limitation on a target re-tracking device in an embodiment of the present invention. In actual applications, a target re-tracking device may include Fig.11 More or fewer components than shown, or combinations of certain components.

[0204] The computer-readable storage medium provided by an embodiment of the present invention is introduced below. The computer-readable storage medium described below and the target re-tracking method described above can be referred to each other.

[0205] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned target re-tracking method are implemented.

[0206] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0207] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0208] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0209] Finally, it should be noted that, in this article, relationships such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0210] The target re-tracking method, device, equipment and computer-readable storage medium provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A target re-tracking method, It is characterized in that include: Use the target detection model to detect the queue and obtain the initial queue target detection result; Using a tracking algorithm to obtain an identification number corresponding to each target in the queue according to the initial queue target detection result; On the basis of obtaining the identification number corresponding to each target, adding speed and direction attributes; wherein the speed and direction attributes include the speed and direction attributes of each target and the speed and direction attributes of the team as a whole; Using the initialized Kalman filter to perform prediction, in the prediction process, using the state vector and the process noise covariance matrix to predict the next frame target position information of the detection target, and obtaining a prediction vector according to the next frame target position information; Use the target detection and tracking algorithm to detect and track the target in the next frame and obtain the observation vector; Determine the Euclidean distance between the prediction vector and the observation vector, and when the Euclidean distance is not greater than a set distance threshold, determine that the observation vector is a target observation vector; Correcting the predicted state vector and state covariance matrix using the target observation vector, observation matrix and observation noise covariance matrix to obtain real-time predicted and corrected target position information; When it is determined that the Euclidean distance is greater than the set distance threshold, it is determined that the current observation vector is incorrect, and the observation vector is recalculated using the speed and direction attributes of the team as a whole.

2. The target re-tracking method according to claim 1, It is characterized in that The target detection model is used to detect the queue to obtain the initial queue target detection result, including: The targets in the queue are detected using a head-shoulder target detection model to obtain the initial queue target detection result; wherein the head-shoulder target detection model is a model obtained by adjusting and training the parameters of the initial target detection model using head-shoulder training enhancement data and transfer learning.

3. The target re-tracking method according to claim 1, It is characterized in that The method of obtaining an identification number corresponding to each target in the queue according to the initial queue target detection result by using a tracking algorithm includes: The DeepSORT model is used to determine the identification number corresponding to each target in each frame according to the initial queue target detection result.

4. The target re-tracking method according to claim 1, It is characterized in that After the predicted state vector and state covariance matrix are corrected by using the target observation vector, the observation matrix and the observation noise covariance matrix to obtain the real-time predicted and corrected target position information, the method further includes: Add the targets that were not tracked in the previous frame to the set to be deleted; The new target given by the current frame tracking algorithm is added to the set to be determined, and the set to be deleted and the set to be determined are used for secondary tracking to obtain the identification rematching result.

5. The target re-tracking method according to claim 1, It is characterized in that The observation vector is a vector including the observed position, velocity and direction angle.

6. A target re-tracking device, It is characterized in that include: An initial queue target detection result determination module is used to detect the queue using a target detection model to obtain an initial queue target detection result; An identification number determination module, used to obtain an identification number corresponding to each target in the queue according to the initial queue target detection result using a tracking algorithm; A re-tracking module is used to add speed and direction attributes based on the identification number corresponding to each target; wherein the speed and direction attributes include the speed and direction attributes of each target and the speed and direction attributes of the team as a whole; using the initialized Kalman filter to make predictions, in the prediction process, using the state vector and the process noise covariance matrix to predict the next frame target position information of the detected target, and obtaining a prediction vector according to the next frame target position information; using the target detection and tracking algorithm to detect and track the next frame target to obtain an observation vector; determining the Euclidean distance between the prediction vector and the observation vector, and when the Euclidean distance is not greater than the set distance threshold, determining that the observation vector is a target observation vector; The target observation vector, observation matrix and observation noise covariance matrix are used to correct the predicted state vector and state covariance matrix to obtain real-time predicted and corrected target position information; when it is determined that the Euclidean distance is greater than the set distance threshold, it is determined that the current observation vector is incorrect, and the observation vector is recalculated using the overall speed and direction attributes of the team.

7. A target re-tracking device, It is characterized in that include: Memory for storing computer programs; A processor, configured to implement the steps of the target re-tracking method as claimed in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the target re-tracking method according to any one of claims 1 to 5 are implemented.

Images