Attitude-enhanced multi-target tracking method and system

Abstract

The invention discloses a posture-enhanced multi-target tracking method and system, and belongs to the technical field of machine vision, and the method comprises the steps: carrying out the target detection of a to-be-detected video frame, and obtaining a detection result; target features are extracted based on the detection result, and appearance feature values and attitude feature values of the target are obtained; calculating an expansion intersection-union ratio of the detection result, and based on the expansion intersection-union ratio and the appearance characteristics and attitude characteristics of the target, performing track matching by adopting a preset track matching strategy to obtain a matching result; and realizing target tracking based on a matching result. By adopting the technical scheme of the invention, the tracking stability and accuracy can be effectively improved, and the tracking problem in a closed environment is solved.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a posture-enhanced multi-target tracking method and system. Background Art

[0002] Multi-Object Tracking (MOT) is an important research topic in the field of computer vision, which aims to track the trajectories of multiple dynamic targets at the same time. The MOT task accurately assigns a unique ID to each target from a video sequence and tracks the position and state of these targets in different time frames. This research problem is widely used in security monitoring, intelligent transportation, autonomous driving, sports analysis, drone monitoring and other fields.

[0003] The research background of MOT can be traced back to the early target tracking technology. The initial multi-target tracking methods relied on region-based tracking models, such as Kalman filtering and particle filtering, which predicted the target's motion trajectory and matched it through physical modeling. However, these methods performed poorly in multi-target environments, especially when there was occlusion or fast movement between targets, the accuracy and stability decreased significantly.

[0004] With the development of deep learning in recent years, the accuracy of deep learning detection algorithms has been greatly improved compared with traditional algorithms, and visual multi-target tracking has made great breakthroughs. With the development of target detection algorithms (such as YOLO and Faster R-CNN based on convolutional neural networks (CNN)), multi-target tracking based on detection (Tracking-by-Detection) has gradually become mainstream. This type of method first locates the target in the video frame through the target detection network, and then uses the data association algorithm (such as the Hungarian algorithm, shortest path matching, etc.) to match the target's motion trajectory. In order to further improve the tracking accuracy and robustness, deep learning technology has been introduced into the MOT field in recent years. Through an end-to-end neural network model, the motion characteristics and appearance information of the target are directly learned from the video frame. These methods can handle complex scene changes and target behaviors by training large neural networks, and are more adaptable to the uncertainty and complexity in practical applications.

[0005] However, although the existing technology has achieved good results in some specific scenarios, it still has several shortcomings. First, although the detection-based multi-target tracking method has become mainstream, it performs poorly when faced with problems such as target occlusion, long-term loss and identity exchange. Especially in high-density target scenes or situations where multiple targets interact rapidly, such as athletes in sports events or vehicles in traffic monitoring, traditional methods are difficult to maintain stable tracking. Secondly, complex backgrounds and severe occlusions further increase the difficulty of target recognition and matching, resulting in reduced tracking accuracy. In addition, objective factors such as camera movement and motion blur also pose challenges to tracking tasks and affect the robustness of the algorithm. Although deep learning has enhanced the model's ability to handle complex scenes, existing algorithms are still limited under certain conditions. For example, in a high-density target environment, the accuracy of feature extraction and data association is easily disturbed; for fast-moving targets, it becomes particularly difficult to predict their future positions. Therefore, improving the accuracy, robustness and real-time performance of the MOT algorithm under various complex conditions remains an urgent problem to be solved.

[0006] In summary, although the existing technologies perform well in some specific scenarios, they still need more improvement and development in a wide range of practical applications, especially in environments with dynamic changes and strong uncertainties. Summary of the invention

[0007] The present invention provides a posture-enhanced multi-target tracking method and system to solve the technical problem that the existing multi-target tracking method has unsatisfactory tracking effect in an environment with dynamic changes and strong uncertainty.

[0008] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0009] On the one hand, the present invention provides a posture-enhanced multi-target tracking method, comprising:

[0010] Perform target detection on the video frame to be detected and obtain the detection result;

[0011] Extract target features based on the detection results, and extract the target's appearance feature values ​​and posture feature values;

[0012] Calculate the extended intersection-over-union ratio of the detection results, and based on the extended intersection-over-union ratio and the appearance eigenvalues ​​and posture eigenvalues ​​of the target, use the preset trajectory matching strategy to perform trajectory matching to obtain the matching result;

[0013] Based on the matching results, target tracking is achieved.

[0014] Furthermore, when the trajectory detection is lost, the method further comprises:

[0015] The trajectory state is determined according to the following formula:

[0016]

[0017] Among them, S i Represents the state corresponding to the i-th trajectory; is the last seen time of the ith trajectory; is the start time when the i-th trajectory is first identified; β is the preset parameter; f c The current frame index.

[0018] Furthermore, based on the extended intersection-over-union ratio and the appearance feature value and posture feature value of the target, a preset trajectory matching strategy is used to perform trajectory matching to obtain a matching result, including:

[0019] Setting a first cost threshold and a second cost threshold;

[0020] The extracted appearance feature values ​​and posture feature values ​​are adjusted: if the original value of the appearance feature value is greater than the first cost threshold or the expanded intersection-and-union ratio is greater than the second cost threshold, the appearance feature value is adjusted to 1, otherwise, the original value of the appearance feature value is retained; if the original value of the posture feature value is greater than the first cost threshold or the expanded intersection-and-union ratio is greater than the second cost threshold, the posture feature value is adjusted to 1, otherwise, the original value of the posture feature value is retained;

[0021] The adjusted appearance eigenvalues ​​and posture eigenvalues ​​are weighted combined to obtain the fused eigenvalues. The formula is:

[0022]

[0023] in, is the fusion feature value corresponding to the i-th trajectory; is the adjusted posture feature value corresponding to the i-th trajectory; is the adjusted appearance feature value corresponding to the i-th track; λ is the weight value;

[0024] Compare the fused feature value with the expanded intersection-union ratio, and take the minimum value as the association cost;

[0025] Based on the association cost, the Hungarian algorithm is used to obtain the best match between the trajectory and the current detection.

[0026] Furthermore, the calculation formula of the weight value λ is:

[0027]

[0028] Among them, λ min Indicates the minimum value of the preset weight value; λ max represents the maximum value of the preset weight value; P represents the occlusion ratio of the occluded objects around the target; T represents the preset occlusion ratio threshold.

[0029] On the other hand, the present invention also provides a posture-enhanced multi-target tracking system, comprising:

[0030] The target detection module is used to perform target detection on the video frame to be detected and obtain the detection result;

[0031] A target feature extraction module is used to extract target features based on the detection results obtained by the target detection module, and extract appearance feature values ​​and posture feature values ​​of the target;

[0032] The trajectory matching module is used to calculate the extended intersection-and-union ratio of the detection results, and based on the extended intersection-and-union ratio and the appearance feature values ​​and posture feature values ​​of the target, a preset trajectory matching strategy is used to perform trajectory matching to obtain a matching result;

[0033] The target tracking module is used to realize target tracking based on the matching result obtained by the trajectory matching module.

[0034] Furthermore, the system further includes a track time adjustment module, which is used to determine the track state according to the following formula when the track detection is lost:

[0035]

[0036] Among them, S i Represents the state corresponding to the i-th trajectory; is the last seen time of the ith trajectory; is the start time when the i-th trajectory is first identified; β is the preset parameter; f c The current frame index.

[0037] Furthermore, the trajectory matching module is specifically used for:

[0038] Setting a first cost threshold and a second cost threshold;

[0039] The extracted appearance feature values ​​and posture feature values ​​are adjusted: if the original value of the appearance feature value is greater than the first cost threshold or the expanded intersection-and-union ratio is greater than the second cost threshold, the appearance feature value is adjusted to 1, otherwise, the original value of the appearance feature value is retained; if the original value of the posture feature value is greater than the first cost threshold or the expanded intersection-and-union ratio is greater than the second cost threshold, the posture feature value is adjusted to 1, otherwise, the original value of the posture feature value is retained;

[0040] The adjusted appearance eigenvalues ​​and posture eigenvalues ​​are weighted combined to obtain the fused eigenvalues. The formula is:

[0041]

[0042] in, is the fusion feature value corresponding to the i-th trajectory; is the adjusted posture feature value corresponding to the i-th trajectory; is the adjusted appearance feature value corresponding to the i-th track; λ is the weight value;

[0043] Compare the fused feature value with the expanded intersection-union ratio, and take the minimum value as the association cost;

[0044] Based on the association cost, the Hungarian algorithm is used to obtain the best match between the trajectory and the current detection.

[0045] Furthermore, the calculation formula of the weight value λ is:

[0046]

[0047] Among them, λ min Indicates the minimum value of the preset weight value; λ max represents the maximum value of the preset weight value; P represents the occlusion ratio of the occluded objects around the target; T represents the preset occlusion ratio threshold.

[0048] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.

[0049] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the above method.

[0050] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0051] 1. This paper develops a new correlation technology to specifically address the tracking challenges caused by dynamically changing motion patterns and appearance similarities. This technology cleverly combines two-dimensional and three-dimensional information, which can effectively address the tracking difficulties caused by targets with changing motion patterns or similar appearances, and improve the stability and accuracy of tracking.

[0052] 2. The solution of the present invention innovatively adds three-dimensional posture features as auxiliary information, and integrates them with appearance weights to form a more sophisticated recognition mechanism, thereby significantly reducing the ID conversion rate, solving the problem of frequent ID conversion, greatly reducing identity confusion, and improving overall tracking accuracy.

[0053] 3. The solution of the present invention proposes a new re-identification strategy, that is, maintaining the track for an adjustable length of time during the tracking process. This method not only improves the accuracy of tracking, but also realizes the effective re-identification of targets that have not appeared in the picture for a long time. It significantly enhances the tracking continuity in complex closed environments and solves the problem of difficulty in re-identifying targets in closed environments after they have been out of view for a long time. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 is a framework diagram of a posture-enhanced multi-target tracking method provided by an embodiment of the present invention;

[0056] Figure 2 is a visualization result diagram of the posture-enhanced multi-target tracking method provided by an embodiment of the present invention;

[0057] Figure 3 It is a system block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0059] First of all, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplarily" is intended to present the concept in a concrete way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0060] First embodiment

[0061] This embodiment provides a posture-enhanced multi-target tracking method that follows the classic paradigm of tracking by detection, which can also achieve online tracking without using future information. The method first applies the object detector YOLOX on each input frame to obtain the detection result (detection box); then, auxiliary information is extracted based on the detection box, including the weighting between the similarity between appearance features and the restored human posture. The weighted features are then fused with the expanded EIoU (intersection over union) to obtain the association cost between the upper and lower frames. After the association cost is obtained, the Hungarian algorithm is performed to obtain the best match between the trajectory and the detection. Finally, the lost trajectory is dynamically adjusted during the tracking process.

[0062] Specifically, the execution process of the method includes the following steps:

[0063] S1, performing target detection on the video frame to be detected and obtaining the detection result;

[0064] Specifically, in this embodiment, YOLOX is used as an object detector to obtain a target detection frame.

[0065] S2, extracting target features based on the detection results, extracting the target's appearance feature value and posture feature value;

[0066] Specifically, in this embodiment, the appearance feature is generated by the appearance extraction network OSNET; and the posture feature is obtained by using the HMR2.0 model; this model can restore the posture, shape and camera parameters of the human body in a given image, and the parameters are based on the SMPL human body model. The SMPL model is a low-dimensional parameter representation of the human body. By providing the posture parameter θ b ∈R 24×3×3 and shape parameter β∈R 10 , the SMPL model generates a 3D mesh M∈R with 6890 vertices 3×N . Body joint X∈R 3×k is defined as a linear combination of these vertices, which can be expressed as X = MW, where W∈R N×k is a fixed weight matrix. π is the camera parameter, and the pose parameter θ includes both the body pose parameter θ b ∈R 24×3×3 , including the global direction θ g ∈R 3×3 Finally, this embodiment integrates these parameters into posture features C p , as shown in formula (1):

[0067] C p =[θ b ,θ g ,β,π] (1)

[0068] S3, calculating the extended intersection-over-union ratio of the detection result, and based on the extended intersection-over-union ratio and the appearance feature value and posture feature value of the target, using a preset trajectory matching strategy to perform trajectory matching to obtain a matching result;

[0069] Specifically, in this embodiment, the implementation process of the above S3 is as follows:

[0070] S31, calculate the extended intersection-over-union (EIoU) of each detection box;

[0071] S32, thresholding the cost filter for association based on appearance and pose using EIoU;

[0072] S33, adjust the extracted appearance feature value and posture feature value: if the original value of the appearance feature value is greater than the threshold τ1 or the expanded intersection-and-union ratio is greater than the threshold τ2, the appearance feature value is adjusted to 1, otherwise, the original value of the appearance feature value is retained; if the original value of the posture feature value is greater than the threshold τ1 or the expanded intersection-and-union ratio is greater than the threshold τ2, the posture feature value is adjusted to 1, otherwise, the original value of the posture feature value is retained; the formula is expressed as:

[0073]

[0074] in, represents the adjusted characteristics; C p,id Indicates the original posture and appearance characteristics; C EIoU represents the extended intersection-over-union ratio; τ1 and τ2 represent the thresholds of the cost filter, which are set to 0.25 and 0.5 respectively in this embodiment.

[0075] In addition, it should be noted that the above-mentioned appearance feature values ​​and posture feature values ​​are essentially cost functions for different target features, which are exact numerical values. Therefore, feature value adjustment can be achieved by comparing them with corresponding thresholds.

[0076] S34, weighted combination of the adjusted appearance feature value and posture feature value to obtain a fusion feature value, the formula is:

[0077]

[0078] in, is the fusion feature value corresponding to the i-th trajectory; is the adjusted posture feature value corresponding to the i-th trajectory; is the adjusted appearance feature value corresponding to the i-th track; λ is the weight value;

[0079] Furthermore, in order to make the auxiliary information more robust during the fusion process, this embodiment proposes an adaptive adjustment mechanism that dynamically balances the human posture feature C according to the degree of occlusion around the target.p and human appearance characteristics C id Specifically, this embodiment introduces an adjustable weight value λ to quantify the impact of occlusion and adjusts the relative importance of posture and appearance accordingly. First, this embodiment uses the overlap degree of EIoU to detect the occlusion ratio (expressed as P). When the proportion of occluded objects around the detected target exceeds a predefined threshold T, the weight on C is automatically reduced. id reliance on C p This adaptive strategy helps to maintain stable target tracking performance under different occlusion conditions.

[0080] Based on the above, the update rule of λ is given by:

[0081]

[0082] Among them, λ max ,λ min There are two thresholds for adjusting λ, which are 0.2 and 0.15 respectively.

[0083] S35, comparing the fused feature value with the expanded intersection-and-union ratio, and taking the minimum value as the association cost;

[0084] S36, based on the association cost, uses the Hungarian algorithm to obtain the best match between the trajectory and the current detection.

[0085] S4, based on the matching results, realize target tracking.

[0086] S5, track life management; it should be noted that, in order to solve the problem of tracking tracks in closed spaces, this embodiment introduces a new strategy for managing track life, which is not only based on static thresholds, but also takes into account the duration of track recognition and its missing time. This strategy helps to effectively re-identify individuals who temporarily exit the video frame due to camera movement or self-motion, ensuring more robust and continuous tracking performance. This embodiment is called track time regulation TTR; its formula is expressed as:

[0087]

[0088] Among them, S i Represents the state corresponding to the i-th trajectory; is the last seen (end) time of the ith trajectory; is the start time when the i-th track is first identified; β is an adjustable parameter set to 60 frames, used as a buffer; f c The current frame index.

[0089] When the track detection is lost, the TTR module will determine whether to delete or retain it.

[0090] In summary, this embodiment provides a multi-target tracking method with posture enhancement, and its overall framework is as follows: Figure 1 As shown in the figure, this method predicts the position, posture and appearance of the human body for tracking: first, the human body is remodeled into a 3D posture and the corresponding appearance features are extracted. After the detection frame obtained by the tracking algorithm is calculated using the intersection-over-union ratio (EIOU), the minimum value is selected from the fusion features of the posture and appearance features to obtain the result. During the tracking process, the information between the upper and lower frames in the video is directly used for correlation analysis, and the visualization result is shown in the figure. Figure 2 shown.

[0091] The programming, training, debugging and test result display of the above algorithm are all implemented in the programming language python. The programming is mainly implemented by PyTorch1.13.0 deep learning framework + python3.9, and the hardware is accelerated by a server configured with Intel i9-10940X CPU and two RTX3090 graphics cards. The training data sets include SportsMOT and SoccerNET-Tracking. Table 1 shows the comparison results between the performance of the method of the present invention (denoted as POSE-TRACK) and the existing methods on the SportsMOT data set.

[0092] Table 1 Comparison results of the performance of the proposed method and the existing methods on the SportsMOT dataset

[0093]

[0094] From the data in Table 1, we can see that POSE-TRACK outperforms the current SOTA tracker in the key indicator HOTA. HOTA is a comprehensive metric that aims to balance detection accuracy and identity consistency. It takes into account three key aspects: detection accuracy (DetA), association accuracy (AssA), and final identity consistency (IDF1). The higher the value, the better the tracking effect. The uniqueness of HOTA is that it can simultaneously evaluate the quality of target detection and identity assignment. As shown in Table 1, the indicators of the proposed method are HOTA of 79.2, IDF1 of 84.1, AssA of 71.0, and the lag indicators are MOTA and DetA, which are 0.6 and 1, respectively. The growing indicators show significant improvement, proving the effectiveness of the proposed method on sports datasets. There are significant improvements in target assignment and ID conversion.

[0095] This example uses the official original detection in the SoccerNET-Tracking dataset. The results are shown in Table 2.

[0096] Table 2 Index data between the method of the present invention (denoted as POSE-TRACK) and the existing method

[0097]

[0098] The data in Table 2 show that the main indicators (HOTA, AssA, DetA) of the method of the present invention are higher than those of the existing tracking algorithms, which further proves the effectiveness of the method of the present invention.

[0099] Second embodiment

[0100] This embodiment provides a posture-enhanced multi-target tracking system, which includes the following modules:

[0101] The target detection module is used to perform target detection on the video frame to be detected and obtain the detection result;

[0102] A target feature extraction module is used to extract target features based on the detection results obtained by the target detection module, and extract appearance feature values ​​and posture feature values ​​of the target;

[0103] The trajectory matching module is used to calculate the extended intersection-and-union ratio of the detection results, and based on the extended intersection-and-union ratio and the appearance feature values ​​and posture feature values ​​of the target, a preset trajectory matching strategy is used to perform trajectory matching to obtain a matching result;

[0104] The target tracking module is used to realize target tracking based on the matching result obtained by the trajectory matching module.

[0105] The track time adjustment module is used to determine the track state according to the following formula when the track detection is lost:

[0106]

[0107] Among them, S i Represents the state corresponding to the i-th trajectory; is the last seen time of the ith trajectory; is the start time when the i-th trajectory is first identified; β is the preset parameter; f c The current frame index.

[0108] It should be noted that the posture-enhanced multi-target tracking system of the present embodiment corresponds to the posture-enhanced multi-target tracking method of the first embodiment; the functions implemented by each functional module in the posture-enhanced multi-target tracking system of the present embodiment correspond one-to-one to each process step in the posture-enhanced multi-target tracking method of the first embodiment; therefore, they will not be repeated here.

[0109] Third embodiment

[0110] This embodiment provides an electronic device, such as Figure 3 As shown, the electronic device includes: a processor and a memory; wherein the processor and the memory can be connected via a communication bus; the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method of the first embodiment. In addition, the electronic device may also include a transceiver, the processor and the transceiver can be connected via a communication bus, and the transceiver is used to communicate with other devices.

[0111] Next, combine Figure 3 The following is a detailed introduction to the various components of the electronic device:

[0112] Among them, the processor is the control center of the electronic device, and the electronic device may include multiple processors, each of which may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPUs), or other general-purpose processors, application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor may execute various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.

[0113] In a specific implementation, as an embodiment, the processor may include one or more CPUs, such as Figure 3 The CPU0 and CPU1 shown in the figure are, of course, only exemplary.

[0114] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0115] Optionally, the memory may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor or exist independently and accessed through the interface circuit ( Figure 3 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0116] The transceiver may include a receiver and a transmitter ( Figure 3 The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function. The transceiver can be integrated with the processor or exist independently and communicate with the electronic device through the interface circuit ( Figure 3 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0117] In addition, it should be noted that Figure 3 The structure of the electronic device shown in the figure does not constitute a limitation on the device, and the actual device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In addition, the technical effects achieved by the electronic device when executing the method of the first embodiment above can refer to the technical effects described in the first embodiment above, so they are not repeated here.

[0118] Fourth embodiment

[0119] This embodiment provides a computer-readable storage medium, which stores at least one instruction, and the instruction is loaded and executed by a processor to implement the method of the first embodiment. The computer-readable storage medium may be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. The instructions stored therein may be loaded by a processor in a terminal to execute the method.

[0120] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiment of the present invention can be in the form of a full or partial hardware embodiment, a full or partial software embodiment or an embodiment combining software and hardware. Moreover, when implemented using software, the embodiment of the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center containing one or more available media sets. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium. The semiconductor medium may be a solid state hard disk.

[0121] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0123] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements. In addition, the term "and / or" is only an association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone, wherein A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding. "At least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can be represented by: a, b, c, ab, ac, bc or abc, where a, b, c can be single or plural.

[0124] In addition, it can be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0125] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0126] In several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of functional modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0127] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0128] Finally, it should be noted that the above is only a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for ordinary technicians in this technical field, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A posture-enhanced multi-target tracking method, characterized in that: include: Perform target detection on the video frame to be detected and obtain the detection result; Extract target features based on the detection results, and extract the target's appearance feature values ​​and posture feature values; Calculate the extended intersection-over-union ratio of the detection results, and based on the extended intersection-over-union ratio and the appearance eigenvalues ​​and posture eigenvalues ​​of the target, use the preset trajectory matching strategy to perform trajectory matching to obtain the matching result; Based on the matching results, target tracking is achieved.

2. The posture-enhanced multi-target tracking method according to claim 1, characterized in that: When the track detection is lost, the method further comprises: The trajectory state is determined according to the following formula: Among them, S i Represents the state corresponding to the i-th trajectory; is the last seen time of the ith trajectory; is the start time when the i-th trajectory is first identified; β is the preset parameter; f c The current frame index.

3. The posture-enhanced multi-target tracking method according to claim 1, characterized in that: The method of performing trajectory matching based on the extended intersection-over-union ratio and the appearance feature value and posture feature value of the target using a preset trajectory matching strategy to obtain a matching result includes: Setting a first cost threshold and a second cost threshold; The extracted appearance feature values ​​and posture feature values ​​are adjusted: if the original value of the appearance feature value is greater than the first cost threshold or the expanded intersection-and-union ratio is greater than the second cost threshold, the appearance feature value is adjusted to 1, otherwise, the original value of the appearance feature value is retained; if the original value of the posture feature value is greater than the first cost threshold or the expanded intersection-and-union ratio is greater than the second cost threshold, the posture feature value is adjusted to 1, otherwise, the original value of the posture feature value is retained; The adjusted appearance eigenvalues ​​and posture eigenvalues ​​are weighted combined to obtain the fused eigenvalues. The formula is: in, is the fusion feature value corresponding to the i-th trajectory; is the adjusted posture feature value corresponding to the i-th trajectory; is the adjusted appearance feature value corresponding to the i-th track; λ is the weight value; Compare the fused feature value with the expanded intersection-union ratio, and take the minimum value as the association cost; Based on the association cost, the Hungarian algorithm is used to obtain the best match between the trajectory and the current detection.

4. The posture-enhanced multi-target tracking method according to claim 3, characterized in that: The calculation formula of weight value λ is: Among them, λ min Indicates the minimum value of the preset weight value; λ max represents the maximum value of the preset weight value; P represents the occlusion ratio of the occluded objects around the target; T represents the preset occlusion ratio threshold.

5. A multi-target tracking system with posture enhancement, characterized in that: include: The target detection module is used to perform target detection on the video frame to be detected and obtain the detection result; A target feature extraction module is used to extract target features based on the detection results obtained by the target detection module, and extract appearance feature values ​​and posture feature values ​​of the target; The trajectory matching module is used to calculate the extended intersection-and-union ratio of the detection results, and based on the extended intersection-and-union ratio and the appearance feature values ​​and posture feature values ​​of the target, a preset trajectory matching strategy is used to perform trajectory matching to obtain a matching result; The target tracking module is used to realize target tracking based on the matching result obtained by the trajectory matching module.

6. The posture-enhanced multi-target tracking system according to claim 5, characterized in that: The system also includes a track time adjustment module, which is used to determine the track state according to the following formula when track detection is lost: Among them, S i Represents the state corresponding to the i-th trajectory; is the last seen time of the ith trajectory; is the start time when the i-th trajectory is first identified; β is the preset parameter; f c The current frame index.

7. The posture-enhanced multi-target tracking system according to claim 5, characterized in that: The trajectory matching module is specifically used for: Setting a first cost threshold and a second cost threshold; The extracted appearance feature values ​​and posture feature values ​​are adjusted: if the original value of the appearance feature value is greater than the first cost threshold or the expanded intersection-and-union ratio is greater than the second cost threshold, the appearance feature value is adjusted to 1, otherwise, the original value of the appearance feature value is retained; if the original value of the posture feature value is greater than the first cost threshold or the expanded intersection-and-union ratio is greater than the second cost threshold, the posture feature value is adjusted to 1, otherwise, the original value of the posture feature value is retained; The adjusted appearance eigenvalues ​​and posture eigenvalues ​​are weighted combined to obtain the fused eigenvalues. The formula is: in, is the fusion feature value corresponding to the i-th trajectory; is the adjusted posture feature value corresponding to the i-th trajectory; is the adjusted appearance feature value corresponding to the i-th track; λ is the weight value; Compare the fused feature value with the expanded intersection-union ratio, and take the minimum value as the association cost; Based on the association cost, the Hungarian algorithm is used to obtain the best match between the trajectory and the current detection.

8. The posture-enhanced multi-target tracking system according to claim 7, characterized in that: The calculation formula of weight value λ is: Among them, λ min Indicates the minimum value of the preset weight value; λ max represents the maximum value of the preset weight value; P represents the occlusion ratio of the occluded objects around the target; T represents the preset occlusion ratio threshold.

Images