Method, apparatus, device, readable medium and program product for target tracking

Abstract

Embodiments of this disclosure provide a method, apparatus, device, readable medium, and program product for target tracking. The method includes: performing target detection on a target frame in a target video to obtain at least one first bounding box; acquiring the motion trajectory of at least one object in the target video; and determining a target bounding box for one or more target objects in the target frame based on the at least one first bounding box and the motion trajectory of the at least one object. In this manner, reliance on the appearance features of the target object for tracking can be avoided, thereby improving the accuracy of target tracking.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computer vision technology, and more specifically, to methods, apparatus, devices, computer-readable storage media, and computer program products for target tracking. Background Technology

[0002] Multi-target tracking refers to the automatic identification and tracking of multiple targets of interest in a video sequence, and the continuous provision of target location data for target determination and trajectory prediction. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for target tracking is provided, comprising: performing target detection on a target frame in a target video to obtain at least one first bounding box, each first bounding box defining an object in the target frame; obtaining a movement trajectory of each of the at least one object in the target video, the movement trajectory of each object indicating the position of the object in a series of consecutive frames preceding the target frame; and determining a target bounding box of one or more target objects in the target frame based on the at least one first bounding box and the respective movement trajectories of the at least one object.

[0004] In a second aspect of this disclosure, an apparatus for target tracking is provided, comprising: a detection module configured to perform target detection on a target frame in a target video to obtain at least one first bounding box, each first bounding box defining an object in the target frame; an acquisition module configured to acquire a movement trajectory of at least one object in the target video, the movement trajectory of each object indicating the position of the object in a plurality of consecutive frames preceding the target frame; and a determination module configured to determine a target bounding box of one or more target objects in the target frame based on the at least one first bounding box and the respective movement trajectory of the at least one object.

[0005] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method of the first aspect of this disclosure when executed by the at least one processing unit.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program that can be executed by a processor to perform the method according to a first aspect of this disclosure.

[0007] In a fifth aspect of this disclosure, a computer program product is provided, including computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method of the first aspect.

[0008] It should be understood that the description in the Summary of the Invention section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] In the following detailed description, in conjunction with the accompanying drawings, the above and other features, advantages, and aspects of the various implementations of this disclosure will become more apparent. In the accompanying drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0011] Figure 2 A flowchart of a process for target tracking according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A flowchart illustrating a target tracking process according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A structural architecture diagram of a predictor according to some embodiments of the present disclosure is shown;

[0014] Figure 5 A block diagram of an apparatus for target tracking according to some embodiments of the present disclosure is shown; and

[0015] Figure 6 A block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented is shown. Detailed Implementation

[0016] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0017] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0018] As used in this paper, the term "model" refers to a system that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. In this paper, "model" may also be referred to as a "machine learning model," "machine learning network," "neural network," or simply "network," and these terms are used interchangeably.

[0019] A neural network is a machine learning network based on deep learning. A neural network processes input and provides a corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning applications often include many hidden layers, thus increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer is provided as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output serves as the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each node processing the input from the layer above.

[0020] Machine learning typically comprises three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating its parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as the input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. In the application phase, the model can be used to process actual inputs based on the trained parameter values ​​to determine the corresponding output.

[0021] It should be noted that the acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0022] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0023] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information, thereby enabling the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers or storage media that perform the operation of the technical solution disclosed herein, based on the prompt message.

[0024] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, for example, via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0025] As briefly mentioned earlier, multi-object tracking refers to the automatic identification and tracking of multiple objects of interest in a video sequence, continuously providing object location data for determining the objects and predicting their trajectories. When tracking multiple objects, a detection-by-detection approach is typically used. This involves first detecting the spatial location of the objects of interest, and then associating the obtained bounding boxes with the objects' behavior or motion information to form a trajectory.

[0026] The detection-then-tracking method relies on the appearance features of the target object when performing data association. In scenarios where the appearance of the target object is consistent and frequently occluded, the appearance features of the target object cannot accurately distinguish between different target objects.

[0027] Embodiments of this disclosure propose a scheme for target tracking. According to various embodiments of this disclosure, target detection is performed on target frames in a target video to obtain at least one first bounding box, each first bounding box defining an object in the target frame. Motion trajectories of at least one object in the target frame are obtained, each object's trajectory indicating its position in several consecutive frames preceding the target frame. Based on the at least one first bounding box and the respective motion trajectories of at least one object, target bounding boxes of one or more target objects within the target frame are determined. In this manner, target bounding boxes of one or more target objects within the target frame can be determined based on the positions of at least one object in the target frame in several consecutive frames preceding the target frame and at least one first bounding box in the target frame, avoiding the use of the object's appearance features for target object tracking, thereby improving the accuracy of target tracking.

[0028] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown.

[0029] In environment 100, acquisition device 110 may include any device used for acquiring video data, such as an image sensor, infrared sensor, etc. Acquisition device 110 is communicatively connected to server device 120, and can transmit the acquired video data to server 120.

[0030] In environment 100, server 120 can detect objects included in the video data collected by acquisition device 110, track each object in the video data collected by acquisition device 110, and output each tracked object.

[0031] In some embodiments, a target tracking system 130 may be deployed in server 120. The target tracking system 130 may include a detector 131, a predictor 132, and a tracker 133. Detector 131 is configured to detect information about various objects contained in video frame 140, such as object identification, bounding boxes, motion trajectories, etc. Predictor 132 is configured to predict the position information of an object in a target frame based on multiple consecutive frames preceding the target frame in video frame 130. Tracker 133 is configured to output a tracking result 150 for tracking various objects in the target video, based on the outputs of detector 131 and predictor 132.

[0032] In some embodiments, server 120 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server-side equipment 120 may include, for example, computing systems / servers, such as mainframes, edge computing nodes, computing devices in a cloud environment, etc.

[0033] The communication connection between the data acquisition device 110 and the server 120 can be established via wired or wireless means. The communication connection may include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus (USB), and Wireless Fidelity (WiFi) connections; the embodiments disclosed herein are not limited in this respect.

[0034] It should be understood that the structure and function of environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0035] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0036] Figure 2 A flowchart of a target tracking process according to some embodiments of the present disclosure is shown. The following is in conjunction with… Figure 1 This disclosure provides a detailed description of the target tracking solution.

[0037] In box 210, detector 131 performs target detection on target frames in the target video to obtain at least one first bounding box, each first bounding box being used to frame an object in the target frame.

[0038] The target frame can be any frame in the target video. The target video can be acquired and transmitted by the acquisition device 110 or transmitted to the server 120 in real time, or it can be preprocessed by the acquisition device 110 before being transmitted to the server 120. For example, preprocessing the acquired video by the acquisition device 110 can include dividing the video into segments of predetermined length, adjusting the video resolution, etc.

[0039] Detector 131 can be used to identify objects in a target frame and output information about at least one object. Specifically, the target frame is used as input to detector 131, and the output is information about at least one identified object. For example, the information about at least one object may include the object's identifier, the object's bounding box, etc.

[0040] Detector 131 may include a pre-selected model or a pre-trained model. For example, detector 131 may include a network for object detection. For example, detector 131 may include convolutional neural networks (CNN), recurrent neural networks (RNN), region-based convolutional neural networks (R-CNN), or fast R-CNN (Faster-RCNN), etc.

[0041] The bounding boxes obtained by detector 131 from detecting objects in the target frame can be called first bounding boxes. For example, at least one first bounding box can be represented as follows:

[0042] D t ={b1,b2,……}

[0043] Among them, D t Let b1, b2, ... represent the set of first bounding boxes in frame t, where b1, b2, ... represent multiple first bounding boxes, and t represents the frame number in the target video.

[0044] In some embodiments, a bounding box can be represented using coordinates within the bounding box region, and the width and height of the bounding box. For example, a bounding box can be represented as follows:

[0045] b = {x, y, w, h}

[0046] Where x and y represent the coordinates of any point in the bounding box in the video frame coordinate system (e.g., the coordinates of the top left corner), and w and h represent the width and height of the bounding box.

[0047] In box 220, detector 131 acquires the movement trajectory of at least one object in the target video, the movement trajectory of each object indicating the position of the object in a series of consecutive frames preceding the target frame.

[0048] Detector 131 can identify multiple consecutive frames in the target video preceding the target frame to obtain information about at least one object in those frames. The information about the at least one object may include, for example, the object's identity and position within the multiple consecutive frames. Detector 131 can determine the movement trajectory of each of the at least one object based on its identity and position (e.g., bounding box) within the multiple consecutive frames.

[0049] In some embodiments, the movement trajectory of each object indicates a sequence of bounding boxes in several consecutive frames preceding the target frame. These bounding boxes may be referred to as second bounding boxes. The order of the second bounding boxes in the sequence can be ordered according to the order of several consecutive frames in the target video. The movement trajectory of an object, i.e., the sequence of second bounding boxes, can be represented as follows:

[0050] T = {T1, T2, ...}

[0051] Where T represents the movement trajectory, and T1, T2, ... represent the second bounding boxes in multiple frames.

[0052] In some embodiments, the sequence of second bounding boxes may include the second bounding box in the previous frame adjacent to the target frame, or it may not include the second bounding box in the previous frame adjacent to the target frame. For ease of discussion, the movement trajectory that includes the second bounding box in the previous frame adjacent to the target frame can be referred to as the movement trajectory of the active state, and the movement trajectory that does not include the second bounding box in the previous frame adjacent to the target frame can also be referred to as the movement trajectory of the active state.

[0053] In some embodiments, it can be determined whether the movement trajectory is an active movement trajectory or a lost movement trajectory by determining whether multiple consecutive frames preceding the target frame contain the previous frame adjacent to the target frame.

[0054] For example, the target video consists of n frames, and the target frame is frame t, where t ≤ n. The movement trajectory of the active state can be T. active ={……,b t-2 ,b t-1 The movement trajectory of the lost state can be T. lost ={……,b t-3 ,b t-2}, or T lost ={……,b t-4 ,b t-3} etc. Among them, T active The trajectory representing the active state, T lost This represents the movement trajectory when the state is lost.

[0055] In box 230, tracker 133 determines the target bounding box of one or more target objects in the target frame based on at least one first bounding box and the respective movement trajectories of at least one object.

[0056] See Figure 3When the movement trajectory only includes the movement trajectory of the active state, the predictor 132 can first determine the first candidate bounding box of the object corresponding to the active state movement trajectory in the target frame based on the active state movement trajectory. Then, the tracker 133 can match the first candidate bounding box of the object corresponding to the active state movement trajectory in the target frame with at least one first bounding box, and determine the first candidate bounding box that successfully matches at least one first bounding box as the target bounding box of the corresponding object in the target frame.

[0057] Specifically, see Figure 4 The predictor 132 may include a temporal embedding layer and multiple coding layers connected in sequence. Figure 4 The example shows three (but more can be used in practice) and an output layer. Each encoding layer can include a forward Mamba block, a reverse Mamba block, a normalization layer, and a fully connected layer. The forward and reverse Mamba blocks are connected to the normalization layer, which is connected to the fully connected layer. The temporal embedding layer can embed trajectory features into a higher-dimensional space to obtain the input embedding. The encoding layer encodes the input embedding, and the output layer predicts the bounding box offset of the object based on the encoded input embedding and decodes the bounding box offset into the first candidate bounding box in the target frame.

[0058] For example, both forward Mamba blocks and reverse Mamba blocks can be a state-space model.

[0059] In some embodiments, predictor 132 may determine trajectory features for at least one object based on the movement trajectory of the active state, each trajectory feature indicating the positional change of the object between adjacent frames in a series of frames (e.g., positional change of a second bounding box). Tracker 132 may also determine a first candidate bounding box for each of the at least one object in a target frame based on the trajectory features.

[0060] For example, the trajectory features of an object corresponding to an active movement trajectory can be represented as follows:

[0061]

[0062] Where O represents the trajectory feature, n represents the time, O n-1 This represents the trajectory feature vector at time n-1.

[0063] For example, the trajectory feature vector at time n-1 can be represented as follows:

[0064]

[0065] in, This represents the change in the x-coordinate of the center point of the bounding box relative to the previous time step. δ represents the change in the ordinate of the center point of the bounding box relative to the previous time step. w δ represents the change in the width of the bounding box relative to the previous time step. h This represents the change in the height of the bounding box relative to the previous time step.

[0066] Using trajectory feature O as input to predictor 132, the position change of the bounding box of the object in the target frame relative to the position change of the bounding box in the adjacent previous frame of the target frame can be predicted by predictor 132, and the bounding box of the object in the target frame can be predicted based on the position change and the position of the bounding box in the adjacent previous frame of the target frame.

[0067] For example, in the first encoding of an input embedding X, its forward process can be represented as follows:

[0068]

[0069]

[0070] Mamba represents the forward Mamba block. denoted as the reverse Mamba block, LN represents the normalization operator, and MLP represents a multilayer perceptron consisting of two fully connected layers.

[0071] Furthermore, predictor 132 can determine the second candidate bounding box of the object corresponding to the movement trajectory of the activation state that failed to match for the first time in the previous frame adjacent to the target frame, based on the movement trajectory of the activation state that failed to match for the first time. Subsequently, tracker 133 can determine the third candidate bounding box of the object corresponding to the movement trajectory of the activation state that failed to match for the first time in the target frame using an autoregressive method based on the second candidate bounding box and the movement trajectory of the activation state that failed to match for the first time. The third candidate bounding box that successfully matches the first candidate bounding box that failed to match for the first time is determined as the target bounding box of the corresponding object in the target frame.

[0072] It should be understood that the process of determining the second candidate bounding box of the object corresponding to the first failed match's movement trajectory in the previous frame adjacent to the target frame from the movement trajectory of the active state is similar to the process of determining the first candidate bounding box of the object corresponding to the active state's movement trajectory in the target frame from the movement trajectory of the active state, and will not be elaborated here.

[0073] Furthermore, if the movement trajectory includes not only the movement trajectory of the active state but also the movement trajectory of the lost state, then the predictor 132 can determine the fourth candidate bounding box of the object corresponding to the movement trajectory of the lost state in the previous frame adjacent to the target frame based on the movement trajectory of the lost state. The tracker 133 can determine the fifth candidate bounding box of the object corresponding to the movement trajectory of the lost state in the target frame using an autoregressive method based on the movement trajectory of the lost state and the fourth candidate bounding box, and then match the fifth candidate bounding box with the first bounding box that failed to match, and determine the fifth candidate bounding box that successfully matches the first bounding box that failed to match as the target bounding box of the corresponding object in the target frame.

[0074] Alternatively, predictor 132 can merge the movement trajectories of the active and lost states that failed to match. After merging, predictor 132 can determine the sixth candidate bounding box of the object corresponding to the merged movement trajectory in the previous frame adjacent to the target frame. Based on the merged movement trajectory and the sixth candidate bounding box, tracker 133 uses an autoregressive method to determine the seventh candidate bounding box of the object corresponding to the merged movement trajectory in the target frame, matches the seventh candidate bounding box with the first bounding box that failed to match, and determines the seventh candidate bounding box that successfully matches the first bounding box that failed to match as the target bounding box of the corresponding object in the target frame.

[0075] In some embodiments, if the confidence level of at least one first bounding box that failed to match again is greater than a confidence threshold, a corresponding movement trajectory is generated based on the at least one first bounding box that failed to match again.

[0076] For example, if at least one first bounding box contains object A, but the object corresponding to the movement trajectory does not contain object A, and the first bounding box that fails to match again corresponds to object A, if the confidence of the first bounding box corresponding to object A is greater than the confidence threshold, then the movement trajectory of object A is generated based on the first bounding box.

[0077] It should be understood that the matching of the candidate bounding boxes and the first bounding box can be based on the Intersection over Union (IoU). When the IoU between the first bounding box and the candidate bounding box is greater than a predetermined threshold, the candidate bounding box is determined as the target bounding box of the target object in the target frame. The IoU is the intersection area of ​​two bounding boxes divided by the union area of ​​the two bounding boxes.

[0078] In some embodiments, after matching is completed, all the obtained target bounding boxes can be merged and output as the final target tracking result.

[0079] According to embodiments of this disclosure, target detection is performed on target frames in a target video to obtain at least one first bounding box, each first bounding box defining an object in the target frame. The movement trajectory of each of the at least one object in the target frame is obtained, the movement trajectory of each object indicating its position in several consecutive frames preceding the target frame. Based on the at least one first bounding box and the respective movement trajectories of the at least one object, target bounding boxes of one or more target objects within the target frame are determined. In this manner, target bounding boxes of one or more target objects within the target frame can be determined based on the positions of at least one object in the target frame in several consecutive frames preceding the target frame and at least one first bounding box in the target frame, avoiding the use of the object's appearance features for tracking, thereby improving the accuracy of target tracking.

[0080] Figure 5 A block diagram of an apparatus 500 for target tracking according to some embodiments of the present disclosure is shown. The apparatus 500 may be implemented as or include a multi-target tracking system 130.

[0081] The apparatus 500 includes a detection module 510 configured to perform target detection on target frames in a target video to obtain at least one first bounding box, each first bounding box defining an object in the target frame. The apparatus 500 also includes an acquisition module 520 configured to acquire a movement trajectory of at least one object in the target frame, the movement trajectory of each object indicating the position of the object in a plurality of consecutive frames preceding the target frame. The apparatus 500 further includes a determination module 530 configured to determine, based on the at least one first bounding box and the respective movement trajectories of the at least one object, a target bounding box in the target frame for one or more target objects among the at least one object.

[0082] The modules included in device 500 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units in device 500 can be implemented at least partially by one or more hardware logic components. By way of example, and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), and so on.

[0083] In some embodiments, the determining module 530 is specifically configured to: in response to determining that the consecutive multiple frames include the previous frame adjacent to the target frame, determine a first candidate bounding box of the object corresponding to the movement trajectory in the target frame based on the movement trajectory; match the first candidate bounding box of the object corresponding to the movement trajectory in the target frame with the at least one first bounding box; and determine the first candidate bounding box that successfully matches the at least one first bounding box as the target bounding box of the corresponding object in the target frame.

[0084] In some embodiments, the determining module 530 is specifically configured to: determine trajectory features of at least one object based on the movement trajectory, the trajectory features of each object indicating the positional change of the object between two adjacent frames in the consecutive plurality of frames; and determine, based on the trajectory features, the first candidate bounding box of each of the at least one object in the target frame.

[0085] In some embodiments, the determining module 530 is specifically configured to: determine a second candidate bounding box in the previous frame adjacent to the target frame based on the failed matching motion trajectory; determine a third candidate bounding box in the target frame based on the failed matching motion trajectory and the second candidate bounding box; match the third candidate bounding box with at least one failed matching first bounding box again; and determine the third candidate bounding box that successfully matches the at least one failed matching first bounding box as the target bounding box of the corresponding object in the target frame.

[0086] In some embodiments, the determining module 530 is specifically configured to: in response to determining that the consecutive multiple frames do not include the previous frame adjacent to the target frame, determine a fourth candidate bounding box of the object corresponding to the movement trajectory in the previous frame adjacent to the target frame based on the movement trajectory; determine a fifth candidate bounding box of the object corresponding to the movement trajectory in the target frame based on the movement trajectory and the fourth candidate bounding box; rematch the fifth candidate bounding box with at least one first bounding box that failed to match; and determine the fifth candidate bounding box that successfully matches the at least one first bounding box that failed to match as the target bounding box of the corresponding object in the target frame.

[0087] In some embodiments, the apparatus 500 further includes a generation module configured to generate a corresponding movement trajectory based on the at least one first bounding box that failed to match again if the confidence level of the first bounding box that failed to match again is greater than a confidence level threshold.

[0088] In some embodiments, the determining module 530 is specifically configured to: in response to determining that the consecutive multiple frames do not include the previous frame adjacent to the target frame, determine a fourth candidate bounding box of the object corresponding to the movement trajectory in the previous frame adjacent to the target frame based on the movement trajectory; determine a fifth candidate bounding box of the object corresponding to the movement trajectory in the target frame based on the movement trajectory and the fourth candidate bounding box; match the fifth candidate bounding box with the at least one first bounding box; and determine the fifth candidate bounding box that successfully matches the at least one first bounding box as the target bounding box of the corresponding object in the target frame.

[0089] In some embodiments, the movement trajectory of each object indicates a sequence of second bounding boxes of the object in each of a series of consecutive frames preceding the target frame.

[0090] Figure 6 A block diagram of an electronic device 600 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 6 The electronic device 600 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein.

[0091] like Figure 6 As shown, electronic device 600 is in the form of a general-purpose computing device. Components of electronic device 600 may include, but are not limited to, one or more processors or processing units 610, memory 620, storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. Processing unit 610 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 620. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 600.

[0092] Electronic device 600 typically includes multiple computer storage media. Such media can be any available media accessible to electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 620 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 630 can be a removable or non-removable medium and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within electronic device 600.

[0093] Electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 6 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 620 may include computer program product 625 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0094] The communication unit 640 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 600 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 600 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0095] Input device 650 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 660 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 600 can also communicate with one or more external devices (not shown) via communication unit 640 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 600, or with any device that enables electronic device 600 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0096] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores one or more computer instructions, wherein the one or more computer instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transient computer-readable medium and includes computer-executable instructions that are executed by a processor to implement the methods described above.

[0097] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0098] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0099] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0101] Various implementations of this disclosure have been described above. The foregoing description is exemplary and not exhaustive, nor is it limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the implementations disclosed herein.

Claims

1. A method for target tracking, comprising: Target detection is performed on target frames in the target video to obtain at least one first bounding box, each first bounding box being used to define an object in the target frame; Acquire the movement trajectory of at least one object in the target video, wherein the movement trajectory of each object indicates the position of the object in a series of consecutive frames preceding the target frame; as well as Based on the at least one first bounding box and the respective movement trajectories of the at least one object, a target bounding box of one or more target objects in the target frame is determined.

2. The method of claim 1, wherein determining the target bounding box of one or more target objects among the at least one object in the target frame comprises: In response to determining that the consecutive multiple frames include the previous frame adjacent to the target frame, a first candidate bounding box of the object corresponding to the movement trajectory in the target frame is determined based on the movement trajectory; Match the first candidate bounding box and the at least one first bounding box of the object corresponding to the movement trajectory in the target frame; as well as The first candidate bounding box that successfully matches the at least one first bounding box is determined as the target bounding box of the corresponding object in the target frame.

3. The method according to claim 2, wherein determining the first candidate bounding box of the object corresponding to the movement trajectory in the target frame comprises: Based on the movement trajectory, determine trajectory features for at least one object, wherein the trajectory features of each object indicate the positional change of that object between two adjacent frames in the consecutive plurality of frames; and Based on the trajectory features, the first candidate bounding box of each of the at least one object in the target frame is determined.

4. The method of claim 2, wherein determining the target bounding box of one or more target objects among the at least one object in the target frame comprises: Based on the failed matching movement trajectory, determine the second candidate bounding box of the object corresponding to the failed matching movement trajectory in the previous frame adjacent to the target frame; Based on the failed matching movement trajectory and the second candidate bounding box, a third candidate bounding box of the object corresponding to the failed matching movement trajectory in the target frame is determined. Match the third candidate bounding box with at least one first bounding box that failed to match; as well as The third candidate bounding box that successfully matches at least one of the first bounding boxes that failed to match is determined as the target bounding box of the corresponding object in the target frame.

5. The method of claim 2, wherein determining the target bounding box of one or more target objects among the at least one object in the target frame comprises: In response to determining that the consecutive multiple frames do not include the previous frame adjacent to the target frame, based on the movement trajectory, a fourth candidate bounding box of the object corresponding to the movement trajectory in the previous frame adjacent to the target frame is determined; Based on the movement trajectory and the fourth candidate bounding box, determine the fifth candidate bounding box of the object corresponding to the movement trajectory in the target frame; The fifth candidate bounding box is matched again with at least one first bounding box that failed to match. as well as The fifth candidate bounding box that successfully matches at least one of the first bounding boxes that failed to match is determined as the target bounding box of the corresponding object in the target frame.

6. The method according to claim 5, further comprising: If the confidence level of at least one first bounding box that failed to match again is greater than the confidence level threshold, then a corresponding movement trajectory is generated based on the at least one first bounding box that failed to match again.

7. The method of claim 1, wherein determining the target bounding box of one or more target objects among the at least one object in the target frame comprises: In response to determining that the consecutive multiple frames do not include the previous frame adjacent to the target frame, based on the movement trajectory, a fourth candidate bounding box of the object corresponding to the movement trajectory in the previous frame adjacent to the target frame is determined; Based on the movement trajectory and the fourth candidate bounding box, determine the fifth candidate bounding box of the object corresponding to the movement trajectory in the target frame; Match the fifth candidate bounding box with the at least one first bounding box; as well as The fifth candidate bounding box that successfully matches the at least one first bounding box is determined as the target bounding box of the corresponding object in the target frame.

8. The method of claim 1, wherein the movement trajectory of each object indicates a sequence of second bounding boxes of the object in each of a plurality of consecutive frames preceding the target frame.

9. An apparatus for target tracking, comprising: The detection module is configured to perform target detection on target frames in the target video to obtain at least one first bounding box, each first bounding box being used to define an object in the target frame; The acquisition module is configured to acquire the movement trajectory of at least one object in the target video, wherein the movement trajectory of each object indicates the position of the object in a series of consecutive frames preceding the target frame. as well as The determination module is configured to determine the target bounding box of one or more target objects in the target frame based on the at least one first bounding box and the respective movement trajectories of the at least one object.

10. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 8 when executed by the at least one processing unit.

11. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 8.

12. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 8.

Images