Video target tracking method, device, storage medium and electronic device

Abstract

The present disclosure relates to a video target tracking method, device, storage medium and electronic device to achieve end-to-end video target tracking and reduce the delay of video target tracking. The method includes: obtaining a video to be tracked; inputting the video to be tracked into a target tracking model to obtain a target tracking result corresponding to the video to be tracked, and the target tracking model is used to perform the following processing: for each frame image of the video to be tracked, determining a feature vector corresponding to the target to be tracked in a target detection image corresponding to the image, the target detection image includes the target to be tracked; performing a first similarity calculation on each feature vector in the feature map corresponding to the image and the feature vector corresponding to the target to be tracked in the target detection image, and determining a target feature vector among all feature vectors in the feature map based on the first similarity calculation result; determining the target to be tracked in the image based on the target feature vector.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a video target tracking method, device, storage medium and electronic device. Background Art

[0002] Video target tracking is the basis of many video application fields such as security monitoring, human behavior analysis, sports video commentary, etc., and has high real-time requirements. However, video target tracking in related technologies is usually based on a process of first detecting the target and then tracking the target. Specifically, the target is detected on the two frames before and after the video, and then the detected targets are matched into pairs to achieve target tracking. This method has a high latency because it needs to perform target detection before performing target tracking. Especially in scenarios with many targets to be tracked, the latency problem is particularly obvious. Summary of the invention

[0003] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, the present disclosure provides a video target tracking method, the method comprising:

[0005] Get the video to be tracked;

[0006] The video to be tracked is input into the target tracking model to obtain the target tracking result corresponding to the video to be tracked. The target tracking model is used to perform the following processing:

[0007] For each frame of the video to be tracked, determining a feature vector corresponding to the target to be tracked in a target detection image corresponding to the image, wherein the target detection image includes the target to be tracked;

[0008] Performing a first similarity calculation on each feature vector in the feature map corresponding to the image and the feature vector corresponding to the target to be tracked in the target detection image, and determining a target feature vector from all feature vectors in the feature map according to the first similarity calculation result;

[0009] A target to be tracked in the image is determined according to the target feature vector.

[0010] In a second aspect, the present disclosure provides a video target tracking device, the device comprising:

[0011] An acquisition module is used to acquire the video to be tracked;

[0012] A tracking module is used to input the video to be tracked into a target tracking model to obtain a target tracking result corresponding to the video to be tracked. The tracking module includes:

[0013] A first determination submodule is used to determine, for each frame of the video to be tracked, a feature vector corresponding to the target to be tracked in the target detection image corresponding to the image, wherein the target detection image includes the target to be tracked;

[0014] A second determination submodule is used to perform a first similarity calculation between each feature vector in the feature map corresponding to the image and the feature vector corresponding to the target to be tracked in the target detection image, and determine a target feature vector from all feature vectors in the feature map according to the first similarity calculation result;

[0015] The third determination submodule is used to determine the target to be tracked in the image according to the target feature vector.

[0016] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when the program is executed by a processing device.

[0017] In a fourth aspect, the present disclosure provides an electronic device, including:

[0018] a storage device having a computer program stored thereon;

[0019] A processing device is used to execute the computer program in the storage device to implement the steps of the method in the first aspect.

[0020] Through the above technical solution, the target tracking model can perform a first similarity calculation between the feature vector corresponding to each frame of the video to be tracked and the feature vector corresponding to the target to be tracked in the target detection image, thereby determining the target to be tracked in each frame according to the first similarity calculation result. As a result, the target to be tracked in each frame output by the target tracking model can correspond one-to-one with the target to be tracked in the target detection image, that is, target detection and target association can be completed at the same time, thereby reducing the delay in the target tracking process.

[0021] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings:

[0023] Figure 1 is a flow chart of a video target tracking method according to an exemplary embodiment of the present disclosure;

[0024] Figure 2 is a schematic diagram of a target tracking process in a video target tracking method according to an exemplary embodiment of the present disclosure;

[0025] Figure 3 is a schematic diagram of a target tracking process in another video target tracking method according to an exemplary embodiment of the present disclosure;

[0026] Figure 4 is a block diagram of a video target tracking device according to an exemplary embodiment of the present disclosure;

[0027] Figure 5 It is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0029] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0030] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0031] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units. It should also be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0032] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0033] As mentioned in the background technology, video target tracking in related technologies is usually based on a process of first detecting the target and then tracking the target. Specifically, the detection module first detects the target in the two frames before and after the video, and then the detected targets are matched into pairs through the association module to achieve target tracking. The model components of this process are relatively complex and have high latency, especially in scenarios with many targets to be tracked, the latency problem is particularly obvious.

[0034] In view of this, the present disclosure proposes a video target tracking method, device, storage medium and electronic device to achieve end-to-end video target tracking and reduce the delay of video target tracking.

[0035] Figure 1 FIG. 1 is a flowchart of a method for tracking a video target according to an exemplary embodiment of the present disclosure. Figure 1 , the video target tracking method includes:

[0036] Step 101: Obtain the video to be tracked.

[0037] For example, obtaining the video to be tracked may be in response to a user's video input operation to obtain the video input by the user, or it may be automatically obtaining the video taken by the image acquisition device from the image acquisition device after receiving a target tracking instruction, etc. The embodiments of the present disclosure are not limited to this.

[0038] Step 102, input the video to be tracked into the target tracking model to obtain the target tracking result corresponding to the video to be tracked. The target tracking model is used to perform the following processing: for each frame of the video to be tracked, determine the feature vector corresponding to the target to be tracked in the target detection image corresponding to the image, and the target detection image includes the target to be tracked; perform a first similarity calculation on each feature vector in the feature map corresponding to the image and the feature vector corresponding to the target to be tracked in the target detection image, and determine the target feature vector from all the feature vectors in the feature map according to the first similarity calculation result; determine the target to be tracked in the image according to the target feature vector.

[0039] In the above manner, the target tracking model can perform a first similarity calculation between the feature vector corresponding to each frame of the video to be tracked and the feature vector corresponding to the target to be tracked in the target detection image, thereby determining the target to be tracked in each frame according to the first similarity calculation result. As a result, the target to be tracked in each frame output by the target tracking model can correspond one-to-one with the target to be tracked in the target detection image, that is, target detection and target association can be completed at the same time, thereby reducing the delay in the target tracking process.

[0040] In order to enable those skilled in the art to better understand the video target tracking method provided by the present disclosure, the above steps are described in detail with examples below.

[0041] For example, after obtaining the video to be tracked, the video to be tracked can be input into the target tracking model. It should be understood that each frame of the video to be tracked has a time sequence, so a video image sequence composed of multiple frames of images arranged in time sequence can be obtained according to the video to be tracked. Therefore, inputting the video to be tracked into the target tracking model can also be inputting the video image sequence corresponding to the video to be tracked into the target tracking model.

[0042] For example, the training of the target tracking model can be performed based on the sample image and the sample target information corresponding to the sample image. Specifically, the sample image can be input into the target tracking model to obtain the predicted target information for the sample image output by the target tracking model, and then the loss function is calculated based on the predicted target information and the sample target information, and finally the parameters of the target tracking model are adjusted based on the calculation result of the loss function, so that the target tracking model outputs more accurate target information. In this way, the target detection function and the target tracking function of the target tracking model can be trained synchronously.

[0043] It should be understood that since the relevant technology requires target detection before target association, the model training method in the relevant technology is to gradually train the detection module and the association module. In the scenario where there are many targets to be tracked, this training method takes a lot of time and it is difficult to achieve the optimal training effect. The method of synchronously training the target detection function and the target tracking function of the target tracking model in the disclosed embodiment not only simplifies the components of the target tracking model, but also simplifies the training process of the target tracking model, which can better meet the scene requirements of multi-target tracking.

[0044] In the application stage, the target tracking model can determine the feature vector corresponding to the target to be tracked in the target detection image corresponding to each frame of the video to be tracked, and then perform a first similarity calculation between each feature vector in the feature map corresponding to the image and the feature vector corresponding to the target to be tracked in the target detection image, and determine the target feature vector from all feature vectors in the feature map based on the first similarity calculation result. Finally, the target to be tracked in the image is determined based on the target feature vector.

[0045] For example, the target detection image may be the previous frame of an image including the target to be tracked, or the target detection image may be a preset input image including the target to be tracked. It should be understood that the video target tracking method provided by the embodiments of the present disclosure can be applied to two application scenarios: the tracking target is given and the tracking target is unknown. Among them, in the scenario where the tracking target is given, the target detection image may be a preset input image including the target to be tracked, for example, if the target to be tracked is person A, then the preset input image may be a full-body photo of person A taken by an image acquisition device. In the scenario where the tracking target is unknown, it is necessary to track all targets in each frame of the image, then the target detection image may be the previous frame of the image including the target to be tracked.

[0046] After determining the target detection image, the feature vector corresponding to the target to be tracked in the target detection image can be determined. The feature vector can be the result obtained by vectorizing the image feature of the central pixel of the target to be tracked, or the feature vector can be the result obtained by vectorizing the image feature of a certain pixel that can distinguish the target to be tracked from other targets, etc., which is not limited in the embodiments of the present disclosure. The method of determining the feature vector is similar to that in the related art and will not be repeated here.

[0047] By way of example, the feature map corresponding to the image may be an image obtained by vectorizing the image features of each pixel in the image. Furthermore, the feature vector corresponding to the target to be tracked in the target detection image is a pixel-level feature vector, so a first similarity calculation may be performed between each feature vector in the feature map corresponding to the image corresponding to the target to be tracked and the feature vector corresponding to the target to be tracked in the target detection image, thereby achieving target tracking based on the first similarity calculation result. By way of example, the first similarity calculation may be a vector dot product calculation, a Euclidean distance calculation, etc. between the feature vector corresponding to each pixel in the image and the feature vector corresponding to the target to be tracked in the target detection image. The disclosed embodiment does not limit the method for calculating the first similarity.

[0048] It should be understood that the target tracking model may include an attention mechanism module, which may perform a first similarity calculation process to determine a feature vector corresponding to a target existing in both the image and the target detection image, that is, to obtain a target feature vector.

[0049] For example, refer to Figure 2 In a given scene of the tracking target, the target detection image is a preset input image including the target to be tracked. Each frame image of the video to be tracked is used as the current frame image, and the feature map of the current frame image is determined. Then, the attention mechanism module can perform a first similarity calculation on each feature vector in the feature map and the feature vector corresponding to the target to be tracked in the preset input image, and output the target feature vector according to the first similarity calculation result, so that the target tracking model can determine the target to be tracked in the current frame image according to the target feature vector.

[0050] In a possible manner, determining the target feature vector among all feature vectors of the feature map based on the first similarity calculation result can be: when the target detection image includes N targets to be tracked, among all the feature vectors of the feature map, N feature vectors with the largest first similarity calculation results are selected as the target feature vector, where N is a positive integer.

[0051] For example, in a given scenario of a tracking target, the target detection image is a preset input image including a target to be tracked. It can be determined by a target detection method in the related art that the target detection image includes N targets to be tracked, that is, the number of feature vectors corresponding to the targets to be tracked is N. In this scenario, for each frame of the video to be tracked, the N feature vectors with the largest first similarity calculation results can be selected as target feature vectors from all feature vectors included in the feature map corresponding to the image to determine the target to be tracked in the image. Among them, all the first similarity calculation results can be sorted from large to small, and then, among all the feature vectors included in the feature map corresponding to the image, the N feature vectors corresponding to the first similarity calculation results with the highest sorting are selected as target feature vectors. Of course, all the first similarity calculation results can also be sorted from small to large, and then, among all the feature vectors included in the feature map corresponding to the image, the N feature vectors corresponding to the last sorted first similarity calculation results are selected as target feature vectors, and the disclosed embodiment is not limited to this.

[0052] In the above manner, the target to be tracked determined according to the selected N feature vectors is a target that exists in each frame image in the video to be tracked and the target detection image, that is, the target to be tracked in each frame image output by the target tracking model can correspond one-to-one to the target to be tracked in the target detection image, so target detection and target association can be completed at the same time, thereby reducing the delay in the target tracking process.

[0053] In a possible manner, in a scenario where the tracking target is unknown, since all targets in the image need to be tracked, the target tracking model can also be used to determine the feature vectors corresponding to all targets in the image based on the pre-trained position vector parameters. Accordingly, according to the first similarity calculation result, the target feature vector can be determined from all feature vectors of the feature map: when the target detection image includes N targets to be tracked, among all feature vectors of the feature map, the N feature vectors with the largest first similarity calculation results are selected as similar feature vectors, where N is a positive integer. Then, the feature vectors corresponding to all targets in the image and the N similar feature vectors are deduplicated to obtain the target feature vector.

[0054] For example, refer to Figure 3 , in a scenario where the target to be tracked is unknown, the target detection image is the previous frame of the image that includes the target to be tracked. Each frame of the video to be tracked is used as the current frame image, and the feature map of the current frame image is first determined. Then the attention mechanism module can perform a first similarity calculation on each feature vector in the feature map with the feature vector corresponding to the target to be tracked in the previous frame image, and determine the similar feature vector based on the first similarity calculation result. At the same time, the attention mechanism module can also determine the feature vectors of all targets in the current frame image based on the pre-trained position vector parameters. Afterwards, the target tracking model can perform feature vector fusion based on the similar feature vectors and the feature vectors of all targets in the current frame image to obtain the target feature vector. Finally, the target tracking model can determine the target to be tracked in the current frame image based on the target feature vector.

[0055] For example, the position vector parameter may include multiple unit position vectors. For example, for an H×W image, the position vector parameter may include equal to or more than H×W unit position vectors to cover each pixel position in the image. It should be understood that the number of unit position vectors included in the position vector parameter may be set larger to adapt to image sizes in different scenarios.

[0056] In a possible manner, the position vector parameters can be trained in the following manner: determining the predicted feature vector corresponding to the target in the sample image based on the initial position vector parameters to obtain the predicted target information corresponding to the sample image, wherein the sample image is pre-annotated with the corresponding sample target information, and then calculating the loss function based on the predicted target information and the sample target information, and adjusting the initial position vector parameters based on the calculation result of the loss function.

[0057] For example, the initial position vector parameter can be a random value, that is, after setting the number of unit position vectors in the position vector, the value of each unit position vector is a random value. Then, according to the loss function result of the target tracking model during the training process, the position vector parameter can be adjusted through the back propagation algorithm so that the position vector parameter can more accurately predict the position of the target in the image.

[0058] In a possible manner, if the image is the first frame of the video to be tracked, the feature vector determined according to the pre-trained position vector parameters can be used as the target feature vector. It should be understood that in a scenario where the tracking target is unknown, the target detection image can be the previous frame of the image, so the first frame cannot be used for similarity calculation. In the embodiment of the present disclosure, the feature vectors corresponding to all targets in the image determined according to the pre-trained position vector parameters can be used as the target feature vector to determine the target to be tracked in the image.

[0059] After determining the target to be tracked in the first frame image according to the position vector parameters, each frame image of the video to be tracked may correspond to the previous frame image including the target to be tracked, so the target feature vector may be determined by combining the feature vectors corresponding to all targets in each frame image with the first similarity calculation result.

[0060] For example, when the target detection image includes N targets to be tracked, the N feature vectors with the largest first similarity calculation results can be selected from all feature vectors in the feature map as similar feature vectors, where N is a positive integer. Then, the feature vectors corresponding to all targets in the image and the N similar feature vectors can be deduplicated to obtain the target feature vector. The process of determining the N similar feature vectors can refer to the above content about selecting the N feature vectors with the largest first similarity calculation results as the target feature vector, which will not be repeated here.

[0061] For example, the N similar feature vectors represent the feature vectors corresponding to the targets existing in each frame image of the video to be tracked and the target detection image. The feature vector determined according to the position vector parameter is the feature vector corresponding to all targets in each frame image of the video to be tracked, and the feature vectors corresponding to the same targets exist in both. For example, a frame image in the video to be tracked includes targets B1, B2 and B3, and the target detection image includes targets B1 and B2. Then the N similar feature vectors are the feature vectors corresponding to targets B1 and B2, and the feature vector determined according to the position vector parameter is the feature vector corresponding to targets B1, B2 and B3. The feature vectors corresponding to the same targets (B1 and B2) exist in both. In this case, in order to avoid vector redundancy, feature vector fusion can be performed, for example, the feature vectors corresponding to all targets in the image and the N similar feature vectors can be deduplicated to obtain the target feature vector for determining the target to be tracked.

[0062] In a possible manner, deduplication processing is performed on the feature vectors corresponding to all targets in the image and the N similar feature vectors to obtain the target feature vector, which may be: for the feature vector corresponding to each target in the image, the feature vector is subjected to a second similarity calculation with the N similar feature vectors, and when the second similarity calculation result is greater than or equal to a preset similarity, the feature vector corresponding to the second similarity calculation result is deleted from the feature vectors corresponding to all targets in the image or from the N similar feature vectors. Then, the feature vectors corresponding to all targets in the image and the remaining feature vectors in the N similar feature vectors after the deletion processing are used as the target feature vector.

[0063] For example, the second similarity calculation may be to perform vector dot product calculation, Euclidean distance calculation, etc. on the feature vector corresponding to each target in the image and the feature vector and N similar feature vectors. The embodiment of the present disclosure does not limit the method of calculating the second similarity. The preset similarity can be customized according to actual conditions, and the present disclosure does not limit this. When the result of the second similarity calculation is greater than or equal to the preset similarity, the feature vector corresponding to a target in the image and a feature vector in the N similar feature vectors can be regarded as the same feature vector, so that a deletion operation can be performed on the feature vector. Among them, considering that the feature vector may exist in the feature vectors corresponding to all targets in the image and in the N similar feature vectors at the same time, the deletion operation can be performed in the feature vectors corresponding to all targets in the image or in the N similar feature vectors.

[0064] For example, in the above example, the N similar feature vectors are feature vectors corresponding to targets B1 and B2, and the feature vector determined according to the position vector parameter is the feature vector corresponding to targets B1, B2 and B3. After the second similarity calculation, the feature vector corresponding to the second similarity calculation result greater than or equal to the preset similarity can be determined as the feature vector corresponding to targets B1 and B2. You can choose to delete the feature vectors corresponding to targets B1 and B2 from the N similar feature vectors, and after deletion, the N similar feature vectors are empty. Alternatively, you can choose to delete the feature vectors corresponding to targets B1 and B2 from the feature vectors determined according to the position vector parameters (that is, the feature vectors corresponding to all targets in the image), and delete the remaining feature vectors as the feature vector corresponding to target B3. It should be understood that after the deletion process, the feature vectors corresponding to the targets existing in both the image and the target detection image can be retained preferentially, so as to ensure target association and achieve target tracking.

[0065] After the deletion process, the feature vectors corresponding to all targets in the image and the remaining feature vectors in the N similar feature vectors can be used as the target feature vector. For example, in the above example, after the feature vectors corresponding to targets B1 and B2 are deleted from the N similar feature vectors, the N similar feature vectors are empty, and the feature vectors corresponding to all targets in the image are the feature vectors corresponding to targets B1, B2 and B3. Then, the feature vectors corresponding to all targets in the image and the remaining feature vectors in the N similar feature vectors are the feature vectors corresponding to targets B1, B2 and B3, that is, the target feature vector is the feature vector corresponding to targets B1, B2 and B3.

[0066] It should be understood that the above-mentioned deduplication processing method is only a possible method of performing feature vector fusion on feature vectors provided by the embodiment of the present disclosure. When the present disclosure is implemented specifically, feature vector fusion of feature vectors corresponding to all targets in the image and N similar feature vectors can also be performed in other ways, and the embodiment of the present disclosure is not limited to this.

[0067] Through the above method, the feature vectors corresponding to all targets in the image can be determined by the position vector parameters, and the similar feature vectors corresponding to the targets existing in both the image and the target detection image can be determined according to the second similarity calculation result. Then, the feature vector fusion is performed on the similar feature vector and the feature vectors corresponding to all targets in the image to remove redundant feature vectors, thereby improving the calculation efficiency and obtaining a more accurate target to be tracked.

[0068] After obtaining the target feature vector by any of the above methods, the target feature vector can be subjected to linear feature transformation to obtain tracking frame information corresponding to the target to be tracked in the image. The tracking frame information includes position information and size information of the tracking frame corresponding to the target to be tracked, so that the target to be tracked can be indicated in the image based on the tracking frame information. This process is similar to the related technology and will not be repeated here.

[0069] Based on the same inventive concept, the embodiment of the present disclosure also provides a video target tracking device, which can be part or all of an electronic device through software, hardware, or a combination of both. Figure 4 , the video target tracking device comprises:

[0070] An acquisition module 401 is used to acquire a video to be tracked;

[0071] The tracking module 402 is used to input the video to be tracked into the target tracking model to obtain the target tracking result corresponding to the video to be tracked. The tracking module includes:

[0072] A first determination submodule 4021 is used to determine, for each frame of the video to be tracked, a feature vector corresponding to the target to be tracked in the target detection image corresponding to the image, wherein the target detection image includes the target to be tracked;

[0073] A second determination submodule 4022 is used to perform a first similarity calculation between each feature vector in the feature map corresponding to the image and the feature vector corresponding to the target to be tracked in the target detection image, and determine a target feature vector from all feature vectors in the feature map according to the first similarity calculation result;

[0074] The third determination submodule 4023 is used to determine the target to be tracked in the image according to the target feature vector.

[0075] Optionally, the target detection image is a previous frame image of the image including the target to be tracked; or, the target detection image is a preset input image including the target to be tracked.

[0076] Optionally, the second determining submodule 4022 is used to:

[0077] When the target detection image includes N targets to be tracked, N feature vectors with the largest first similarity calculation results are selected as target feature vectors from all feature vectors in the feature map, where N is a positive integer.

[0078] Optionally, the target tracking model is further used to determine feature vectors corresponding to all targets in the image according to pre-trained position vector parameters, and the second determination submodule is used to:

[0079] When the target detection image includes N targets to be tracked, among all the feature vectors in the feature map, N feature vectors with the largest first similarity calculation results are selected as similar feature vectors, where N is a positive integer;

[0080] Deduplication processing is performed on the feature vectors corresponding to all targets in the image and the N similar feature vectors to obtain a target feature vector.

[0081] Optionally, the image is a first frame image of the video to be tracked, and the device further includes:

[0082] The fourth determination submodule is used to use the feature vector determined according to the pre-trained position vector parameters as the target feature vector.

[0083] Optionally, the second determining submodule 4022 is used to:

[0084] For each feature vector corresponding to the target in the image, performing a second similarity calculation between the feature vector and the N similar feature vectors;

[0085] When the second similarity calculation result is greater than or equal to a preset similarity, deleting a feature vector corresponding to the second similarity calculation result from the feature vectors corresponding to all objects in the image or from the N similar feature vectors;

[0086] The feature vectors corresponding to all the targets in the image after the deletion process and the remaining feature vectors in the N similar feature vectors are used as target feature vectors.

[0087] Optionally, the device 400 further includes the following module for training to obtain the position vector parameters:

[0088] A first training module is used to determine a prediction feature vector corresponding to a target in a sample image according to an initial position vector parameter to obtain prediction target information corresponding to the sample image, wherein the sample image is pre-labeled with corresponding sample target information;

[0089] The second training module is used to calculate the loss function according to the predicted target information and the sample target information, and adjust the initial position vector parameters according to the calculation result of the loss function.

[0090] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0091] Based on the same inventive concept, an embodiment of the present disclosure further provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processing device, the steps of any of the above-mentioned video target tracking methods are implemented.

[0092] Based on the same inventive concept, the present disclosure also provides an electronic device, including:

[0093] a storage device having a computer program stored thereon;

[0094] A processing device is used to execute the computer program in the storage device to implement the steps of any of the above-mentioned video target tracking methods.

[0095] Reference below Figure 5 , which shows a schematic diagram of the structure of an electronic device (electronic device) 500 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0096] like Figure 5 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0097] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0098] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0099] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0100] In some embodiments, any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol) can be used for communication, and can be interconnected with any form or medium of digital data communication (e.g., communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), internets (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

[0101] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0102] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains a video to be tracked; inputs the video to be tracked into a target tracking model to obtain a target tracking result corresponding to the video to be tracked, and the target tracking model is used to perform the following processing: for each frame image of the video to be tracked, determine the feature vector corresponding to the target to be tracked in the target detection image corresponding to the image, and the target detection image includes the target to be tracked; perform a first similarity calculation on each feature vector in the feature map corresponding to the image and the feature vector corresponding to the target to be tracked in the target detection image, and determine the target feature vector from all the feature vectors in the feature map according to the first similarity calculation result; and determine the target to be tracked in the image according to the target feature vector.

[0103] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0105] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a module does not, in some cases, limit the module itself.

[0106] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0107] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0108] According to one or more embodiments of the present disclosure, Example 1 provides a video target tracking method, the method comprising:

[0109] Get the video to be tracked;

[0110] The video to be tracked is input into the target tracking model to obtain the target tracking result corresponding to the video to be tracked. The target tracking model is used to perform the following processing:

[0111] For each frame of the video to be tracked, determining a feature vector corresponding to the target to be tracked in a target detection image corresponding to the image, wherein the target detection image includes the target to be tracked;

[0112] Performing a first similarity calculation on each feature vector in the feature map corresponding to the image and the feature vector corresponding to the target to be tracked in the target detection image, and determining a target feature vector from all feature vectors in the feature map according to the first similarity calculation result;

[0113] A target to be tracked in the image is determined according to the target feature vector.

[0114] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein the target detection image is a previous frame image of the image including the target to be tracked; or

[0115] The target detection image is a preset input image including the target to be tracked.

[0116] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 1, wherein determining the target feature vector from all feature vectors of the feature graph according to the first similarity calculation result includes:

[0117] When the target detection image includes N targets to be tracked, N feature vectors with the largest first similarity calculation results are selected as target feature vectors from all feature vectors in the feature map, where N is a positive integer.

[0118] According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 1, wherein the target tracking model is further used to determine feature vectors corresponding to all targets in the image according to pre-trained position vector parameters, and the target feature vector is determined from all feature vectors of the feature map according to the first similarity calculation result, including:

[0119] When the target detection image includes N targets to be tracked, among all the feature vectors in the feature map, N feature vectors with the largest first similarity calculation results are selected as similar feature vectors, where N is a positive integer;

[0120] Deduplication processing is performed on the feature vectors corresponding to all targets in the image and the N similar feature vectors to obtain a target feature vector.

[0121] According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 4, wherein the image is a first frame image of the video to be tracked, and the method further includes:

[0122] The feature vector determined according to the pre-trained position vector parameters is used as the target feature vector.

[0123] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 4, wherein deduplication processing is performed on the feature vectors corresponding to all targets in the image and the N similar feature vectors to obtain the target feature vector, including:

[0124] For each feature vector corresponding to the target in the image, performing a second similarity calculation between the feature vector and the N similar feature vectors;

[0125] When the second similarity calculation result is greater than or equal to a preset similarity, deleting a feature vector corresponding to the second similarity calculation result from the feature vectors corresponding to all objects in the image or from the N similar feature vectors;

[0126] The feature vectors corresponding to all the targets in the image after the deletion process and the remaining feature vectors in the N similar feature vectors are used as target feature vectors.

[0127] According to one or more embodiments of the present disclosure, Example 7 provides the method of any one of Examples 4-6, wherein the position vector parameter is obtained by training in the following manner:

[0128] Determining a predicted feature vector corresponding to a target in a sample image according to the initial position vector parameter to obtain predicted target information corresponding to the sample image, wherein the sample image is pre-labeled with corresponding sample target information;

[0129] A loss function is calculated according to the predicted target information and the sample target information, and the initial position vector parameters are adjusted according to the calculation result of the loss function.

[0130] According to one or more embodiments of the present disclosure, Example 8 provides a video target tracking device, the device comprising:

[0131] An acquisition module is used to acquire the video to be tracked;

[0132] A tracking module is used to input the video to be tracked into a target tracking model to obtain a target tracking result corresponding to the video to be tracked. The tracking module includes:

[0133] A first determination submodule is used to determine, for each frame of the video to be tracked, a feature vector corresponding to the target to be tracked in the target detection image corresponding to the image, wherein the target detection image includes the target to be tracked;

[0134] A second determination submodule is used to perform a first similarity calculation between each feature vector in the feature map corresponding to the image and the feature vector corresponding to the target to be tracked in the target detection image, and determine a target feature vector from all feature vectors in the feature map according to the first similarity calculation result;

[0135] The third determination submodule is used to determine the target to be tracked in the image according to the target feature vector.

[0136] According to one or more embodiments of the present disclosure, Example 9 provides the device of Example 8, wherein the target detection image is a previous frame image of the image including the target to be tracked; or, the target detection image is a preset input image including the target to be tracked.

[0137] According to one or more embodiments of the present disclosure, Example 10 provides the apparatus of Example 8, wherein the second determining submodule is configured to:

[0138] When the target detection image includes N targets to be tracked, N feature vectors with the largest first similarity calculation results are selected as target feature vectors from all feature vectors in the feature map, where N is a positive integer.

[0139] According to one or more embodiments of the present disclosure, Example 11 provides the apparatus of Example 8, wherein the target tracking model is further used to determine feature vectors corresponding to all targets in the image according to pre-trained position vector parameters, and the second determination submodule is used to:

[0140] When the target detection image includes N targets to be tracked, among all the feature vectors in the feature map, N feature vectors with the largest first similarity calculation results are selected as similar feature vectors, where N is a positive integer;

[0141] Deduplication processing is performed on the feature vectors corresponding to all targets in the image and the N similar feature vectors to obtain a target feature vector.

[0142] According to one or more embodiments of the present disclosure, Example 12 provides the device of Example 11, wherein the image is a first frame image of the video to be tracked, and the device further includes:

[0143] The fourth determination submodule is used to use the feature vector determined according to the pre-trained position vector parameters as the target feature vector.

[0144] According to one or more embodiments of the present disclosure, Example 13 provides the apparatus of Example 11, wherein the second determining submodule is configured to:

[0145] For each feature vector corresponding to the target in the image, performing a second similarity calculation between the feature vector and the N similar feature vectors;

[0146] When the second similarity calculation result is greater than or equal to a preset similarity, deleting a feature vector corresponding to the second similarity calculation result from the feature vectors corresponding to all objects in the image or from the N similar feature vectors;

[0147] The feature vectors corresponding to all the targets in the image after the deletion process and the remaining feature vectors in the N similar feature vectors are used as target feature vectors.

[0148] According to one or more embodiments of the present disclosure, Example 14 provides the device of any one of Examples 11-13, wherein the device further includes the following module for training to obtain the position vector parameter:

[0149] A first training module is used to determine a prediction feature vector corresponding to a target in a sample image according to an initial position vector parameter to obtain prediction target information corresponding to the sample image, wherein the sample image is pre-labeled with corresponding sample target information;

[0150] The second training module is used to calculate the loss function according to the predicted target information and the sample target information, and adjust the initial position vector parameters according to the calculation result of the loss function.

[0151] According to one or more embodiments of the present disclosure, Example 15 provides a computer-readable medium having a computer program stored thereon, which implements the steps of any of the methods described in Examples 1-7 when executed by a processing device.

[0152] According to one or more embodiments of the present disclosure, Example 16 provides an electronic device, including:

[0153] a storage device having a computer program stored thereon;

[0154] A processing device is used to execute the computer program in the storage device to implement the steps of the method described in any one of Examples 1-7.

[0155] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0156] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0157] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. A video target tracking method, characterized in that: The method comprises: Get the video to be tracked; The video to be tracked is input into the target tracking model to obtain the target tracking result corresponding to the video to be tracked. The target tracking model is used to perform the following processing: For each frame of the video to be tracked, determining a feature vector corresponding to the target to be tracked in a target detection image corresponding to the image, wherein the target detection image includes the target to be tracked; Perform a first similarity calculation on each feature vector in the feature map corresponding to the image and the feature vector corresponding to the target to be tracked in the target detection image, and when the target detection image includes N targets to be tracked, select N feature vectors with the largest first similarity calculation results from all feature vectors in the feature map as similar feature vectors, where N is a positive integer; For each feature vector corresponding to the target in the image, performing a second similarity calculation between the feature vector and the N similar feature vectors; When the second similarity calculation result is greater than or equal to a preset similarity, deleting a feature vector corresponding to the second similarity calculation result from the feature vectors corresponding to all objects in the image or from the N similar feature vectors; The feature vectors corresponding to all the targets in the image after the deletion process and the remaining feature vectors in the N similar feature vectors are used as target feature vectors, wherein the feature vectors corresponding to all the targets in the image are determined according to the pre-trained position vector parameters and the target tracking model; A target to be tracked in the image is determined according to the target feature vector.

2. The method according to claim 1, characterized in that: The target detection image is a previous frame of the image including the target to be tracked; or The target detection image is a preset input image including the target to be tracked.

3. The method according to claim 1, characterized in that The image is a first frame image of the video to be tracked, and the method further includes: The feature vector determined according to the pre-trained position vector parameters is used as the target feature vector.

4. The method according to any one of claims 1 to 3, characterized in that: The position vector parameters are obtained by training in the following way: Determining a predicted feature vector corresponding to a target in a sample image according to the initial position vector parameter to obtain predicted target information corresponding to the sample image, wherein the sample image is pre-labeled with corresponding sample target information; A loss function is calculated according to the predicted target information and the sample target information, and the initial position vector parameters are adjusted according to the calculation result of the loss function.

5. A video target tracking device, characterized in that: The device comprises: An acquisition module is used to acquire the video to be tracked; A tracking module is used to input the video to be tracked into a target tracking model to obtain a target tracking result corresponding to the video to be tracked. The tracking module includes: A first determination submodule is used to determine, for each frame of the video to be tracked, a feature vector corresponding to the target to be tracked in the target detection image corresponding to the image, wherein the target detection image includes the target to be tracked; A second determination submodule is used to perform a first similarity calculation between each feature vector in the feature map corresponding to the image and the feature vector corresponding to the target to be tracked in the target detection image, and when the target detection image includes N targets to be tracked, select N feature vectors with the largest first similarity calculation results from all feature vectors in the feature map as similar feature vectors, where N is a positive integer; for each feature vector corresponding to the target in the image, perform a second similarity calculation between the feature vector and the N similar feature vectors; when the second similarity calculation result is greater than or equal to a preset similarity, delete the feature vector corresponding to the second similarity calculation result from the feature vectors corresponding to all targets in the image or from the N similar feature vectors; and use the feature vectors corresponding to all targets in the image after the deletion process and the remaining feature vectors in the N similar feature vectors as target feature vectors, wherein the feature vectors corresponding to all targets in the image are determined based on pre-trained position vector parameters and the target tracking model; The third determination submodule is used to determine the target to be tracked in the image according to the target feature vector.

6. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processing device, the steps of the method described in any one of claims 1 to 4 are implemented.

7. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 4.

Images