A target tracking method, device, apparatus and storage medium

Abstract

The application discloses a target tracking method and device, equipment and storage medium, and relates to the technical field of computers. The application comprises the following steps: acquiring a target image, wherein the target image comprises at least one tracking target; performing feature extraction on the target image to obtain a plurality of feature images, wherein the image resolutions of the feature images are different; fusing feature information in the plurality of feature images to determine a target feature image corresponding to the target image; performing target detection on the target feature image to obtain target position information of the tracking target, and performing target re-identification on the target feature image to obtain re-identified feature information; and performing target correlation tracking according to the target position information and the re-identified feature information. Through sharing of the target feature image and simultaneous target detection and target re-identification, the timeliness of target tracking can be greatly improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a target tracking method, apparatus, device, and storage medium. Background Technology

[0002] Multi-object tracking is a long-term goal of computer vision. It involves detecting multiple targets to be tracked in each frame of a video to determine the position of each target within the video image. The trajectory of the target can then be analyzed based on the correlation between the target positions in different video images. It is widely used in scenarios such as target action recognition and motion video analysis.

[0003] In existing technologies, multi-target tracking methods typically perform target detection first, followed by target ID matching. Because this is done in two separate stages, its computational efficiency is low, and it significantly reduces the timeliness of target tracking. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a target tracking method, apparatus, device and storage medium that overcomes or at least partially solves the above problems.

[0005] Based on a first aspect of the present invention, a target tracking method is provided, the method comprising:

[0006] Acquire a target image, wherein the target image includes at least one tracking target;

[0007] Feature extraction is performed on the target image to obtain multiple feature images, wherein the image resolutions of the feature images are different.

[0008] By fusing feature information from multiple feature images, the target feature image corresponding to the target image is determined;

[0009] Target detection is performed on the target feature image to obtain the target location information of the tracked target;

[0010] The target feature image is re-identified to obtain re-identified feature information;

[0011] Based on the target location information and re-identification feature information, target association tracking is performed.

[0012] Based on a second aspect of the present invention, a target tracking device is also provided, the device comprising:

[0013] An image acquisition module is used to acquire a target image, wherein the target image includes at least one tracking target;

[0014] The feature image determination module is used to extract features from the target image to obtain multiple feature images, wherein the image resolutions of the feature images are different.

[0015] The feature fusion module fuses feature information from multiple feature images to determine the target feature image corresponding to the target image;

[0016] The target detection module is used to perform target detection on the target feature image to obtain the target location information of the tracked target;

[0017] The re-identification module is used to perform target re-identification on the target feature image to obtain re-identification feature information;

[0018] The association tracking module is used to perform target association tracking based on the target location information and re-identification feature information.

[0019] Based on a third aspect of the present invention, an electronic device is also provided, comprising:

[0020] One or more processors;

[0021] Memory;

[0022] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform any of the methods described above.

[0023] Based on a fourth aspect of the invention, a computer-readable storage medium is also provided for storing a computer program for use in conjunction with an electronic device, the computer program being executable by a processor to perform any of the methods described above.

[0024] Compared with existing technologies, this invention includes first acquiring a target image, wherein the target image includes at least one tracking target. Then, feature extraction is performed on the target image to obtain multiple feature images, wherein the image resolutions of the feature images differ. Next, feature information from the multiple feature images is fused to determine the target feature image corresponding to the target image. Subsequently, target detection is performed on the target feature image to obtain the target location information of the tracking target, and target re-identification is performed on the target feature image to obtain re-identified feature information. Finally, target association tracking is performed based on the target location information and the re-identified feature information. By sharing the target feature image and simultaneously performing target detection and target re-identification, the timeliness of target tracking can be greatly improved.

[0025] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0026] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0027] In the attached diagram:

[0028] Figure 1 This is a flowchart illustrating the steps of a target tracking method provided in an embodiment of the present invention;

[0029] Figure 2 This is a flowchart illustrating the steps of another target tracking method provided in an embodiment of the present invention;

[0030] Figure 3 This is a logical architecture diagram of the target tracking method provided in the embodiments of the present invention;

[0031] Figure 4 This is a schematic diagram of the steps for generating a target feature image according to an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of the structure of a target tracking device provided in an embodiment of the present invention. Detailed Implementation

[0033] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0034] Reference Figure 1 This illustrates a target tracking method provided by an embodiment of the present invention, the method comprising:

[0035] S101. Acquire a target image, wherein the target image includes at least one tracking target.

[0036] In this embodiment of the invention, the tracking target may be a person, and the image resolution of the target image is preset. For example, the width or height of the target image is 640 pixels.

[0037] S102. Perform feature extraction on the target image to obtain multiple feature images, wherein the image resolutions of the feature images are different.

[0038] In this embodiment of the invention, a feature extraction network can be used to extract features from the target image. In one example, the feature extraction network can employ a convolutional neural network structure to output multiple extracted feature images at different image resolutions. For example, the feature images can be 1 / 8, 1 / 16, or 1 / 32 of the target image's resolution.

[0039] S103. Merge feature information from multiple feature images to determine the target feature image corresponding to the target image.

[0040] In this embodiment of the invention, the fusion feature can employ a feature fusion network, which is used to fuse feature images at different scales to obtain a target feature image. This target feature image, obtained through feature fusion, has better spatial and scale representation capabilities, thereby facilitating improved subsequent target recognition performance.

[0041] S104. Perform target detection on the target feature image to obtain the target location information of the tracked target.

[0042] S105. Perform target re-identification on the target feature image to obtain re-identification feature information.

[0043] S106. Based on the target location information and re-identification feature information, perform target association tracking.

[0044] In this embodiment of the invention, the target location information refers to location-related information corresponding to the detected target. Target re-identification can be understood as extracting feature information from each pixel in the target feature image and obtaining re-identification feature information. Finally, based on the feature correlation between the target location information and the corresponding re-identification feature information, target association tracking is performed. Therefore, by sharing the target feature image and simultaneously performing target detection and target re-identification, the timeliness of target tracking can be greatly improved.

[0045] Reference Figure 2 This illustrates another target tracking method provided by an embodiment of the present invention, the method comprising:

[0046] S201. Extract video frame images from the video.

[0047] S202. Preprocess the video frame image to obtain a target image, wherein the target image includes at least one tracking target.

[0048] In this embodiment of the invention, the tracking target can be a person, and the video can be a real-time surveillance video or a recorded video, etc. Video frames can be extracted periodically from the video, and preprocessed. For example, the image resolutions of different video acquisition devices may differ. Therefore, preprocessing may include interpolation of the video frame images to obtain the target image. In one example, the image resolution of the target image can be preset. For example, the width or height of the target image is 640 pixels to facilitate subsequent input into the feature extraction network.

[0049] S203. Based on the feature extraction network, feature extraction is performed on the target image to obtain multiple feature images, wherein the image resolutions of the feature images are different.

[0050] In this embodiment of the invention, a feature extraction network can be used to extract features from the target image. (Refer to...) Figure 3 As shown, the feature extraction network can employ a convolutional neural network structure to output multiple feature images with different image resolutions. For example, the feature images can be 1 / 8, 1 / 16, and 1 / 32 of the target image's resolution, respectively. Figure 3 Feature image A, feature image B, and feature image C, etc.

[0051] The feature extraction network may include multiple cascaded residual modules (e.g., a deep residual network ResNet34). Based on these cascaded residual modules, features are extracted from the target image sequentially. Features extracted by several residual modules are selected as feature images, thus enabling the feature extraction network to obtain multiple feature images. Furthermore, these feature images have different image resolutions. For example, the feature images may be 1 / 8, 1 / 16, or 1 / 32 of the target image's resolution.

[0052] Reference Figure 3 As shown, a feature fusion network can be used to fuse feature information from multiple feature images. This network fuses feature images at different scales to obtain a single target feature image. The target feature image, obtained through feature fusion, has better spatial and scale representation capabilities, thus improving subsequent target recognition performance. In one example, the feature fusion network may include a first fusion module and a second fusion module.

[0053] S204. Input the multiple feature images into the first fusion module to perform first feature fusion and determine multiple sets of fused images corresponding to the target image.

[0054] In this embodiment of the invention, the multiple first fused images in each fused image set have the same image resolution. The first fusion module is used to fuse multiple feature images at different spaces and scales to obtain multiple sets of fused image sets, wherein the number of images in the first fused image in each set of fused image sets is consistent with the number of images in the multiple feature images.

[0055] In an optional embodiment of the invention, the first fusion module includes a self-conversion unit, an implantation conversion unit, and a rendering conversion unit. The step of inputting the plurality of feature images into the first fusion module, performing first feature fusion, and determining multiple sets of fused image sets corresponding to the target image includes:

[0056] The multiple feature images are input into the self-conversion unit, the implantation conversion unit, and the rendering conversion unit to perform first feature fusion and obtain multiple first fused images.

[0057] The multiple first fused images are divided according to different image resolutions to obtain multiple sets of fused images corresponding to the target image.

[0058] In this embodiment of the invention, the self-transformer (ST) is used to extract the features of objects that appear simultaneously in the same feature image, and its input image and output image have the same scale (or can be understood as having the same image resolution).

[0059] The grounding-transformer (GT) is used to implant conceptual attributes from high-level feature images into pixels of low-level feature images, thereby enhancing the feature information in the low-level feature images. Its output image has the same scale as the lower-level feature image. The level of a feature image is related to its scale; the larger the scale (higher image resolution), the lower the corresponding level. Conversely, the smaller the scale (lower image resolution), the higher the corresponding level.

[0060] The rendering-transformer (RT) is used to render the conceptual attributes of pixels in a higher-level feature image from the visual attributes of pixels in a lower-level feature image, thereby enhancing the feature information in the higher-level feature image. The output image has the same scale as the higher-level feature image.

[0061] For example, if there are three feature images, and the target image is 640*640 pixels, with corresponding image resolutions of 1 / 8, 1 / 16, and 1 / 32 of the target image, then feature image A has a resolution of 80*80 pixels; feature image B has a resolution of 40*40 pixels; and feature image C has a resolution of 20*20 pixels. (Refer to...) Figure 4 As shown, by inputting the multiple feature images into the self-conversion unit for feature extraction, a first fused image with the same scale as each feature image can be obtained.

[0062] Two feature images from adjacent layers of multiple feature images are input into the implantation conversion unit for feature extraction, resulting in multiple first fused images. For example, refer to... Figure 4 As shown, feature image A and feature image B, and feature image B and feature image C are respectively input into the implantation unit for feature extraction, and a first fused image with the same scale as feature image A and a first fused image with the same scale as feature image B are obtained respectively.

[0063] Two feature images from adjacent layers of multiple feature images are input into the rendering conversion unit for feature extraction, resulting in multiple first fused images. For example, refer to... Figure 4 As shown, feature image A and feature image B, and feature image B and feature image C are respectively input into the rendering conversion unit for feature extraction, and a first fused image with the same scale as feature image B and a first fused image with the same scale as feature image C are obtained respectively.

[0064] Furthermore, multiple feature pyramids can be formed by copying several of the first fused images (meaning the multiple first fused images are arranged sequentially from top to bottom according to their hierarchical level). Then, the multiple first fused images are divided according to different image resolutions, that is, first fused images at the same level are grouped together. After dividing the multiple first fused images, multiple sets of fused images corresponding to the target image are obtained.

[0065] In this embodiment of the invention, the self-transformer (ST) is used to extract the features of objects that appear simultaneously in the same feature image, and its input image and output image have the same scale (or can be understood as having the same image resolution).

[0066] S205. The multiple feature images are respectively stitched into a fusion image set with the same resolution as the corresponding images to obtain multiple sets of stitched images.

[0067] S206. Input multiple sets of stitched images into the second fusion module to perform second feature fusion and determine the target feature image.

[0068] In this embodiment of the invention, after obtaining the fused image set, each feature image is stitched together into the fused image set with the same image resolution, resulting in multiple stitched image sets. For example, each fused image set includes multiple feature images of the same scale.

[0069] In one example, the second feature fusion may include steps such as convolution and downsampling, and the second fusion module performs the following steps:

[0070] Convolution operations are performed on multiple sets of stitched images to reduce the number of channels in each set, thereby obtaining a second feature image with the same resolution as each individual set. After obtaining multiple second feature images, the higher-level second feature images are downsampled, and then stitched together with the next higher-level second feature images to obtain a stitched image. The stitched image and the next higher-level second feature images have the same scale.

[0071] The stitched image is then downsampled, and the downsampled stitched image is stitched together with a second feature image at a lower level than the stitched image to obtain an updated stitched image. The updated stitched image has the same scale as the second feature image at a lower level than the stitched image.

[0072] Determine whether the updated stitched image is at the lowest level among multiple second feature images. If the updated stitched image is at the lowest level among multiple second feature images, then the updated stitched image is used as the target feature image. For example, when the image resolution corresponding to the lowest level of the feature images is 80 pixels * 80 pixels, the target feature image is also 80 pixels * 80 pixels. Since the target feature image is obtained through feature fusion, it has better expressive power in terms of space and scale, thereby facilitating improved subsequent target recognition performance.

[0073] In another example, if the updated stitched image is not at the lowest level among the multiple second feature images, the updated stitched image is downsampled again, and the downsampled stitched image is stitched together with the second feature image at a level one level lower than the level corresponding to the updated stitched image to obtain the updated stitched image, and it is determined whether the updated stitched image is at the lowest level among the multiple second feature images.

[0074] S207. Perform target detection on the target feature image to obtain the target location information of the tracked target.

[0075] In this embodiment of the invention, to facilitate the separate output of the same target feature image for target detection and target re-identification, for example, the target feature image can be input into a preset target detection network for target detection. This target detection network is trained based on sample images including different tracking targets and corresponding bounding boxes. The target location information of the tracking targets is obtained; if there are more than one tracking target, multiple target location information is output. Furthermore, since the target feature image is essentially a downsampled version of the target image, referencing... Figure 4 As shown, when the sampling factor is 8, each pixel in the target feature image corresponds to an 8*8 pixel image block in the target image.

[0076] In one example, the target location information includes at least: center point confidence, center point offset, and target size. The center point confidence can be understood as the probability value that the corresponding image patch is the center point of the tracked target; the center point offset can be understood as the offset of the corresponding image patch relative to the center point of the tracked target; and the target size can be understood as the bounding box size of the predicted bounding box corresponding to the tracked target.

[0077] In one optional embodiment of the invention, the step of performing target detection on the target feature image to obtain the target location information of the tracked target includes:

[0078] Obtain the initial reference point set and the corresponding point coordinates.

[0079] Based on the initial reference point set and point set coordinates, target detection is performed on the target image to obtain the center point confidence, center point offset, and target size corresponding to the tracked target.

[0080] In one example, for each image patch, the coordinates of an initial reference point set are pre-set. The initial reference point set in each image patch is centrally distributed, with one initial center point at the center of each patch. This generates the initial reference point set corresponding to the target image and determines its coordinates. After initializing the initial reference point set, the target detection network performs the following steps: based on the coordinates of each initial center point in the initial reference point set, it predicts the center point offset of the tracked target and determines the coordinates of the target's center point. Then, for each initial reference point, it predicts the target size and the confidence level of the corresponding center point. This yields the target location information of the tracked target.

[0081] S208. Perform target re-identification on the target feature image to obtain re-identification feature information.

[0082] In this embodiment of the invention, target re-identification can be understood as extracting the feature information of each pixel in the target feature image and obtaining re-identification feature information. For example, multi-level convolution operations can be performed on the target feature image, such as using 256 convolutional layers for feature extraction, thereby extracting the feature information corresponding to each pixel in the target feature image (corresponding to each image block in the target image).

[0083] S209. Based on the target location information and re-identification feature information, perform target association tracking.

[0084] In this embodiment of the invention, a corresponding cost matrix is ​​calculated based on the target location information and the corresponding re-identification feature information. The cost matrix can be understood as the intersection-union ratio (IUR) between the predicted position of the tracking target in the current target image and the predicted bounding box of the tracking target in the target image. It can be calculated using Kalman filtering. Finally, target association tracking is performed based on the cost matrix. The Hungarian algorithm can be used for target association tracking. That is, based on the cost matrix, a set of matches is found that minimizes the total cost corresponding to multiple tracking targets; that is, the smaller the distance between the IURs of two consecutive target images, the smaller the total cost. Therefore, according to the target allocation scheme corresponding to the minimum total cost, a corresponding identifier (ID) is assigned to the tracking target, thereby completing the target association tracking. By sharing target feature images and simultaneously performing target detection and target re-identification, the timeliness of target tracking can be greatly improved.

[0085] In summary, this invention discloses a target tracking method. The method may include first acquiring a target image, wherein the target image includes at least one tracking target. Then, feature extraction is performed on the target image to obtain multiple feature images, wherein the image resolutions of the feature images are different. Next, feature information from the multiple feature images is fused to determine the target feature image corresponding to the target image. Subsequently, target detection is performed on the target feature image to obtain the target location information of the tracking target, and target re-identification is performed on the target feature image to obtain re-identified feature information. Finally, target association tracking is performed based on the target location information and the re-identified feature information. By sharing the target feature image and simultaneously performing target detection and target re-identification, the timeliness of target tracking can be greatly improved.

[0086] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0087] Reference Figure 5 This illustration shows a target tracking device provided by an embodiment of the present invention, the device may include:

[0088] Image acquisition module 501 is used to acquire a target image, wherein the target image includes at least one tracking target.

[0089] The feature image determination module 502 is used to extract features from the target image to obtain multiple feature images, wherein the image resolutions of the feature images are different.

[0090] The feature fusion module 503 fuses feature information from multiple feature images to determine the target feature image corresponding to the target image.

[0091] The target detection module 504 is used to perform target detection on the target feature image to obtain the target location information of the tracked target.

[0092] The re-identification module 505 is used to perform target re-identification on the target feature image to obtain re-identification feature information.

[0093] The association tracking module 506 is used to perform target association tracking based on the target location information and re-identification feature information.

[0094] In an optional embodiment of the invention, the feature fusion module 503 may also be used for:

[0095] The multiple feature images are input into a feature fusion network to fuse feature information and determine the target feature image corresponding to the target image.

[0096] In an optional embodiment of the invention, the feature fusion network includes a first fusion module and a second fusion module, and the feature fusion module 503 may further include:

[0097] The first fusion submodule is used to input the multiple feature images into the first fusion module, perform first feature fusion, and determine multiple sets of fused image sets corresponding to the target image, wherein the multiple first fused images in each fused image set have the same image resolution.

[0098] The image stitching submodule is used to stitch the multiple feature images into a fusion image set with the same resolution as the corresponding images, thereby obtaining multiple stitched image sets.

[0099] The second fusion submodule is used to input multiple sets of stitched images into the second fusion module to perform second feature fusion and determine the target feature image.

[0100] In one optional embodiment of the invention, the second fusion submodule may include:

[0101] The second image determination unit is used to perform convolution operations on multiple sets of stitched images to obtain a second feature image with the same image resolution as each set of stitched images.

[0102] The first downsampling unit is used to downsample the second feature image at a higher level, and then stitch the downsampled second feature image with the second feature image at the next higher level to obtain a stitched image.

[0103] The second downsampling unit is used to downsample the stitched image and stitch the downsampled stitched image with a second feature image that is one level lower than the level corresponding to the stitched image to obtain an updated stitched image.

[0104] The target feature determination unit is used to determine the updated stitched image as the target feature image when the updated stitched image is at the lowest level among a plurality of second feature images.

[0105] In an optional embodiment of the invention, the first fusion module includes a self-conversion unit, an implantation conversion unit, and a rendering conversion unit, and the first fusion sub-module may include:

[0106] The first fusion unit is used to input the multiple feature images into the self-conversion unit, the implantation conversion unit, and the rendering conversion unit to perform first feature fusion and obtain multiple first fused images.

[0107] The image segmentation unit is used to segment the multiple first fused images according to different image resolutions to obtain multiple sets of fused images corresponding to the target image.

[0108] In an optional embodiment of the invention, the feature image determination module 502 may also be used for:

[0109] The target image is subjected to feature extraction using a feature extraction network to obtain multiple feature images.

[0110] In an optional embodiment of the invention, the feature extraction network includes a multi-layered cascaded residual module, and the feature image determination module 502 may include:

[0111] The feature extraction submodule is used to extract features from the target image based on the multi-layered cascaded residual modules.

[0112] The feature image determination submodule is used to select images extracted by multiple residual modules as feature images.

[0113] In one optional embodiment of the invention, the target location information includes at least: center point confidence, center point offset, and target size.

[0114] In an optional embodiment of the invention, the target detection module 504 may include:

[0115] The initial data acquisition submodule is used to obtain the initial reference point set and the corresponding point set coordinates.

[0116] The target detection submodule is used to perform target detection on the target image based on the initial reference point set and point set coordinates, and obtain the center point confidence, center point offset and target size corresponding to the tracked target.

[0117] In an optional embodiment of the invention, the re-identification module 505 may also be used for:

[0118] Multi-level convolution operations are performed on the target feature image to extract the features of each pixel in the target feature image, thereby obtaining re-identification feature information.

[0119] In an optional embodiment of the invention, the association tracking module 506 may include:

[0120] The matrix calculation submodule is used to calculate the corresponding cost matrix based on the target location information and the re-identification feature information.

[0121] The association tracking submodule is used to perform target association tracking based on the cost matrix.

[0122] In an optional embodiment of the invention, the image acquisition module 501 can also be used for:

[0123] Extract video frame images from the video.

[0124] The video frame images are preprocessed to obtain the target image.

[0125] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0126] It will be readily apparent to those skilled in the art that any combination of the above embodiments is feasible, and therefore any combination of the above embodiments is an implementation scheme of the present invention. However, due to space limitations, this specification will not describe them in detail here.

[0127] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0128] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in fewer than all features of the single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0129] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0130] An electronic device, comprising:

[0131] One or more processors;

[0132] Memory;

[0133] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the methods described in the above embodiments.

[0134] A computer-readable storage medium stores a computer program for use in conjunction with an electronic device, the computer program being executable by a processor to perform the methods described in the embodiments above.

[0135] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0137] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0139] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0140] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0141] The target tracking method and target tracking device provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A target tracking method, characterized in that, The method includes: Acquire a target image, wherein the target image includes at least one tracking target; Feature extraction is performed on the target image to obtain multiple feature images, wherein the image resolutions of the feature images are different. By fusing feature information from multiple feature images, the target feature image corresponding to the target image is determined; Target detection is performed on the target feature image to obtain the target location information of the tracked target; The target feature image is re-identified to obtain re-identified feature information; Based on the target location information and re-identification feature information, target association tracking is performed; The step of fusing feature information from multiple feature images to determine the target feature image corresponding to the target image includes: The multiple feature images are input into the first fusion module to perform first feature fusion, thereby determining multiple sets of fused image sets corresponding to the target image, wherein the multiple first fused images in each fused image set have the same image resolution; The multiple feature images are respectively stitched together into a fusion image set with the same resolution as the corresponding images to obtain multiple sets of stitched images; Multiple sets of stitched images are input into the second fusion module to perform second feature fusion and determine the target feature image; The first fusion module includes a self-conversion unit, an implantation conversion unit, and a rendering conversion unit. The step of inputting the multiple feature images into the first fusion module, performing first feature fusion, and determining multiple sets of fused image sets corresponding to the target image includes: The multiple feature images are input into the self-conversion unit, the implantation conversion unit, and the rendering conversion unit to perform first feature fusion, resulting in multiple first fused images. The self-conversion unit is used to extract object features that appear simultaneously in the same feature image. The implantation conversion unit is used to extract features from two feature images at adjacent levels in the multiple feature images to implant the conceptual attributes in the higher-level feature image into the pixels of the lower-level feature image. The rendering conversion unit is used to extract features from two feature images at adjacent levels in the multiple feature images to render the visual attributes of the pixels in the lower-level feature image into the conceptual attributes of the higher-level feature image. The multiple first fused images are divided according to different image resolutions to obtain multiple sets of fused images corresponding to the target image.

2. The target tracking method according to claim 1, characterized in that, The step of inputting multiple sets of stitched image sets into the second fusion module for second feature fusion to determine the target feature image includes: Perform convolution operations on multiple sets of stitched images to obtain a second feature image with the same resolution as each set of stitched images; The second feature image at a high level is downsampled, and the downsampled second feature image is then stitched together with the second feature image at the next higher level to obtain a stitched image. The stitched image is downsampled, and the downsampled stitched image is stitched together with a second feature image at a lower level than the stitched image to obtain an updated stitched image. If the updated stitched image is at the lowest level among multiple second feature images, then the updated stitched image is used as the target feature image.

3. The target tracking method according to claim 1, characterized in that, The feature extraction of the target image yields multiple feature images, including: The target image is subjected to feature extraction using a feature extraction network to obtain multiple feature images.

4. The target tracking method according to claim 3, characterized in that, The feature extraction network includes multiple cascaded residual modules. The feature extraction network is used to extract features from the target image to obtain multiple feature images, including: Feature extraction is performed on the target image based on the multi-layered cascaded residual modules; Images extracted from multiple residual modules are selected as feature images.

5. The target tracking method according to claim 1, characterized in that, The target location information includes at least: center point confidence, center point offset, and target size.

6. The target tracking method according to claim 5, characterized in that, The step of performing target detection on the target feature image to obtain the target location information of the tracked target includes: Obtain the initial reference point set and the corresponding point set coordinates; Based on the initial reference point set and point set coordinates, target detection is performed on the target image to obtain the center point confidence, center point offset, and target size corresponding to the tracked target.

7. The target tracking method according to claim 1, characterized in that, The process of re-identifying the target feature image to obtain re-identified feature information includes: Multi-level convolution operations are performed on the target feature image to extract the features of each pixel in the target feature image, thereby obtaining re-identification feature information.

8. The target tracking method according to claim 1, characterized in that, The step of performing target association tracking based on the target location information and re-identification feature information includes: Based on the target location information and re-identification feature information, the corresponding cost matrix is ​​calculated; Based on the cost matrix, target association tracking is performed.

9. The target tracking method according to claim 1, characterized in that, The acquisition of the target image includes: Extract video frame images from the video; The video frame images are preprocessed to obtain the target image.

10. A target tracking device, characterized in that, The device includes: An image acquisition module is used to acquire a target image, wherein the target image includes at least one tracking target; The feature image determination module is used to extract features from the target image to obtain multiple feature images, wherein the image resolutions of the feature images are different. The feature fusion module is used to fuse feature information from multiple feature images to determine the target feature image corresponding to the target image; The target detection module is used to perform target detection on the target feature image to obtain the target location information of the tracked target; The re-identification module is used to perform target re-identification on the target feature image to obtain re-identification feature information; The association tracking module is used to perform target association tracking based on the target location information and re-identification feature information; The feature fusion module is used to: input the multiple feature images into the feature fusion network, fuse feature information, and determine the target feature image corresponding to the target image; The feature fusion network includes a first fusion module and a second fusion module, and the feature fusion module further includes: The first fusion submodule is used to input the multiple feature images into the first fusion module, perform first feature fusion, and determine multiple sets of fused image sets corresponding to the target image, wherein the multiple first fused images in each fused image set have the same image resolution; The image stitching submodule is used to stitch the multiple feature images into a fusion image set with the same resolution as the corresponding images, thereby obtaining multiple stitched image sets; The second fusion submodule is used to input multiple sets of stitched images into the second fusion module, perform second feature fusion, and determine the target feature image; The first fusion module includes a self-conversion unit, an implantation conversion unit, and a rendering conversion unit. The first fusion sub-module includes: The first fusion unit is used to input the multiple feature images into the self-conversion unit, the implantation conversion unit, and the rendering conversion unit to perform first feature fusion and obtain multiple first fused images. The self-conversion unit is used to extract object features that appear simultaneously in the same feature image. The implantation conversion unit is used to extract features from two feature images at adjacent levels in the multiple feature images to implant the conceptual attributes in the higher-level feature image into the pixels of the lower-level feature image. The rendering conversion unit is used to extract features from two feature images at adjacent levels in the multiple feature images to render the visual attributes of the pixels in the lower-level feature image into the conceptual attributes of the higher-level feature image. The image segmentation unit is used to segment the multiple first fused images according to different image resolutions to obtain multiple sets of fused images corresponding to the target image.

11. An electronic device, comprising: One or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method of any one of claims 1-9.

12. A computer-readable storage medium storing a computer program for use in conjunction with an electronic device, said computer program being executable by a processor to perform the method of any one of claims 1-9.

Images