Target tracking method, device, terminal equipment and computer readable storage medium

Abstract

The present application is applicable to the field of image processing technology, and provides a target tracking method, device, terminal equipment and computer-readable storage medium, the method comprising: obtaining the first feature data of the target object in the initial frame and the second feature data of the target object in the current frame respectively through the first feature module of the preset model; obtaining the third feature data in the first feature data and the fourth feature data in the second feature data respectively through the second feature module of the preset model; detecting the center position point of the target object in the current frame according to the third feature data and the fourth feature data; generating multiple candidate detection frames of the target object corresponding to the center position point; determining the target detection frame of the target object in the current frame from the multiple candidate detection frames according to the first feature data and the second feature data. Through the above method, the algorithm complexity of target tracking can be effectively reduced, thereby improving the efficiency of target tracking.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular, relates to a target tracking method, apparatus, terminal device and computer-readable storage medium. Background Art

[0002] Target tracking refers to the technology of associating target objects detected in multiple frames of images. It is one of the important research directions in the field of computer vision. This technology is widely used in sports broadcasting, security monitoring, robotics and other fields. The algorithm complexity of existing target tracking methods is relatively high, and the performance requirements of computing devices are relatively high. For computing devices with lower performance, such as embedded devices, running existing target tracking methods will result in slower computing speeds, which will affect the target tracking effect. Summary of the invention

[0003] The embodiments of the present application provide a target tracking method, apparatus, terminal device and computer-readable storage medium, which can effectively reduce the algorithm complexity of target tracking and thereby improve the efficiency of target tracking.

[0004] In a first aspect, an embodiment of the present application provides a target tracking method, wherein the preset model includes a first feature extraction module and a second feature extraction module connected in sequence, and the method includes:

[0005] Acquire, by the first feature module, first feature data of the target object in the initial frame and second feature data of the target object in the current frame, respectively, wherein the initial frame is the first frame captured image of the target object, the current frame is the Nth frame captured image of the target object, and N is a positive integer greater than 1;

[0006] Respectively acquiring third feature data in the first feature data and fourth feature data in the second feature data through the second feature module;

[0007] Detecting a center point of the target object in the current frame according to the third feature data and the fourth feature data;

[0008] Generate multiple candidate detection boxes of the target object corresponding to the center position point;

[0009] A target detection frame of the target object in the current frame is determined from the multiple candidate detection frames according to the first feature data and the second feature data.

[0010] In an embodiment of the present application, in the process of detecting the center position point of the target object, the first feature extraction module and the second feature extraction module in the preset model are used to sequentially obtain feature data, and in the process of determining the target detection frame from multiple candidate frames, the feature data output by the first feature extraction module in the preset model is used to determine the target detection frame. This is equivalent to using deeper feature data when determining the center position point, and using shallower feature data when determining the target detection frame. Since the accuracy of the center position point determines the detection position of the target object in the current frame to a greater extent, the use of deeper feature data can ensure that a more accurate center point position is determined; and when the target detection frame is subsequently determined, shallower feature data is used, which helps to reduce the amount of calculation and improve detection efficiency. Through the above method, while ensuring the accuracy of target tracking, the algorithm complexity of target tracking can be effectively reduced and the efficiency of target tracking can be improved.

[0011] In a possible implementation manner of the first aspect, respectively acquiring, by the first feature module, first feature data of the target object in the initial frame and second feature data of the target object in the current frame, includes:

[0012] Performing data enhancement processing on the initial frame to obtain enhanced data;

[0013] Inputting the enhanced data into the first feature extraction module to obtain the first feature data;

[0014] The current frame is input into the first feature extraction module to obtain the second feature data.

[0015] In a possible implementation manner of the first aspect, performing data enhancement processing on the initial frame to obtain enhanced data includes:

[0016] Performing M kinds of preprocessing on the initial frame respectively to obtain processed images corresponding to the M kinds of preprocessing, wherein M is a positive integer greater than or equal to 1;

[0017] Extracting first image data of the target object in the initial frame;

[0018] extracting second image data of the target object in each of the processed images;

[0019] The enhancement data is generated based on the first image data and the second image data.

[0020] In a possible implementation manner of the first aspect, detecting a center position point of the target object in the current frame according to the third feature data and the fourth feature data includes:

[0021] Inputting the third feature data and the fourth feature data into the trained classification model to obtain respective confidences of a plurality of key points of the target object in the current frame;

[0022] The key point corresponding to the maximum value of the confidences of the multiple key points is determined as the center position point of the target object in the current frame.

[0023] In a possible implementation manner of the first aspect, generating a plurality of candidate detection boxes of the target object corresponding to the center position point includes:

[0024] When N is equal to 2, K aspect ratios are obtained within a preset first aspect ratio range, and candidate detection frames corresponding to the K aspect ratios are generated according to the center point position, where K is a positive integer greater than 1;

[0025] When N is greater than 2, a second aspect ratio range of the target detection frame of the target object in the target frame is counted; L aspect ratios are obtained according to the second aspect ratio range; and candidate detection frames corresponding to the L aspect ratios are generated according to the center point position, where L is a positive integer greater than 1, and the target frame includes N-1 frames of images captured before the current frame.

[0026] In a possible implementation manner of the first aspect, determining, from the multiple candidate detection frames, a target detection frame of the target object in the current frame according to the first feature data and the second feature data includes:

[0027] Performing pooling processing according to the first feature data and the second feature data to obtain pooled data;

[0028] Calculate the intersection-and-union ratios (IoU) corresponding to each of the plurality of candidate detection frames according to the pooled data, wherein the IoU represents an overlap ratio between the candidate detection frame and a reference detection frame, where the reference detection frame is a detection frame of the target object in the initial frame;

[0029] The candidate detection frame corresponding to the maximum value of the calculated intersection-over-union ratio is determined as the target detection frame of the target object in the current frame.

[0030] In a possible implementation manner of the first aspect, the method further includes: performing model acceleration on the preset model using a preset acceleration algorithm.

[0031] In a second aspect, an embodiment of the present application provides a target tracking device, which is applied to a preset model, wherein the preset model includes a first feature extraction module and a second feature extraction module connected in sequence, and the device includes:

[0032] A first acquisition module, used to acquire first feature data of a target object in an initial frame and second feature data of the target object in a current frame respectively through the first feature module, wherein the initial frame is a first frame captured image of the target object, and the current frame is an Nth frame captured image of the target object, where N is a positive integer greater than 1;

[0033] A second acquisition module, used to respectively acquire third feature data in the first feature data and fourth feature data in the second feature data through the second feature module;

[0034] A center detection module, used for detecting a center position point of the target object in the current frame according to the third feature data and the fourth feature data;

[0035] A candidate box generation module, used to generate multiple candidate detection boxes of the target object corresponding to the central position point;

[0036] A target frame determination module is used to determine a target detection frame of the target object in the current frame from the multiple candidate detection frames according to the first feature data and the second feature data.

[0037] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a target tracking method as described in any one of the first aspects above when executing the computer program.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and wherein the computer program, when executed by a processor, implements a target tracking method as described in any one of the first aspects above.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the target tracking method described in any one of the above-mentioned first aspects.

[0040] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0042] Figure 1 is a flowchart of a target tracking method provided by an embodiment of the present application;

[0043] Figure 2 is a schematic diagram of the target tracking algorithm structure provided in an embodiment of the present application;

[0044] Figure 3(a) to Figure 3(e) is a schematic diagram of the preprocessing provided in the embodiment of the present application;

[0045] Figure 4 is a structural block diagram of a target tracking device provided in an embodiment of the present application;

[0046] Figure 5 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0048] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0049] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0050] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0051] See also Figure 1 , is a flow chart of the target tracking method provided in the embodiment of the present application. The embodiment of the present application is applied to a preset model, and the preset model includes a first feature extraction module and a second feature extraction module connected in sequence. Optionally, the preset model in the embodiment of the present application can be a neural network model, such as a neural network model such as ResNet18 and ResNet50.

[0052] Taking ResNet18 as an example, the model includes 18 modules. If the 18 modules are used to extract features in sequence, the amount of calculation is large. In the embodiment of the present application, in order to reduce the amount of data processing, some of the modules are used for feature extraction.

[0053] For example, see Figure 2 , is a schematic diagram of the target tracking algorithm structure provided in the embodiment of the present application. Figure 2 As shown, in the embodiment of the present application, block1-2 (module 1 and module 2) in ResNet18 is used as the first feature extraction module, and block3 (module 3) is used as the second feature extraction module.

[0054] It should be noted that Figure 2 The above is only an example of a preset model and does not specifically limit the network structure of the preset model or the number of modules in the preset model.

[0055] The following methods are all introduced using ResNet18 as an example. As an example but not a limitation, the method may include the following steps:

[0056] S101, obtaining first feature data of a target object in an initial frame and second feature data of a target object in a current frame respectively through a first feature module.

[0057] The initial frame is the first frame of the captured image of the target object, the current frame is the Nth frame of the captured image of the target object, and N is a positive integer greater than 1.

[0058] Optionally, before S101, data conversion processing may be performed on the initial frame and the current frame. Exemplarily, by calling the OpenCV library, the initial frame and the current frame are converted into image data respectively, and then feature data in the converted image data is extracted. In this way, the image reading and conversion method can be optimized, and large matrix calculations can be migrated from the CPU to the GPU.

[0059] like Figure 2 As shown, the first feature module includes block1 and block2 in ResNet18.

[0060] There are two existing target tracking algorithms. One is to detect the target object in each frame of the collected image separately. The correlation of the target objects in each frame detected by this method is poor, and the target tracking accuracy is low. Another method is to collect a reference image in advance, and then use the reference image to detect the target object in each frame of the real-time collected image. In this method, the detection accuracy often depends on the reference image. If the reference image is not selected properly (such as the target object is blocked or deformed, etc.), the subsequent detection accuracy will be low.

[0061] In the embodiment of the present application, the first frame image (i.e., the initial frame) of the target object collected in real time is used as the reference image. Since the features of the target object are relatively uniform in a specific application scenario, using the initial frame as the reference image can effectively reduce the feature difference between the reference image and the subsequent frames to be detected, thereby effectively improving the detection accuracy.

[0062] In one embodiment, an implementation of S101 may be: inputting the initial frame into a first feature extraction module to obtain first feature data; and inputting the current frame into the first feature extraction module to obtain second feature data.

[0063] Since the initial frame is used as a reference image in the embodiment of the present application, the detection accuracy of the target object in the subsequent frames can only be guaranteed if the detection accuracy of the target object in the initial frame is guaranteed. In order to improve the detection accuracy of the target object in the initial frame. In one embodiment, S101 may include the following steps:

[0064] Perform data enhancement processing on the initial frame to obtain enhanced data; input the enhanced data into the first feature extraction module to obtain first feature data; input the current frame into the first feature extraction module to obtain second feature data.

[0065] Of course, data enhancement processing can also be performed on the current frame, but this will increase the amount of data calculation. Therefore, in the embodiment of the present application, data enhancement processing is performed on the initial frame to improve the detection accuracy of the target object in the initial frame; and data enhancement processing is not performed on the current frame to reduce the amount of data calculation.

[0066] Optionally, the current frame may be input into the first feature extraction module to obtain the second feature data. Alternatively, local image data corresponding to the target object in the current frame may be first extracted and the local image data may be input into the first feature extraction module to obtain the second feature data.

[0067] Optionally, an implementation method of performing data enhancement processing on the initial frame may include:

[0068] Perform M kinds of preprocessing on the initial frame respectively to obtain processed images corresponding to the M kinds of preprocessing, where M is an integer greater than or equal to 1; extract the first image data of the target object in the initial frame; extract the second image data of the target object in each processed image; and generate enhanced data based on the first image data and the second image data.

[0069] In the embodiment of the present application, the preprocessing method may include image rotation processing, image flipping processing, image blur processing, image translation processing, image scale change processing, image brightness change processing, and image noise processing.

[0070] See also Figure 3(a) to Figure 3(e) , is a schematic diagram of the preprocessing provided by an embodiment of the present application. As shown in Figure 3(a), image rotation processing refers to rotating the image clockwise or counterclockwise by a preset angle. Image flip processing includes mirror flipping (reversing the left and right of the image) and vertical flipping (rotating the image 180 degrees). The image flip shown in Figure 3(b) is a vertical flip. Image blur processing may include Gaussian blur and mean filter modules, etc. As shown in Figure 3(c), the left side is the original image, and the right side image is the image after Gaussian blur processing. Image brightness change processing may include image brightness enhancement and image brightness reduction. As shown in Figure 3(d), the left side image is an image after image brightness enhancement processing, and the right side image is an image after image brightness reduction processing. Image noise processing may include adding Gaussian noise, white noise, and salt and pepper noise, etc. By adding different types of noise, the picture noise caused by various reasons can be imitated to prevent the convolution result errors caused by noise. As shown in Figure 3(e), the left side image is an image after adding Gaussian noise processing, and the right side image is an image after adding salt and pepper noise processing. Image translation processing includes up and down movement and left and right movement of the image. Image scale change processing includes zooming in or out of the image and adding data to local areas.

[0071] The initial frame may be processed separately by any one of the above-mentioned preprocessing methods; the initial frame may also be processed in combination by any of the above-mentioned multiple preprocessing methods. For example, after the initial frame is rotated, the rotated image is subjected to image brightness enhancement, and then Gaussian noise is superimposed on the brightness enhanced image.

[0072] Generating enhanced data according to the first image data and the second image data may be to superimpose the first image data and the second image data. For example, the first image data includes a pixel value corresponding to each pixel point on the target object in the initial frame, the second image data includes a pixel value corresponding to each pixel point on the target object in the processed image, and the pixel values ​​of corresponding pixels in the first image data and the second image data are added.

[0073] The enhancement data may be generated according to the first image data and the second image data, or the first image data and the second image data may be combined. Continuing with the above example, the first image data may be generated into a matrix, the second image data may be generated into a matrix, and then the two matrices may be concatenated.

[0074] Inputting the enhanced data into the classification model and the preset model can improve the preset model's ability to recognize target features and improve the classification model's ability to discriminate target features. In addition, in the embodiment of the present application, the network depth of the preset model is reduced, and shallow image features are retained. Through the above-mentioned data enhancement processing of the initial frame, the accuracy loss caused by the shallow network depth can be effectively compensated, while reducing the amount of calculation, ensuring the recognition accuracy of the features.

[0075] S102, respectively obtaining the third feature data in the first feature data and the fourth feature data in the second feature data through the second feature module.

[0076] like Figure 2 As shown, the second feature module includes block3 in ResNet18. Through the second feature module, features can be further extracted from the feature data output by the first feature module. Compared with the feature data output by the first feature module, the feature data output by the second feature module represents deeper image features.

[0077] In one embodiment, when extracting feature data using a preset model, a preset acceleration algorithm can be used to accelerate the preset model. For example, Nvidia TensorRT is used to accelerate the deep learning network to accelerate the preset model, that is, the algorithm is accelerated during the operation process of extracting feature data from the preset model.

[0078] exist Figure 2 In the target tracking algorithm structure shown, it is equivalent to using a twin network to extract feature data, that is, the model parameters of the preset model used when extracting feature data in the initial frame are the same as the model parameters used when extracting feature data in the current frame (parameter sharing). The twin network can be used to evaluate the similarity between two inputs. In other words, the third feature data and the fourth feature data extracted by the preset model can be used to determine the similarity between the initial frame and the current frame.

[0079] S103: Detect a center point of the target object in the current frame according to the third feature data and the fourth feature data.

[0080] In the process of detecting the center position of the target object, the first feature extraction module and the second feature extraction module in the preset model are used to sequentially obtain feature data, which is equivalent to using deeper feature data when determining the center position. Since the accuracy of the center position determines the detection position of the target object in the current frame to a greater extent, the use of deeper feature data can ensure that a more accurate center point position is determined.

[0081] In one embodiment, the method of detecting the center point in S103 may include:

[0082] The third feature data and the fourth feature data are input into the trained classification model to obtain the confidence of each of the multiple key points of the target object in the current frame; the key point corresponding to the maximum value of the confidence of each of the multiple key points is determined as the center position point of the target object in the current frame.

[0083] The classification model can be Figure 2 Inputting the third feature data and the fourth feature data into the classifier is equivalent to detecting the fourth feature data with reference to the third feature data, that is, detecting the target object in the current frame with reference to the feature data of the target object in the initial frame.

[0084] In order to improve the classification accuracy of the classifier, the trained classifier may be initialized in advance. Specifically, the initialization process may include: using the enhanced data obtained by performing data enhancement processing on the initial frame as described in S101, performing iterative optimization processing on the classifier to obtain an optimized classifier.

[0085] It should be noted that in order to simplify the calculation steps in the target tracking process, the processing result of the initial frame can be saved, that is, the first feature data of the target object in the initial frame and the third feature data obtained based on the first feature data are saved. When detecting each frame of the image after the initial frame, the saved processing result of the initial frame can be read without repeatedly processing the initial frame.

[0086] S104, generating multiple candidate detection boxes of the target object corresponding to the center position point.

[0087] In one embodiment, a first aspect ratio range may be preset, and within the first aspect ratio range, a plurality of aspect ratios may be randomly selected, and then a candidate detection frame corresponding to each aspect ratio may be generated according to a center position point.

[0088] However, the candidate detection frame generated by the above method may be quite different from the shape of the target detection frame of the previous frame. In practical applications, the frame rate of tracking the target object is usually high, and the detection frame of the target object in two adjacent frames will not be greatly deformed.

[0089] In order to obtain more accurate candidate detection frames, in one embodiment, a method of generating multiple candidate detection frames may include:

[0090] When N is equal to 2, K aspect ratios are obtained within a preset first aspect ratio range, and candidate detection frames corresponding to the K aspect ratios are generated according to the center point position, where K is a positive integer greater than 1.

[0091] When N is greater than 2, the second aspect ratio range of the target detection frame of the target object in the target frame is counted; L aspect ratios are obtained according to the second aspect ratio range; and candidate detection frames corresponding to L aspect ratios are generated according to the position of the center point, where L is a positive integer greater than 1, and the target frame includes the N-1 frames of images captured before the current frame.

[0092] Exemplarily, assuming that the current frame is the 6th image of the target object, the aspect ratios of the target detection frames of the previous 5 frames are 1, 1.5, 1.2, 1.4 and 2 respectively. The second aspect ratio range is 1 to 2. Within this range, 3 aspect ratios are randomly selected. Assuming that the selected aspect ratios are 1, 1.5 and 2, correspondingly, candidate detection frames corresponding to the three aspect ratios are generated according to the center position point. It should be noted that the more aspect ratios are randomly selected, the greater the amount of calculation required to determine the target detection frame later, but the accuracy is also higher.

[0093] By using the second aspect ratio range, a sudden change in the shape of the candidate detection box can be avoided.

[0094] Optionally, after determining the second aspect ratio range, the range may be appropriately compressed or expanded to obtain a transformed second aspect ratio range, and then multiple aspect ratios may be selected within the transformed second aspect ratio range. Continuing with the above example, the second aspect ratio range 1 to 2 is compressed by half to obtain a transformed second aspect ratio range of 1.25 to 1.75.

[0095] Optionally, when N is greater than 2, the aspect ratios of the target detection frames in the images captured in H frames before the current frame can also be obtained, and candidate detection frames corresponding to the H aspect ratios are generated according to the center position point, where H is greater than 1 and less than N-1.

[0096] S105: Determine a target detection frame of the target object in the current frame from a plurality of candidate detection frames according to the first feature data and the second feature data.

[0097] In one embodiment, the step of determining the target detection frame in S105 may include:

[0098] Pooling processing is performed according to the first feature data and the second feature data to obtain pooled data; the intersection-and-union ratios corresponding to each of the multiple candidate detection frames are calculated according to the pooled data; and the candidate detection frame corresponding to the maximum value in the calculated intersection-and-union ratios is determined as the target detection frame of the target object in the current frame.

[0099] The IoU represents the overlap ratio between the candidate detection frame and the reference detection frame, and the reference detection frame is the detection frame of the target object in the initial frame.

[0100] Pooling can be maximum pooling, average pooling, or Figure 2 prpool (Precise ROIPooling, precise pooling of the region of interest) shown in . The pooled data is fully connected (such as Figure 2 fc) shown in the figure, the intersection-over-union ratio corresponding to each candidate box can be obtained. Figure 2 As shown in FIG. 1 , three candidate detection frames are detected in the current frame, and their corresponding IoU ratios are 0.81, 0.72, and 0.58. The candidate detection frame with the largest IoU ratio, that is, the candidate detection frame corresponding to 0.81, is determined as the target detection frame.

[0101] Optionally, the first feature data may be convolved (eg Figure 2 conv shown), pooling (such as Figure 2 prpool) and full connection processing (such as Figure 2 fc shown in the figure) to mine deeper feature information. The second feature data can be subjected to secondary convolution processing. It should be noted that the computational complexity of the convolution, pooling and full connection processing performed in this step is much lower than the computational complexity of the second feature extraction module.

[0102] In one embodiment, before step S105, multiple candidate detection frames may be refined. For example, if the aspect ratios of the determined candidate detection frames are 1 and 2, a more accurate aspect ratio, such as 1.2 and 2.3, may be obtained through refinement.

[0103] It can be refined by iterative optimization, but the computational cost can be controlled by controlling the number of iterative optimizations. The technical cost can also be controlled by cutting out unnecessary parameter calls during the iterative optimization process.

[0104] S105 is equivalent to using shallower feature data when determining the target detection frame. Since deeper feature data has been used to determine a more accurate center position point in S101 and S102, using deeper feature data when determining the target detection frame is beneficial to reducing the amount of calculation and improving detection efficiency. Through the above method, while ensuring the accuracy of target tracking, the algorithm complexity of target tracking can be effectively reduced and the efficiency of target tracking can be improved.

[0105] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0106] Corresponding to the target tracking method described in the above embodiment, Figure 4 This is a structural block diagram of the target tracking device provided in an embodiment of the present application. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.

[0107] Reference Figure 4 , the device comprises:

[0108] The first acquisition module 41 is used to obtain the first feature data of the target object in the initial frame and the second feature data of the target object in the current frame through the first feature module, wherein the initial frame is the first frame captured image of the target object, and the current frame is the Nth frame captured image of the target object, and N is a positive integer greater than 1.

[0109] A second acquisition module 42, configured to respectively acquire third feature data in the first feature data and fourth feature data in the second feature data through the second feature module;

[0110] The center detection module 43 is used to detect the center position point of the target object in the current frame according to the third feature data and the fourth feature data.

[0111] The candidate box generation module 44 is used to generate multiple candidate detection boxes of the target object corresponding to the central position point.

[0112] The target frame determination module 45 is used to determine the target detection frame of the target object in the current frame from the multiple candidate detection frames according to the first feature data and the second feature data.

[0113] Optionally, the first acquisition module 41 is further used for:

[0114] Performing data enhancement processing on the initial frame to obtain enhanced data; inputting the enhanced data into the first feature extraction module to obtain the first feature data; and inputting the current frame into the first feature extraction module to obtain the second feature data.

[0115] Optionally, the first acquisition module 41 is further used for:

[0116] Perform M types of preprocessing on the initial frame respectively to obtain processed images corresponding to the M types of preprocessing, wherein M is a positive integer greater than or equal to 1; extract the first image data of the target object in the initial frame; extract the second image data of the target object in each of the processed images; and generate the enhanced data according to the first image data and the second image data.

[0117] Optionally, the central detection module 43 is further used for:

[0118] The third feature data and the fourth feature data are input into the trained classification model to obtain the confidence scores of the multiple key points of the target object in the current frame; and the key point corresponding to the maximum value of the confidence scores of the multiple key points is determined as the center position point of the target object in the current frame.

[0119] Optionally, the candidate box generating module 44 is further used for:

[0120] When N is equal to 2, K aspect ratios are obtained within a preset first aspect ratio range, and candidate detection frames corresponding to the K aspect ratios are generated according to the center point position, where K is a positive integer greater than 1;

[0121] When N is greater than 2, a second aspect ratio range of the target detection frame of the target object in the target frame is counted; L aspect ratios are obtained according to the second aspect ratio range; and candidate detection frames corresponding to the L aspect ratios are generated according to the center point position, where L is a positive integer greater than 1, and the target frame includes N-1 frames of images captured before the current frame.

[0122] Optionally, the target frame determination module 45 is further used for:

[0123] Performing pooling processing according to the first feature data and the second feature data to obtain pooled data;

[0124] Calculate the intersection-and-union ratios (IoU) corresponding to each of the plurality of candidate detection frames according to the pooled data, wherein the IoU represents an overlap ratio between the candidate detection frame and a reference detection frame, where the reference detection frame is a detection frame of the target object in the initial frame;

[0125] The candidate detection frame corresponding to the maximum value of the calculated intersection-over-union ratio is determined as the target detection frame of the target object in the current frame.

[0126] Optionally, the device 4 further comprises:

[0127] The acceleration module 46 is used to accelerate the preset model using a preset acceleration algorithm.

[0128] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0129] in addition, Figure 4 The target tracking device shown may be a software unit, a hardware unit, or a combination of software and hardware, which is built into an existing terminal device, or may be integrated into the terminal device as an independent pendant, or may exist as an independent terminal device.

[0130] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0131] Figure 5 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Figure 5 As shown, the terminal device 5 of this embodiment includes: at least one processor 50 ( Figure 5 Only one is shown in the figure) a processor, a memory 51, and a computer program 52 stored in the memory 51 and executable on the at least one processor 50, and when the processor 50 executes the computer program 52, the steps in any of the above-mentioned target tracking method embodiments are implemented.

[0132] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 5 It is only an example of the terminal device 5 and does not constitute a limitation on the terminal device 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0133] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0134] In some embodiments, the memory 51 may be an internal storage unit of the terminal device 5, such as a hard disk or memory of the terminal device 5. In other embodiments, the memory 51 may also be an external storage device of the terminal device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 5. Further, the memory 51 may also include both an internal storage unit and an external storage device of the terminal device 5. The memory 51 is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0135] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0136] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0138] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0139] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0140] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0141] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A target tracking method, characterized in that: Applied to a preset model, the preset model includes a first feature extraction module and a second feature extraction module connected in sequence, the method includes: Acquire, by the first feature extraction module, first feature data of the target object in the initial frame and second feature data of the target object in the current frame, respectively, wherein the initial frame is the first frame captured image of the target object, and the current frame is the Nth frame captured image of the target object, where N is a positive integer greater than 1; Respectively acquiring third feature data in the first feature data and fourth feature data in the second feature data through the second feature extraction module; Inputting the third feature data and the fourth feature data into the trained classification model to obtain respective confidences of a plurality of key points of the target object in the current frame; Determine the key point corresponding to the maximum value of the confidence scores of the multiple key points as the center position point of the target object in the current frame; Generate multiple candidate detection boxes of the target object corresponding to the center position point; A target detection frame of the target object in the current frame is determined from the multiple candidate detection frames according to the first feature data and the second feature data.

2. The target tracking method according to claim 1, characterized in that: The first feature extraction module is used to obtain first feature data of the target object in the initial frame and second feature data of the target object in the current frame, including: Performing data enhancement processing on the initial frame to obtain enhanced data; Inputting the enhanced data into the first feature extraction module to obtain the first feature data; The current frame is input into the first feature extraction module to obtain the second feature data.

3. The target tracking method according to claim 2, characterized in that: The performing data enhancement processing on the initial frame to obtain enhanced data includes: Performing M kinds of preprocessing on the initial frame respectively to obtain processed images corresponding to the M kinds of preprocessing, wherein M is a positive integer greater than or equal to 1; Extracting first image data of the target object in the initial frame; extracting second image data of the target object in each of the processed images; The enhancement data is generated based on the first image data and the second image data.

4. The target tracking method according to claim 1, characterized in that: The generating a plurality of candidate detection frames of the target object corresponding to the center position point comprises: When N is equal to 2, K aspect ratios are obtained within a preset first aspect ratio range, and candidate detection frames corresponding to the K aspect ratios are generated according to the center position point, where K is a positive integer greater than 1; When N is greater than 2, a second aspect ratio range of the target detection frame of the target object in the target frame is counted; L aspect ratios are obtained according to the second aspect ratio range; and candidate detection frames corresponding to the L aspect ratios are generated according to the center position point, where L is a positive integer greater than 1, and the target frame includes N-1 frames of images captured before the current frame.

5. The target tracking method according to claim 1, characterized in that: The determining, according to the first feature data and the second feature data, a target detection frame of the target object in the current frame from the multiple candidate detection frames comprises: Performing pooling processing according to the first feature data and the second feature data to obtain pooled data; Calculate the intersection-and-union ratios (IoU) corresponding to each of the plurality of candidate detection frames according to the pooled data, wherein the IoU represents an overlap ratio between the candidate detection frame and a reference detection frame, where the reference detection frame is a detection frame of the target object in the initial frame; The candidate detection frame corresponding to the maximum value of the calculated intersection-over-union ratio is determined as the target detection frame of the target object in the current frame.

6. The target tracking method according to claim 1, characterized in that: The method further includes: performing model acceleration on the preset model using a preset acceleration algorithm.

7. A target tracking device, characterized in that: Applied to a preset model, the preset model includes a first feature extraction module and a second feature extraction module connected in sequence, the device includes: A first acquisition module, used to respectively acquire first feature data of a target object in an initial frame and second feature data of the target object in a current frame through the first feature extraction module, wherein the initial frame is a first frame captured image of the target object, and the current frame is an Nth frame captured image of the target object, where N is a positive integer greater than 1; A second acquisition module, used for respectively acquiring third feature data in the first feature data and fourth feature data in the second feature data through the second feature extraction module; A center detection module, used for inputting the third feature data and the fourth feature data into the trained classification model to obtain the confidence of each of the multiple key points of the target object in the current frame; determining the key point corresponding to the maximum value of the confidence of each of the multiple key points as the center position point of the target object in the current frame; A candidate box generation module, used to generate multiple candidate detection boxes of the target object corresponding to the central position point; A target frame determination module is used to determine a target detection frame of the target object in the current frame from the multiple candidate detection frames according to the first feature data and the second feature data.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images