Target tracking method, device and equipment, tracking holder and medium

Abstract

The invention discloses a target tracking method, device and equipment, a tracking holder and a medium, and relates to the field of machine vision, and the method comprises the steps: recognizing a target object in a current monitoring image; the preset target detection model is a model obtained after the initial model learns the relevance between the sheltered target and the unsheltered target in the preset data set; determining a to-be-tracked target object from the target objects, and determining a template area corresponding to the to-be-tracked target object in the current monitoring picture; generating a target response diagram by using the template area and the next frame of monitoring picture; and predicting position information of the to-be-tracked target object in the next frame of monitoring picture based on the target response diagram to obtain a corresponding tracking result. Therefore, the detection effect under the condition of object shielding can be improved by paying attention to the importance of the unshielded area, the template area is generated through the monitoring image collected in real time, the reusability of pedestrian detection and tracking is improved, and the situation that the tracking precision is not high due to the preset detection area is avoided.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a target tracking method, device, equipment, tracking pan / tilt and medium. Background Art

[0002] With the rapid development of artificial intelligence technology, target tracking, as an important branch of computer vision, has shown great application potential in many fields such as security monitoring, autonomous driving, and smart home. After the embedded devices deploy deep learning algorithms, localized intelligent applications can be realized. However, in practical applications, target tracking faces many challenges, such as target occlusion, lighting changes, motion blur, etc. in complex scenes. At the same time, it needs to be effectively deployed on edge devices to ensure the accuracy and efficiency of target tracking while meeting the lightweight requirements. In addition, the existing target tracking technology cannot effectively detect the situation of multiple overlapping targets, and requires a preset detection area. The detection reusability is poor, and the tracking accuracy is not high.

[0003] It can be seen that how to improve the pedestrian tracking effect under multi-target overlapping monitoring images and improve the reusability of detection is a problem that technical personnel in this field need to solve. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a target tracking method, device, equipment, tracking gimbal and medium, which can improve the detection effect under object occlusion by paying attention to the importance of unobstructed areas, and generate template areas by real-time collected monitoring images, thereby improving the reusability of pedestrian detection and tracking, and avoiding the situation where the tracking accuracy is low due to the preset detection area.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a target tracking method, comprising:

[0006] The target object in the current monitoring screen is identified by using a preset target detection model; the preset target detection model is a model obtained after the initial target detection model learns the correlation between the obscured target and the unobscured target in the preset data set;

[0007] Determine a target object to be tracked from the identified target objects, and use a monitoring screen area where the target object to be tracked is located in the current monitoring screen as a template area;

[0008] Generate a target response map using the template area and the next frame of monitoring picture; the target response map is used to represent the similarity between the template area and the search area in the next frame of monitoring picture;

[0009] The position information of the target object to be tracked in the next frame of monitoring picture is predicted based on the target response graph to obtain the target tracking result of the target object to be tracked in the next frame of monitoring picture.

[0010] Optionally, before using a preset target detection model to identify a target object in the current monitoring screen, the method further includes:

[0011] Performing format conversion and label detection on the first pedestrian data set and the second pedestrian data set in the preset data set, respectively, to obtain the corresponding first pre-processed pedestrian data set and the second pre-processed pedestrian data set; wherein the first pedestrian data set is a data set obtained by collecting images of pedestrians under preset low-light conditions;

[0012] Merging the first preprocessed pedestrian dataset and the second preprocessed pedestrian dataset to obtain a target dataset, and splitting the target dataset based on a preset split ratio to obtain a target training set, a target test set, and a target verification set;

[0013] The current target detection model to be trained is trained based on the target training set, so that the current target detection model learns the correlation between the occluded target and the unoccluded target in each training sample in the target training set through a self-supervised equivariant attention mechanism, and the parameters of the current target detection model are adjusted through the target test set to obtain the target detection model to be determined;

[0014] The target detection model to be determined is verified by the target verification set. If the verification parameter corresponding to the target detection model to be determined meets the preset verification condition threshold, the target detection model to be determined is used as the preset target detection model;

[0015] If the verification parameters corresponding to the target detection model to be determined do not meet the preset verification condition threshold, the target detection model to be determined is determined as the new current target detection model, and the process jumps to the step of training the initial target detection model based on the target training set until the verification parameters corresponding to the target detection model to be determined meet the preset verification condition threshold.

[0016] Optionally, before using a preset target detection model to identify a target object in the current monitoring screen, the method further includes:

[0017] Determine the target model format supported locally, and convert the model format of the preset target detection model through the preset model conversion tool to convert the model format of the preset target detection model into the target model format.

[0018] Optionally, a preset target detection model is used to identify a target object in the current monitoring image, including:

[0019] The current monitoring picture is collected by a local image acquisition device, and the unobstructed area in the current monitoring picture is feature enhanced by a self-supervised equivariant attention mechanism in a preset target detection model to obtain an enhanced monitoring picture;

[0020] The enhanced monitoring picture is subjected to convolution processing, and all target objects in the current monitoring picture are determined according to the feature information obtained after the convolution processing, and the object coordinates corresponding to the target objects in the current monitoring picture are determined.

[0021] Optionally, determining a target object to be tracked from the identified target objects, and using a monitoring screen area where the target object to be tracked is located in the current monitoring screen as a template area, includes:

[0022] Selecting a target object to be tracked from the target objects based on the received object selection instruction, and selecting a target object coordinate to be tracked corresponding to the target object to be tracked from the object coordinates based on the target object to be tracked;

[0023] The monitoring screen area corresponding to the coordinates of the target object to be tracked in the current monitoring screen is used as the template area.

[0024] Optionally, a target response graph is generated using the template area and the next frame of monitoring screen, including:

[0025] Input the template area and the next frame of monitoring picture into a preset target tracking network, so as to extract features of the template area and the next frame of monitoring picture through the target tracking network, and obtain a first feature map corresponding to the template area and a second feature map corresponding to the next frame of monitoring picture;

[0026] The first feature map and the second feature map are pixel-multiplied to generate a target response map.

[0027] Optionally, predicting the position information of the target object to be tracked in the next frame of the monitoring screen based on the target response graph to obtain the target tracking result of the target object to be tracked in the next frame of the monitoring screen includes:

[0028] Determine the similarity between the template area and the next frame of monitoring screen based on the target response map, and infer the position information and size information of the target object to be tracked in the next frame of monitoring screen based on the target response map and the similarity through a preset tracker;

[0029] Generate a bounding box corresponding to the target object to be tracked in the next frame of monitoring screen based on the position information and size information through a preset regression network;

[0030] The next frame of monitoring screen is used as the new current monitoring screen, and the process jumps to the step of using a preset target detection model to identify the target object in the current monitoring screen.

[0031] In a second aspect, the present application discloses a target tracking device, comprising:

[0032] A target information determination module is used to identify the target object in the current monitoring screen using a preset target detection model; the preset target detection model is a model obtained after the initial target detection model learns the correlation between the obscured target and the unobscured target in the preset data set;

[0033] A template region determination module is used to determine the target object to be tracked from the identified target objects, and use the monitoring screen area where the target object to be tracked is located in the current monitoring screen as the template area;

[0034] A response map generation module is used to collect and utilize the template area and the next frame of monitoring picture to generate a target response map; the target response map is used to represent the similarity between the template area and the search area in the next frame of monitoring picture;

[0035] The bounding box generation module is used to predict the position information of the target object to be tracked in the next frame of the monitoring picture based on the target response map, so as to obtain the target tracking result of the target object to be tracked in the next frame of the monitoring picture.

[0036] In a third aspect, the present application discloses an electronic device, comprising:

[0037] Memory for storing computer programs;

[0038] A processor is used to execute a computer program to implement the steps of the target tracking method as described above.

[0039] In a fourth aspect, the present application discloses a tracking gimbal, comprising an image acquisition device and a gimbal motor, and also comprising a processor component connected to the image acquisition device and the gimbal motor; wherein the processor component is used to execute a computer program to implement the steps of the aforementioned target tracking method.

[0040] Optionally, the pan / tilt motor also has a built-in position sensor for collecting position data;

[0041] Correspondingly, the processor component is also used to send corresponding control instructions to the gimbal motor based on the position data sent by the position sensor to control the tracking gimbal to perform rotation and / or deflection operations.

[0042] Optional, tracking gimbal, also includes:

[0043] a removable chassis connected to the processor assembly;

[0044] Correspondingly, the processor component is also used to generate a movement control signal when the target object to be tracked is in motion, and send the movement control signal to the movable chassis to control the movable chassis to move so as to track the target object to be tracked in motion.

[0045] In a fifth aspect, the present application discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the target tracking method as described above are implemented.

[0046] It can be seen that the method of the present application can use a preset target detection model to identify the target object in the current monitoring screen; the preset target detection model is a model obtained after the initial target detection model learns the correlation between the occluded target and the unoccluded target in the preset data set; the target object to be tracked is determined from the identified target objects, and the monitoring screen area where the target object to be tracked in the current monitoring screen is located is used as the template area; a target response map is generated using the template area and the next frame of monitoring screen; the target response map is used to characterize the similarity between the template area and the search area in the next frame of monitoring screen; based on the target response map, the position information of the target object to be tracked in the next frame of monitoring screen is predicted to obtain the target tracking result of the target object to be tracked in the next frame of monitoring screen.

[0047] Beneficial effect: In the present application, the target object in the current monitoring screen is identified by a preset target detection model, and the preset target detection model for identification is a model obtained after learning the correlation between the occluded targets and the violators in the preset data set, and then the target object to be tracked and the corresponding template area of ​​the target object to be tracked are determined from the identified objects, and a target response map for reflecting the similarity of the screen is generated according to the next frame of the monitoring screen collected and the template area, and finally the position information of the target object to be tracked in the next frame of the monitoring screen is predicted according to the target response map. In this way, by paying attention to the importance of the unobstructed area, the overall feature representation can be improved, and the detection effect under object occlusion can be improved, thereby improving the accuracy of pedestrian tracking, and the template area for detection is generated in real time through the real-time collected monitoring screen, which effectively improves the reusability of pedestrian detection and tracking, and avoids the situation where the tracking accuracy is low due to the preset detection area. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A flow chart of a target tracking method provided by an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of image processing based on a convolutional network provided by an embodiment of the present invention;

[0051] Figure 3 A schematic diagram of a tracking gimbal provided in an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of the structure of a target tracking device provided by an embodiment of the present invention;

[0053] Figure 5 A structural diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] Currently, target tracking faces many challenges, such as target occlusion, lighting changes, motion blur, etc. in complex scenes. At the same time, it needs to be effectively deployed on edge devices to ensure target tracking accuracy and efficiency while meeting lightweight requirements. Existing target tracking technologies cannot effectively detect multiple overlapping targets, and require preset detection areas. The detection reusability is poor, and the tracking accuracy is not high.

[0056] In order to overcome the above-mentioned technical problems, the present application discloses a target tracking method, device, equipment, tracking gimbal and medium, which can improve the detection effect under object occlusion by focusing on the importance of unobstructed areas, and generate template areas by real-time collected monitoring images, thereby improving the reusability of pedestrian detection and tracking, and avoiding the situation where the tracking accuracy is low due to the preset detection area.

[0057] The terms "including" and "having" in the specification of the present invention and the above-mentioned drawings, as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0058] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0059] See also Figure 1 As shown, the embodiment of the present application discloses a target tracking method, including:

[0060] Step S11, using a preset target detection model to identify the target object in the current monitoring screen; the preset target detection model is a model obtained after the initial target detection model learns the correlation between the obscured target and the unobscured target in the preset data set.

[0061] In this embodiment, it is necessary to track the target by a tracking device. The tracking device used in this embodiment is a tracking gimbal. The current monitoring screen needs to be collected by the tracking gimbal, and then the target object in the current monitoring screen is identified by a preset target detection model. It should be noted that before performing target detection, the initial target detection model needs to be trained. Specifically, it is necessary to first determine a preset data set for model training. The preset data set includes a first pedestrian data set and a second pedestrian data set, where the first pedestrian data set is a NightOwls dataset low-light pedestrian detection data set, and the second data pedestrian data set is a WiderPerson dataset pedestrian detection data set. Then, it is necessary to convert the label format of the data set, convert the label of the NightOwls dataset from Json format to txt format, and then convert the label of the WiderPerson dataset from [class_label, x1, y1, x2, y2] to yolo format.<class_id><x_center><y_center> , and perform label detection after the conversion is completed to check whether the labels are correctly marked to obtain the first preprocessed pedestrian data set and the second preprocessed pedestrian data set, and then the data sets need to be merged to obtain the target data set.

[0062] After obtaining the target data set, the target data set needs to be split according to a preset ratio. In this embodiment, the split ratio is 8:1:1, and then the corresponding target training set, target test set, and target verification set are obtained. Then, the current target detection model to be trained is trained by the obtained target training set. It should be noted that the current target detection model in this embodiment is an improved YOLOv8 (You Only Look Once) model. The YOLOv8 model in this embodiment integrates the SEAM (spatially enhanced attention module, attention network) module, adopts the strategy of fusing the outputs of different depth convolution layers through point-by-point convolution, and uses a two-layer fully connected network to integrate the information of each channel, so as to strengthen the correlation between all channels. When training the model, the correlation between the occluded target and the unoccluded target in each training sample in the target training set can be learned through the self-supervised equivariant attention mechanism of the SEAM module to compensate for the detection performance loss caused by occlusion. Finally, the output of the SEAM module is multiplied by the original feature as the attention weight to improve the detection ability of the model when dealing with target occlusion, thereby enhancing the overall accuracy of the model. And after training with the target training set model, it is also necessary to adjust the parameters of the current target detection model through the target test set to obtain the target detection model to be determined.

[0063] Furthermore, it is necessary to verify the target detection model to be determined through the target verification set to determine whether the verification parameters corresponding to the target detection model to be determined meet the preset verification condition threshold. Specifically, it is necessary to determine the accuracy, recall rate, average precision value and frames per second of the model. The calculation formula of accuracy Precision is as follows:

[0064] ;

[0065] The calculation formula of recall rate Recall is as follows:

[0066] ;

[0067] The calculation formula of the average precision value mAP is as follows:

[0068] ;

[0069] ;

[0070] Among them, TP represents the number of positive samples successfully detected by the model; FP represents the number of negative samples incorrectly detected as positive samples by the model; FN represents the number of positive samples not detected by the model; Q represents the number of categories; APi represents the average accuracy; Precision represents how many of the samples predicted as positive by the model are correct, Recall represents how many of the truly positive samples are correctly predicted as positive, mAP represents the average precision, and AP represents the average of the average precision calculated under different IoU (Intersection over Union) thresholds.

[0071] If the verification parameters corresponding to the target detection model to be determined meet the preset verification condition threshold, the target detection model to be determined is used as the preset target detection model. If the verification parameters corresponding to the target detection model to be determined do not meet the preset verification condition threshold, the target detection model to be determined is determined as the new current target detection model, and jumps to the step of training the initial target detection model based on the target training set until the verification parameters corresponding to the target detection model to be determined meet the preset verification condition threshold. In this way, by introducing the self-supervised equivariant attention mechanism, the relationship between the occluded target and the unoccluded target is learned, the loss in the occlusion scene is compensated, and the model performance can be improved.

[0072] Furthermore, since the tracking gimbal in this application is based on K230, and the YOLO model is usually not directly compatible with K230, it is necessary to determine the target model format supported locally, and convert the model format of the preset target detection model through the preset model conversion tool to convert the model format of the preset target detection model into the target model format. The preset model conversion tool can be selected according to user needs. The preset model conversion tool in this application is nncase (Neural Network Compiler and Accelerator), which is a deep learning model compiler and optimization tool designed for embedded devices. It supports the import, conversion and optimization of multiple deep learning models to generate model files suitable for embedded device operation. In addition, by utilizing the high performance and low power consumption characteristics of the K230 chip, long-term stable operation of tracking can be achieved.

[0073] In this embodiment, it is necessary to capture the current monitoring screen by tracking the local image acquisition device of the gimbal, and then perform feature enhancement on the unobstructed area in the current monitoring screen through the self-supervised equivariant attention mechanism in the preset target detection model to obtain the enhanced monitoring screen, and then perform convolution processing on the enhanced monitoring screen, determine all target objects in the current monitoring screen based on the feature information obtained after the convolution processing, and determine the object coordinates corresponding to the target object in the current monitoring screen.

[0074] Step S12: determining a target object to be tracked from the identified target objects, and taking a monitoring screen area where the target object to be tracked is located in the current monitoring screen as a template area.

[0075] In this embodiment, it is necessary to determine the target object to be tracked from the identified target objects, and then determine the template area according to the target object to be tracked. Specifically, it is necessary to select the target object to be tracked from the target objects based on the received object selection instruction, and select the target object coordinates corresponding to the target object to be tracked from the object coordinates based on the target object to be tracked. Further, it is necessary to use the NanoTrack model as a tracker for target tracking, and realize the tracking of the object to be tracked in combination with the target object coordinates. And it is necessary to use the monitoring screen area corresponding to the target object coordinates to be tracked in the current monitoring screen as the template area. It should be noted that the NanoTrack model also needs to be converted, and the NanoTrack model is converted into a format suitable for the K230 chip in the tracking pan-tilt through a preset model conversion tool. In this way, the lightweight and efficient NanoTrack target tracking method can be combined to make it easier to deploy on resource-constrained devices, and the multi-target detection capability of K230 and the efficient tracking capability of NanoTrack can be combined to achieve multi-target tracking in complex scenes.

[0076] Step S13, generating a target response map using the template region and the next frame of monitoring picture; the target response map is used to represent the similarity between the template region and the search region in the next frame of monitoring picture.

[0077] In this embodiment, the template area and the collected data are used to generate a target response map for representing the similarity between the template area and the search area in the next frame of the monitoring screen. Specifically, the template area and the collected next frame of the monitoring screen need to be input into the preset target tracking network, that is, NanoTrack. Then NanoTrack will use the template area as the model input and the next frame of the monitoring screen as the image input, and Figure 2The two identical convolutional neural networks shown in the figure process two images respectively to extract deep features by convolution operation on the two images, so as to obtain a first feature map corresponding to the template area and a second feature map corresponding to the next frame of monitoring screen, wherein the input dimension of the template area is [1,3,127,127], and the input dimension of the next frame of monitoring screen is [1,3,255,255]. After the convolution operation, the first feature map corresponding to the template area with a feature dimension of [1,48,8,8] and the second feature map corresponding to the next frame of monitoring screen with a feature dimension of [1,48,16,16] are obtained. The input channel is fixed to 3. After the convolution is completed, the number of output channels is set to 48, and 48 feature maps will be generated. The stride of the pooling operation is 16, that is, the number of pixels of the pooling window moved each time is 16. Then, the first feature map and the second feature map are pixel-multiplied to generate a target response map. The position of each pixel in the generated response map reflects the similarity between the template and the search area features at that position, indicating the possible position of the target in the search area. In this way, the similarity between two images can be reflected based on the generated response map, thereby improving the accuracy of target tracking.

[0078] Step S14: predicting the position information of the target object to be tracked in the next frame of monitoring picture based on the target response graph to obtain the target tracking result of the target object to be tracked in the next frame of monitoring picture.

[0079] In this embodiment, it is necessary to predict the position information of the target object to be tracked in the next frame of the monitoring screen based on the target response map. Specifically, it is necessary to determine the similarity between the template area and the next frame of the monitoring screen based on the obtained target response map, so as to perform regression analysis based on the target response map and the similarity through a preset tracker (such as Pointwise-Correlation) to infer the position information and size information of the target object to be tracked in the next frame of the monitoring screen, and then pass the obtained position information and size information to the preset regression network. After receiving the information, the preset regression network will generate a bounding box corresponding to the target object to be tracked in the next frame of the monitoring screen based on the position information and size information.

[0080] Furthermore, after the bounding box of the target object to be tracked is generated in the next frame of the monitoring screen, the next frame of the monitoring screen needs to be used as the new current monitoring screen, and then the next frame of the monitoring screen is continuously collected, and the step of using the preset target detection model to identify the target object in the current monitoring screen is jumped to continue the target tracking until the instruction to stop tracking is received. In this way, the detection template area can be generated in real time through the real-time collected monitoring screen, which effectively improves the reusability of pedestrian detection and tracking, and avoids the situation where the tracking accuracy is low due to the preset detection area.

[0081] It should be further explained that when tracking a target, you can collect pictures or videos, and extract video frames from the video as the current monitoring screen. However, the method of collecting videos may cause frame loss, which in turn affects the similarity between the two frames. Therefore, the video can be interpolated. It should be explained that when performing frame interpolation, it is necessary to determine the motion vector intersection pixels of the two frames before and after that need to be interpolated, and then determine the extracted pixels based on the determined motion vector intersection pixels of the two frames, and generate a supplementary screen by extracting the pixels, and insert the supplementary screen between the two frames to ensure the stability of target tracking.

[0082] It can be seen that in this embodiment, the target object in the current monitoring screen is identified by a preset target detection model, and the preset target detection model for identification is a model obtained after learning the correlation between the internal occluded target and the violator in the preset data set, and then the target object to be tracked and the corresponding template area of ​​the target object to be tracked are determined from the identified object, and a target response map for reflecting the similarity of the screen is generated according to the next frame of the monitoring screen collected and the template area, and finally the position information of the target object to be tracked in the next frame of the monitoring screen is predicted according to the target response map. In this way, on the one hand, the importance of the unoccluded area can be paid attention to by adding the SEAM attention mechanism to the YOLOv8 model, the overall feature representation can be improved, the detection effect under object occlusion can be improved, and the accuracy of pedestrian tracking can be improved; on the other hand, the night low-light pedestrian detection data set and the pedestrian data set can be used to mix and train the model, enhance the model's detection accuracy of pedestrian targets in lighting changes, and improve the robustness of the model; on the other hand, the template area for detection can be generated in real time through the real-time collected monitoring screen, which effectively improves the reusability of pedestrian detection and tracking, and avoids the situation where the tracking accuracy is not high due to the preset detection area.

[0083] See also Figure 3 As shown, an embodiment of the present application discloses a tracking gimbal, including: an image acquisition device and a gimbal motor, and also includes a processor component connected to the image acquisition device and the gimbal motor; wherein the processor component includes an STM32 microcontroller and a K230 chip.

[0084] The tracking gimbal provided in this embodiment corresponds to the above method and has the same beneficial effects as the above method. Therefore, for the embodiments of the tracking gimbal part, please refer to the description of the embodiments of the method part.

[0085] As an optional embodiment, STM32 as the main control module exchanges data with K230 through RS485 bus, receives position sensor data and K230 module visual information, and controls the deflection of the two motors by tracking the position of the target in the field of view, so that the tracked target is always in the center of the field of view. Finally, the TX (Transmit) interface and RX (Receive) interface are exposed as external modules to realize serial communication with other devices or systems.

[0086] As an optional embodiment, the pan-tilt motor is also built with a position sensor for collecting position data; and the processor component is also used to send corresponding control instructions to the pan-tilt motor based on the position data sent by the position sensor to control the tracking pan-tilt to perform rotation and / or deflection operations. Specifically, the pan-tilt control system is deployed in the STM32 microcontroller in the processor component, which is responsible for receiving sensor data, executing control algorithms, and outputting control signals. Through position closed-loop control, PWM waves are output to control the deflection angles of the two motors. The motor uses a GM6020 pan-tilt motor with a built-in position sensor to achieve position closed-loop control, and the motor can be selected according to actual conditions. In this application, the motor uses a GM6020 pan-tilt motor.

[0087] As an optional embodiment, the tracking gimbal also includes a removable chassis connected to the processor component, and the removable chassis is connected to the STM32 microcontroller in the processor component through the TX interface and RX interface exposed by the STM32 microcontroller, and the processor component is also used to generate a mobile control signal when the target object to be tracked is in motion, and send the mobile control signal to the removable chassis to control the movement of the removable chassis so as to track the target object to be tracked in motion. Further, the removable chassis is a detachable chassis connected to the gimbal through an interface, which can be replaced according to demand, and can be replaced with other chassis such as a fixed tripod.

[0088] As an optional embodiment, the pan-tilt motor also has a built-in position PID (Proportional Integral Derivative) controller for controlling the pan-tilt to rotate and / or deflect. After receiving the set position signal, the pan-tilt motor converts the position signal into a PWM (Pulse width modulation) signal for motor rotation through the position PID, and controls the motor rotation through the PWM signal, and then reads the accumulated pulse value through the encoder to determine the actual rotation / deflection position of the motor for adjustment. The position PID controller can make the motor run more stable.

[0089] See also Figure 4 As shown, the embodiment of the present application discloses a target tracking device, comprising:

[0090] The target information determination module 11 is used to identify the target object in the current monitoring screen by using a preset target detection model; the preset target detection model is a model obtained after the initial target detection model learns the correlation between the obscured target and the unobscured target in the preset data set;

[0091] The template region determination module 12 is used to determine the target object to be tracked from the identified target objects, and use the monitoring screen area where the target object to be tracked is located in the current monitoring screen as the template area;

[0092] The response graph generation module 13 is used to collect and utilize the template area and the next frame of monitoring picture to generate a target response graph; the target response graph is used to represent the similarity between the template area and the search area in the next frame of monitoring picture;

[0093] The bounding box generation module 14 is used to predict the position information of the target object to be tracked in the next frame of the monitoring picture based on the target response map, so as to obtain the target tracking result of the target object to be tracked in the next frame of the monitoring picture.

[0094] It can be seen that in this embodiment, the target object in the current monitoring screen is identified by a preset target detection model, and the preset target detection model for identification is a model obtained by learning the correlation between the occluded target and the violator target in the preset data set, and then the target object to be tracked and the corresponding template area of ​​the target object to be tracked are determined from the identified objects, and a target response map for reflecting the similarity of the screen is generated according to the next frame of the monitoring screen collected and the template area, and finally the position information of the target object to be tracked in the next frame of the monitoring screen is predicted according to the target response map. In this way, by paying attention to the importance of the unobstructed area, the overall feature representation can be improved, the detection effect under object occlusion can be improved, and the accuracy of pedestrian tracking can be improved. The template area for detection is generated in real time through the real-time collected monitoring screen, which effectively improves the reusability of pedestrian detection and tracking, and avoids the situation where the tracking accuracy is low due to the preset detection area.

[0095] In some embodiments, the target tracking device may further include:

[0096] A data set preprocessing unit, used to perform format conversion and label detection on a first pedestrian data set and a second pedestrian data set in a preset data set, respectively, to obtain a corresponding first preprocessed pedestrian data set and a second preprocessed pedestrian data set; wherein the first pedestrian data set is a data set obtained by collecting images of pedestrians under preset low-light conditions;

[0097] A data set splitting unit, used for merging the first preprocessed pedestrian data set and the second preprocessed pedestrian data set to obtain a target data set, and splitting the target data set based on a preset splitting ratio to obtain a target training set, a target test set, and a target verification set;

[0098] A model training unit is used to train a current target detection model to be trained based on a target training set, so that the current target detection model learns the correlation between the occluded target and the unoccluded target in each training sample in the target training set through a self-supervised equivariant attention mechanism, and adjusts the parameters of the current target detection model through a target test set to obtain a target detection model to be determined;

[0099] A model verification unit is used to verify the target detection model to be determined through a target verification set. If the verification parameter corresponding to the target detection model to be determined meets a preset verification condition threshold, the target detection model to be determined is used as a preset target detection model;

[0100] A step jump unit is used to determine the target detection model to be determined as the new current target detection model if the verification parameters corresponding to the target detection model to be determined do not meet the preset verification condition threshold, and jump to the step of training the initial target detection model based on the target training set until the verification parameters corresponding to the target detection model to be determined meet the preset verification condition threshold.

[0101] In some embodiments, the target tracking device may further include:

[0102] The format conversion unit is used to determine the target model format supported locally, and perform model format conversion on the preset target detection model through a preset model conversion tool to convert the model format of the preset target detection model into the target model format.

[0103] In some embodiments, the target information determination module 11 may specifically include:

[0104] An image enhancement unit is used to acquire the current monitoring picture through a local image acquisition device, and to enhance the features of the unobstructed area in the current monitoring picture through a self-supervised equivariant attention mechanism in a preset target detection model, so as to obtain an enhanced monitoring picture;

[0105] The target information determination unit is used to perform convolution processing on the enhanced monitoring picture, determine all target objects in the current monitoring picture according to the feature information obtained after the convolution processing, and determine the object coordinates corresponding to the target objects in the current monitoring picture.

[0106] In some embodiments, the template region determination module 12 may specifically include:

[0107] a coordinate determining unit, configured to select a target object to be tracked from the target objects based on the received object selection instruction, and select a target object coordinate to be tracked corresponding to the target object to be tracked from the object coordinates based on the target object to be tracked;

[0108] The template area determination unit is used to use the monitoring screen area corresponding to the coordinates of the target object to be tracked in the current monitoring screen as the template area.

[0109] In some embodiments, the response graph generating module 13 may specifically include:

[0110] A feature extraction unit is used to input the template area and the next frame of monitoring picture into a preset target tracking network, so as to extract features of the template area and the next frame of monitoring picture through the target tracking network, and obtain a first feature map corresponding to the template area and a second feature map corresponding to the next frame of monitoring picture;

[0111] The response map generating unit is used to perform pixel multiplication on the first feature map and the second feature map to generate a target response map.

[0112] In some embodiments, the bounding box generation module 14 may specifically include:

[0113] An information inference unit is used to determine the similarity between the template area and the next frame of monitoring picture based on the target response map, and infer the position information and size information of the target object to be tracked in the next frame of monitoring picture based on the target response map and the similarity through a preset tracker;

[0114] A bounding box generation unit, used to generate a bounding box corresponding to the target object to be tracked in the next frame of monitoring picture based on the position information and size information through a preset regression network;

[0115] The step jump unit is used to take the next frame of monitoring picture as the new current monitoring picture and jump to the step of using a preset target detection model to identify the target object in the current monitoring picture.

[0116] Furthermore, the present application also discloses an electronic device. Figure 5 It is a structural diagram of an electronic device according to an exemplary embodiment, and the content in the figure cannot be regarded as any limitation on the scope of use of this application. The electronic device may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input and output interface 25 and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the target tracking method disclosed in any of the aforementioned embodiments. In addition, the electronic device in this embodiment may specifically be an electronic computer.

[0117] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0118] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0119] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the target tracking method performed by the electronic device disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0120] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the target tracking method disclosed above. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the above embodiments, and no further description will be given here.

[0121] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0122] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0123] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0124] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0125] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A target tracking method, characterized in that: include: Use the preset target detection model to identify the target object in the current monitoring screen; The preset target detection model is a model obtained after the initial target detection model learns the correlation between the occluded targets and the unoccluded targets in the preset data set; Determine a target object to be tracked from the identified target objects, and use a monitoring screen area where the target object to be tracked is located in the current monitoring screen as a template area; Generate a target response map using the template area and the next frame of monitoring picture; the target response map is used to characterize the similarity between the template area and the search area in the next frame of monitoring picture; The position information of the target object to be tracked in the next frame of monitoring picture is predicted based on the target response map to obtain the target tracking result of the target object to be tracked in the next frame of monitoring picture.

2. The target tracking method according to claim 1, characterized in that: Before using the preset target detection model to identify the target object in the current monitoring screen, the method also includes: Performing format conversion and label detection on the first pedestrian data set and the second pedestrian data set in the preset data set, respectively, to obtain a corresponding first pre-processed pedestrian data set and a second pre-processed pedestrian data set; wherein the first pedestrian data set is a data set obtained by collecting images of pedestrians under preset low-light conditions; Merging the first preprocessed pedestrian dataset and the second preprocessed pedestrian dataset to obtain a target dataset, and splitting the target dataset based on a preset splitting ratio to obtain a target training set, a target test set, and a target verification set; Training a current target detection model to be trained based on the target training set, so that the current target detection model learns the correlation between the occluded target and the unoccluded target in each training sample in the target training set through a self-supervised equivariant attention mechanism, and adjusting parameters of the current target detection model through the target test set to obtain a target detection model to be determined; The target detection model to be determined is verified by using the target verification set, and if the verification parameter corresponding to the target detection model to be determined meets the preset verification condition threshold, the target detection model to be determined is used as the preset target detection model; If the verification parameters corresponding to the target detection model to be determined do not meet the preset verification condition threshold, the target detection model to be determined is determined as the new current target detection model, and the process jumps to the step of training the initial target detection model based on the target training set until the verification parameters corresponding to the target detection model to be determined meet the preset verification condition threshold.

3. The target tracking method according to claim 2, characterized in that: Before using the preset target detection model to identify the target object in the current monitoring screen, the method also includes: Determine a target model format supported locally, and perform model format conversion on the preset target detection model through a preset model conversion tool to convert the model format of the preset target detection model into the target model format.

4. The target tracking method according to claim 2, characterized in that: The method of using a preset target detection model to identify a target object in the current monitoring screen includes: The current monitoring picture is collected by a local image acquisition device, and features of unobstructed areas in the current monitoring picture are enhanced by the self-supervised equivariant attention mechanism in the preset target detection model to obtain an enhanced monitoring picture; The enhanced monitoring picture is subjected to convolution processing, and all target objects in the current monitoring picture are determined according to feature information obtained after the convolution processing, and object coordinates corresponding to the target objects in the current monitoring picture are determined.

5. The target tracking method according to claim 4, characterized in that: The step of determining the target object to be tracked from the identified target objects, and using the monitoring screen area where the target object to be tracked is located in the current monitoring screen as a template area, includes: Selecting a target object to be tracked from the target objects based on the received object selection instruction, and selecting a target object coordinate to be tracked corresponding to the target object to be tracked from the object coordinates based on the target object to be tracked; The monitoring screen area corresponding to the coordinates of the target object to be tracked in the current monitoring screen is used as a template area.

6. The target tracking method according to claim 1, characterized in that: The method of generating a target response graph by using the template area and the next frame monitoring picture includes: Inputting the template area and the next frame of monitoring picture into a preset target tracking network, so as to extract features of the template area and the next frame of monitoring picture through the target tracking network, and obtaining a first feature map corresponding to the template area and a second feature map corresponding to the next frame of monitoring picture; The first feature map and the second feature map are pixel-multiplied to generate a target response map.

7. The target tracking method according to any one of claims 1 to 6, characterized in that: The predicting the position information of the target object to be tracked in the next frame of monitoring picture based on the target response graph to obtain the target tracking result of the target object to be tracked in the next frame of monitoring picture includes: Determine the similarity between the template area and the next frame of monitoring picture based on the target response map, and infer the position information and size information of the target object to be tracked in the next frame of monitoring picture based on the target response map and the similarity through a preset tracker; Generate a bounding box corresponding to the target object to be tracked in the next frame of monitoring picture based on the position information and the size information through a preset regression network; The next frame of monitoring screen is used as the new current monitoring screen, and the process jumps to the step of using the preset target detection model to identify the target object in the current monitoring screen.

8. A target tracking device, characterized in that: include: A target information determination module is used to identify the target object in the current monitoring screen using a preset target detection model; The preset target detection model is a model obtained after the initial target detection model learns the correlation between the occluded targets and the unoccluded targets in the preset data set; A template region determination module, used to determine a target object to be tracked from the identified target objects, and use a monitoring screen region in the current monitoring screen where the target object to be tracked is located as a template region; A response graph generation module, used to collect and generate a target response graph using the template area and the next frame of monitoring picture; the target response graph is used to represent the similarity between the template area and the search area in the next frame of monitoring picture; The bounding box generation module is used to predict the position information of the target object to be tracked in the next frame monitoring picture based on the target response map to obtain the target tracking result of the target object to be tracked in the next frame monitoring picture.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the target tracking method according to any one of claims 1 to 7.

10. A tracking gimbal, characterized in that: It includes an image acquisition device and a pan-tilt motor, and also includes a processor component connected to the image acquisition device and the pan-tilt motor; wherein the processor component is used to execute a computer program to implement the steps of the target tracking method as claimed in any one of claims 1 to 7.

11. The tracking pan-tilt head according to claim 10, characterized in that: The pan / tilt motor is also equipped with a position sensor for collecting position data; Correspondingly, the processor component is also used to send corresponding control instructions to the gimbal motor based on the position data sent by the position sensor, so as to control the tracking gimbal to perform rotation and / or deflection operations.

12. The tracking pan-tilt head according to claim 10, characterized in that: Also includes: a removable chassis connected to the processor assembly; Correspondingly, the processor component is also used to generate a movement control signal when the target object to be tracked is in motion, and send the movement control signal to the movable chassis to control the movable chassis to move so as to track the target object to be tracked in motion.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the target tracking method according to any one of claims 1 to 7 are implemented.

Images