Target tracking method and device, computer device and storage medium

Abstract

The application relates to a target tracking method and device, computer equipment and a storage medium. The method comprises the following steps: when a target object is detected to be lost, a preset number of historical images are acquired, the historical images comprising the target object; the moving speed of the target object is determined based on the historical images; the coordinate position of the target object is predicted based on the moving speed and moving time of the target object; the moving time comprises a loss time and a delay time of an image acquisition device, and the delay time comprises a start-up delay of the image acquisition device; and the pose of the image acquisition device is adjusted based on the coordinate position. The method can realize target object position prediction based on historical images, and the introduction of the delay time in the prediction calculation can improve the accuracy of target object position prediction, improve the robustness of the prediction algorithm, and improve the tracking accuracy after the target is lost.

Description

Technical Field

[0001] This application relates to the field of robot following technology, and in particular to a target tracking method, apparatus, computer device, and storage medium. Background Technology

[0002] Service robots are often used in home companionship scenarios, proactively identifying the elderly or children and continuously following them to address their physiological and emotional needs in a timely manner.

[0003] In practical applications of robots, target loss often occurs. This can happen when the target's average or instantaneous speed exceeds the robot's speed, or when the robot cannot reach a speed matching the target's within a short time, causing the target to move out of the robot's field of view. Traditional techniques for handling target loss include stopping robot pose adjustment and resuming tracking when the target re-enters the field of view; tracking a certain distance based on the target's direction of movement and stopping pose adjustment if the target still hasn't re-entered the field of view; or restoring the pose to its state before the target was lost. However, these measures are still relatively ineffective in restoring target tracking.

[0004] This shows that current service robots still have a low success rate in tracking targets that have been lost. Summary of the Invention

[0005] Therefore, it is necessary to provide a target tracking method, apparatus, computer equipment, and storage medium that can improve the tracking accuracy after the target is lost, in order to address the above-mentioned technical problems.

[0006] Firstly, this embodiment provides a target tracking method applied to an image acquisition device, the method comprising:

[0007] When the target object is detected to be missing, a preset number of historical images are acquired, and the historical images include the target object;

[0008] The movement speed of the target object is determined based on the historical images;

[0009] The coordinate position of the target object is predicted based on the target object's moving speed and moving time; the moving time includes the loss time and the delay time of the image acquisition device, and the delay time includes the startup delay of the image acquisition device;

[0010] The pose of the image acquisition device is adjusted based on the coordinate position.

[0011] In one embodiment, determining the movement speed of the target object based on the historical image includes:

[0012] Obtain the historical target human body detection bounding box of the target object in the historical image;

[0013] The moving speed of the target object is calculated based on the position of the historical target human body detection box in the historical image and the historical horizontal moving speed of the image acquisition device.

[0014] In one embodiment, calculating the movement speed of the target object based on the position of the historical target human detection box in the historical image and the historical horizontal movement speed of the image acquisition device includes:

[0015] Obtain the first and second historical images of adjacent frames, as well as the time difference between adjacent frames;

[0016] Based on the position of the historical target human body detection box in the first historical image and the second historical image, the horizontal movement difference of the historical target human body detection box relative to the image acquisition device is determined;

[0017] The movement speed of the target object is determined based on the horizontal movement difference, the time difference, and the historical horizontal movement speed of the image acquisition device at the time of the first historical image.

[0018] In one embodiment, calculating the movement speed of the target object based on the position of the historical target human detection box in the historical image and the historical horizontal movement speed of the image acquisition device further includes:

[0019] The average moving speed of the target object at any given moment is calculated based on multiple historical images.

[0020] The average value is determined as the moving speed of the target object.

[0021] In one embodiment, the process of predicting the coordinate position of the target object based on its moving speed and moving time includes:

[0022] The servo response time and algorithm running time of the image acquisition device are obtained; the servo response time is the time it takes for the servo to accelerate to a preset horizontal movement speed, and the algorithm running time is the calculation time from acquiring the historical image to adjusting the pose of the image acquisition device;

[0023] The startup delay is determined based on the servo response time and the algorithm execution time.

[0024] In one embodiment, adjusting the pose of the image acquisition device based on the coordinate position includes:

[0025] Get the current real-time image;

[0026] A real-time target human detection box is determined based on the real-time image;

[0027] The pose of the image acquisition device is adjusted based on the difference between the coordinate position of the real-time target human detection box in the real-time image and the center point of the real-time image.

[0028] In one embodiment, determining the real-time target human detection box based on the real-time image includes:

[0029] Based on the real-time image, determine whether the target object's face exists;

[0030] If the target face exists in the real-time image, a target face detection box is determined based on the target face; a real-time target human body detection box is determined based on the target face detection box.

[0031] If the target face is not present in the real-time image, a human body detection box is determined based on the real-time image, and a real-time target human body detection box is determined based on the matching between the human body detection box and the reference human body detection box.

[0032] In one embodiment, determining whether a target face of the target object exists based on the real-time image includes:

[0033] A face detection bounding box is determined based on the real-time image;

[0034] The first feature vector is determined based on the face detection box;

[0035] The existence of the target face is determined based on the first feature vector and the reference face feature vector in the database.

[0036] Secondly, this embodiment provides a target tracking device applied to an image acquisition device, the device comprising:

[0037] The acquisition module is used to acquire a preset number of historical images when the target object is detected to be lost, and the historical images include the target object;

[0038] A calculation module is used to determine the moving speed of the target object based on the historical image;

[0039] The prediction module is used to predict the coordinate position of the target object based on the target object's moving speed and moving time; the moving time includes the loss time and the delay time of the image acquisition device, the delay time including the startup delay of the image acquisition device;

[0040] An adjustment module is used to adjust the pose of the image acquisition device based on the coordinate position.

[0041] Thirdly, this embodiment provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0042] Fourthly, this embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0043] The aforementioned target tracking method, apparatus, computer device, and storage medium include the following steps: when a target object is detected to be lost, acquiring a preset number of historical images, the historical images including the target object; determining the moving speed of the target object based on the historical images; predicting the coordinate position of the target object based on the moving speed and moving time; the moving time including the loss time and the delay time of the image acquisition device, the delay time including the start-up delay of the image acquisition device; and adjusting the pose of the image acquisition device based on the coordinate position. Specifically, determining the moving speed of the target object through historical images and predicting the coordinate position of the target object based on the moving speed enables target object position prediction based on historical images. Introducing the delay time parameter of the image acquisition device during the prediction process can improve the accuracy of target object position prediction, enhance the robustness of the prediction algorithm, and ultimately improve the tracking accuracy after target loss. Attached Figure Description

[0044] Figure 1 This is a diagram illustrating the application environment of a target tracking method in one embodiment.

[0045] Figure 2 This is a flowchart illustrating a target tracking method in one embodiment;

[0046] Figure 3 This is a schematic diagram of a target face detection bounding box in one embodiment;

[0047] Figure 4 This is a schematic diagram of a human detection box in one embodiment;

[0048] Figure 5 This is a schematic diagram of a real-time target human body detection box in one embodiment;

[0049] Figure 6 This is a schematic diagram of a face detection bounding box in one embodiment;

[0050] Figure 7This is a flowchart illustrating the target tracking method in another embodiment;

[0051] Figure 8 This is a structural diagram of a target tracking device in one embodiment;

[0052] Figure 9 This is a structural diagram of an image acquisition device in one embodiment;

[0053] Figure 10 This is a structural diagram of an image acquisition device in another embodiment;

[0054] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0056] The target tracking method provided in this application can be applied to a stand-alone terminal, which includes an image acquisition device, a memory, and a processor. The image acquisition device acquires images within its field of view, and the processor stores these images in the memory. When a target object is detected to be lost, the processor acquires a preset number of historical images stored in the memory, determines the target object's movement speed based on the historical images, predicts the target object's coordinate position based on the target object's movement speed, and adjusts the pose of the image acquisition device based on the coordinate position.

[0057] For practical needs, the target tracking method provided in this application embodiment can also be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another network server. When a target object is detected as missing, terminal 102 can acquire a preset number of historical images from server 104, determine the target object's movement speed based on the historical images, predict the target object's coordinate position based on the movement speed, and adjust the pose of the image acquisition device based on the coordinate position. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0058] In one embodiment, such as Figure 2 The diagram shown is a flowchart of the target tracking method provided in this embodiment, applied to... Figure 1Taking terminal 102 as an example, which can be an image acquisition device, the method includes the following steps:

[0059] Step S100: When the target object is detected to be lost, acquire a preset number of historical images, the historical images including the target object.

[0060] The target object is the object to be tracked, which can be a person, an animal, or a moving object. The determination of the target object can be based on pre-set parameters, or it can be an object that is close to the image acquisition device, or an object with a large display area in the acquired image, or other methods of determining the target object. This paper does not limit these methods.

[0061] The determination of whether a target object is lost can be based on visual recognition, detection of the target object's status record by a host computer or other devices, determination of target object loss based on received tracking instructions, or other methods to detect target object loss. This article does not limit these methods.

[0062] The preset frame rate of historical images is captured by an image acquisition device. After acquisition, the images can be stored in the device's memory or on a server. Similarly, historical images can be retrieved from the memory of the image acquisition device or the server. The preset frame rate can be set according to actual needs and can be a positive integer not less than 1.

[0063] Step S200: Determine the moving speed of the target object based on the historical image.

[0064] The moving speed can be the moving speed of the target object in the reference frame of the image acquisition device, or the moving speed in the reference frame of the plane in which the image acquisition device is located.

[0065] Determining the movement speed of the target object based on the historical images can be done in several ways. First, it can be based on a single frame of historical imagery. For example, the movement speed can be determined by analyzing the afterimages created by the target object's movement in the historical images and calculating the distance the target object travels within a single frame of exposure time on the image acquisition device. Second, it can be based on multiple frames of historical imagery. For example, the movement speed can be determined by the difference in the target object's movement distance and the difference in shooting time between different historical images. Other methods based on historical images can also be used, which are not limited in this document. It is understood that when multiple frames of historical imagery are acquired, the historical images can be adjacent frames or non-adjacent frames. The calculation method can also be set to acquire some adjacent frames and some non-adjacent frames according to actual needs.

[0066] Step S300: Predict the coordinate position of the target object based on the target object's moving speed and moving time; the moving time includes the loss time and the delay time of the image acquisition device, and the delay time includes the startup delay of the image acquisition device.

[0067] It is understandable that image acquisition devices have a certain lag when the target object is lost. Due to limitations in hardware and other aspects, they cannot instantly complete the processes of detecting whether the target object is lost, calculating the movement speed, and controlling the pose adjustment of the image acquisition device. Therefore, in order to improve the possibility of the image acquisition device achieving target tracking again when predicting the coordinate position of the target object, it is necessary to incorporate the delay time of the image acquisition device into the coordinate position prediction algorithm.

[0068] The startup delay can be preset based on prior knowledge or determined based on the current moving speed of the image acquisition device. For example, the time required to reach the preset moving speed can be determined based on the current moving speed of the image acquisition device. This article does not limit this.

[0069] Step S400: Adjust the pose of the image acquisition device based on the coordinate position.

[0070] The pose adjustment of the image acquisition device can be achieved by controlling the motion execution unit to adjust the yaw or pitch angle of the image acquisition device, or by controlling the motion execution unit to adjust the position of the image acquisition device in the plane reference frame.

[0071] This embodiment provides a target tracking method that determines the moving speed of a target object through historical images and predicts the target object's coordinate position based on the moving speed. This enables target object position prediction based on historical images. Introducing the delay time parameter of the image acquisition device during the prediction process can improve the accuracy of target object position prediction, enhance the robustness of the prediction algorithm, and ultimately improve the tracking accuracy after the target is lost.

[0072] In one embodiment, determining the movement speed of the target object based on the historical image includes:

[0073] Obtain the historical target human body detection bounding box of the target object in the historical image;

[0074] The moving speed of the target object is calculated based on the position of the historical target human body detection box in the historical image and the historical horizontal moving speed of the image acquisition device.

[0075] In this embodiment, the moving speed refers to the moving speed of the target object within the reference frame of the plane where the image acquisition device is located. Horizontal movement can be the yaw rotation of the image acquisition device itself; the historical horizontal moving speed of the image acquisition device can be the yaw rotation speed of the image acquisition device at a historical moment. It is understandable that when the image acquisition device is also in a horizontal moving state, calculating the moving speed of the target object using historical images also requires combining the historical motion state of the image acquisition device.

[0076] The historical target human detection box can be marked in the historical image as a pixel box, or it can be stored in the corresponding record information of the historical image as coordinates. The content stored in the record information can include the acquisition data of a single frame or multiple frames of historical images, and can include the image information of the historical images, such as the acquisition time, exposure time, acquisition device information, aperture size, resolution, etc. It can also include the status information of the image acquisition device when the historical image was acquired, such as the yaw angle movement speed, pitch angle movement speed, displacement speed, etc. of the image acquisition device. This paper does not limit this.

[0077] The historical horizontal movement speed of the image acquisition device can be recorded in the record information corresponding to the historical image, or it can be obtained from the motion log file of the image acquisition device. Based on the acquisition time of the historical image, the corresponding motion parameters in the motion log file are obtained to determine the historical horizontal movement speed of the image acquisition device. Alternatively, the historical horizontal movement speed of the image acquisition device can be obtained through other methods, which are not limited in this article.

[0078] The movement speed of the target object is calculated based on the position of the historical target human body detection box in the historical image and the historical horizontal movement speed of the image acquisition device. This can be achieved by determining the first movement speed of the target object in the reference frame of the image acquisition device based on the movement distance of the historical target human body detection box in the historical image; and by determining the movement speed of the target object in the plane reference frame of the image acquisition device based on the first movement speed and the historical horizontal movement speed of the image acquisition device.

[0079] Furthermore, horizontal movement can also include the movement of the image acquisition device within its plane, and the horizontal movement speed also includes the movement speed of the image acquisition device within its plane. It is understood that at some historical moments, the image acquisition device may be tracking the target object in real time, moving forward or away from the target object. Calculating the target object's movement speed based on the position of the historical target human detection box in the historical image and the historical horizontal movement speed of the image acquisition device can also include calculating the target object's movement speed based on the image acquisition device's positional movement speed at a corresponding moment in the historical image and the position of the historical target human detection box in the historical image.

[0080] This embodiment provides a target tracking method that determines the moving speed of a target object by using the position of a historical target human body detection box in a historical image and the horizontal moving speed of the image acquisition device. This method can calculate the moving speed of the target object in the reference frame of its plane, thereby improving the accuracy of target tracking.

[0081] In one embodiment, calculating the movement speed of the target object based on the position of the historical target human detection box in the historical image and the historical horizontal movement speed of the image acquisition device includes:

[0082] Obtain the first and second historical images of adjacent frames, as well as the time difference between adjacent frames;

[0083] Based on the position of the historical target human body detection box in the first historical image and the second historical image, the horizontal movement difference of the historical target human body detection box relative to the image acquisition device is determined;

[0084] The movement speed of the target object is determined based on the horizontal movement difference, the time difference, and the historical horizontal movement speed of the image acquisition device at the time of the first historical image.

[0085] The moving speed can be angular velocity, which is the angular velocity of the target object moving in the reference frame of the image acquisition device.

[0086] Based on the position of the historical target human body detection box in the first historical image and the second historical image, the horizontal movement difference of the historical target human body detection box relative to the image acquisition device is determined, which may be to determine the horizontal movement angle of the historical target human body detection box relative to the image acquisition device.

[0087] Given the field of view of the image acquisition device, the angle of the target object in the historical images can be determined based on its position and the device's field of view. The target object's movement angle can be obtained by the angle difference between adjacent historical images. Furthermore, based on the target object's movement angle and the time difference between the historical images, the angular velocity of the target object relative to the image acquisition device can be determined. Based on the angular velocity and the historical horizontal movement speed of the image acquisition device at the first historical image moment, the relative movement speed of the target object and the image acquisition device in the reference frame of its plane can be determined.

[0088] In one specific embodiment, the i-th frame and the (i-1)-th frame of historical images are acquired. The i-th frame is the last historical image from which a historical target human detection box can be obtained. Based on the field of view of the image acquisition device and the position of the historical target human detection box in the historical images, the yaw angle of the target object in the i-th and (i-1)-th frames is determined, respectively. The acquisition time difference between the i-th and (i-1)-th frames and the historical horizontal movement speed of the image acquisition device in the i-th frame are obtained. Based on the yaw angle difference, the acquisition time difference, and the historical horizontal movement speed, the yaw angle movement speed of the target object is determined. The calculation formula is as follows:

[0089]

[0090] in, Let yaw angle be the yaw rate of the target object in the i-th frame of the historical image. Let yaw angle be the yaw angle of the target object in the i-th frame of the historical image. Let i be the image acquisition time of the i-th historical image. Let be the historical horizontal movement speed of the image acquisition device in the i-th frame.

[0091] This embodiment provides a target tracking method that determines the moving speed of the target object based on the horizontal movement difference, the time difference, and the historical horizontal movement speed of the image acquisition device at the time of the first historical image. This method can calculate the relative moving speed of the target object and the image acquisition device in the reference frame of the plane it is located in. By calculating the yaw angle, compared with establishing a spatial model to calculate the actual displacement of the target object, the amount of calculation data and calculation time can be reduced, thereby improving the target tracking speed.

[0092] In one embodiment, calculating the movement speed of the target object based on the position of the historical target human detection box in the historical image and the historical horizontal movement speed of the image acquisition device further includes:

[0093] The average moving speed of the target object at any given moment is calculated based on multiple historical images.

[0094] The average value is determined as the moving speed of the target object.

[0095] The timestamps of multiple historical images can be the timestamps of consecutive frames, or the timestamps of partial frames can be selected according to preset rules.

[0096] In one specific embodiment, taking the yaw rate of the target object as an example, the formula for calculating the average value is as follows:

[0097]

[0098] in, The mean yaw rate, Let yaw angular velocity be the yaw rate of the target object at the i-th frame of historical image time.

[0099] This embodiment provides a target tracking method that calculates the average moving speed of the target object based on the moving speed of the target object at different times in multiple historical images, and uses the average value as the moving speed of the target object. This reduces the amount of data required to calculate the coordinate position of the target object and can improve the tracking speed.

[0100] In one embodiment, the process of predicting the coordinate position of the target object based on its moving speed and moving time includes:

[0101] The servo response time and algorithm running time of the image acquisition device are obtained; the servo response time is the time it takes for the servo to accelerate to a preset horizontal movement speed, and the algorithm running time is the calculation time from acquiring the historical image to adjusting the pose of the image acquisition device;

[0102] The startup delay is determined based on the servo response time and the algorithm execution time.

[0103] Understandably, after a target is determined to be lost, the image acquisition device needs time to obtain relevant information, such as historical images, and to calculate the target object's coordinates. Similarly, the servo motor needs time to accelerate to the preset horizontal movement speed after receiving control commands, and then, after reaching the predicted target object's coordinates, the algorithm needs time to recapture the target object and perform real-time tracking. Therefore, the delays in algorithm execution and servo motor response can affect the accuracy of target object coordinate determination to some extent.

[0104] The servo response time can be the delay response time from receiving the re-tracking command to re-capturing the target object for real-time tracking. The servo response time can include the time it takes for the servo to accelerate to a preset horizontal movement speed. The algorithm runtime is the calculation time from acquiring historical images to adjusting the pose of the image acquisition device. Both the servo response time and the algorithm runtime can be obtained based on prior knowledge or set according to actual needs. The preset horizontal movement speed can be the servo's maximum horizontal movement speed or set according to actual needs. Furthermore, the servo response time can be the time it takes for the servo to accelerate from a stationary state to the preset horizontal movement speed, or it can be the time it takes for the servo to accelerate from its current operating speed to the preset horizontal movement speed. For practical purposes, if the servo is set to decelerate when approaching the predicted coordinate position, the servo response time can also include the time required for deceleration when approaching the predicted coordinate position. In addition, the servo response time can also include the response time required to re-capture the target object after approaching or reaching the predicted coordinate position.

[0105] The startup delay can be measured in units of time, determined based on the servo response time and algorithm execution time, or it can be determined by adding the servo response time and algorithm execution time together. Alternatively, the startup delay can be measured in units of frames, determined based on the servo response time and algorithm execution time, or it can be determined by multiplying the sum of the servo response time and algorithm execution time by the sampling frequency of the image acquisition device. In a specific embodiment, the formula for calculating the startup delay is as follows:

[0106]

[0107] in, To initiate a delay, For servo response time, For the algorithm's running time, The sampling frequency of the image acquisition device.

[0108] Furthermore, if the servo response time is the time it takes for the servo to accelerate from its current moving speed to a preset horizontal moving speed, the servo response time can be determined based on the ratio of the time it takes for the current moving speed to reach the preset horizontal moving speed to the time it takes to reach the preset horizontal moving speed from a stationary state. In addition, the servo response time can be determined based on the statistical results obtained by previously calculating the servo response time at different initial moving speeds.

[0109] This embodiment provides a target tracking method that determines the start delay based on the servo motor response time and algorithm running time, thereby realizing the calculation of the start delay and improving the accuracy of position prediction after target loss.

[0110] In one embodiment, adjusting the pose of the image acquisition device based on the coordinate position includes:

[0111] Get the current real-time image;

[0112] A real-time target human detection box is determined based on the real-time image;

[0113] The pose of the image acquisition device is adjusted based on the difference between the coordinate position of the real-time target human detection box in the real-time image and the center point of the real-time image.

[0114] The real-time image can be acquired by an image acquisition device. Determining the real-time target human detection box based on the real-time image can be based on image recognition to determine the location of the target object within the real-time target human detection box.

[0115] Based on the difference between the coordinate position of the real-time target human detection box in the real-time image and the center point of the real-time image, the pose of the image acquisition device is adjusted. This adjustment may involve adjusting one or more of the following: the yaw angle, pitch angle, and distance from the target object of the image acquisition device.

[0116] Adjusting the yaw angle of the image acquisition device can be achieved by calculating the average horizontal coordinate of the real-time target human body detection box, determining the yaw angle difference between the average horizontal coordinate of the real-time target human body detection box and the horizontal coordinate at the center of the field of view of the image acquisition device, and adjusting the pose of the image acquisition device based on this yaw angle difference.

[0117] Adjusting the pitch angle of the image acquisition device can be achieved by calculating the average ordinate of the real-time target human detection box, target face, or other parts, and then adjusting the pose of the image acquisition device based on the pitch angle difference between the average ordinate and the ordinate at the center of the field of view of the image acquisition device.

[0118] Adjusting the distance between the image acquisition device and the target object can be achieved by calculating the pixel width of the real-time target human body detection box, converting the required radial displacement based on the difference between the real-time target human body detection box pixel width and the reference pixel width, and then adjusting the position of the image acquisition device based on this radial displacement, i.e., moving closer to or further away from the target in a forward or backward manner; alternatively, it can be achieved by converting the real-time distance between the image acquisition device and the target object based on the pixel width of the real-time target human body detection box, determining the required radial displacement based on the difference between the real-time distance and the reference distance, and then adjusting the position of the image acquisition device based on this radial displacement.

[0119] Furthermore, adjusting the pose of the image acquisition device can also achieve parallel movement with the target object, i.e., movement in the same direction as the target object's movement. This can be done by determining the distance between the image acquisition device and the target based on the pixel width of the real-time target human detection box, and determining the target's real-time displacement based on the distance the real-time target human detection box moves in the real-time image. The position of the image acquisition device can then be adjusted based on the target's movement direction and real-time displacement to achieve parallel movement with the target object.

[0120] This embodiment provides a target tracking method that, when the target object is within the field of view of the image acquisition device, determines a real-time target human body detection box based on the real-time image, and adjusts the pose of the image acquisition device based on the real-time target human body detection box, thereby achieving the effect of continuous tracking of the target object.

[0121] In one embodiment, determining the real-time target human detection box based on the real-time image includes:

[0122] Based on the real-time image, determine whether the target object's face exists;

[0123] If the target face exists in the real-time image, then a target face detection box is determined based on the target face; and a real-time target human body detection box is determined based on the target face detection box.

[0124] If the target face is not present in the real-time image, a human body detection box is determined based on the real-time image, and a real-time target human body detection box is determined based on the matching between the human body detection box and the reference human body detection box.

[0125] It is understandable that the features of a human face are richer than those of a human body. Therefore, detecting a target face can more accurately determine whether a target object exists in a real-time image. Determining the presence of a target face based on the real-time image can be done through image recognition. For example, the face recognition model provided by Rockchip's Rock-XSDK library can be used. With the acceleration of the RK3399PRO chip's NPU, the recognition result can be calculated within 40ms. Other face recognition models or algorithm libraries such as arcface, MobilefaceNet, deepface, openface, and dlib can also be used for arbitrary face recognition; this paper does not limit the scope of these methods.

[0126] Determining the target face detection box based on the target face can be achieved by marking the target face in the real-time image with a pixel box of a preset color, or by determining the coordinate position of the target face detection box in the image and recording that coordinate position in the recording information corresponding to the real-time image. For example... Figure 3As shown, in one specific embodiment, after determining the target face, the coordinates of the upper left corner and the lower right corner of the rectangle containing the target face can be used as the coordinate positions of the target face detection box.

[0127] Determining the real-time target human body detection box based on the target face detection box can be achieved by matching the position of the target face detection box with the coordinate positions of the human body detection boxes already detected in the real-time image.

[0128] Human detection bounding boxes can be obtained based on image recognition, such as the human detection models provided by the Rock-X SDK library mentioned above. With the acceleration of the RK3399PRO chip's NPU, the recognition result can be calculated within 70ms. Alternatively, high-performance object detection models can be used, such as fast-RCNN, SSD, and the YOLO series. Human pose detection models or algorithm libraries can also be employed, such as Mediapipe Pose, OpenPose, and AlphaPose. After obtaining the human detection bounding boxes, multiple bounding boxes can be numbered, such as... Figure 4 As shown, in one specific embodiment, three human detection boxes are determined in the current image based on a human detection model and numbered NO.1, NO.2 and NO.3 respectively.

[0129] The human detection frame can either only indicate a part of a person's body, or it can include the entire body, including the head. For example... Figure 5 As shown, the human body detection box that matches the target face detection box can be determined based on the relative position of the human body detection box and the target face detection box, and thus the human body detection box is determined as the real-time target human body detection box.

[0130] Furthermore, when the target object is too close to the image acquisition device or is obstructed by obstacles, the target face may be outside the acquisition range of the image acquisition device or the line of sight may be limited and unable to be acquired, making it impossible to judge the target object based on facial features.

[0131] If no target face is found in the real-time image, a human body detection box is determined based on the real-time image. The method for determining the human body detection box can be based on image recognition, which will not be elaborated in this paper.

[0132] The reference human detection box can be determined by predicting the coordinate position of the target object in the current image based on the displacement distance and pixel volume change of the real-time target human detection box in historical images, and then determining the reference human detection box based on this coordinate position. The real-time target human detection box can be determined based on the matching result between the human detection box and the reference human detection box, using the coordinate distance between them. In one embodiment, when determining the real-time target human detection box based on the target face detection box, a face-target tracking key-value pair is determined based on the target face detection box and the real-time target human detection box. If the current image does not contain a target face, the real-time target human detection box is determined based on the target-tracked human detection box.

[0133] The reference human detection box can also be a reference image determined based on real-time target human detection boxes in historical images containing the target face. This reference image can be generated when the target face is lost, or it can be stored in a database. The matching between the human detection box and the reference human detection box can be based on comparing human features with the features of the reference image to determine the real-time target human detection box. In one embodiment, the reference human detection box in the database is updated based on the image of the real-time target human detection box. By updating the reference human detection box, it is possible to determine the real-time target human detection box using the most recently recorded human image of the target object when the target face is not present in the real-time image, thereby improving the accuracy of target human recognition and achieving the effect of improving target tracking accuracy.

[0134] This embodiment provides a target tracking method that first determines whether a target face exists, and then determines a real-time target human body detection box based on the target face. This enables accurate determination of the real-time target human body detection box using facial features, thereby improving the target tracking accuracy. By matching the human body detection box in the real-time image with a reference human body detection box determined based on historical images, the real-time target human body detection box is determined. This allows for the identification of the target object's human body detection box even when the target face is lost, further improving the target tracking accuracy.

[0135] In one embodiment, determining whether a target face of the target object exists based on the real-time image includes:

[0136] A face detection bounding box is determined based on the real-time image;

[0137] The first feature vector is determined based on the face detection box;

[0138] The existence of the target face is determined based on the first feature vector and the reference face feature vector in the database.

[0139] The determination of face detection boxes based on the real-time image can be achieved through image recognition, such as using the face detection model provided by Rockchip's Rock-X SDK library. Alternatively, it can utilize other face detection models or algorithm libraries such as MTCNN, RetinaFace, YOLO5Face, and SCRFD to detect faces in the image; this paper does not limit the specific method used. After determining the face detection boxes, multiple face detection boxes can be numbered, such as... Figure 6 As shown, in one specific embodiment, three face detection boxes are determined in the current image based on a face detection model and numbered NO.1, NO.2 and NO.3 respectively.

[0140] After determining the first feature vector based on the face detection bounding box, the existence of the target face is determined based on the first feature vector and the reference face feature vector in the database. This can be done by obtaining the reference face features in the database, determining the face feature with the shortest Euclidean distance based on the face features in the real-time image and the reference face features, and if the Euclidean distance is less than a preset threshold, the face detection bounding box corresponding to that face feature can be determined as the target face detection bounding box. In a specific embodiment, the preset threshold for the Euclidean distance can be 1.0. Figure 3 As shown, after the target face detection box is determined, the number of the face detection box can also be updated.

[0141] This embodiment provides a target tracking method that compares the feature vector of a face detection box with a reference feature vector to determine whether a target face exists, thereby enabling target face recognition.

[0142] To more clearly illustrate the technical solution of this application, a detailed embodiment is also provided. For example... Figure 6 and Figure 7 As shown, the target tracking method of this embodiment is applied to an image acquisition device. The image acquisition device includes a base, in which a chassis, drive wheels, casters, servo motor 2, controller, processor, and memory are disposed. A platform extends upward from the base, and servo motor 1 is mounted on the platform. Servo motor 1 is connected to a monocular camera via a linkage. Servo motor 1 controls the pitch angle acquired by the monocular camera, servo motor 2 controls the yaw angle acquired by the monocular camera, and the drive wheels and casters control the movement of the image acquisition device on its surface. The target tracking method of this embodiment includes:

[0143] During image acquisition, the timestamp of the image captured by the monocular camera and the rotation speed of the yaw angle servo at that time are recorded. For example, the timestamp of the i-th frame is... The rotational speed of the yaw angle servo is Based on experimental data, the servo motor response speed is defined as the time from when the controller sends a command until the servo motor reaches 73% of the target speed. Record the algorithm's running speed, that is, the speed from reading the i-th frame of the image to sending the command to the servo motor. .

[0144] When the target object is lost, i.e., a target human detection box can be obtained in the i-th frame but not in the (i+1)-th frame, the velocity of the target object at the yaw angle in the N frames before the target disappears is estimated using the following formula:

[0145]

[0146] in It is the yaw angle of the target motion in the i-th frame image. It is the yaw rate of the target motion in the i-th frame of the image that can be calculated.

[0147] Calculate the average yaw rate of the target in the N frames before the target object disappears. for:

[0148]

[0149] Based on the sampling frequency of the monocular camera The startup delay caused by algorithm execution and servo response is .

[0150] Determining the first based on startup delay The yaw angle of the target object relative to the image acquisition device is: And adjust the yaw angle of servo 2 to .

[0151] Before or after the target is lost and then recaptured, the controller also includes target human visual servoing tracking based on lightweight model inference, such as... Figure 7 As shown, it includes:

[0152] Step 1: The monocular camera captures the current image and runs a face recognition algorithm to obtain the target face detection box;

[0153] Step 2: Run the human detection algorithm and the multi-target tracking algorithm to obtain human tracking detection boxes. The human detection algorithm includes taking the currently acquired image as input, running a human detection model based on a lightweight deep neural network for inference, and obtaining a human detection rectangle containing the coordinates of four points: upper left, lower left, upper right, and lower right. The multi-target tracking algorithm includes using the human detection boxes obtained by the human detection algorithm as the initial tracking target, running the multi-target tracking algorithm, and obtaining human tracking detection boxes for multiple consecutive frames of images, as well as the target tracking number.

[0154] Step 3: Match the target face detection box with the human body tracking detection box to obtain the target human body detection box;

[0155] Step 4: Input the position information of the target face detection box and the target human body detection box into the PID controller to obtain the motion control signal for adjusting the position of the monocular camera;

[0156] Step 5: The motion control unit adjusts the angle and position of the monocular camera according to the control signal.

[0157] The face recognition algorithm in step one includes:

[0158] Step 1.1: Take the currently acquired image as input, run the face detection model based on a lightweight deep neural network for inference, and obtain a face detection rectangle containing the coordinates of four points: top left, bottom left, top right, and bottom right.

[0159] Step 1.2: Crop the image based on the face detection bounding box, use the cropped image as input, and run the face recognition model based on a lightweight deep neural network to obtain the feature vector of the face;

[0160] Step 1.3: Find the face feature vector in the database that has the shortest Euclidean distance to the face feature vector output in Step 1.2, and this distance is less than a threshold. If the target face is successfully identified, the label corresponding to the vector is retrieved from the database and combined with the face detection box to obtain the "face label-face detection box key-value pair", which is the target face detection box.

[0161] Step 3 involves position matching between the target face detection bounding box and the human body tracking detection bounding box to obtain the target human body detection bounding box, including:

[0162] Step 3.1: Compare the positions of all human body tracking detection boxes obtained from the image with the target face detection boxes from Step 1. If there exists a set of face detection boxes that satisfy the following set of inequalities:

[0163]

[0164]

[0165]

[0166]

[0167] in Let the minimum and maximum x-coordinates of the target face detection bounding box be denoted as ; The minimum and maximum ordinates of the target face detection bounding box; The minimum and maximum x-coordinates of the human body tracking detection box; The minimum and maximum ordinates of the human body tracking detection box; Let be the height of the human body tracking detection box; if there exists a set of detection boxes that satisfy the above set of inequalities, then the target face detection box and the human body tracking detection box are successfully matched, and the target human body detection box is obtained.

[0168] Step 3.2: If the target face detection bounding box in Step 3.1 successfully matches the human body tracking detection bounding box, combine the face label of the target face detection bounding box with the target tracking number obtained by the human body detection algorithm to obtain a "face label - target tracking number key-value pair". At the same time, delete the "face label - face detection bounding box key-value pair" in Step 1.3.

[0169] Step 3.3: For image frames where human bodies can be detected but faces cannot, use the face label-target tracking number key-value pair obtained in 3.2 to combine the face label with the human body tracking detection box into a new data structure to obtain the target human body detection box.

[0170] Step four involves inputting the position information of the target face detection bounding box and the target human body detection bounding box into the PID controller to obtain the motion control signal for adjusting the position of the monocular camera, including:

[0171] Step 4.1: Calculate the average ordinate of the target face detection bounding box using the following formula. :

[0172]

[0173] The required adjustment of the visual sensor pitch angle difference for tracking the target is calculated using the following formula. :

[0174]

[0175] in The height of the image in pixels. It refers to the longitudinal field of view of a monocular camera.

[0176] Step 4.2: If the target tracking requires adjustment of the visual sensor pitch angle difference... The absolute value is greater than the threshold Then As input, the PID control algorithm is run to obtain the control signal needed to control the tilt angle of the monocular camera. Otherwise, let .

[0177] Step 4.3: Calculate the width of the target human detection box using the following formula. and average x-axis :

[0178]

[0179]

[0180] Step 4.4: Calculate the visual sensor yaw angle difference that needs to be adjusted to track the target using the following formula. :

[0181]

[0182] in The width of the image in pixels. It refers to the horizontal field of view of a monocular camera.

[0183] Step 4.5: If the target tracking requires adjustment of the visual sensor yaw angle difference... The absolute value is greater than the threshold Then As input, a PID control algorithm is run to obtain the control signal needed to control the yaw angle of the vision sensor. Otherwise, let .

[0184] Step 4.6: Calculate the required visual human width adjustment value for tracking the target using the following formula. :

[0185]

[0186] in This is a reference value for the width of the target human detection box, representing the reasonable distance that the vision sensor and the target should maintain.

[0187] Step 4.7: Adjust the visual human body width value The absolute value is greater than the threshold Then As input, a PID control algorithm is run to obtain the control signal needed to control the distance between the vision sensor and the target. Otherwise, let .

[0188] Step five, the motion control unit, adjusts the angle and position of the monocular camera according to the control signal, including:

[0189] Step 5.1: If the control signal for controlling the pitch angle of the vision sensor... If the value is less than 0, then the direction of the visual sensor's pitch angle movement is set to upward, and the rotation speed is... ; like If the value is greater than 0, then the direction of the visual sensor's pitch angle movement is set to downward, and the rotation speed is... ;like Then the pitch angle movement of the vision sensor will stop.

[0190] Step 5.2: If the control signal controlling the yaw angle of the vision sensor... If the value is greater than 0, then the yaw angle movement direction of the visual sensor is set to right rotation, and the rotation speed is... ; like If the value is less than 0, then the yaw angle movement direction of the visual sensor is set to a leftward rotation, and the rotation speed is [value missing]. ;like If so, the yaw angle movement of the visual sensor will stop.

[0191] Step 5.3: If the control signal controlling the distance between the vision sensor and the target... If the value is greater than 0, then the movement direction of the vision sensor platform is set to forward, and the movement speed is [value missing]. ; like If the value is less than 0, then the movement direction of the vision sensor platform is set to backward, and the movement speed is [value missing]. ;like If this happens, the forward and backward movement of the vision sensor platform will stop.

[0192] This application provides a target tracking method that determines the moving speed of a target object through historical images and predicts the target object's coordinate position based on that speed, enabling target object position prediction based on historical images. By introducing servo motor response time and algorithm execution time as delay time parameters for the image acquisition device during the prediction process, the accuracy of target object position prediction and the robustness of the prediction algorithm can be improved, thereby enhancing tracking accuracy after target loss. Furthermore, by determining the target object's moving speed based on the horizontal movement difference, the time difference, and the historical horizontal movement speed of the image acquisition device at the time of the first historical image, the method can achieve target-to-target tracking. The calculation of the relative movement speed of the object with respect to the image acquisition device in the reference frame of its plane, by calculating the yaw angle, can reduce the amount of computational data and computation time compared to establishing a spatial model to calculate the actual displacement of the target object, thereby improving the target tracking speed. By calculating the average movement speed of the target object based on the movement speed of the target object at multiple historical images at different times, and using the average value as the movement speed of the target object, the amount of computational data for the coordinate position of the target object can be reduced, thereby improving the tracking speed. By determining the real-time target human detection box through the target face or reference human detection box, the target object can be identified during real-time tracking, improving the accuracy of target tracking.

[0193] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0194] Based on the same inventive concept, this application also provides a target tracking device for implementing the target tracking method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more target tracking device embodiments provided below can be found in the limitations of the target tracking method described above, and will not be repeated here.

[0195] In one embodiment, such as Figure 8 As shown, a target tracking device is provided, including: an acquisition module 100, a calculation module 200, a prediction module 300, and an adjustment module 400, wherein:

[0196] The acquisition module 100 is used to acquire a preset number of historical images when the target object is detected to be lost, wherein the historical images include the target object;

[0197] Calculation module 200 is used to determine the moving speed of the target object based on the historical image;

[0198] Prediction module 300 is used to predict the coordinate position of the target object based on the target object's moving speed and moving time; the moving time includes the loss time and the delay time of the image acquisition device, the delay time including the start-up delay of the image acquisition device;

[0199] The adjustment module 400 is used to adjust the pose of the image acquisition device based on the coordinate position.

[0200] In one embodiment, the calculation module is further configured to: obtain a historical target human body detection box of the target object in the historical image; and calculate the moving speed of the target object based on the position of the historical target human body detection box in the historical image and the historical horizontal moving speed of the image acquisition device.

[0201] In one embodiment, the calculation module is further configured to: acquire a first historical image and a second historical image of adjacent frames and the time difference between the adjacent frames; determine the horizontal movement difference of the historical target human body detection box relative to the image acquisition device based on the position of the historical target human body detection box in the first historical image and the second historical image; and determine the movement speed of the target object based on the horizontal movement difference, the time difference, and the historical horizontal movement speed of the image acquisition device at the time of the first historical image.

[0202] In one embodiment, the calculation module is further configured to: calculate an average value of the moving speed of the target object at a given moment based on multiple historical images; and determine the average value as the moving speed of the target object.

[0203] In one embodiment, the process of predicting the coordinate position of the target object based on its moving speed and moving time includes:

[0204] The servo response time and algorithm running time of the image acquisition device are obtained; the servo response time is the time it takes for the servo to accelerate to a preset horizontal movement speed, and the algorithm running time is the calculation time from acquiring the historical image to adjusting the pose of the image acquisition device;

[0205] The startup delay is determined based on the servo response time and the algorithm execution time.

[0206] In one embodiment, the device further includes:

[0207] A real-time tracking module is used to acquire the current real-time image; determine a real-time target human body detection box based on the real-time image; and adjust the pose of the image acquisition device based on the difference between the coordinate position of the real-time target human body detection box in the real-time image and the center point of the real-time image.

[0208] In one embodiment, the real-time tracking module is further configured to determine whether a target face of the target object exists based on the real-time image; if the target face exists in the real-time image, a target face detection box is determined based on the target face; a real-time target human body detection box is determined based on the target face detection box; if the target face does not exist in the real-time image, a human body detection box is determined based on the real-time image, and a real-time target human body detection box is determined based on the matching situation between the human body detection box and a reference human body detection box.

[0209] In one embodiment, the real-time tracking module is further configured to: determine a face detection box based on the historical image; determine a first feature vector based on the face detection box; and determine whether the target face exists based on the first feature vector and a reference face feature vector in the database.

[0210] Each module in the aforementioned target tracking device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0211] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a target tracking method. The display screen can be an LCD screen or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0212] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0213] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0214] When the target object is detected to be missing, a preset number of historical images are acquired, and the historical images include the target object;

[0215] The movement speed of the target object is determined based on the historical images;

[0216] The coordinate position of the target object is predicted based on the target object's moving speed and moving time; the moving time includes the loss time and the delay time of the image acquisition device, and the delay time includes the startup delay of the image acquisition device;

[0217] The pose of the image acquisition device is adjusted based on the coordinate position.

[0218] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0219] When the target object is detected to be missing, a preset number of historical images are acquired, and the historical images include the target object;

[0220] The movement speed of the target object is determined based on the historical images;

[0221] The coordinate position of the target object is predicted based on the target object's moving speed and moving time; the moving time includes the loss time and the delay time of the image acquisition device, and the delay time includes the startup delay of the image acquisition device;

[0222] The pose of the image acquisition device is adjusted based on the coordinate position.

[0223] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0224] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0225] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0226] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A target tracking method, applied to an image acquisition device, characterized in that, The method includes: When the target object is detected to be missing, a preset number of historical images are acquired, and the historical images include the target object; The movement speed of the target object is determined based on the historical images; The coordinate position of the target object is predicted based on its moving speed and moving time; the moving time includes the loss time and the delay time of the image acquisition device, the delay time including the startup delay of the image acquisition device; before predicting the coordinate position of the target object based on its moving speed and moving time, the process includes: acquiring the servo response time and algorithm running time of the image acquisition device; the servo response time is determined by the ratio of the time it takes for the current moving speed to reach a preset horizontal moving speed to the time it takes to reach a preset horizontal moving speed from a stationary state, and the algorithm running time is the calculation time from acquiring the historical image to adjusting the pose of the image acquisition device; the startup delay is determined by multiplying the sum of the servo response time and the algorithm running time by the sampling frequency of the image acquisition device; The pose of the image acquisition device is adjusted based on the coordinate position.

2. The target tracking method according to claim 1, characterized in that, Determining the movement speed of the target object based on the historical image includes: Obtain the historical target human body detection bounding box of the target object in the historical image; The moving speed of the target object is calculated based on the position of the historical target human body detection box in the historical image and the historical horizontal moving speed of the image acquisition device.

3. The target tracking method according to claim 2, characterized in that, The calculation of the target object's movement speed based on the position of the historical target human body detection box in the historical image and the historical horizontal movement speed of the image acquisition device includes: Obtain the first and second historical images of adjacent frames, as well as the time difference between adjacent frames; Based on the position of the historical target human body detection box in the first historical image and the second historical image, the horizontal movement difference of the historical target human body detection box relative to the image acquisition device is determined; The movement speed of the target object is determined based on the horizontal movement difference, the time difference, and the historical horizontal movement speed of the image acquisition device at the time of the first historical image.

4. The target tracking method according to claim 2, characterized in that, The step of calculating the movement speed of the target object based on the position of the historical target human body detection box in the historical image and the historical horizontal movement speed of the image acquisition device further includes: The average moving speed of the target object at any given moment is calculated based on multiple historical images. The average value is determined as the moving speed of the target object.

5. The target tracking method according to claim 1, characterized in that, After adjusting the pose of the image acquisition device based on the coordinate position, the following steps are included: Get the current real-time image; A real-time target human detection box is determined based on the real-time image; The pose of the image acquisition device is adjusted based on the difference between the coordinate position of the real-time target human detection box in the real-time image and the center point of the real-time image.

6. The target tracking method according to claim 5, characterized in that, The step of determining the real-time target human detection box based on the real-time image includes: Based on the real-time image, determine whether the target object's face exists; If the target face exists in the real-time image, a target face detection box is determined based on the target face; a real-time target human body detection box is determined based on the target face detection box. If the target face is not present in the real-time image, a human body detection box is determined based on the real-time image, and a real-time target human body detection box is determined based on the matching between the human body detection box and the reference human body detection box.

7. The target tracking method according to claim 6, characterized in that, The step of determining whether a target face of the target object exists based on the real-time image includes: A face detection bounding box is determined based on the real-time image; The first feature vector is determined based on the face detection box; The existence of the target face is determined based on the first feature vector and the reference face feature vector in the database.

8. A target tracking device, characterized in that, The device includes: The acquisition module is used to acquire a preset number of historical images when the target object is detected to be lost, and the historical images include the target object; A calculation module is used to determine the moving speed of the target object based on the historical image; A prediction module is used to predict the coordinate position of the target object based on its moving speed and moving time. The moving time includes the loss time and the delay time of the image acquisition device, and the delay time includes the startup delay of the image acquisition device. Before predicting the coordinate position of the target object based on its moving speed and moving time, the module includes: acquiring the servo response time and algorithm running time of the image acquisition device. The servo response time is determined by the ratio of the time it takes for the current moving speed to reach a preset horizontal moving speed to the time it takes to reach a preset horizontal moving speed from a stationary state. The algorithm running time is the calculation time from acquiring the historical image to adjusting the pose of the image acquisition device. The startup delay is determined by multiplying the sum of the servo response time and the algorithm running time by the sampling frequency of the image acquisition device. An adjustment module is used to adjust the pose of the image acquisition device based on the coordinate position.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images