Target tracking method, device, system and program product based on pan-tilt linkage

Abstract

The present invention discloses a target tracking method, device, system and program product based on pan-tilt linkage, and relates to the technical field of target tracking. In order to solve the defects that the existing pan-tilt linkage imaging devices use single coordinate and position information when tracking targets, which may lead to target loss and poor automatic tracking accuracy, the method includes: obtaining the tracking target; obtaining the target movement speed of the tracking target relative to the search device or tracking device that collects the tracking target; obtaining the pan-tilt movement speed of the pan-tilt device connected to the tracking device; and controlling the movement of the tracking device according to the target movement speed and the pan-tilt movement speed. The present invention is mainly used for tracking targets.

Description

Technical Field

[0001] The present invention relates to the technical field of target tracking, and in particular to a target tracking method based on pan-tilt linkage, a tracking device, a host computer, a target tracking system based on pan-tilt linkage, and a computer program product. Background Art

[0002] Currently, there is an increasing demand for system functions based on gimbal-type linked imaging devices for automatic target tracking. However, since the target states that need to be tracked by such products are different and the application scenarios are complex and diverse, in the overall system linkage process, it is necessary to set up imaging devices and gimbal devices according to different states for accurate and effective control, thereby ensuring that the target to be tracked can be accurately tracked.

[0003] However, in actual applications, there may be a variety of target motion states and image states, which will affect the control accuracy of the pan-tilt device and the imaging device, and also affect the effectiveness and accuracy of automatic tracking.

[0004] The automatic tracking solutions in the prior art generally use single coordinate and position information (for example, only use the position of the target on the imaging device) to control the pan / tilt, which may cause the imaging device to lose the target to be tracked, so the accuracy of automatic tracking is not high. Summary of the invention

[0005] In order to solve the defects of existing pan-tilt linkage imaging devices that use single coordinate and position information when tracking targets, which may lead to target loss and poor automatic tracking accuracy, the present invention provides a target tracking method based on pan-tilt linkage, a tracking device, a host computer, a target tracking system based on pan-tilt linkage and a computer program product, which can reduce the probability of tracking target loss, thereby improving the accuracy of automatic tracking.

[0006] In a first aspect, a target tracking method based on pan-tilt linkage is provided, comprising: obtaining a tracking target; obtaining a target movement speed of the tracking target relative to a search device or a tracking device that collects the tracking target; obtaining a pan-tilt movement speed of a pan-tilt device connected to the tracking device; and controlling the movement of the tracking device according to the target movement speed and the pan-tilt movement speed.

[0007] In a second aspect, a tracking device is provided, which includes a storage device and a processing device, wherein the storage device stores a computer program, and when the computer program is executed by the processing device, the processing device executes any one of the target tracking methods based on pan-tilt linkage described in the present application.

[0008] In a third aspect, a host computer is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the target tracking method based on pan-tilt linkage described in any embodiment of the present application.

[0009] In a fourth aspect, a target tracking system based on pan-tilt linkage is provided, the system comprising a tracking device and a pan-tilt device as described in any one of the second aspects of the present application; or

[0010] The system includes the tracking device, search device and pan-tilt device described in any one of the second aspects of the present application, wherein the search device is used to obtain the tracking target, and the pan-tilt device is used to connect with the tracking device to assist the movement of the tracking device.

[0011] In a fifth aspect, a target tracking system based on pan-tilt linkage is provided, the system comprising a host computer, a tracking device and a pan-tilt device as described in any one of the third aspects of the present application, the host computer is used to communicate with the tracking device and the pan-tilt device, the tracking device is used to obtain the tracking target and track the tracking target according to the control instructions issued by the host computer, and the pan-tilt device is used to connect with the tracking device to assist the movement of the tracking device; or

[0012] The system includes a host computer, a search device, a tracking device and a pan-tilt device as described in any one of the third aspects of the present application, the host computer is used to communicate with the search device, the tracking device and the pan-tilt device, the search device is used to obtain the tracking target, the tracking device is used to track the tracking target according to the control instructions issued by the host computer, and the pan-tilt device is used to connect with the tracking device to assist the movement of the tracking device.

[0013] In a sixth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements a target tracking method based on pan-tilt linkage as described in any one of the first aspects of the present application.

[0014] The embodiment of the present application obtains a tracking target, obtains the target movement speed of the tracking target relative to the device that collects its image, and obtains the pan-tilt movement speed of the pan-tilt device connected to the tracking device, and controls the movement of the tracking device in combination with the target movement speed and the pan-tilt speed. This can ensure that the tracking target is always located in the tracking area of ​​the tracking device, reduce the probability of losing the tracking target, and thus improve the accuracy of automatic tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 1. is a diagram of an application environment of a target tracking method based on pan / tilt linkage in an embodiment;

[0016] Figure 2 This is a diagram of an application environment of a target tracking method based on pan / tilt linkage in another embodiment;

[0017] Figure 3 is a flow chart of a target tracking method based on pan / tilt linkage in an embodiment;

[0018] Figure 4 It is a flow chart of a target tracking method based on pan-tilt linkage in an application scenario;

[0019] Figure 5 is a flow chart of a target tracking method based on pan-tilt linkage in another application scenario;

[0020] Figure 6 is a schematic diagram of a target tracking device based on pan / tilt linkage in an embodiment;

[0021] Figure 7 A schematic diagram of a host computer in an embodiment;

[0022] Figure 8 A schematic diagram of a tracking device in one embodiment. DETAILED DESCRIPTION

[0023] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the technical field of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0025] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0026] See also Figure 1, is an application environment diagram of a target tracking method based on pan-tilt linkage in an embodiment. The target tracking system based on pan-tilt linkage in the application environment diagram includes a host computer 10, a search device 20, a tracking device 30 and a pan-tilt device 40. The host computer 10 can communicate with the search device 20, the tracking device 30 and the pan-tilt device 40 in a wireless or wired manner. The host computer 10 obtains the operation data of the search device 20, the tracking device 30 and the pan-tilt device 40, and issues control instructions to the search device 20, the tracking device 30 and the pan-tilt device 40, etc. The target tracking method based on pan-tilt linkage is applied to the host computer 10 to control the tracking device 30 to track the tracking target. The search device 20 can be used to search for the target to be tracked or collect scene images, and the host computer 10 obtains the tracking target according to the scene image, and issues it to the tracking device 30 to track the tracking target. The pan-tilt device 40 is used to support the rotation of the tracking device 30. One pan-tilt device 40 can be connected to one or more tracking devices 30. In some optional implementations, the search device 20 can also be connected to a pan-tilt device and move through the connected pan-tilt device.

[0027] The host computer 10 may be a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a terminal device (e.g., a mobile phone, etc.), a wearable device (e.g., a pair of smart glasses or a smart watch), various embedded devices, an industrial computer, etc. The host computer 10 may provide a user interface to interact with the user. The search device 20 includes but is not limited to a device with an image acquisition function, such as a search device 20 that is a full-scan image acquisition device, etc. The search device 20 may be one or more. The tracking device 30 includes but is not limited to a device with an image acquisition function, and the tracking device 30 may be one or more. The image sensor in the search device 20 and the tracking device 30 may be a monocular vision sensor or a multi-eye vision sensor. For example, it may be a combination of one or more sensors among a thermal imaging sensor, a visible light image sensor, a millimeter wave sensor, a lidar sensor, an infrared thermal imaging sensor, and a depth sensor.

[0028] In some embodiments, the target tracking system based on pan-tilt linkage in the application environment diagram may not include the search device 20. The host computer 10 directly determines the target to be tracked through the scene image collected by the tracking device 30, and then sends a tracking instruction to the tracking device 30 for tracking.

[0029] See also Figure 2, is an application environment diagram of a target tracking method based on pan-tilt linkage in another embodiment. The target tracking system based on pan-tilt linkage in the application environment diagram includes a search device 20, a tracking device 30 and a pan-tilt device 40. The target tracking method based on pan-tilt linkage is applied to the tracking device 30 to track the tracking target. The tracking device 30 can communicate with the search device 20 and the pan-tilt device 40 in a wireless or wired manner. The tracking device 30 obtains the operating data of the search device 20 and the pan-tilt device 40, and issues control instructions to the search device 20 and the pan-tilt device 40, etc. The search device 20 can be used to search for targets to be tracked or collect scene images, and the tracking device 30 obtains the tracking target according to the tracking target or scene image searched by the search device 20, and tracks the tracking target. The pan-tilt device 40 is used to support the rotation of the tracking device 30. One pan-tilt device 40 can be connected to one or more tracking devices 30. In some optional implementations, the search device 20 can also be connected to a pan-tilt device and move through the connected pan-tilt device.

[0030] In some embodiments, the target tracking system based on pan-tilt linkage in the application environment diagram may not include the search device 20. The tracking device 30 determines the target to be tracked based on the captured scene image and then tracks it.

[0031] See also Figure 3 , is a flow chart of a target tracking method based on pan-tilt linkage provided by an embodiment of the present application, and the target tracking method based on pan-tilt linkage includes the following steps:

[0032] S11. Acquire the tracking target.

[0033] In this embodiment, the tracking target indicates a target that needs to be continuously tracked. In some optional implementations, the host computer 10 obtains a scene image captured by the search device 20 or the tracking device 30, performs image analysis on the scene image, and automatically screens the tracking target that meets the tracking conditions. In some optional implementations, after the host computer 10 obtains the scene image, it displays the scene image on the user interface, and the user specifies the tracking target. In an optional method, the tracking device 30 obtains a scene image captured by the search device 20 or itself, performs image analysis on the scene image, and automatically screens the tracking target that meets the tracking conditions.

[0034] S12: Acquire the target movement speed of the tracked target relative to the search device or the tracking device that collects the tracked target.

[0035] In this embodiment, for any search device 20 or tracking device 30, there is a corresponding coordinate system, and the target motion speed is relative to the coordinate system of the device that collects the tracking target. When the device that collects the image of the tracking target is different from the device that tracks the tracking target, it is necessary to convert the target motion speed to the coordinate system of the device that tracks the tracking target. In an optional implementation, the host computer 10 communicates with the search device 20 or the tracking device 30, collects the operation data of the search device 20 or the tracking device 30, and determines the target motion speed according to the operation data of the search device 20 or the tracking device 30. In an optional implementation, the tracking device 30 communicates with the search device 20, and obtains the target motion speed according to the operation data of the search device 20. In an optional implementation, the tracking device 30 obtains the target motion speed according to its own operation data. The operation data includes but is not limited to: collected multi-frame scene images, motion speed, etc.

[0036] S13, obtaining a pan / tilt movement speed of a pan / tilt device connected to the tracking device.

[0037] In this embodiment, the rotational movement, movement, etc. of the tracking device 30 can be controlled by controlling the pan-tilt device 40, wherein the rotational movement includes but is not limited to horizontal rotational movement and vertical rotational movement. In this way, the tracking device 30 can move to the target position to track the tracking target. For the tracking device and the pan-tilt device, different coordinate systems are also corresponding, wherein the pan-tilt movement speed is the movement speed relative to the pan-tilt coordinate system. In a specific optional implementation, the pan-tilt rotation speed can be calculated according to the rotational angular velocity of the pan-tilt device 40, the focal length of the tracking device 30, and the distance between the pan-tilt device 40 and the tracking device 30, thereby obtaining the pan-tilt movement speed. In an optional implementation, this step can be implemented in the host computer 10. In an optional implementation, this step can also be implemented in the tracking device 30.

[0038] S14. Control the movement of the tracking device according to the target movement speed and the pan / tilt movement speed.

[0039] In this embodiment, according to the target movement speed and the pan-tilt movement speed, the tracking device 30 is controlled to move to the target position and maintain a certain movement speed, so that the tracking target can be located in the tracking area of ​​the tracking device 30. The tracking area can be the central area part or other preset areas in the scene image collected by the tracking device 30. When the tracking target can always be in the tracking area of ​​the tracking device 30, the probability of losing the tracking target can be reduced. In an optional implementation, this step can be implemented in the host computer 10, and the host computer 10 generates a control instruction of the tracking device 30 according to the target movement speed and the pan-tilt movement speed and sends it to the tracking device 30. In an optional implementation, this step can also be implemented in the tracking device 30, and the tracking device 30 analyzes the acquired target movement speed and pan-tilt movement speed to control its own movement.

[0040] In the above embodiment, the tracking target is obtained, and the target movement speed of the tracking target relative to the device that collects its image is obtained, and the pan-tilt movement speed of the pan-tilt device connected to the tracking device is obtained. In combination with the target movement speed and the pan-tilt speed, the movement of the tracking device is controlled. The tracking target can be always located in the tracking area of ​​the tracking device, which can reduce the probability of losing the tracking target, thereby improving the accuracy of automatic tracking.

[0041] In some embodiments, obtaining the tracking target includes at least one of the following:

[0042] Taking a target meeting the tracking condition in the scene image collected by the search device or the tracking device as the tracking target;

[0043] The target corresponding to the target designation operation based on the displayed scene image is determined as the tracking target.

[0044] In this embodiment, the tracking conditions include but are not limited to targets located in the center area of ​​the scene image, targets with the largest size, and the like. In some optional implementations, the host computer 10 obtains the scene image captured by the search device 20 or the tracking device 30, performs image analysis on the scene image, and automatically screens the tracking targets that meet the tracking conditions. In some optional implementations, after the host computer 10 obtains the scene image, it displays the scene image on the user interface, and the user specifies the tracking target. In an optional method, the tracking device 30 obtains the scene image captured by the search device 20 or itself, performs image analysis on the scene image, and automatically screens the tracking targets that meet the tracking conditions.

[0045] In the above embodiment, by determining the tracking target in various forms, the target for automatic tracking can be made more diverse, and it is convenient to find the target in time, which can reduce the probability of losing the tracking target, thereby improving the accuracy of automatic tracking.

[0046] In some embodiments, when the tracking target is a target acquired according to an image collected by the tracking device, controlling the movement of the tracking device according to the target movement speed and the pan / tilt movement speed includes:

[0047] Based on the coordinate conversion relationship between the pan-tilt device and the tracking device, the pan-tilt motion speed is converted into the coordinate system corresponding to the tracking device to obtain the pan-tilt relative speed;

[0048] Calculating the relative speed of the target based on the target speed and the relative speed of the gimbal;

[0049] The movement of the tracking device is controlled according to the relative movement speed.

[0050] In this embodiment, when the tracking target is a target obtained from the scene image collected by the tracking device 30, the target movement speed is relative to the coordinate system of the tracking device 30, and no coordinate conversion is required. However, the pan-tilt movement speed needs to be converted to the coordinate system of the tracking device 30 to obtain the pan-tilt relative speed. Based on the conversion relationship between the coordinate system of the tracking device 30 and the coordinate system of the pan-tilt device, the pan-tilt movement speed is converted to obtain the pan-tilt relative speed.

[0051] Since the pan-tilt device 40 moves, the tracking target also moves, so it is necessary to calculate the relative speed of the tracking target based on the target speed and the pan-tilt relative speed, and control the movement of the tracking device 30 according to the relative speed, so that the tracking target is always located in the tracking area, which will reduce the probability of losing the target. The relative speed is the difference between the target speed and the pan-tilt relative speed. The relative speed has a direction, and the direction of the movement of the tracking device 30 can be controlled according to the direction of the relative speed, and the speed value of the tracking device 30 can be controlled according to the value of the relative speed.

[0052] In the above embodiment, when the tracking target is obtained based on the scene image captured by the tracking device, the pan-tilt movement speed is converted into the coordinate system corresponding to the tracking device to obtain the pan-tilt relative speed, and based on the target movement speed and the pan-tilt relative speed, the relative movement speed of the tracking target is obtained. According to the relative movement speed, the movement of the tracking device is controlled so that the tracking target is always located in the tracking area, which can reduce the probability of losing the target and thus improve the accuracy of automatic tracking.

[0053] In some embodiments, when the tracking target is a target acquired according to an image collected by the search device, controlling the movement of the tracking device according to the target movement speed and the pan / tilt movement speed includes:

[0054] Based on the coordinate conversion relationship between the pan-tilt device and the tracking device, the pan-tilt motion speed is converted into the coordinate system corresponding to the tracking device to obtain the pan-tilt relative speed;

[0055] Based on the coordinate conversion relationship between the search device and the tracking device, the target movement speed is converted into the coordinate system corresponding to the tracking device to obtain the target relative speed;

[0056] Calculating the relative motion speed of the tracking target based on the target relative speed and the pan / tilt relative speed;

[0057] The movement of the tracking device is controlled according to the relative movement speed.

[0058] In this embodiment, when the tracking target is a target obtained based on the scene image collected by the search device 20, it is necessary not only to convert the pan-tilt motion speed to the coordinate system of the tracking device 30 to obtain the pan-tilt relative speed, but also to convert the target motion speed to the coordinate system corresponding to the tracking device to obtain the target relative speed. In this way, the pan-tilt relative speed and the target motion speed are in the same coordinate system, which is convenient for calculating the relative motion speed of the tracking target. In this way, when the tracking device 30 continuously tracks the tracking target, the tracking target is always located in the tracking area, which can reduce the probability of losing the target, thereby improving the accuracy of automatic tracking. The relative motion speed is the difference between the target motion speed and the pan-tilt relative speed. The relative motion speed has a direction, and the motion direction of the tracking device 30 can be controlled according to the direction of the relative motion speed, and the speed value of the tracking device 30 can be controlled according to the value of the relative speed.

[0059] In the above embodiment, when the tracking target is obtained based on the scene image captured by the search device, the pan-tilt movement speed is converted into the coordinate system corresponding to the tracking device to obtain the pan-tilt relative speed, and based on the target movement speed and the pan-tilt relative speed, the relative movement speed of the tracking target is obtained. According to the relative movement speed, the movement of the tracking device is controlled so that the tracking target is always located in the tracking area, which can reduce the probability of losing the target and thus improve the accuracy of automatic tracking.

[0060] In some embodiments, the method further comprises:

[0061] Acquire the imaging position of the tracking target in the scene image collected by the search device;

[0062] Based on the coordinate conversion relationship between the search device and the tracking device, the imaging position is converted to the coordinate system corresponding to the tracking device, the target position is determined, and the tracking device is controlled to move to the target position.

[0063] In this embodiment, when the tracking target is a target obtained based on the scene image collected by the search device 20, in order for the tracking device 30 to quickly find the tracking target, it is necessary to obtain the imaging position of the tracking target relative to the search device 20, and then based on the coordinate conversion relationship between the search device 20 and the tracking device 30, the imaging position is converted to the coordinate system between the tracking devices 30 to obtain the target position, so as to directly control the tracking device 30 to move to the target position to track the tracking target. In an optional implementation, this step can be performed in the host computer 10. When there are multiple tracking devices 30, the host computer 10 pre-stores the position correspondence between the search device and each tracking device. The host computer 10 determines the tracking device that matches the search device based on the acquired imaging position and the stored position correspondence, and sends the control instruction containing the target position and the relative motion speed to the tracking device that matches the search device 20, so that the tracking device 30 that matches the search device 20 tracks the tracking target. In an optional implementation, this step can also be implemented in the tracking device 30, which is a tracking device that matches the search device 20. In each search device, the position correspondence relationship with each tracking device can be pre-stored. According to the acquired imaging position and the stored position correspondence relationship, the tracking device that matches the search device is determined, and the relevant information of the tracking target is sent to the tracking device that matches the search device. The tracking device that matches the search device executes the target tracking method based on pan-tilt linkage provided in the embodiment of the present application to track the tracking target. The relevant information includes but is not limited to the imaging position, the coordinate system data of the search device, and the like.

[0064] In the above embodiment, when the tracking target is a target obtained based on a scene image captured by a search device, the target position of the tracking target at the coordinates corresponding to the tracking device is calculated based on the imaging position of the tracking target in the search device. Based on the coordinates of the target position, the tracking device can be controlled to rotate quickly to the target position so that the tracking target is located in the tracking area, which can reduce the probability of losing the target and thus improve the accuracy of automatic tracking.

[0065] In some embodiments, the method further comprises:

[0066] Acquire the imaging size of the tracking target in the image collected by the tracking device;

[0067] The tracking area of ​​the tracking device is adjusted according to the imaging size.

[0068] In an optional implementation, adjusting the tracking area according to the imaging size includes:

[0069] When the imaging size is larger than the size of the tracking area, expanding the tracking area;

[0070] When the imaging size is smaller than the size of the tracking area, the tracking area is reduced.

[0071] In this embodiment, when the tracking device tracks the tracking target, since the scene images collected in real time are different and the tracking target is also moving, the imaging size of the tracking target in the scene images collected in real time may also be different. The imaging size may be relatively large, and the tracking area cannot completely frame the tracking target, or the imaging size is too small, and it is also very small in the tracking area and cannot be seen clearly, etc. Therefore, in order to ensure that the tracking target can be located in the central area of ​​the tracking area and adapt to the size of the tracking area, it is necessary to adaptively adjust the imaging size. The imaging size includes but is not limited to at least one combination of the following: width, height, and area. The size of the tracking area includes but is not limited to at least one combination of the following: width, height, and area. When the imaging size is larger than the size of the tracking area, the tracking area is enlarged so that the tracking area can frame the imaging size; when the imaging size is smaller than the size of the tracking area, the tracking area is reduced to frame the imaging size and the tracking target is located in the central area of ​​the tracking area.

[0072] In an optional implementation, this step may be performed in the host computer 10, which obtains the imaging size of the tracking target in the tracking device 30, adaptively adjusts the tracking area, and sends the adjusted data to the tracking device 30. In an optional implementation, this step may also be performed directly in the tracking device 30, which directly analyzes the imaging size of the tracking target and adaptively adjusts the tracking area.

[0073] In the above embodiment, when continuously tracking a target, the tracking area can be adaptively adjusted according to the imaging size of the target, so that in various scenarios, the tracking area can be matched with tracking targets of different imaging sizes, so that the tracking target is located in the tracking area, which can reduce the probability of losing the target, thereby improving the accuracy of automatic tracking.

[0074] In some embodiments, the method further comprises:

[0075] Acquire the imaging size of the tracking target in the image collected by the tracking device;

[0076] When the imaging size is smaller than a size threshold, controlling the tracking device to reduce the imaging field of view, and controlling the tracking device to increase the movement speed;

[0077] When the imaging size is larger than the size threshold, the tracking device is controlled to expand the imaging field of view, and the tracking device is controlled to slow down the movement speed.

[0078] In this embodiment, when the imaging size is less than the size threshold, the tracking device 30 is controlled to reduce the imaging field of view and speed up the movement speed in order to track the target more accurately. Reducing the imaging field of view means that the focal length of the image acquisition device becomes longer and the field of view angle becomes smaller, so that the tracking target can occupy a larger pixel area on the image sensor, thereby improving the imaging quality of the target and making it easier for the tracking algorithm to identify and lock the target. At the same time, speeding up the movement speed can improve the responsiveness of the tracking device, enabling it to quickly adapt to the movement of the target and maintain stable tracking of the target.

[0079] When the imaging size is larger than the set size threshold, the tracking device is controlled to expand the imaging field of view and slow down the movement speed in order to capture the target in a wider field of view while maintaining the stability and accuracy of the tracking. Expanding the imaging field of view means that the focal length of the image acquisition device becomes shorter, thereby increasing the field of view angle. In this way, capturing images in a wider area allows the tracking device 30 to observe a larger environment or scene. When the field of view is expanded, the speed at which the tracked target moves in the image may slow down relative to the size of the image. Therefore, slowing down the movement speed of the tracking device 30 can help more accurately predict the target's movement trajectory and reduce tracking errors caused by rapid movement.

[0080] In an optional implementation, this step can be executed in the host computer 10, and a corresponding control command is issued to the tracking device 30. For example, when the imaging size is smaller than the size threshold, a control command is issued to the tracking device 30 to control the tracking device 30 to reduce the imaging field of view and control the tracking device 30 to speed up the movement. In an optional implementation, this step can also be executed in the tracking device 30, and the tracking device 30 analyzes the above steps by itself and controls its own movement.

[0081] In the above embodiment, the imaging field of view and movement speed of the tracking device are adaptively adjusted according to the imaging size, which can improve the imaging quality of the target, make it easier for the tracking algorithm to identify and lock the target, reduce the probability of losing the target, and thus improve the accuracy of automatic tracking.

[0082] In some embodiments, the method further comprises:

[0083] Obtaining the tracking status of the tracking target;

[0084] When the tracking state indicates a normal tracking state, controlling the pan / tilt device and the tracking device to continue tracking;

[0085] When the tracking status indicates a lost state, the pan-tilt device and the tracking device are controlled to stop tracking.

[0086] In this embodiment, the tracking state includes a normal tracking state and a lost state. The normal tracking state indicates that the target can be continuously tracked during the tracking process. The lost state indicates that the target cannot be continuously tracked during the tracking process. In an optional implementation, this step can be performed in the host computer 10. When the tracking state indicates a normal tracking state, the host computer 10 sends a first control command to the pan-tilt device 40 and the tracking device 30 to control the pan-tilt device 40 and the tracking device 30 to continue tracking and report that the tracking target is not lost; when the tracking state indicates a lost state, the host computer 10 sends a second control command to the pan-tilt device 40 and the tracking device 30 to control the pan-tilt device 40 and the tracking device 30 to stop tracking and report that the tracking target is lost. In an optional implementation, this step can be executed in the tracking device 30. When the tracking status indicates a normal tracking status, the tracking device 30 sends a first control command to the gimbal device 40, controls the gimbal device 40 to continue tracking based on the first control command, controls itself to continue tracking, and reports that the tracking target is not lost; when the tracking status indicates a lost status, the tracking device 30 sends a second control command to the gimbal device 40 based on the second control command, controls the gimbal device 40 to stop tracking, controls itself to stop tracking, and reports that the tracking target is lost.

[0087] In an optional implementation, the acquiring the tracking state of the tracking target includes:

[0088] Obtaining a tracking score of the tracking target;

[0089] When the tracking score is greater than or equal to a preset score threshold, determining that the tracking target is in a normal tracking state;

[0090] When the tracking score is less than a preset score threshold, it is determined that the tracking target is in a lost state.

[0091] In this embodiment, when the tracking target is continuously tracked, a tracking score is calculated based on the tracking data, wherein the tracking data includes but is not limited to whether the tracking target is tracked in multiple frames of images, the image quality of the tracking target, and the like.

[0092] In the above embodiment, during the process of tracking the target, the tracking state of the target is analyzed in real time, and different actions are performed according to different tracking states, thereby improving the accuracy of automatic tracking.

[0093] like Figure 4 As shown, Figure 4 The flowchart is a target tracking method based on pan-tilt linkage in an application scenario. In the application scenario, the target tracking system based on pan-tilt linkage includes at least a tracking device 30 and a pan-tilt device 40. The flowchart includes:

[0094] S41: Acquire a tracking target in a scene image collected by a tracking device.

[0095] S42: Obtain the target movement speed of the tracking target relative to the tracking device.

[0096] S43, obtaining the pan / tilt movement speed of the pan / tilt device connected to the tracking device.

[0097] S44, calculating the relative motion speed of the tracking target based on the target motion speed and the pan / tilt motion speed.

[0098] S45. Control the movement of the tracking device according to the relative movement speed.

[0099] In the above steps, S42 and S43 are executed in no particular order. Figure 4 The flowchart shown in can be applied to the tracking device 30, and can also be applied to the host computer 10. The method in one or more of the above embodiments can also be applied to the flowchart in this application scenario, which will not be repeated here.

[0100] like Figure 5 As shown, Figure 5 This is a flow chart of a target tracking method based on pan-tilt linkage in another application scenario. In this application scenario, the target tracking system based on pan-tilt linkage includes at least a search device 20, a tracking device 30 and a pan-tilt device 40. The flow chart includes:

[0101] S51: Acquire a tracking target in a scene image collected by a search device.

[0102] S52, obtaining the target movement speed of the tracking target relative to the collection device, and based on the coordinate conversion relationship between the search device and the tracking device, converting the target movement speed to the coordinate system corresponding to the tracking device to obtain the target relative speed.

[0103] S53, obtaining the pan / tilt movement speed of the pan / tilt device connected to the tracking device, and based on the coordinate conversion relationship between the pan / tilt device and the tracking device, converting the pan / tilt movement speed into the coordinate system corresponding to the tracking device to obtain the pan / tilt relative speed.

[0104] S54: Calculate the relative motion speed of the tracked target based on the target relative speed and the pan / tilt relative speed.

[0105] S55. Control the movement of the tracking device according to the relative movement speed.

[0106] In the above steps, S52 and S53 may be executed in any order. Figure 5The flowchart shown in can be applied to the tracking device 30, and can also be applied to the host computer 10. The method in one or more of the above embodiments can also be applied to the flowchart in this application scenario, which will not be repeated here.

[0107] On the other hand, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the target tracking method based on pan-tilt linkage described in any embodiment of the present application.

[0108] Among them, in the computer program product, an optional implementation form of the program module architecture of the computer program that implements each step of the target recognition method can be a target tracking device based on pan-tilt linkage.

[0109] See also Figure 6 An embodiment of the present application provides a target tracking device based on pan-tilt linkage, including: an acquisition module 61, used to acquire the tracking target; the acquisition module 61 is also used to acquire the target movement speed of the tracking target relative to the search device or tracking device that collects the tracking target; the acquisition module 61 is also used to acquire the pan-tilt movement speed of the pan-tilt device connected to the tracking device; a control module 62 is used to control the movement of the tracking device according to the target movement speed and the pan-tilt movement speed.

[0110] Optionally, the acquisition module 61 is further used for at least one of the following:

[0111] Taking a target meeting the tracking condition in the scene image collected by the search device or the tracking device as the tracking target;

[0112] The target corresponding to the target designation operation based on the displayed scene image is determined as the tracking target.

[0113] Optionally, in the case where the tracking target is a target acquired according to an image collected by the tracking device, the control module 62 is further configured to:

[0114] Based on the coordinate conversion relationship between the pan-tilt device and the tracking device, the pan-tilt motion speed is converted into the coordinate system corresponding to the tracking device to obtain the pan-tilt relative speed;

[0115] Calculating the relative speed of the target based on the target speed and the relative speed of the gimbal;

[0116] The movement of the tracking device is controlled according to the relative movement speed.

[0117] Optionally, in the case where the tracking target is a target acquired based on an image collected by the search device, the control module 62 is further configured to:

[0118] Based on the coordinate conversion relationship between the pan-tilt device and the tracking device, the pan-tilt motion speed is converted into the coordinate system corresponding to the tracking device to obtain the pan-tilt relative speed;

[0119] Based on the coordinate conversion relationship between the search device and the tracking device, the target movement speed is converted into the coordinate system corresponding to the tracking device to obtain the target relative speed;

[0120] Calculating the relative motion speed of the tracking target based on the target relative speed and the pan / tilt relative speed;

[0121] The movement of the tracking device is controlled according to the relative movement speed.

[0122] Optionally, the control module 62 is further configured to:

[0123] Acquire the imaging position of the tracking target in the scene image collected by the search device;

[0124] Based on the coordinate conversion relationship between the search device and the tracking device, the imaging position is converted to the coordinate system corresponding to the tracking device, the target position is determined, and the movement of the tracking device is controlled according to the target position.

[0125] Optionally, the control module 62 is further configured to:

[0126] Acquire the imaging size of the tracking target in the image collected by the tracking device;

[0127] The tracking area of ​​the tracking device is adjusted according to the imaging size.

[0128] Optionally, the control module 62 is further configured to:

[0129] When the imaging size is larger than the size of the tracking area, expanding the tracking area;

[0130] When the imaging size is smaller than the size of the tracking area, the tracking area is reduced.

[0131] Optionally, the control module 62 is further configured to:

[0132] Acquire the imaging size of the tracking target in the image collected by the tracking device;

[0133] When the imaging size is smaller than a size threshold, controlling the tracking device to reduce the imaging field of view, and controlling the tracking device to increase the movement speed;

[0134] When the imaging size is larger than the size threshold, the tracking device is controlled to expand the imaging field of view, and the tracking device is controlled to slow down the movement speed.

[0135] Optionally, the control module 62 is further configured to:

[0136] Obtaining the tracking status of the tracking target;

[0137] When the tracking state indicates a normal tracking state, controlling the pan / tilt device and the tracking device to continue tracking;

[0138] When the tracking status indicates a lost state, the pan-tilt device and the tracking device are controlled to stop tracking.

[0139] Optionally, the control module 62 is further configured to:

[0140] Obtaining a tracking score of the tracking target;

[0141] When the tracking score is greater than or equal to a preset score threshold, determining that the tracking target is in a normal tracking state;

[0142] When the tracking score is less than a preset score threshold, it is determined that the tracking target is in a lost state.

[0143] It can be understood by those skilled in the art that Figure 6 The structure of the target tracking device based on pan-tilt linkage does not constitute a limitation on the target tracking device based on pan-tilt linkage, and the various modules can be implemented in whole or in part by software, hardware, and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. In other embodiments, the target tracking device based on pan-tilt linkage may include more or fewer modules than shown in the figure.

[0144] See also Figure 7 In another aspect of the embodiment of the present application, a host computer 10 is provided, including a processor 13 and a memory 14, wherein the memory 14 stores a computer program, and when the computer program is executed by the processor, the processor 13 executes the steps of the target tracking method based on the pan-tilt linkage provided in any of the above embodiments of the present application. The host computer 10 may include a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a terminal device (e.g., a mobile phone, etc.), a wearable device (e.g., a pair of smart glasses or a smart watch), and other computer devices.

[0145] The processor 13 is the control center, which uses various interfaces and lines to connect various parts of the entire computer device, and executes various functions of the computer device and processes data by running or executing software programs and / or modules stored in the memory 14, and calling data stored in the memory 14. Optionally, the processor 13 may include one or more processing cores; preferably, the processor 13 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 13.

[0146] The memory 14 can be used to store software programs and modules. The processor 13 executes various functional applications and data processing by running the software programs and modules stored in the memory 14. The memory 14 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 14 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 14 can also include a memory processor to provide the processor 13 with access to the memory 14.

[0147] On the other hand, the embodiment of the present application Figure 8 As shown, a tracking device 30 is also provided, the tracking device 30 includes a storage device 301 and a processing device 302, the storage device 301 stores a computer program, and when the computer program is executed by the processing device 302, the processing device 302 executes the target tracking method based on pan-tilt linkage described in any embodiment of the present application. The storage device 301 has the same function as the memory 14, and the processing device 302 has the same function as the processor 13.

[0148] In another aspect, an embodiment of the present application provides a target tracking system based on pan-tilt linkage, the system comprising a tracking device and a pan-tilt device as described in any embodiment of the present application; or

[0149] The system includes the tracking device, search device and pan-tilt device described in any embodiment of the present application, wherein the search device is used to obtain the tracking target, and the pan-tilt device is used to connect with the tracking device to assist the movement of the tracking device.

[0150] On the other hand, an embodiment of the present application provides a target tracking system based on pan-tilt linkage, the system comprising a host computer, a tracking device and a pan-tilt device as described in any embodiment of the present application, the host computer is used to communicate with the tracking device and the pan-tilt device, the tracking device is used to obtain the tracking target and track the tracking target according to the control instructions issued by the host computer, and the pan-tilt device is used to connect with the tracking device to assist the movement of the tracking device; or

[0151] The system includes the host computer, search device, tracking device and pan-tilt device described in any embodiment of the present application, the host computer is used to communicate with the search device, the tracking device and the pan-tilt device, the search device is used to obtain the tracking target, the tracking device is used to track the tracking target according to the control instructions issued by the host computer, and the pan-tilt device is used to connect with the tracking device to assist the movement of the tracking device.

[0152] On the other hand, an embodiment of the present application further provides a storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the target tracking method based on pan-tilt linkage provided in any of the above embodiments of the present application.

[0153] Those skilled in the art can understand that all or part of the processes in the methods provided in the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0154] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A target tracking method based on pan-tilt linkage, characterized in that: include: Get the tracking target; Acquire the target motion speed of the tracked target relative to the search device or the tracking device that collects the tracked target; Obtaining the pan / tilt movement speed of the pan / tilt device connected to the tracking device; The movement of the tracking device is controlled according to the target movement speed and the pan / tilt movement speed.

2. The target tracking method based on pan-tilt linkage according to claim 1, characterized in that: The obtaining of the tracking target includes at least one of the following: Taking a target meeting the tracking condition in the scene image collected by the search device or the tracking device as the tracking target; The target corresponding to the target designation operation based on the displayed scene image is determined as the tracking target.

3. The target tracking method based on pan-tilt linkage according to claim 1, characterized in that: In the case where the tracking target is a target acquired according to an image collected by the tracking device, controlling the movement of the tracking device according to the movement speed of the target and the movement speed of the pan / tilt head includes: Based on the coordinate conversion relationship between the pan-tilt device and the tracking device, the pan-tilt motion speed is converted into the coordinate system corresponding to the tracking device to obtain the pan-tilt relative speed; Calculating the relative speed of the target based on the target speed and the relative speed of the gimbal; The movement of the tracking device is controlled according to the relative movement speed.

4. The target tracking method based on pan-tilt linkage according to claim 1, characterized in that: In the case where the tracking target is a target acquired according to an image collected by the search device, controlling the movement of the tracking device according to the movement speed of the target and the movement speed of the pan / tilt head includes: Based on the coordinate conversion relationship between the pan-tilt device and the tracking device, the pan-tilt motion speed is converted into the coordinate system corresponding to the tracking device to obtain the pan-tilt relative speed; Based on the coordinate conversion relationship between the search device and the tracking device, the target movement speed is converted into the coordinate system corresponding to the tracking device to obtain the target relative speed; Calculating the relative motion speed of the tracking target based on the target relative speed and the pan / tilt relative speed; The movement of the tracking device is controlled according to the relative movement speed.

5. The target tracking method based on pan-tilt linkage as claimed in claim 4, characterized in that: The method further comprises: Acquire the imaging position of the tracking target in the scene image collected by the search device; Based on the coordinate conversion relationship between the search device and the tracking device, the imaging position is converted to the coordinate system corresponding to the tracking device, the target position is determined, and the movement of the tracking device is controlled according to the target position.

6. The target tracking method based on pan-tilt linkage according to claim 1, characterized in that: The method further comprises: Acquire the imaging size of the tracking target in the image collected by the tracking device; The tracking area of ​​the tracking device is adjusted according to the imaging size.

7. The target tracking method based on pan-tilt linkage according to claim 6, characterized in that: The adjusting the tracking area according to the imaging size comprises: When the imaging size is larger than the size of the tracking area, expanding the tracking area; When the imaging size is smaller than the size of the tracking area, the tracking area is reduced.

8. The target tracking method based on pan-tilt linkage according to claim 1, characterized in that: The method further comprises: Acquire the imaging size of the tracking target in the image collected by the tracking device; When the imaging size is smaller than a size threshold, controlling the tracking device to reduce the imaging field of view, and controlling the tracking device to increase the movement speed; When the imaging size is larger than the size threshold, the tracking device is controlled to expand the imaging field of view, and the tracking device is controlled to slow down the movement speed.

9. The target tracking method based on pan-tilt linkage according to claim 1, characterized in that: The method further comprises: Obtaining the tracking status of the tracking target; When the tracking state indicates a normal tracking state, controlling the pan / tilt device and the tracking device to continue tracking; When the tracking status indicates a lost state, the pan-tilt device and the tracking device are controlled to stop tracking.

10. The target tracking method based on pan / tilt linkage according to claim 9, characterized in that: The obtaining the tracking state of the tracking target includes: Obtaining a tracking score of the tracking target; When the tracking score is greater than or equal to a preset score threshold, determining that the tracking target is in a normal tracking state; When the tracking score is less than a preset score threshold, it is determined that the tracking target is in a lost state.

11. A tracking device, characterized in that: The tracking device includes a storage device and a processing device, wherein the storage device stores a computer program, and when the computer program is executed by the processing device, the processing device executes the target tracking method based on pan-tilt linkage as claimed in any one of claims 1 to 10.

12. A host computer, characterized in that: It includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the target tracking method based on pan-tilt linkage as described in any one of claims 1 to 10.

13. A target tracking system based on pan-tilt linkage, characterized in that: The system comprises the tracking device and the pan-tilt device as claimed in claim 11; or The system comprises the tracking device, the search device and the pan-tilt device as claimed in claim 11, wherein the search device is used to obtain the tracking target, and the pan-tilt device is used to connect with the tracking device to assist the movement of the tracking device.

14. A target tracking system based on pan-tilt linkage, characterized in that: The system comprises a host computer, a tracking device and a pan-tilt device as claimed in claim 12, wherein the host computer is used to communicate with the tracking device and the pan-tilt device, the tracking device is used to obtain the tracking target and track the tracking target according to the control instructions issued by the host computer, and the pan-tilt device is used to connect with the tracking device to assist the movement of the tracking device; or The system includes a host computer, a search device, a tracking device and a pan-tilt device as described in claim 12, wherein the host computer is used to communicate with the search device, the tracking device and the pan-tilt device, the search device is used to obtain the tracking target, the tracking device is used to track the tracking target according to the control instructions issued by the host computer, and the pan-tilt device is used to connect with the tracking device to assist the movement of the tracking device.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the target tracking method based on pan-tilt linkage as described in any one of claims 1 to 10 is implemented.

Images