Target tracking shooting method, device and storage medium

Abstract

This application discloses a target tracking and shooting method, device, and storage medium, relating to the field of multi-axis gimbal camera technology and applied to a gimbal control system. The method includes: if a target tracking process is triggered, selecting a target tracking source based on priority and the confidence level of the target detection results corresponding to each tracking source, wherein the target tracking source is a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source; reading the tracking object position output by the target tracking process corresponding to the target tracking source, wherein the tracking object position is the position of the tracking object in the video frame corresponding to the target tracking source; determining the pose adjustment parameters of the multi-axis gimbal based on the tracking object position, and controlling the rotation of the multi-axis gimbal based on the pose adjustment parameters. This solves the technical problem in the prior art where the inability to effectively switch alternative sources when a single tracking source detection is unreliable leads to tracking interruption, thereby improving tracking accuracy and tracking stability.

Description

Technical Field

[0001] This application relates to the field of multi-axis gimbal camera technology, and in particular to a target tracking and shooting method, device, and storage medium. Background Technology

[0002] When viewers use mobile devices to live stream sports events, the process typically involves directly holding the device and automatically tracking and filming the athlete or moving object. However, in real-world live streaming scenarios, the target is often in a complex environment with high-speed movement and frequent obstructions. A single video source can easily lead to unstable target detection or even target loss, resulting in discontinuous tracking, image shifting, or interruptions, thus affecting the smoothness and stability of the live stream. Summary of the Invention

[0003] The main purpose of this application is to provide a target tracking and imaging method, device and storage medium, which aims to solve the technical problem in the prior art that the inability to effectively switch to alternative sources when the detection of a single tracking source is unreliable, resulting in tracking interruption.

[0004] To achieve the above objectives, this application proposes a target tracking and shooting method applied to a gimbal control system. The gimbal control system includes a multi-axis gimbal, a mobile terminal, and a data processor. The multi-axis gimbal is equipped with a telephoto lens, a wide-angle lens, and a mobile terminal fixing component. The mobile terminal is detachably connected to the multi-axis gimbal via the mobile terminal fixing component. The target tracking and shooting method includes:

[0005] If the target tracking process is triggered, the target tracking source is selected according to the priority and the confidence level of the target detection result corresponding to each tracking source. The target tracking source is a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source.

[0006] Read the tracking object position output by the target tracking process corresponding to the target tracking source, wherein the tracking object position is the position of the tracking object in the video frame corresponding to the target tracking source;

[0007] The pose adjustment parameters of the multi-axis gimbal are determined based on the position of the tracked object, and the rotation of the multi-axis gimbal is controlled based on the pose adjustment parameters.

[0008] In some embodiments, the step of selecting a target tracking source based on priority and the confidence level of the target detection result corresponding to each tracking source includes:

[0009] Obtain the confidence level of the target detection result corresponding to each of the tracking sources;

[0010] Determine the priority of each of the aforementioned tracking sources;

[0011] The confidence level of the target detection result corresponding to each tracking source is determined in descending order of priority to determine whether it is greater than or equal to the first preset threshold.

[0012] If the initial judgment result is yes, the tracking source corresponding to the judgment result is determined as the target tracking source;

[0013] The step of determining whether to proceed to the step of judging whether the confidence level of the target detection result corresponding to the non-highest-level tracking source is greater than or equal to the first preset threshold is determined by the judgment result of the previous priority tracking source corresponding to the non-highest-level tracking source.

[0014] In some embodiments, the step of determining the priority of each of the tracking sources includes:

[0015] In response to the user's scene selection, determine the current scene category;

[0016] The priority of each tracking source under the current scene category is determined based on the current scene category and the preset priority of each tracking source under each scene category.

[0017] In some embodiments, the step of determining the pose adjustment parameters of the multi-axis gimbal based on the position of the tracked object includes:

[0018] Calculate the offset between the position of the tracked object and the center of the image of the target tracking source;

[0019] Obtain the current field of view parameters of the multi-axis gimbal, and determine the corresponding pose adjustment parameters based on the current field of view parameters and the offset.

[0020] In some embodiments, the step of determining the corresponding pose adjustment parameters based on the current field of view parameter and the offset includes:

[0021] The current horizontal field of view and the current vertical field of view are determined based on the current field of view parameters, and the horizontal offset vector and the vertical offset vector are determined based on the offset amount;

[0022] Based on the current horizontal field of view and the horizontal offset vector, calculate the corresponding yaw angle adjustment vector, and based on the current vertical field of view and the vertical offset vector, calculate the corresponding pitch angle adjustment vector;

[0023] The yaw angle adjustment vector and the pitch angle adjustment vector are determined as the attitude adjustment parameters.

[0024] In some embodiments, after the steps of determining the pose adjustment parameters of the multi-axis gimbal based on the position of the tracked object and controlling the rotation of the multi-axis gimbal based on the pose adjustment parameters, the method further includes:

[0025] Based on the target tracking source, a tracking algorithm is used to control the multi-axis gimbal in order to track and capture images of the target.

[0026] In some embodiments, when the target tracking source is a mobile terminal video source or a telephoto lens video source, the step of controlling the multi-axis gimbal using a tracking algorithm based on the target tracking source to track and capture the target includes:

[0027] Read the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source;

[0028] Based on the tracking object data corresponding to the current frame and the tracking object data corresponding to multiple historical frames, the motion trajectory of the tracking object is predicted to obtain the predicted motion trajectory of the tracking object in the next frame.

[0029] Based on the tracking object data corresponding to the current frame and the motion trajectory prediction, the pose adjustment parameters of the multi-axis gimbal are generated, and the rotation of the multi-axis gimbal is controlled according to the pose adjustment parameters.

[0030] The continuous video frames include the current frame and multiple historical frames, and the tracking object data includes the tracking object location and the tracking object data confidence level.

[0031] In some embodiments, before the step of predicting the motion trajectory of the tracked object based on the tracked object data corresponding to the current frame and the tracked object data corresponding to multiple historical frames, the method further includes:

[0032] If the confidence level of the tracking object data corresponding to the current frame is less than the second preset threshold or the tracking object data corresponding to the current frame is empty, read the first candidate tracking object data and the second candidate tracking object data corresponding to the current frame output by the target tracking process that is not the target tracking source;

[0033] Based on the first candidate tracking object data and the second candidate tracking object data, switch the target tracking source and return to the step of reading the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source;

[0034] Wherein, the first candidate tracking object data is the tracking object data corresponding to the tracking source with higher priority among the other tracking sources besides the target tracking source, and the second candidate tracking object data is the tracking object data corresponding to the tracking source with lower priority among the other tracking sources besides the target tracking source.

[0035] In some embodiments, the step of switching the target tracking source based on the first candidate tracking object data and the second candidate tracking object data includes:

[0036] The confidence level of the first candidate detection result is determined based on the data of the first candidate tracking object.

[0037] Compare the confidence level of the first alternative detection result with the value of the second preset threshold;

[0038] If the confidence level of the first candidate detection result is greater than or equal to the second preset threshold, the tracking source corresponding to the first candidate tracking object data is switched to the target tracking source.

[0039] If the confidence level of the first candidate detection result is less than the second preset threshold or the data of the second candidate tracking object is empty, the confidence level of the second candidate detection result is determined based on the data of the second candidate tracking object.

[0040] Compare the confidence level of the second alternative detection result with the value of the second preset threshold;

[0041] If the confidence level of the second alternative detection result is greater than or equal to the second preset threshold, the tracking source corresponding to the second alternative tracking object data is switched to the target tracking source.

[0042] In some embodiments, when the target tracking source is a wide-angle lens video source, the step of controlling the multi-axis gimbal based on the target tracking source using a tracking algorithm to track and capture the target includes:

[0043] Read the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source, and read the first alternative tracking object data and the second alternative tracking object data corresponding to the continuous video frames output by the target tracking process that are not corresponding to the target tracking source.

[0044] If the detection result of either the first candidate tracking object data or the second candidate tracking object data is not empty, the target tracking source is switched, and based on the switched target tracking source, the multi-axis gimbal is controlled by a tracking algorithm to track and capture the target.

[0045] Furthermore, to achieve the above objectives, this application also proposes a target tracking and imaging device, which includes:

[0046] The tracking source determination module is used to select a target tracking source based on priority and the confidence level of the target detection result corresponding to each tracking source if the target tracking process is triggered. The target tracking source is a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source.

[0047] The position determination module is used to read the position of the tracked object output by the target tracking process corresponding to the target tracking source, wherein the position of the tracked object is the position of the tracked object in the video frame corresponding to the target tracking source;

[0048] The control module is used to determine the pose adjustment parameters of the multi-axis gimbal based on the position of the tracked object, and control the rotation of the multi-axis gimbal based on the pose adjustment parameters.

[0049] In addition, to achieve the above objectives, this application also proposes a target tracking and imaging device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the target tracking and imaging method described above.

[0050] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the target tracking and shooting method described above.

[0051] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the target tracking and shooting method described above.

[0052] One or more technical solutions proposed in this application have at least the following technical effects: The target tracking and shooting method is applied to a gimbal control system, which includes a multi-axis gimbal (equipped with a telephoto lens and a wide-angle lens), a mobile terminal, and a data processor. The mobile terminal can be connected to the gimbal through a dedicated fixed component to form a flexible and scalable shooting platform. If a target tracking process is triggered, the most suitable video source is selected from the mobile terminal video source, the telephoto lens video source, or the wide-angle lens video source based on priority and the confidence level of the target detection results corresponding to each tracking source. Combining the advantages of different cameras, the most suitable video source for the current scene is selected according to actual needs, which can avoid tracking interruption caused by the failure of a single device. The position of the tracked object output by the target tracking process corresponding to the target tracking source is read, where the position of the tracked object is the position of the tracked object in the video frame corresponding to the target tracking source, which can provide a basis for subsequent adjustment of the gimbal attitude. The pose adjustment parameters of the multi-axis gimbal are determined based on the position of the tracked object, and the rotation of the multi-axis gimbal is controlled based on these parameters to ensure that the tracked object is always centered in the image of the target tracking source. This solves the technical problem in existing technologies where the inability to effectively switch to alternative sources when a single tracking source is unreliable leads to tracking interruption, and improves tracking accuracy and stability. The target tracking and shooting method disclosed in this application integrates video from three sources: a mobile terminal, a telephoto lens, and a wide-angle lens, covering different scene requirements. After tracking is triggered, the optimal tracking source is selected as the target tracking source by combining preset priorities and the real-time target detection confidence of each tracking source. Based on the position of the tracked object output by the target tracking process corresponding to the target tracking source, the pose adjustment parameters of the multi-axis gimbal are calculated and the multi-axis gimbal is driven to rotate, ensuring that the target is always centered in the image. When a single tracking source fails, it can automatically switch to other tracking sources to avoid tracking interruption, significantly improving tracking stability and continuity in complex scenes. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating an embodiment of the target tracking and imaging method of this application.

[0056] Figure 2 A schematic diagram of the entire device involved in the target tracking of this application;

[0057] Figure 3 An assembly diagram of a mobile terminal and a multi-axis gimbal provided in this application;

[0058] Figure 4 An assembly diagram of another mobile terminal and multi-axis gimbal provided in this application;

[0059] Figure 5 A schematic diagram illustrating the transmission process of the various test results provided in this application;

[0060] Figure 6 A schematic diagram of the first test result provided for this application;

[0061] Figure 7 A flowchart illustrating another target tracking and imaging method provided in this application;

[0062] Figure 8 A flowchart illustrating yet another target tracking and imaging method provided in this application;

[0063] Figure 9 This is a schematic diagram of the module structure of the target tracking and imaging device according to an embodiment of this application;

[0064] Figure 10 This is a schematic diagram of the device structure of the hardware operating environment involved in the target tracking and shooting method in the embodiments of this application.

[0065] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0066] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0067] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0068] The main solution of this application embodiment is as follows: if the target tracking process is triggered, a target tracking source is selected according to the priority and the confidence level of the target detection result corresponding to each tracking source, wherein the target tracking source is a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source; the tracking object position output by the target tracking process corresponding to the target tracking source is read, wherein the tracking object position is the position of the tracking object in the video frame corresponding to the target tracking source; the pose adjustment parameters of the multi-axis gimbal are determined according to the tracking object position, and the multi-axis gimbal is controlled to rotate based on the pose adjustment parameters.

[0069] In this embodiment, for ease of description, the gimbal control system will be used as the execution subject in the following description.

[0070] In current technology, when viewers use mobile devices to live stream sports events, the process primarily involves directly holding the device and automatically tracking and filming athletes or moving objects. However, in real-world live streaming scenarios, the target is often in a complex environment with high-speed movement and frequent obstructions. A single video source can easily lead to unstable target detection or even target loss, resulting in discontinuous tracking, image shifting, or interruptions, thus affecting the smoothness and stability of the live stream.

[0071] This application provides a solution that integrates video from mobile terminals, telephoto lenses, and wide-angle lenses to cover various scenario requirements. After tracking is triggered, the optimal tracking source is selected based on preset priorities and the real-time target detection confidence of each tracking source. The tracking object position is then calculated based on the target tracking process output from the corresponding tracking source, and the attitude adjustment parameters of the multi-axis gimbal are calculated to drive its rotation, ensuring the target remains centered in the frame. When a single tracking source fails, it automatically switches to another, preventing tracking interruptions and significantly improving tracking stability and continuity in complex scenarios.

[0072] It should be noted that the executing entity in this embodiment is a gimbal control system or similar device capable of performing the aforementioned functions. The following description uses a gimbal control system as an example to illustrate this embodiment and the subsequent embodiments.

[0073] Based on this, this application provides a target tracking and shooting method applied to a gimbal control system. The gimbal control system includes a multi-axis gimbal, a mobile terminal, and a data processor. The multi-axis gimbal is equipped with a telephoto lens, a wide-angle lens, and a mobile terminal fixing component. The mobile terminal is detachably connected to the multi-axis gimbal through the mobile terminal fixing component.

[0074] Reference Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the target tracking and imaging method of this application.

[0075] In this embodiment, the target tracking and shooting method includes steps 101-103:

[0076] Step 101: If the target tracking process is triggered, select the target tracking source according to the priority and the confidence of the target detection result corresponding to each tracking source. The target tracking source is a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source.

[0077] The target tracking and shooting method provided in this application is applied to a gimbal control system. It enables continuous and stable tracking of the target object by intelligently selecting and switching different video sources (such as mobile terminal video sources, telephoto lens video sources, or wide-angle lens video sources). The gimbal control system consists of a multi-axis gimbal, a mobile terminal, and a data processor. The multi-axis gimbal is equipped with both telephoto and wide-angle lenses, and the mobile terminal can be connected and fixed to the gimbal via a mobile terminal fixing component. The mobile terminal fixing component is a mechanical device used to securely mount the mobile terminal on the multi-axis gimbal, ensuring its stable position and allowing it to rotate with the multi-axis gimbal. The data processor processes the target detection results from different video sources, selects the target tracking source, and generates pose adjustment parameters for the multi-axis gimbal. In this application, the data processor can be located within the multi-axis gimbal or in the mobile terminal.

[0078] Specifically, the mobile terminal is a smart terminal equipped with a camera, such as a mobile phone, which can be mounted on a multi-axis gimbal. Optionally, when the mobile terminal is a mobile phone, the schematic diagram of the overall device involved in the target tracking and shooting in this application is as follows. Figure 2As shown, the actual application may differ. When a mobile phone is mounted on a multi-axis gimbal, the phone can participate in tracking, shooting, live streaming, and storage in two ways: When the phone has a working mobile terminal camera, the camera participates in target detection and target tracking scene shooting, supporting live streaming or local storage; when the phone has no camera or the camera is not working properly, target detection relies entirely on the multi-axis gimbal's built-in camera. The images captured by the multi-axis gimbal's camera are transmitted to the phone for live streaming or storage. Alternatively, storage can be performed locally on the multi-axis gimbal. These two methods can automatically switch based on the presence and working status of the mobile terminal camera, or can be customized by the user. The multi-axis gimbal of this application is at least a two-axis gimbal, i.e., a gimbal including a yaw axis and a pitch axis. The yaw axis of the multi-axis gimbal rotates around the Z-axis to achieve horizontal tracking of the object, and the pitch axis rotates around the Y-axis to achieve vertical tracking of the object. When the multi-axis gimbal is a three-axis gimbal, it also includes a roll axis. The roll axis rotates around the X-axis, which helps maintain the phone's real-time parallelism with the ground, improving shooting stability. The shooting effect is better when the roll axis is present. This application's multi-axis gimbal is equipped with two lenses: a wide-angle lens and a telephoto lens. Specifically, the telephoto lens has a narrow angle of view, making it more suitable for shooting distant scenes and improving the tracking accuracy of distant objects; the wide-angle lens has a wide angle of view, suitable for shooting global scenes. When the target is lost within the field of view of the mobile terminal lens or the telephoto lens, a global search and adaptive tracking can be performed. The communication method between the mobile phone and the multi-axis gimbal is unrestricted and can be achieved via Universal Serial Bus (USB) wired, Bluetooth, Wi-Fi, or other communication protocols.

[0079] Before performing target detection on the tracked object, the mobile terminal (such as a mobile phone) needs to be fixed on a multi-axis gimbal, such as... Figure 2 As shown. Figure 2 The mobile terminal shown is placed flat and fixed on a multi-axis gimbal, which is only one embodiment. This application can realize the installation and placement of a mobile terminal on a multi-axis gimbal at any angle, as long as the mobile terminal lens is ensured to be perpendicular to the multi-axis gimbal. For example, as shown... Figure 3 and Figure 4 The installation shown is also applicable. Figure 3 and Figure 4 This is a schematic diagram of the assembly of a mobile terminal and a multi-axis gimbal.

[0080] To achieve the tracking function for objects at long distances, this application fully utilizes the built-in camera of a mobile terminal (such as a mobile phone) in its hardware design, combining it with the telephoto and wide-angle lenses on a multi-axis gimbal. The three-axis gimbal and the mobile terminal work together to perform distributed target detection, detect and track objects, and obtain target detection results corresponding to each tracking source. The target detection results corresponding to each tracking source include a first detection result, a second detection result, and a third detection result.

[0081] The first detection result is the output of the wide-angle lens on the multi-axis gimbal for detecting the tracked object, including the first center coordinates of the tracked object, the first scale data of the tracked object, and the confidence score of the first detection result. The second detection result is the output of the telephoto lens on the multi-axis gimbal for detecting the tracked object, including the second center coordinates of the tracked object, the second scale data of the tracked object, and the confidence score of the second detection result. The third detection result is the output of the mobile terminal lens on the mobile terminal, including the third center coordinates of the tracked object, the third scale data of the tracked object, and the confidence score of the third detection result. Specifically, the first detection result is based on target detection using the wide-angle video source captured by the wide-angle lens of the multi-axis gimbal; the second detection result is based on target detection using the telephoto video source captured by the telephoto lens of the multi-axis gimbal; and the third detection result is based on target detection using the mobile terminal video source captured by the mobile terminal lens on the mobile terminal. The tracked object is an object that needs to be continuously filmed, such as an athlete, vehicle, or soccer ball, and its position changes over time.

[0082] The above priorities are preset tracking source selection rules used to quickly determine the target tracking source among multiple video sources (mobile terminal, telephoto lens, wide-angle lens). The target detection result confidence score is the reliable probability of the target detection model's target detection result for a video frame; its value range is typically 0 to 1, and it can be used to quantitatively evaluate the reliability of the target detection result. The target tracking source is the video input source ultimately selected for target tracking, which can be a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source. A mobile terminal video source is a video stream acquired through a mobile terminal lens; a telephoto lens video source is a video stream acquired through a telephoto lens of a multi-axis gimbal; and a wide-angle lens video source is a video stream acquired through a wide-angle lens of a multi-axis gimbal.

[0083] Optionally, target detection results from different cameras are acquired. Specifically, detection results for three independent tracking objects are acquired, including a first detection result, a second detection result, and a third detection result. After the mobile terminal independently completes the target detection task from its respective viewpoint, it transmits the third detection result to the multi-axis gimbal via a communication protocol. This eliminates the need to transmit the complete detected image, effectively saving transmission bandwidth, improving transmission efficiency, and achieving low-latency transmission. The flowchart for the transmission of each detection result can be shown below. Figure 5As shown, the gimbal tracking module needs to receive each detection result. The first and second detection results on the multi-axis gimbal can be directly obtained from the local detection module, while the third detection result of the mobile terminal is obtained through communication protocol transmission. This application does not restrict the communication method and supports USB wired, Bluetooth, Wi-Fi and other protocols.

[0084] Optionally, in the PTZ control system, after receiving instructions from the user or the system, the system triggers a target tracking process, analyzes the target detection results of each video source, and obtains the confidence level of the target detection results for each tracking source. Based on preset priority rules and combined with the confidence levels of the target detection results for each tracking source, the optimal tracking source, i.e., the target tracking source, is dynamically selected.

[0085] In some embodiments, the step of obtaining a first detection result of a tracked object using a multi-axis gimbal includes:

[0086] Acquire wide-angle video frames captured by the wide-angle lens through a multi-axis gimbal;

[0087] A first image coordinate system is established with the center point of the wide-angle lens video frame as the origin. The width and height of the wide-angle lens video frame are linearly normalized to obtain the processed wide-angle lens video frame, so that the absolute value of the horizontal axis coordinate or the absolute value of the vertical axis coordinate of each vertex of the processed wide-angle lens video frame is one.

[0088] The processed wide-angle lens video frame is input into the target detection model to detect and identify the tracked object, and the first detection result of the tracked object in the first image coordinate system of the processed wide-angle lens video frame is obtained. The first detection result includes the first center coordinates of the tracked object, the first scale data of the tracked object, and the confidence level of the first detection result in the first image coordinate system.

[0089] Specifically, the first image coordinate system is a normalized coordinate system established with the center of the wide-angle lens video frame as the origin (0,0) and the boundary coordinates of the wide-angle lens video frame as ±1. The target detection model can be a deep learning model such as You Only Look Once (YOLO), Single Shot MultiBox Detector (SSD), or Region-based Convolutional Neural Networks (R-CNN), used to identify targets in the image and output their position and size information. The first detection result is the target detection result of the tracked object output by the multi-axis gimbal in the first image coordinate system, including the first center coordinates of the target, the first scale data of the tracked object, and the confidence score of the first detection result. A schematic diagram of the first detection result can be shown as follows: Figure 6As shown. The first center coordinates are the position of the tracked object in the first image coordinate system, such as (x=0.3, y=0.2). The first scale data are the normalized values ​​of the width and height of the tracked object relative to the wide-angle lens video frame size, such as w=0.25, h=0.3. In this application, the normalized value range of the width is w∈[0,2], the normalized value range of the height is h∈[0,2], and the normalized value range of the center coordinates of the tracked object is {(x,y)|x∈[-1,1],y∈[-1,1]}. The first detection result confidence score is used to represent the reliability of the first detection result; the higher the first detection result confidence score, the more reliable it is.

[0090] As an example, wide-angle video frames are captured using a multi-axis gimbal's wide-angle lens. The wide-angle lens can be a fisheye or ultra-wide-angle lens. Wide-angle video frames have a large field of view, covering a wider scene range, suitable for global target search and initial localization. The resolution of the wide-angle video frame can be 1920×1080 or higher. Subsequently, a first image coordinate system is established with the image center point of the wide-angle video frame as the origin. This coordinate system has the image center as (0,0), and the width and height of the wide-angle video frame are linearly normalized, converting the image pixel positions into relative coordinates. This ensures that the absolute values ​​of the horizontal or vertical coordinates of the four vertices of the wide-angle video frame are all 1, meaning the entire wide-angle video frame is mapped to a standardized coordinate range of [-1,1]. This normalization process eliminates scale differences between images of different resolutions, providing a unified representation basis for image data captured by different cameras. The normalized wide-angle lens video frames are input into the target detection model to detect and identify the tracked objects in the processed wide-angle lens video frames, and output the first detection result in the first image coordinate system. Through the above process of acquisition-coordinate system establishment-normalization-model detection, the original image of the wide-angle lens is transformed into a standardized target detection result, i.e., the first detection result, eliminating the influence of image size differences on localization and providing unified benchmark data for subsequent multi-source fusion (detection results from telephoto lenses and mobile terminal lenses).

[0091] In some embodiments, the step of obtaining a second detection result of a tracked object using a multi-axis gimbal includes:

[0092] Acquire telephoto video frames captured by the telephoto lens through a multi-axis gimbal;

[0093] A second image coordinate system is established with the center point of the telephoto lens video frame as the origin. The width and height of the telephoto lens video frame are linearly normalized to obtain the processed telephoto lens video frame, so that the absolute value of the horizontal axis coordinate or the absolute value of the vertical axis coordinate of each vertex of the processed telephoto lens video frame is one.

[0094] The processed telephoto lens video frame is input into the target detection model to detect and identify the tracked object, and a second detection result of the tracked object in the image coordinate system of the processed telephoto lens video frame is obtained. The second detection result includes the second center coordinates of the tracked object in the second image coordinate system, the second scale data of the tracked object, and the confidence of the second detection result.

[0095] Specifically, the second image coordinate system is a normalized coordinate system established with the center of the telephoto lens video frame as the origin (0,0) and the boundary coordinates of the telephoto lens video frame as ±1. The second detection result is the target detection result of the tracked object output by the gimbal in the second image coordinate system, including the second center coordinate of the target, the second scale data of the tracked object, and the confidence level of the second detection result. The second center coordinate is the position of the tracked object in the second image coordinate system, such as (x=-0.3, y=0.2). The second scale data is the normalized value of the width and height of the tracked object relative to the size of the telephoto lens video frame, such as w=0.5, h=0.6. In this application, the normalized value range of the width is w∈[0,2], the normalized value range of the height is h∈[0,2], and the normalized value range of the center coordinate of the tracked object is {(x,y)|x∈[-1,1],y∈[-1,1]}. The confidence level of the second test result is used to indicate the reliability of the second test result. The higher the confidence level of the second test result, the more reliable it is.

[0096] As an example, telephoto video frames are captured using a multi-axis gimbal's telephoto lens. The telephoto lens can be a fisheye or super-telephoto lens. Telephoto video frames have a narrow field of view and high spatial resolution, suitable for accurate identification and localization of distant targets. The resolution of the telephoto video frame can be 3840×2160 (4K) or higher. While the field of view is narrow, the pixel density is high. Subsequently, a second image coordinate system is established with the image center point of the telephoto video frame as the origin. This coordinate system uses the image center as its origin, and the width and height of the telephoto video frame are linearly normalized, converting the image pixel positions into relative coordinates. This ensures that the absolute values ​​of the horizontal or vertical coordinates of the four vertices of the telephoto video frame are all 1, meaning the entire telephoto video frame is mapped to a standardized coordinate range of [-1, 1]. This normalization process eliminates scale differences between images of different resolutions, providing a unified representation basis for image data acquired by different cameras. The normalized telephoto lens video frames are input into the target detection model to detect and identify the tracked objects in the processed telephoto lens video frames, and output a second detection result in a second image coordinate system. Through the above process of acquisition-coordinate system establishment-normalization-model detection, the original telephoto lens image is transformed into a standardized target detection result, i.e., the second detection result, eliminating the influence of image size differences on localization and providing unified benchmark data for subsequent multi-source fusion.

[0097] In some embodiments, the step of obtaining a third detection result of a tracked object, applied to a mobile terminal, includes:

[0098] Acquire mobile terminal video frames captured by the mobile terminal's camera.

[0099] A third image coordinate system is established with the center point of the mobile terminal video frame as the origin. The width and height of the mobile terminal video frame are linearly normalized to obtain the processed mobile terminal video frame, so that the absolute value of the horizontal axis coordinate or the absolute value of the vertical axis coordinate of each vertex of the processed mobile terminal video frame is one.

[0100] The processed mobile terminal video frame is input into the target detection model to detect and identify the tracked object, and the third detection result of the tracked object in the third image coordinate system of the processed mobile terminal video frame is obtained. The third detection result includes the third center coordinate of the tracked object in the third image coordinate system, the third scale data of the tracked object, and the confidence score of the third detection result.

[0101] The third detection result is sent to the multi-axis gimbal.

[0102] Specifically, the third image coordinate system is a normalized coordinate system established with the center of the mobile terminal video frame as the origin (0,0) and the boundary coordinates of the mobile terminal video frame as ±1. The third detection result is the target detection result of the tracked object output by the mobile terminal in the third image coordinate system, including the third center coordinates of the target, the third scale data of the tracked object, and the confidence score of the third detection result. The third center coordinates are the position of the tracked object in the third image coordinate system, such as (x=0.3, y=0.2). The third scale data are the normalized values ​​of the width and height of the tracked object relative to the size of the mobile terminal video frame, such as w=0.45, h=0.5. In this application, the normalized value range of the width is w∈[0,2], the normalized value range of the height is h∈[0,2], and the normalized value range of the center coordinates of the tracked object is {(x,y)|x∈[-1,1],y∈[-1,1]}. The confidence level of the third test result is used to indicate the reliability of the third test result. The higher the confidence level of the third test result, the more reliable it is.

[0103] As an example, video frames are captured by the mobile terminal's camera. A third image coordinate system is then established with the image center point of the video frame as the origin. This coordinate system uses the image center as its origin, and the width and height of the video frame are linearly normalized, converting the image pixel positions into relative coordinates. This ensures that the absolute values ​​of the horizontal or vertical coordinates of the four vertices of the video frame are all 1, meaning the entire video frame is mapped to a standardized coordinate range of [-1, 1]. This normalization process eliminates scale differences between images of different resolutions, providing a unified representation for image data captured by different cameras. The normalized video frame is then input into a target detection model to detect and identify the tracked object in the processed video frame. The model outputs a third detection result in the third image coordinate system and sends this result to the multi-axis gimbal. Through the above process of acquisition, coordinate system establishment, normalization, and model detection, the original image from the mobile terminal camera is transformed into a standardized target detection result, i.e., the third detection result, eliminating the impact of image size differences on positioning and providing unified benchmark data for subsequent multi-source fusion.

[0104] Step 102: Read the tracking object position output by the target tracking process corresponding to the target tracking source, where the tracking object position is the position of the tracking object in the video frame corresponding to the target tracking source.

[0105] Specifically, the target tracking source is the optimal video input source selected by the system based on priority and confidence, used to provide the target detection result in the current frame. The target tracking process is the target detection and tracking algorithm flow running internally within the system, responsible for identifying the tracking object from the video source and outputting the position (center coordinates), scale data, and confidence score of the tracking object in the corresponding video frame of the tracking source. In this application, the mobile terminal video source, the telephoto lens video source, and the wide-angle lens video source each have their own corresponding target tracking process. The target tracking processes corresponding to the mobile terminal video source, the telephoto lens video source, and the wide-angle lens video source are independent of each other and can be executed separately or jointly. The tracking object position is the coordinate of the tracking object in the corresponding video frame of the tracking source, represented by normalized coordinates. Optionally, the tracking object position can reflect the position of the tracking object relative to the center of the corresponding video frame of the tracking source, or alternatively, the tracking object position can reflect the position of the tracking object relative to the vertex of the corresponding video frame of the tracking source.

[0106] Step 103: Determine the pose adjustment parameters of the multi-axis gimbal based on the position of the tracked object, and control the rotation of the multi-axis gimbal based on the pose adjustment parameters.

[0107] Specifically, the pose adjustment parameters are the angles that the multi-axis gimbal needs to adjust based on the position of the tracked object, including yaw and pitch adjustment vectors, used to reposition the tracked object back to the center of the target tracking source's view. The position of the tracked object is determined, and the angle parameters that the multi-axis gimbal needs to adjust are calculated based on this position. This is done by converting the offset of the target center coordinates into actual spatial angular offsets, considering the current attitude parameters and field of view parameters of the multi-axis gimbal, to determine the yaw and pitch adjustment vectors. For example, if the tracked object is located to the left of the center of the tracking source's view, the yaw angle of the multi-axis gimbal needs to be increased to rotate it to the left until the tracked object is back to the center of the tracking source's view. The calculated gimbal adjustment parameters are then sent to the gimbal controller to drive the yaw and pitch axis motors. By converting the position information of the tracked object into gimbal motion, the tracked object is always kept in the center of the tracking source's view, solving the tracking instability problem caused by dynamic movement or initial position deviation of the tracked object, and improving the accuracy and robustness of tracking.

[0108] The target tracking and shooting method proposed in this application is applied to a gimbal control system. The gimbal control system includes a multi-axis gimbal, a mobile terminal, and a data processor. The mobile terminal can be connected to the gimbal via a dedicated fixed component to form a flexible and scalable shooting platform. If a target tracking process is triggered, the most suitable video source is selected from the mobile terminal video source, telephoto lens video source, or wide-angle lens video source based on priority and the confidence level of the target detection results corresponding to each tracking source. Combining the advantages of different cameras, the most suitable video source for the current scene is selected according to actual needs, avoiding tracking interruption caused by the failure of a single device. The position of the tracked object output by the target tracking process corresponding to the target tracking source is read. The position of the tracked object is the location of the tracked object in the video frame corresponding to the target tracking source, providing a basis for subsequent gimbal attitude adjustment. The pose adjustment parameters of the multi-axis gimbal are determined based on the position of the tracked object, and the multi-axis gimbal is rotated based on the pose adjustment parameters to ensure that the tracked object is always located in the center of the target tracking source's frame. This solves the technical problem in the prior art where the inability to effectively switch to alternative sources when a single tracking source detection is unreliable leads to tracking interruption, and improves tracking accuracy and stability. The target tracking and shooting method disclosed in this application integrates video from three sources: a mobile terminal, a telephoto lens, and a wide-angle lens. This allows it to cover various scene requirements. After tracking is triggered, the optimal tracking source is selected based on preset priorities and the real-time target detection confidence of each tracking source. The attitude adjustment parameters of the multi-axis gimbal are calculated based on the target tracking process output from the corresponding tracking source, and the multi-axis gimbal is driven to rotate, ensuring the target remains centered in the frame. When a single tracking source fails, it can automatically switch to another tracking source, avoiding tracking interruption and significantly improving tracking stability and continuity in complex scenes.

[0109] In some embodiments, after determining the pose adjustment parameters of the multi-axis gimbal based on the position of the tracked object, and controlling the rotation of the multi-axis gimbal based on the pose adjustment parameters, the method further includes:

[0110] Based on the target tracking source, a tracking algorithm is used to control a multi-axis gimbal in order to track and capture images of the target.

[0111] Specifically, the tracking algorithm is used to analyze the positional changes of a target in a video stream in real time and predict the future position of the target accordingly. The tracking algorithm in this application can be optical flow, Kalman filtering, kernel-based correlation filters (KCF), deep learning-based sorting (DeepSORT), etc.

[0112] In some embodiments, based on a determined tracking source (such as a telephoto lens), a tracking algorithm, such as KCF, is fused with a target tracking algorithm based on a Siamese network architecture (SiamRPN++) to continuously locate the target. KCF quickly generates a target response map through frequency domain correlation filtering, while SiamRPN++ extracts depth features and regresses bounding boxes using the Siamese network. The results of both are fused by weighting according to the tracking source type; for example, in a telephoto scene, the weight of SiamRPN++ is ≥0.8. Simultaneously, Kalman filtering predicts the target's position in the next frame, generating a feedforward control variable to compensate for delay. Closed-loop proportional-integral-derivative (PID) control converts the target position deviation into gimbal motor commands, dynamically adjusting the proportional coefficient to adapt to changes in the field of view, ensuring consistent control sensitivity. If the tracked object is lost, the tracking source is switched. When switching tracking sources, smooth transition of gimbal movement can be achieved through Bézier curve interpolation.

[0113] In some embodiments, the step of selecting a target tracking source based on priority and the confidence level of the target detection result corresponding to each tracking source includes:

[0114] Obtain the confidence level of the target detection results corresponding to each tracking source;

[0115] Determine the priority of each tracking source;

[0116] The confidence level of the target detection result corresponding to each tracking source is determined in descending order of priority to determine whether it is greater than or equal to the first preset threshold.

[0117] If the initial judgment result is yes, the tracking source corresponding to the judgment result is identified as the target tracking source;

[0118] The step of determining whether to proceed to the step of judging whether the confidence level of the target detection result corresponding to the non-highest-level tracking source is greater than or equal to the first preset threshold is determined by the judgment result of the previous priority tracking source corresponding to the non-highest-level tracking source.

[0119] Specifically, the tracking source is the video input source used for target detection, such as a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source. The first preset threshold is a preset confidence threshold (e.g., 0.7) used to determine whether the target detection result is reliable. A non-highest priority tracking source refers to a tracking source that is not the highest priority. For example, in a priority ranking of telephoto lens video source > mobile terminal video source > wide-angle lens video source, the mobile terminal video source and the wide-angle lens video source are non-highest priority tracking sources.

[0120] Optionally, the confidence level of the target detection result corresponding to each tracking source (such as mobile terminal video source, telephoto lens video source, and wide-angle lens video source) is obtained. The confidence level reflects the reliability of target detection in the video frame. Then, according to the priority of each tracking source, for example, telephoto lens video source > mobile terminal video source > wide-angle lens video source, the target detection confidence level of each tracking source is judged in descending order of priority to see if it is greater than or equal to a first preset threshold (e.g., 0.7). Once the target detection confidence level of a tracking source is greater than or equal to the first preset threshold, it is identified as a target tracking source, and subsequent judgments stop. If the currently judged tracking source does not meet the threshold, the judgment continues to see if the target detection result confidence level of the next priority tracking source is greater than or equal to the first preset threshold. The condition for entering the judgment of a non-highest priority tracking source depends on the judgment result of the previous priority tracking source. That is, only when the current level tracking source does not meet the confidence level requirement can the judgment continue to see if the target detection result confidence level of the next level tracking source is greater than or equal to the first preset threshold.

[0121] In some embodiments, the step of determining the priority of each tracking source includes:

[0122] In response to the user's scene selection, determine the current scene category;

[0123] The priority of each tracking source in the current scene category is determined based on the current scene category and the preset priority of each tracking source in each scene category.

[0124] Specifically, the scene selection operation allows users to actively choose the current shooting environment type through the interface or commands. For example, the scene category can be sports events, stage performances, outdoor hiking, etc. The current scene category is the specific application scenario identified based on the user's operation, used to match a preset priority strategy. The preset priority is the order in which each tracking source is used in different scenarios. For example, in sports event scenarios, telephoto lens video sources have the highest priority; in large-scale stage performance scenarios, wide-angle lens video sources have the highest priority; and in meeting recording scenarios, mobile terminal video sources have the highest priority.

[0125] As an example, when a user selects the current shooting scene (e.g., "sports event," "stage performance," "outdoor hiking," etc.) through an interface (such as a mobile application or gimbal control panel), the system responds to this scene selection by identifying and determining the current scene category. The system has a pre-set scene-priority mapping table that stores the preset priority strategies for each tracking source under different scenes. For example, in a sports event scene, telephoto lens video sources have the highest priority; while in a stage performance scene, wide-angle lens video sources have the highest priority. Based on the current scene category, the system looks up the corresponding priority configuration and determines the specific priority order of each tracking source within that category. This ensures that the system prioritizes the video source most suitable for the current scene for target recognition and tracking, achieving scene-adaptive configuration of tracking source priorities. This helps improve the intelligence, adaptability, and stability of the target tracking system, making it particularly suitable for diverse and complex application scenarios such as live streaming and sports shooting.

[0126] Optionally, refer to Figure 7 , Figure 7 A flowchart of a target tracking and imaging method is provided to determine the location of the tracked object and the tracking source.

[0127] In the priority ranking of mobile terminal video source > telephoto lens video source > wide-angle lens video source, when the first detection result is the detection output of the wide-angle lens on the multi-axis gimbal for the tracked object, the second detection result is the detection output of the telephoto lens on the multi-axis gimbal for the tracked object, and the third detection result is the detection output of the mobile terminal lens on the mobile terminal, the confidence level of the target detection result corresponding to each tracking source is judged in descending order of priority to determine whether it is greater than or equal to the first preset threshold; when the first judgment result is yes, the step of determining the tracking source corresponding to the judgment result as the target tracking source includes:

[0128] The confidence level of the third detection result in the third image coordinate system is determined based on the third detection result.

[0129] Compare the confidence level of the third detection result with the magnitude of the first preset threshold;

[0130] If the confidence level of the third detection result is greater than or equal to the first preset threshold, the mobile terminal video source corresponding to the third detection result will be identified as the target tracking source.

[0131] If the confidence level of the third detection result is less than the first preset threshold, compare the confidence level of the second detection result with the first preset threshold.

[0132] If the confidence level of the second detection result is greater than or equal to the first preset threshold, the telephoto lens video source corresponding to the second detection result will be determined as the target tracking source.

[0133] If the confidence level of the second detection result is less than the first preset threshold, the wide-angle lens video source corresponding to the third detection result is determined as the target tracking source.

[0134] Specifically, the first preset threshold is a preset confidence threshold (e.g., 0.7) used to determine the reliability of the detection results. The mobile terminal's main control chip has high computing power, enabling it to process detection results captured by the mobile terminal's camera in real time at a high frame rate. While the multi-axis gimbal's main control chip has relatively limited computing power, it can fully leverage the hardware advantages of its telephoto and wide-angle lenses. When the mobile terminal's camera does not detect the tracked object, the gimbal's dual cameras assist in target detection and tracking, thus achieving full-scene target detection and coverage. Based on this, when the confidence level of the mobile terminal's target detection result is higher than the preset threshold and the task is reliable, the mobile terminal's video source is preferentially selected as the target tracking source.

[0135] As an example, the confidence level of the third detection result (representing the reliability of the detection result, typically a value between 0 and 1) is determined based on the third detection result. Then, the confidence level of the third detection result is compared with a first preset threshold (e.g., 0.7). If the confidence level of the third detection result is greater than or equal to the first preset threshold, it indicates that the current target recognition result of the mobile terminal lens has high reliability. The mobile terminal video source is then determined as the target tracking source, meaning that subsequent adjustments to the gimbal will be based on the video source provided by the mobile terminal.

[0136] If the confidence level of the third detection result is less than the first preset threshold, it indicates that the current detection result of the mobile terminal lens is not reliable enough. At this time, the next level of screening logic will be entered, that is, the confidence level of the second detection result is compared with the first preset threshold. If the confidence level of the second detection result is greater than or equal to the first preset threshold, the telephoto lens video source corresponding to the second detection result will be determined as the target tracking source. That is, the subsequent gimbal adjustment will be based on the video source provided by the telephoto lens.

[0137] If the confidence level of the second detection result is less than the first preset threshold, it indicates that even the telephoto lens cannot provide reliable target information. In this case, the wide-angle lens video source corresponding to the third detection result will be identified as the target tracking source by default. Although the wide-angle lens is not as accurate as a mobile phone or telephoto lens in terms of target recognition, it has the advantages of a large field of view and wide coverage. In the case of target loss or unstable detection, it can provide basic tracking capabilities to prevent the target from completely leaving the frame. Through the above confidence level evaluation and step-by-step screening mechanism, the multi-axis gimbal is always adjusted based on the most reliable detection data to achieve stable and accurate target tracking in large-scale motion scenarios, which is suitable for scenarios such as live sports broadcasts and security monitoring.

[0138] In some embodiments, the step of determining the pose adjustment parameters of the multi-axis gimbal based on the position of the tracked object includes:

[0139] Calculate the offset between the position of the tracked object and the center of the image of the target tracking source;

[0140] Obtain the current field of view parameters of the multi-axis gimbal, and determine the corresponding pose adjustment parameters based on the current field of view parameters and the offset.

[0141] Specifically, the current field of view parameters include the current horizontal field of view and the current vertical field of view, which determine the viewing angle range that the multi-axis gimbal can cover. The offset is the difference between the center coordinates of the tracked object and the center of the image.

[0142] By calculating the adjustment parameters of the multi-axis gimbal based on the position of the tracked object, and controlling the rotation of the multi-axis gimbal accordingly, accurate and stable tracking of the tracked object can be achieved in dynamic environments, improving the accuracy and robustness of tracking, as well as enhancing the flexibility and adaptability of the multi-axis gimbal, making it suitable for target tracking and shooting tasks in complex scenarios.

[0143] In some embodiments, the step of determining the corresponding pose adjustment parameters based on the current field of view angle parameters and the offset includes:

[0144] The current horizontal and vertical field of view angles are determined based on the current field of view angle parameters, and the horizontal and vertical offset vectors are determined based on the offset.

[0145] Based on the current horizontal field of view and horizontal offset vector, calculate the corresponding yaw angle adjustment vector, and based on the current vertical field of view and vertical offset vector, calculate the corresponding pitch angle adjustment vector;

[0146] The yaw angle adjustment vector and pitch angle adjustment vector are defined as attitude adjustment parameters.

[0147] Specifically, the horizontal offset vector is the offset in the horizontal direction, and the vertical offset vector is the offset in the vertical direction. The yaw angle adjustment vector is the angle change that the multi-axis gimbal needs to adjust in the horizontal direction to bring the tracked object back to the center of the video frame corresponding to the target tracking source. The pitch angle adjustment vector is the angle change that the multi-axis gimbal needs to adjust in the vertical direction to bring the tracked object back to the center of the video frame corresponding to the target tracking source.

[0148] As an example, the data processor reads the current field of view parameters stored internally by the gimbal, or measures the current field of view parameters in real time through sensors. The current horizontal field of view reflects the range of angles that the camera can cover in the horizontal direction, while the current vertical field of view corresponds to the coverage range in the vertical direction. Together, they determine the shooting range of the image. The tracking object position is the current center coordinate of the tracking object in the video frame corresponding to the target tracking source, representing the position of the tracking object relative to the center. By calculating the difference between the target center coordinates and the center of the video frame corresponding to the target tracking source, horizontal and vertical offset vectors are obtained. The larger the absolute value of the offset vector, the greater the degree of deviation of the tracking object from the center of the frame. The horizontal offset vector reflects the proportion of the tracking object's deviation from the center in the horizontal direction, while the vertical offset vector reflects the proportion of deviation in the vertical direction. Furthermore, the yaw angle adjustment vector is calculated based on the horizontal field of view and the horizontal offset vector. The yaw angle adjustment vector represents the angle that the multi-axis gimbal needs to rotate in the horizontal direction to move the tracking object to the center of the frame. Similarly, the pitch angle adjustment vector is calculated based on the vertical field of view and the vertical offset vector. The pitch angle adjustment vector represents the angle that the multi-axis gimbal needs to rotate in the vertical direction. The yaw and pitch adjustment vectors are defined as the adjustment parameters for the multi-axis gimbal, used to drive the motor rotation and bring the tracked object back to the center of the video frame corresponding to the target tracking source. Through dynamic conversion of the field of view and offset, a precise mapping from the target position to the gimbal's movement is achieved, which helps improve tracking accuracy. Simultaneously, it dynamically adapts to different shooting scenarios to ensure the tracked object remains at the center of the video frame corresponding to the target tracking source, providing reliable input for subsequent closed-loop control.

[0149] refer to Figure 8 , Figure 8This document provides a flowchart illustrating another target tracking and shooting method for determining pose adjustment parameters. The method involves: a mobile terminal acquiring video data and performing target detection; then, based on this video data, performing target detection on the tracked object to obtain the confidence level of the target detection result and the position of the tracked object corresponding to the mobile terminal video data; finally, a multi-axis gimbal acquiring video data from a telephoto lens and performing target detection on the tracked object to obtain the confidence level of the target detection result and the position of the tracked object corresponding to the telephoto lens video data; and finally, a multi-axis gimbal acquiring video data from a wide-angle lens and performing target detection on the tracked object to obtain the confidence level of the target detection result and the position of the tracked object corresponding to the wide-angle lens video data. The data processor reads the target detection result confidence scores corresponding to the mobile terminal video source, the telephoto lens video source, and the wide-angle lens video source, as well as the priority of each tracking source. Based on these scores and priorities, it selects a target tracking source from among the mobile terminal video source, the telephoto lens video source, and the wide-angle lens video source. The data processor then reads the tracking object position output by the target tracking process corresponding to this tracking source, i.e., it obtains one of the following: the tracking object position corresponding to the mobile terminal video source, the tracking object position corresponding to the telephoto lens video source, or the target detection result confidence score corresponding to the wide-angle lens video source. The data processor then determines the pose adjustment parameters of the multi-axis gimbal based on the tracking object position.

[0150] In some embodiments, when the target tracking source is a mobile terminal video source or a telephoto lens video source, the step of controlling a multi-axis gimbal using a tracking algorithm based on the target tracking source to track and capture images of the target includes:

[0151] Read the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source;

[0152] Based on the tracking object data corresponding to the current frame and the tracking object data corresponding to multiple historical frames, the motion trajectory of the tracking object is predicted to obtain the predicted motion trajectory of the tracking object in the next frame.

[0153] Based on the tracking object data and motion trajectory prediction corresponding to the current frame, the pose adjustment parameters of the multi-axis gimbal are generated, and the rotation of the multi-axis gimbal is controlled according to the pose adjustment parameters.

[0154] The continuous video frames include the current frame and multiple historical frames, and the tracking object data includes the tracking object location and the tracking object data confidence level.

[0155] Specifically, the tracking object data includes the tracking object's position, scale data, and confidence information in each video frame.

[0156] As an example, when the tracking source is a mobile terminal lens or a telephoto lens of a multi-axis gimbal, the mobile terminal lens or telephoto lens acquires continuous video frames of the tracked object in the moving scene at a fixed frame rate (e.g., 30fps). For example, it acquires the current frame and 30 historical frames from the most recent second. The tracking algorithm processes each frame and outputs the tracked object data corresponding to the continuous video frames, including the tracked object data corresponding to the current frame (frame N) and the tracked object data corresponding to multiple historical frames (frames N-1 to N-30). The tracking algorithm combines the tracked object data of the current frame and historical frames to predict the motion trajectory of the next frame (frame N+1), obtaining the predicted motion trajectory for the next frame. After obtaining the tracked object data corresponding to the current frame and the predicted motion trajectory for the next frame, the algorithm combines the predicted motion trajectory with the tracked object data corresponding to the current frame to generate a feedforward control quantity, i.e., pose adjustment parameters, to rotate the gimbal in advance so that the multi-axis gimbal is already in a pre-adjusted state when the tracked object enters the center of the video frame corresponding to the target tracking source. Furthermore, the pose adjustment parameters are input into the PID controller to drive the yaw and pitch axis motors to rotate until the position of the tracked object coincides with the center of the video frame corresponding to the target tracking source. During the target tracking process, the adjustment parameters are updated and the gimbal position is corrected for each frame of image processed, forming a real-time control loop.

[0157] In addition, the control parameters need to be automatically scaled according to the field of view of the tracking source. For example, the adjustment angle corresponding to the same offset is larger under a telephoto lens, while the adjustment angle is smaller under a mobile terminal lens, so as to achieve consistent control response speed under different focal lengths.

[0158] In some embodiments, before the step of predicting the motion trajectory of the tracked object based on the tracked object data corresponding to the current frame and the tracked object data corresponding to multiple historical frames, the method further includes:

[0159] If the confidence level of the tracking object data corresponding to the current frame is less than the second preset threshold or the tracking object data corresponding to the current frame is empty, read the first candidate tracking object data and the second candidate tracking object data corresponding to the current frame output by the target tracking process corresponding to the non-target tracking source.

[0160] Based on the first and second candidate tracking object data, switch the target tracking source and return to the step of reading the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source;

[0161] The first candidate tracking object data is the tracking object data corresponding to the tracking source with higher priority among the other tracking sources besides the target tracking source, and the second candidate tracking object data is the tracking object data corresponding to the tracking source with lower priority among the other tracking sources besides the target tracking source.

[0162] Specifically, in this application, the priority order of each tracking source can be: mobile terminal video source > telephoto lens video source > wide-angle lens video source. The second preset threshold is a preset confidence threshold (e.g., 0.6), used to determine the reliability of the tracking object data corresponding to the current frame. Null tracking object data for the current frame indicates that no tracking object was detected in the current frame, and the tracking algorithm outputs no valid data, such as empty fields for tracking object position or scale data. The first candidate tracking object data is the tracking object data corresponding to a higher-priority tracking source among the other tracking sources besides the target tracking source, and the second candidate tracking object data is the tracking object data corresponding to a lower-priority tracking source among the other tracking sources besides the target tracking source. Optionally, in the case of a tracking source priority order of mobile terminal video source > telephoto lens video source > wide-angle lens video source, when the target tracking source is a mobile terminal video source, the first candidate tracking object data is the tracking object data corresponding to the telephoto lens video source, and the second candidate tracking object data is the tracking object data corresponding to the wide-angle lens video source. Optionally, when the target tracking source is a telephoto lens video source, the first alternative tracking object data is the tracking object data corresponding to the mobile terminal video source, and the second alternative tracking object data is the tracking object data corresponding to the wide-angle lens video source.

[0163] As an example, before predicting the target motion trajectory, the reliability of the tracking object data in the current frame is assessed. Specifically, if the confidence level of the tracking object data in the current frame is lower than a second preset threshold (e.g., 0.6), or if no tracking object is detected in the current frame (i.e., the tracking object data is empty), it indicates that the video source information provided by the currently used camera is unreliable or missing and cannot be used for subsequent trajectory prediction and gimbal control. To avoid target loss or gimbal malfunction, a multi-camera collaborative mechanism is adopted to acquire first and second candidate tracking object data. If the confidence level of the first candidate data is high and the target information is complete, the tracking source corresponding to the first candidate tracking object data is set as the new target tracking source. If the confidence level of the first candidate detection result is lower than the second preset threshold or the second candidate tracking object data is empty, but the second candidate data is available (i.e., the confidence level of the second candidate detection result is greater than or equal to the second preset threshold), the tracking source corresponding to the second candidate tracking object data is set as the new target tracking source. After the target tracking source switch is completed, the target detection process based on the tracking algorithm is returned to execute. The continuous video frames acquired by the new tracking source are re-analyzed to obtain the tracking object data corresponding to the continuous video frames output by the target tracking process of the target tracking source. Subsequent trajectory prediction and gimbal control operations are then performed. By introducing the above-mentioned multi-camera collaboration mechanism and intelligent switching strategy, when the current target tracking source fails or becomes unstable, it automatically switches to other tracking sources and uses the most reliable tracking object data for tracking decisions. This avoids tracking interruptions caused by the failure of a single camera, providing technical support for achieving continuous, stable, and high-quality automatic tracking and shooting.

[0164] In some embodiments, the step of switching the target tracking source based on first alternative tracking object data and second alternative tracking object data includes:

[0165] The confidence level of the first candidate detection result is determined based on the data of the first candidate tracking object.

[0166] Compare the confidence level of the first alternative detection result with the value of the second preset threshold;

[0167] If the confidence level of the first candidate detection result is greater than or equal to the second preset threshold, the tracking source corresponding to the first candidate tracking object data will be switched to the target tracking source.

[0168] If the confidence level of the first alternative detection result is less than the second preset threshold or the data of the second alternative tracking object is empty, the confidence level of the second alternative detection result is determined based on the data of the second alternative tracking object.

[0169] Compare the confidence level of the second alternative detection result with the second preset threshold.

[0170] If the confidence level of the second alternative detection result is greater than or equal to the second preset threshold, the tracking source corresponding to the second alternative tracking object data will be switched to the target tracking source.

[0171] Specifically, when the confidence level of the tracked object data corresponding to the current frame is detected to be lower than the second preset threshold, or when no tracked object is detected in the current frame, first candidate tracked object data is obtained, and the confidence level of the first candidate detection result is extracted from the first candidate tracked object data. The confidence level of the first candidate detection result is compared with the second preset threshold. If the first candidate confidence level is greater than or equal to the second preset threshold, it indicates that the target detection result provided by the tracking source corresponding to the first candidate tracked object data has sufficient reliability. At this time, the tracking source corresponding to the first candidate tracked object data is switched to the target tracking source, and the target detection and tracking process based on the tracking algorithm is restarted. If the confidence level of the first candidate detection result is still lower than the second preset threshold, or the first candidate tracked object data is empty, the confidence level of the second candidate detection result corresponding to the second candidate tracked object data is extracted, and the confidence level of the second candidate detection result is compared with the second preset threshold. If the confidence level of the second candidate detection result is greater than or equal to the second preset threshold, the tracking source corresponding to the second candidate tracked object data is switched to the target tracking source. If the requirements are still not met, a global search mechanism is initiated or the user is prompted for intervention.

[0172] In some embodiments, when the target tracking source is a wide-angle lens video source, the step of controlling a multi-axis gimbal with a tracking algorithm based on the target tracking source to track and capture images of the target includes:

[0173] Read the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source, and read the first and second candidate tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the non-target tracking source.

[0174] If the detection result of either the first or second candidate tracking object data is not empty, the target tracking source is switched, and based on the switched target tracking source, the tracking algorithm is used to control the multi-axis gimbal to track and capture the target.

[0175] Specifically, when the target tracking source is a wide-angle lens video source, wide-angle lenses are typically used for searching large-scale targets. While wide-angle lens video source frames have a wide coverage area, their detail recognition capability is relatively weak. Using the wide-angle lens video source as the current target tracking source, the tracking algorithm processes the video frames captured by the wide-angle lens to generate tracking object data for the current frame (including center coordinates, scale, and confidence level). Simultaneously, the system calls upon the remaining cameras (such as telephoto lenses and mobile terminal lenses) to perform parallel detection on the same tracking object, generating first candidate tracking object data (the detection result from the mobile terminal) and second candidate tracking object data (the detection result corresponding to the telephoto lens video source). If either the first or second candidate tracking object data is not empty, indicating a target has been detected, the target tracking source is switched, and the tracking algorithm process is reset.

[0176] If either the first or second candidate tracking object data is not empty, optionally, if the first candidate tracking object data is not empty and the second candidate tracking object data is empty, the tracking source corresponding to the first candidate tracking object data is switched to the target tracking source. Alternatively, if the first candidate tracking object data is empty and the second candidate tracking object data is not empty, the tracking source corresponding to the second candidate tracking object data is switched to the target tracking source.

[0177] Optionally, if both the first and second candidate tracking object data are not empty, the confidence level of the first candidate detection result is determined based on the first candidate tracking object data; the confidence level of the first candidate detection result is compared with a second preset threshold; if the confidence level of the first candidate detection result is greater than or equal to the second preset threshold, the tracking source corresponding to the first candidate tracking object data is switched to the target tracking source; if the confidence level of the first candidate detection result is less than the second preset threshold, the confidence level of the second candidate detection result is determined based on the second candidate tracking object data; the confidence level of the second candidate detection result is compared with the second preset threshold; if the confidence level of the second candidate detection result is greater than or equal to the second preset threshold, the tracking source corresponding to the second candidate tracking object data is switched to the target tracking source.

[0178] After the switch is complete, the tracking process will restart based on the new tracking source. This means the tracking algorithm will continue to analyze the continuous video frames captured by the new camera and generate gimbal control commands to drive the multi-axis gimbal to adjust its attitude, re-centering the tracked object and continuing tracking. By introducing high-precision detection results from other cameras, the shortcomings of wide-angle lenses in target recognition accuracy are compensated for, enabling intelligent switching of the tracking source and dynamic optimization of the tracking strategy.

[0179] In addition, during the tracking and filming of the target, the mobile terminal can obtain its camera images through the application development interface to realize real-time tracking, shooting, storage and live streaming functions. Multi-axis gimbals can also achieve the same functions.

[0180] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the target tracking and shooting method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0181] This application also provides a target tracking and imaging device, please refer to... Figure 9 The target tracking and imaging device includes:

[0182] The tracking source determination module 901 is used to select a target tracking source based on priority and the confidence level of the target detection result corresponding to each tracking source if the target tracking process is triggered. The target tracking source is a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source.

[0183] The position determination module 902 is used to read the position of the tracked object output by the target tracking process corresponding to the target tracking source, wherein the position of the tracked object is the position of the tracked object in the video frame corresponding to the target tracking source;

[0184] The control module 903 is used to determine the pose adjustment parameters of the multi-axis gimbal based on the position of the tracked object, and to control the rotation of the multi-axis gimbal based on the pose adjustment parameters.

[0185] The target tracking and imaging device provided in this application, employing the target tracking and imaging method described in the above embodiments, can solve the technical problem in the prior art where the inability to effectively switch alternative sources due to unreliable detection of a single tracking source leads to tracking interruption. Compared with the prior art, the beneficial effects of the target tracking and imaging device provided in this application are the same as those of the target tracking and imaging method described in the above embodiments, and other technical features in the target tracking and imaging device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0186] This application provides a target tracking and imaging device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the target tracking and imaging method in Embodiment 1 above.

[0187] The following is for reference. Figure 10 It shows a schematic diagram of a target tracking and shooting device suitable for implementing the embodiments of this application. Figure 10 The target tracking and shooting device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0188] like Figure 10 As shown, the target tracking and imaging device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The random access memory 1004 also stores various programs and data required for the operation of the target tracking and imaging device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the target tracking and imaging device to communicate wirelessly or wiredly with other devices to exchange data. Although a target tracking and imaging device with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0189] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0190] The target tracking and imaging device provided in this application, employing the target tracking and imaging method described in the above embodiments, can solve the technical problem in the prior art where the inability to effectively switch alternative sources due to unreliable detection of a single tracking source leads to tracking interruption. Compared with the prior art, the beneficial effects of the target tracking and imaging device provided in this application are the same as those of the target tracking and imaging method described in the above embodiments, and other technical features of this target tracking and imaging device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0191] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0192] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0193] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the target tracking and imaging method described in the above embodiments.

[0194] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0195] The aforementioned computer-readable storage medium may be included in the target tracking and imaging device; or it may exist independently and not be assembled into the target tracking and imaging device.

[0196] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the target tracking and imaging device, the target tracking and imaging device: if a target tracking process is triggered, selects a target tracking source according to priority and the confidence level of the target detection result corresponding to each tracking source, wherein the target tracking source is a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source; reads the tracking object position output by the target tracking process corresponding to the target tracking source, wherein the tracking object position is the position of the tracking object in the video frame corresponding to the target tracking source; determines the pose adjustment parameters of the multi-axis gimbal based on the tracking object position, and controls the rotation of the multi-axis gimbal based on the pose adjustment parameters.

[0197] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0198] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0199] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0200] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described target tracking and imaging method. This solves the technical problem in the prior art where the inability to effectively switch alternative sources due to unreliable detection of a single tracking source leads to tracking interruption. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the target tracking and imaging method provided in the above embodiments, and will not be repeated here.

[0201] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the target tracking and shooting method described above.

[0202] The computer program product provided in this application can solve the technical problem in the prior art where the inability to effectively switch alternative sources when the detection of a single tracking source is unreliable leads to tracking interruption. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the target tracking and imaging method provided in the above embodiments, and will not be repeated here.

[0203] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A target tracking and shooting method, characterized in that, An application is made in a gimbal control system, the gimbal control system including a multi-axis gimbal, a mobile terminal, and a data processor. The multi-axis gimbal is equipped with a telephoto lens, a wide-angle lens, and a mobile terminal fixing component. The mobile terminal is detachably connected to the multi-axis gimbal via the mobile terminal fixing component. The target tracking and shooting method includes: If the target tracking process is triggered, the target tracking source is selected according to the priority and the confidence level of the target detection result corresponding to each tracking source. The target tracking source is a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source. Read the tracking object position output by the target tracking process corresponding to the target tracking source, wherein the tracking object position is the position of the tracking object in the video frame corresponding to the target tracking source; The pose adjustment parameters of the multi-axis gimbal are determined based on the position of the tracked object, and the rotation of the multi-axis gimbal is controlled based on the pose adjustment parameters.

2. The target tracking and shooting method as described in claim 1, characterized in that, The step of selecting a target tracking source based on priority and the confidence level of the target detection results corresponding to each tracking source includes: Obtain the confidence level of the target detection result corresponding to each of the tracking sources; In response to the user's scene selection, determine the current scene category; Based on the current scene category and the preset priorities of each tracking source under each scene category, the priority of each tracking source under the current scene category is determined; The confidence level of the target detection result corresponding to each tracking source is determined in descending order of priority to determine whether it is greater than or equal to the first preset threshold. If the initial judgment result is yes, the tracking source corresponding to the judgment result is determined as the target tracking source; The step of determining whether to proceed to the step of judging whether the confidence level of the target detection result corresponding to the non-highest-level tracking source is greater than or equal to the first preset threshold is determined by the judgment result of the previous priority tracking source corresponding to the non-highest-level tracking source.

3. The target tracking and shooting method as described in claim 1, characterized in that, The step of determining the pose adjustment parameters of the multi-axis gimbal based on the position of the tracked object includes: Calculate the offset between the position of the tracked object and the center of the image of the target tracking source; Obtain the current field of view parameters of the multi-axis gimbal, determine the current horizontal field of view and the current vertical field of view based on the current field of view parameters, and determine the horizontal offset vector and the vertical offset vector based on the offset. Based on the current horizontal field of view and the horizontal offset vector, calculate the corresponding yaw angle adjustment vector, and based on the current vertical field of view and the vertical offset vector, calculate the corresponding pitch angle adjustment vector; The yaw angle adjustment vector and the pitch angle adjustment vector are determined as the attitude adjustment parameters.

4. The target tracking and shooting method as described in claim 1, characterized in that, After the steps of determining the pose adjustment parameters of the multi-axis gimbal based on the position of the tracked object, and controlling the rotation of the multi-axis gimbal based on the pose adjustment parameters, the method further includes: Based on the target tracking source, a tracking algorithm is used to control the multi-axis gimbal in order to track and capture images of the target.

5. The target tracking and shooting method as described in claim 4, characterized in that, When the target tracking source is a mobile terminal video source or a telephoto lens video source, the step of controlling the multi-axis gimbal using a tracking algorithm based on the target tracking source to track and capture images of the target includes: Read the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source; Based on the tracking object data corresponding to the current frame and the tracking object data corresponding to multiple historical frames, the motion trajectory of the tracking object is predicted to obtain the predicted motion trajectory of the tracking object in the next frame. Based on the tracking object data corresponding to the current frame and the motion trajectory prediction, the pose adjustment parameters of the multi-axis gimbal are generated, and the rotation of the multi-axis gimbal is controlled according to the pose adjustment parameters. The continuous video frames include the current frame and multiple historical frames, and the tracking object data includes the tracking object location and the tracking object data confidence level.

6. The target tracking and shooting method as described in claim 5, characterized in that, Before the step of predicting the motion trajectory of the tracked object based on the tracked object data corresponding to the current frame and the tracked object data corresponding to multiple historical frames, the method further includes: If the confidence level of the tracking object data corresponding to the current frame is less than the second preset threshold or the tracking object data corresponding to the current frame is empty, read the first candidate tracking object data and the second candidate tracking object data corresponding to the current frame output by the target tracking process that is not the target tracking source; Based on the first candidate tracking object data and the second candidate tracking object data, switch the target tracking source and return to the step of reading the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source; Wherein, the first candidate tracking object data is the tracking object data corresponding to the tracking source with higher priority among the other tracking sources besides the target tracking source, and the second candidate tracking object data is the tracking object data corresponding to the tracking source with lower priority among the other tracking sources besides the target tracking source.

7. The target tracking and shooting method as described in claim 6, characterized in that, The step of switching the target tracking source based on the first candidate tracking object data and the second candidate tracking object data includes: The confidence level of the first candidate detection result is determined based on the data of the first candidate tracking object. Compare the confidence level of the first alternative detection result with the value of the second preset threshold; If the confidence level of the first candidate detection result is greater than or equal to the second preset threshold, the tracking source corresponding to the first candidate tracking object data is switched to the target tracking source. If the confidence level of the first candidate detection result is less than the second preset threshold or the data of the second candidate tracking object is empty, the confidence level of the second candidate detection result is determined based on the data of the second candidate tracking object. Compare the confidence level of the second alternative detection result with the value of the second preset threshold; If the confidence level of the second alternative detection result is greater than or equal to the second preset threshold, the tracking source corresponding to the second alternative tracking object data is switched to the target tracking source.

8. The target tracking and shooting method as described in claim 5, characterized in that, When the target tracking source is a wide-angle lens video source, the step of controlling the multi-axis gimbal using a tracking algorithm based on the target tracking source to track and capture images of the target includes: Read the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source, and read the first alternative tracking object data and the second alternative tracking object data corresponding to the continuous video frames output by the target tracking process that are not corresponding to the target tracking source. If the detection result of either the first candidate tracking object data or the second candidate tracking object data is not empty, the target tracking source is switched, and based on the switched target tracking source, the multi-axis gimbal is controlled by a tracking algorithm to track and capture the target.

9. A target tracking and imaging device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the target tracking and imaging method as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the target tracking and shooting method as described in any one of claims 1 to 8.

Images