Target tracking shooting method and device and storage medium

Abstract

The invention discloses a target tracking shooting method and device and a storage medium, relates to the technical field of multi-axis pan-tilt cameras, is applied to a pan-tilt control system, and comprises the following steps: if a target tracking process is triggered, selecting a target tracking source according to a priority and target detection result confidence corresponding to each tracking source, the target tracking source is a mobile terminal video source, a telephoto lens video source or a wide-angle lens video source; the position of a tracking object output by a target tracking process corresponding to the target tracking source is read, and the position of the tracking object is the position of the tracking object in a video picture corresponding to the target tracking source; and determining a pose adjustment parameter of the multi-axis holder according to the position of the tracking object, and controlling the multi-axis holder to rotate based on the pose adjustment parameter, thereby solving the technical problem of tracking interruption caused by incapability of effectively switching alternative sources when single tracking source detection is unreliable in the prior art, and improving the tracking precision and the tracking stability.

Description

Technical Field

[0001] The present application relates to the technical field of multi-axis pan-tilt camera, and in particular to a target tracking and shooting method, device, and storage medium. Background Art

[0002] When viewers use mobile devices to live stream sports, they typically automatically track and capture athletes or moving objects with their handheld devices. However, in real-world live streams, the target is often in complex backgrounds, often experiencing high-speed movement and frequent obstructions. Using a single video source can lead to unstable target detection or even target loss, resulting in inconsistent tracking, image offset, or interruptions, impacting the smoothness and stability of live broadcasts. Summary of the Invention

[0003] The main purpose of this application is to provide a target tracking shooting method, device and storage medium, aiming to solve the technical problem in the prior art that when a single tracking source detection is unreliable, it is impossible to effectively switch to an alternative source, resulting in tracking interruption.

[0004] To achieve the above objectives, the present application proposes a target tracking and shooting method, which is applied to a pan-tilt control system. The pan-tilt control system includes a multi-axis pan-tilt, a mobile terminal, and a data processor. The multi-axis pan-tilt is provided with a telephoto lens, a wide-angle lens, and a mobile terminal fixing component. The mobile terminal is detachably connected to the multi-axis pan-tilt via the mobile terminal fixing component. The target tracking and shooting method includes: If the target tracking process is triggered, a target tracking source is selected based on the priority and the confidence level of the target detection results corresponding to each tracking source, wherein the target tracking source is a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source; Reading a tracking object position output by a target tracking process corresponding to the target tracking source, wherein the tracking object position is a position of the tracking object in a video image corresponding to the target tracking source; The multi-axis gimbal is controlled to rotate based on the posture adjustment parameters.

[0005] In some embodiments, the step of selecting a target tracking source based on the priority and the confidence level of the target detection result corresponding to each tracking source includes: Obtaining the target detection result confidence level corresponding to each tracking source; Determining the priority of each of the tracking sources; Determining, in descending order of priority, whether the confidence level of the target detection result corresponding to each tracking source is greater than or equal to a first preset threshold; When the first judgment result is yes, determining the tracking source corresponding to the judgment result as the target tracking source; Whether to proceed to the step of determining whether the confidence level of the target detection result corresponding to the non-highest-level tracking source is greater than or equal to the first preset threshold is determined by the determination result of the previous priority tracking source corresponding to the non-highest-level tracking source.

[0006] In some embodiments, the step of determining the priority of each of the tracking sources includes: In response to a scene selection operation by a user, determining a current scene category; Based on the current scene category and the preset priorities of the tracking sources under the preset scene categories, the priorities of the tracking sources under the current scene category are determined.

[0007] In some embodiments, the step of determining the posture adjustment parameters of the multi-axis gimbal according to the position of the tracked object includes: Calculating the offset between the position of the tracked object and the center of the image of the target tracking source; Obtain a current field of view angle parameter of the multi-axis gimbal, and determine the corresponding posture adjustment parameter based on the current field of view angle parameter and the offset.

[0008] In some embodiments, the step of determining the corresponding posture adjustment parameter based on the current field of view angle parameter and the offset includes: Determining a current horizontal field of view angle and a current vertical field of view angle based on the current field of view angle parameter, and determining a horizontal offset vector and a vertical offset vector based on the offset; Calculating a corresponding yaw angle adjustment vector based on the current horizontal field of view angle and the horizontal offset vector, and calculating a corresponding pitch angle adjustment vector based on the current vertical field of view angle and the vertical offset vector; The yaw angle adjustment vector and the pitch angle adjustment vector are determined as the posture adjustment parameters.

[0009] In some embodiments, after the steps of determining the posture adjustment parameters of the multi-axis gimbal according to the position of the tracked object and controlling the rotation of the multi-axis gimbal based on the posture adjustment parameters, the method further includes: Based on the target tracking source, a tracking algorithm is used to control the multi-axis pan-tilt head to track and shoot the tracked object.

[0010] In some embodiments, when the target tracking source is a mobile terminal video source or the telephoto lens video source, the step of controlling the multi-axis gimbal using a tracking algorithm based on the target tracking source to track and shoot the tracked object includes: Reading tracking object data corresponding to continuous video frames output by the target tracking process corresponding to the target tracking source; Predicting a motion trajectory of the tracked object based on the tracked object data corresponding to the current frame and the tracked object data corresponding to multiple historical frames to obtain a predicted motion trajectory of the tracked object in the next frame; generating a posture adjustment parameter of the multi-axis gimbal based on the tracking object data corresponding to the current frame and the motion trajectory prediction, and controlling the rotation of the multi-axis gimbal according to the posture adjustment parameter; The continuous video frames include a current frame and multiple historical frames, and the tracked object data includes a tracked object position and a tracked object data confidence level.

[0011] In some embodiments, before the step of predicting the motion trajectory of the tracked object based on the tracked object data corresponding to the current frame and the tracked object data corresponding to the plurality of historical frames, the step further includes: If the confidence level of the tracking object data corresponding to the current frame is less than a second preset threshold or the tracking object data corresponding to the current frame is empty, reading first candidate tracking object data corresponding to the current frame and second candidate tracking object data corresponding to the current frame output by the target tracking process that does not correspond to the target tracking source; Switching the target tracking source based on the first candidate tracking object data and the second candidate tracking object data, and returning to the step of reading the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source; The first candidate tracking object data is tracking object data corresponding to a tracking source with a higher priority among other tracking sources except the target tracking source, and the second candidate tracking object data is tracking object data corresponding to a tracking source with a lower priority among other tracking sources except the target tracking source.

[0012] In some embodiments, the step of switching the target tracking source based on the first candidate tracking object data and the second candidate tracking object data includes: determining a first candidate detection result confidence level based on the first candidate tracking object data; Comparing the confidence level of the first candidate detection result with the second preset threshold; If the confidence level of the first candidate detection result is greater than or equal to the second preset threshold, switching the tracking source corresponding to the first candidate tracking object data to the target tracking source; If the confidence level of the first candidate detection result is less than the second preset threshold or the second candidate tracking object data is empty, determining the confidence level of the second candidate detection result based on the second candidate tracking object data; Comparing the confidence level of the second candidate detection result with the second preset threshold; If the confidence level of the second candidate detection result is greater than or equal to the second preset threshold, the tracking source corresponding to the second candidate tracking object data is switched to the target tracking source.

[0013] In some embodiments, when the target tracking source is a wide-angle lens video source, the step of controlling the multi-axis gimbal using a tracking algorithm based on the target tracking source to track and shoot the tracked object includes: Reading the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source, and reading the first candidate tracking object data and the second candidate tracking object data corresponding to the continuous video frames output by the target tracking process not corresponding to the target tracking source; When it is detected that the detection and identification result of either the first candidate tracking object data or the second candidate tracking object data is not empty, the target tracking source is switched, and based on the switched target tracking source, a tracking algorithm is used to control the multi-axis gimbal to track and shoot the tracking object.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a target tracking and shooting device, which includes: A tracking source determination module is configured to select a target tracking source based on a priority and a confidence level of a target detection result corresponding to each tracking source when a target tracking process is triggered, wherein the target tracking source is a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source; a position determination module, configured to read a tracking object position output by a target tracking process corresponding to the target tracking source, wherein the tracking object position is a position of the tracking object in a video image corresponding to the target tracking source; A control module is used to determine the posture adjustment parameters of the multi-axis gimbal according to the position of the tracked object, and control the rotation of the multi-axis gimbal based on the posture adjustment parameters.

[0015] In addition, to achieve the above-mentioned purpose, the present application also proposes a target tracking and shooting device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is configured to implement the steps of the target tracking and shooting method as described above.

[0016] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the target tracking and shooting method as described above are implemented.

[0017] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the target tracking and shooting method as described above.

[0018] One or more technical solutions proposed in this application have at least the following technical effects: the target tracking shooting method is applied to a pan-tilt control system, which includes a multi-axis pan-tilt (equipped with a telephoto lens and a wide-angle lens), a mobile terminal, and a data processor. The mobile terminal can be connected to the pan-tilt through a dedicated fixed component to form a flexible and expandable shooting platform. If the target tracking process is triggered, the most suitable video source is selected from the mobile terminal video source, the telephoto lens video source, or the wide-angle lens video source as the current target tracking source based on the priority and the confidence of the target detection result corresponding to each tracking source. Combining the advantages of different cameras, the video source that best suits the current scene is selected according to actual needs, which can avoid tracking interruptions caused by the failure of a single device. The tracking object position output by the target tracking process corresponding to the target tracking source is read, where the tracking object position is the position of the tracking object in the video image corresponding to the target tracking source, which can provide a basis for subsequent adjustment of the pan-tilt posture. The posture adjustment parameters of the multi-axis gimbal are determined according to the position of the tracked object, and the rotation of the multi-axis gimbal is controlled based on the posture adjustment parameters to ensure that the tracked object is always located in the center of the screen of the target tracking source, solving the technical problem in the prior art that the alternative source cannot be effectively switched when the detection of a single tracking source is unreliable, resulting in tracking interruption, and improving the tracking accuracy and tracking stability. The target tracking shooting method disclosed in the present application can cover the needs of different scenarios by integrating three-source video of a mobile terminal, a telephoto lens, and a wide-angle lens. After triggering the tracking, the optimal tracking source is comprehensively selected as the target tracking source based on the preset priority and the real-time target detection confidence of each tracking source. Based on the tracking object position output by the target tracking process corresponding to the target tracking source, the posture adjustment parameters of the multi-axis gimbal are calculated and the multi-axis gimbal is driven to rotate to ensure that the target is always located in the center of the screen. When a single tracking source fails, it can automatically switch to other tracking sources to avoid tracking interruption, significantly improving the tracking stability and continuity in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 A flowchart of the first embodiment of the target tracking and shooting method of the present application is provided; Figure 2 Schematic diagram of the entire device involved in target tracking and shooting in this application; Figure 3 A schematic diagram of the assembly of a mobile terminal and a multi-axis gimbal provided in this application; Figure 4 A schematic diagram of an assembly of another mobile terminal and a multi-axis gimbal provided in this application; Figure 5 A schematic diagram of the process of transmitting each test result provided in this application; Figure 6 A schematic diagram of the first test result provided in this application; Figure 7 A flowchart of another target tracking and shooting method provided by this application; Figure 8 A flowchart of another target tracking and shooting method provided by this application; Figure 9 This is a schematic diagram of the module structure of the target tracking and shooting device according to an embodiment of the present application; Figure 10 Schematic diagram of the device structure of the hardware operating environment involved in the target tracking and shooting method in the embodiment of the present application.

[0022] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0023] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0024] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0025] The main solution of the embodiment of the present application is: if the target tracking process is triggered, the target tracking source is selected according to the priority and the confidence of the target detection result corresponding to each tracking source, where the target tracking source is a mobile terminal video source, a telephoto lens video source or a wide-angle lens video source; the tracking object position output by the target tracking process corresponding to the target tracking source is read, where the tracking object position is the position of the tracking object in the video picture corresponding to the target tracking source; the posture adjustment parameters of the multi-axis gimbal are determined according to the tracking object position, and the rotation of the multi-axis gimbal is controlled based on the posture adjustment parameters.

[0026] In this embodiment, for ease of description, the following description is made with the pan-tilt control system as the execution entity.

[0027] In existing technologies, when viewers use mobile devices to live stream sports scenes, they typically use their handheld devices to automatically track and capture athletes or moving objects. However, in real-world live broadcasts, the target is often in complex background environments, such as high-speed movement and frequent occlusions. A single video source can easily lead to unstable target detection or even target loss, resulting in inconsistent tracking, image offsets, or interruptions, affecting the smoothness and stability of live broadcasts.

[0028] This application provides a solution that integrates three video sources: mobile terminal, telephoto, and wide-angle. It can cover the needs of different scenarios. After triggering tracking, the optimal tracking source is selected based on the preset priority and the real-time target detection confidence of each tracking source. Based on the tracking object position output by the target tracking process corresponding to the target tracking source, the multi-axis gimbal's attitude adjustment parameters are calculated and the multi-axis gimbal is driven to rotate, ensuring that the target is always in the center of the image. If a single tracking source fails, it can automatically switch to other tracking sources to avoid tracking interruption, significantly improving tracking stability and continuity in complex scenarios.

[0029] It should be noted that the execution subject of this embodiment is a pan-tilt control system that can realize the above functions. The following takes the pan-tilt control system as an example to illustrate this embodiment and the following embodiments.

[0030] Based on this, an embodiment of the present application provides a target tracking and shooting method, which is applied to a gimbal control system. The gimbal control system includes a multi-axis gimbal, a mobile terminal and a data processor. The multi-axis gimbal is provided with a telephoto lens, a wide-angle lens and a mobile terminal fixing component. The mobile terminal is detachably connected to the multi-axis gimbal through the mobile terminal fixing component.

[0031] Reference Figure 1 , Figure 1 This is a flow chart of the first embodiment of the target tracking and shooting method of this application.

[0032] In this embodiment, the target tracking and shooting method includes steps 101 to 103: Step 101: If the target tracking process is triggered, a target tracking source is selected based on the priority and the confidence level of the target detection result corresponding to each tracking source. The target tracking source is a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source.

[0033] The target tracking and shooting method provided in the present application is applied to a pan-tilt control system, and can realize continuous and stable tracking of the tracked object by intelligently selecting and switching different video sources (such as mobile terminal video source, telephoto lens video source or wide-angle lens video source). The pan-tilt control system consists of a multi-axis pan-tilt, a mobile terminal and a data processor, wherein the multi-axis pan-tilt is equipped with a telephoto lens and a wide-angle lens, and the mobile terminal can be connected and fixed to the pan-tilt through a mobile terminal fixing component. The mobile terminal fixing component is a mechanical device used to firmly mount the mobile terminal on the multi-axis pan-tilt to ensure that its position is stable and can rotate with the multi-axis pan-tilt. The data processor is used to process the target detection results of data from different video sources, select the target tracking source, and generate the posture adjustment parameters of the multi-axis pan-tilt. In the present application, the data processor can be set in the multi-axis pan-tilt or in the mobile terminal.

[0034] Specifically, the mobile terminal is a smart terminal equipped with a camera, such as a mobile phone, which can be installed on a multi-axis gimbal. Optionally, when the mobile terminal is a mobile phone, the schematic diagram of the entire device involved in the target tracking and shooting of this application is as follows: Figure 2As shown, the actual application may be different. When the mobile phone is installed on the multi-axis gimbal, the mobile phone can participate in tracking, shooting, live broadcast and storage in the following two ways: when the mobile phone has a normally working mobile terminal lens, the mobile terminal lens participates in target detection and is used for target tracking and shooting scene shooting, and supports live broadcast or local storage; when the mobile phone has no camera or the camera cannot work properly, it completely relies on the multi-axis gimbal's built-in camera for target detection, and the image captured by the multi-axis gimbal's camera is transmitted to the mobile phone for live broadcast or storage. In addition, it can also be selected to be stored locally on the multi-axis gimbal. The above two methods can be automatically switched according to the presence or absence and working status of the mobile phone's mobile terminal lens, and can also be customized by the user. The multi-axis gimbal of the present application is at least a two-axis gimbal, that is, a gimbal including a yaw axis and a pitch axis. The yaw axis of the multi-axis gimbal rotates around the Z axis to achieve horizontal tracking of the tracked object, and the pitch axis of the multi-axis gimbal rotates around the Y axis, which can be used to achieve vertical tracking of the tracked object. When the multi-axis gimbal is a three-axis gimbal, the multi-axis gimbal also includes a roll axis. The roll axis of the multi-axis gimbal rotates around the X-axis and can be used to keep the mobile phone parallel to the ground in real time, thereby improving the stability of the shooting. When the roll axis exists, the shooting effect is better. The multi-axis gimbal of the present application is equipped with two lenses, including a wide-angle lens and a telephoto lens. Specifically, the telephoto lens has a small viewing angle and is more suitable for shooting long-distance scenes, thereby improving the tracking accuracy of distant tracking objects; the wide-angle lens has a large viewing angle and is suitable for shooting global scenes. When the target is lost in the field of view of the mobile terminal lens or the telephoto lens, a global search and adaptive tracking can be performed. The communication method between the mobile phone and the multi-axis gimbal is not restricted and can be achieved through Universal Serial Bus (USB) wired, Bluetooth, Wi-Fi or other communication protocols.

[0035] Before performing target detection on the tracked object, the mobile terminal (such as a mobile phone) needs to be fixed on the multi-axis gimbal, such as Figure 2 shown. Figure 2 The mobile terminal shown is placed flat and fixed on the multi-axis platform. This is only one embodiment. The present application can realize the installation of the mobile terminal on the multi-axis platform at any angle. It is only necessary to ensure that the mobile terminal lens of the mobile terminal is perpendicular to the multi-axis platform. For example, Figure 3 and Figure 4 The installation shown is also applicable. Figure 3 and Figure 4 This is a schematic diagram of the assembly of the mobile terminal and the multi-axis gimbal.

[0036] To achieve tracking of distant objects, this application utilizes the built-in mobile terminal camera (such as a mobile phone) in hardware design. Combined with the telephoto and wide-angle lenses on a multi-axis gimbal, the three-axis gimbal and the mobile terminal collaborate to perform distributed target detection, detecting and tracking objects and obtaining target detection results corresponding to each tracking source. The target detection results corresponding to each tracking source include a first detection result, a second detection result, and a third detection result.

[0037] The first detection result is the detection output of the tracked object by the wide-angle lens on the multi-axis gimbal, including the first center coordinates of the tracked object, the first scale data of the tracked object, and the confidence level of the first detection result. The second detection result is the detection output of the tracked object by the telephoto lens on the multi-axis gimbal, including the second center coordinates of the tracked object, the second scale data of the tracked object, and the confidence level of the second detection result. The third detection result is the detection output of the mobile terminal's mobile terminal lens, including the third center coordinates of the tracked object, the third scale data of the tracked object, and the confidence level of the third detection result. The first detection result is the target detection result based on the wide-angle lens video source captured by the wide-angle lens of the multi-axis gimbal, the second detection result is the target detection result based on the telephoto lens video source captured by the telephoto lens of the multi-axis gimbal, and the third detection result is the target detection result based on the mobile terminal's mobile terminal lens video source. The tracked object is an object that requires continuous capture and can be an athlete, vehicle, football, etc., whose position changes over time.

[0038] The above priorities represent the preset tracking source selection rules, used to quickly determine the target tracking source among multiple video sources (mobile, telephoto, and wide-angle). The target detection result confidence level represents the probability that the target detection model reliably detects the target in a video frame. The value typically ranges from 0 to 1 and can be used to quantitatively assess the reliability of the target detection result. The target tracking source is the video input source ultimately selected for target tracking. It can be a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source. The mobile terminal video source is the video stream captured by the mobile terminal lens, the telephoto lens video source is the video stream captured by the telephoto lens of a multi-axis gimbal, and the wide-angle lens video source is the video stream captured by the wide-angle lens of a multi-axis gimbal.

[0039] Optionally, target detection results from different cameras are obtained, specifically, detection results of three independent tracking objects are obtained, including the first detection result, the second detection result, and the third detection result. After the mobile terminal independently completes the target detection task under its own perspective, the third detection result is transmitted to the multi-axis gimbal through the communication protocol transmission, without the need to transmit the complete detected image, effectively saving transmission bandwidth, improving transmission efficiency, and achieving low-latency transmission. The flow chart for the transmission of each detection result can be as follows Figure 5As shown, the gimbal tracking module needs to receive each detection result. The first detection result and the second detection result on the multi-axis gimbal can be obtained directly from the local detection module, and the third detection result of the mobile terminal is obtained through communication protocol transmission. This application does not limit the communication method and supports USB wired, Bluetooth, Wi-Fi and other protocols.

[0040] Optionally, in the PTZ control system, upon receiving a command from the user or the system, the system triggers the target tracking process, analyzes the target detection results from each video source, and determines the confidence level of the target detection results corresponding to each tracking source. Based on a preset priority rule and the confidence levels of the target detection results corresponding to each tracking source, the optimal tracking source is dynamically selected, i.e., the target tracking source.

[0041] In some embodiments, when applied to a multi-axis gimbal, the step of obtaining a first detection result of a tracked object includes: Obtain wide-angle lens video frames captured by the wide-angle lens of the multi-axis gimbal; Establishing a first image coordinate system with the center point of the wide-angle lens video frame as the origin, performing linear normalization processing on the width and height of the wide-angle lens video frame to obtain a processed wide-angle lens video frame, so that the absolute value of the horizontal axis coordinate or the absolute value of the vertical axis coordinate of each vertex of the processed wide-angle lens video frame is a value of one; The processed wide-angle lens video frame is input into the target detection model to detect and identify the tracked object, and a first detection result of the tracked object in a first image coordinate system of the processed wide-angle lens video frame is obtained, wherein the first detection result includes a first center coordinate of the tracked object in the first image coordinate system, first scale data of the tracked object, and a confidence level of the first detection result.

[0042] Specifically, the first image coordinate system is a normalized coordinate system established with the center of the wide-angle lens video frame as the origin (0,0) and the boundary coordinates of the wide-angle lens video frame as ±1. The target detection model can be a deep learning model such as You Only Look Once (YOLO), Single Shot MultiBox Detector (SSD), Region-based Convolutional Neural Networks (R-CNN), etc., which is used to identify targets in the image and output position and size information. The first detection result is the target detection result of the tracked object output by the multi-axis gimbal in the first image coordinate system, including the first center coordinates of the target, the first scale data of the tracked object and the confidence of the first detection result. The schematic diagram of the first detection result can be as follows Figure 6As shown. The first center coordinate is the position of the tracked object in the first image coordinate system, such as (x=0.3, y=0.2), and the first scale data is the normalized value of the width and height of the tracked object relative to the wide-angle lens video frame size, such as w=0.25, h=0.3. In this application, the normalized value range of the width is w∈[0,2], the normalized value range of the height is h∈[0,2], and the normalized value range of the center coordinate of the tracked object is {(x,y)|x∈[-1,1],y∈[-1,1]}. The first detection result confidence is used to indicate the reliability of the first detection result. The higher the value of the first detection result confidence, the more credible it is.

[0043] As an example, a wide-angle video frame is captured using the wide-angle lens of a multi-axis gimbal. The wide-angle lens can be a fisheye or ultra-wide-angle lens. Wide-angle video frames have a large field of view, covering a wider range of scenes and suitable for global search and preliminary target localization. The resolution of the wide-angle video frame can be 1920×1080 or higher. Subsequently, a first image coordinate system is established with the image center of the wide-angle video frame as the origin. This coordinate system is centered at (0,0). The width and height of the wide-angle video frame are linearly normalized, converting the image pixel positions into relative coordinates. This ensures that the absolute values ​​of the horizontal and vertical coordinates of the four vertices of the wide-angle video frame are all 1. In other words, the entire wide-angle video frame is mapped to the standardized coordinate interval [-1,1]. This normalization eliminates scale differences between images of different resolutions, ensuring a unified representation of image data collected by different cameras. The normalized wide-angle lens video frames are fed into the target detection model, which detects and identifies the tracked objects in the processed wide-angle lens video frames, outputting a first detection result in the first image coordinate system. Through the aforementioned acquisition-coordinate system establishment-normalization-model detection process, the wide-angle lens raw images are converted into standardized target detection results, i.e., the first detection results. This eliminates the impact of image size differences on positioning and provides unified baseline data for subsequent multi-source fusion (telephoto lens and mobile terminal lens detection results).

[0044] In some embodiments, when applied to a multi-axis gimbal, the step of obtaining a second detection result of a tracked object includes: Obtaining telephoto lens video frames captured by the telephoto lens of the multi-axis gimbal; establishing a second image coordinate system with the center point of the telephoto lens video frame as the origin, and performing linear normalization processing on the width and height of the telephoto lens video frame to obtain a processed telephoto lens video frame, so that the absolute value of the horizontal axis coordinate or the absolute value of the vertical axis coordinate of each vertex of the processed telephoto lens video frame is a value of one; The processed telephoto lens video frame is input into the target detection model to detect and identify the tracked object, thereby obtaining a second detection result of the tracked object in the image coordinate system of the processed telephoto lens video frame, wherein the second detection result includes a second center coordinate of the tracked object in the second image coordinate system, second scale data of the tracked object, and a second detection result confidence level.

[0045] Specifically, the second image coordinate system is a normalized coordinate system established with the center of the telephoto lens video frame as the origin (0,0) and the boundary coordinates of the telephoto lens video frame as ±1. The second detection result is the target detection result of the tracked object output by the gimbal in the second image coordinate system, including the second center coordinates of the target, the second scale data of the tracked object and the confidence of the second detection result. The second center coordinates are the position of the tracked object in the second image coordinate system, such as (x=-0.3, y=0.2), and the second scale data are the normalized values ​​of the width and height of the tracked object relative to the size of the telephoto lens video frame, such as w=0.5, h=0.6. In this application, the normalized value range of the width is w∈[0,2], the normalized value range of the height is h∈[0,2], and the normalized value range of the center coordinates of the tracked object is {(x,y)|x∈[-1,1], y∈[-1,1]}. The second detection result confidence level is used to indicate the reliability of the second detection result. A higher second detection result confidence level indicates a more reliable result.

[0046] As an example, a multi-axis gimbal captures telephoto video frames using a telephoto lens. The telephoto lens can be a fisheye or ultra-telephoto lens. Telephoto video frames have a narrow field of view and high spatial resolution, making them suitable for accurately identifying and locating distant targets. The resolution of telephoto video frames can be 3840×2160 (4K) or higher. While the field of view is narrow, the pixel density is high. Subsequently, a second image coordinate system is established with the image center of the telephoto video frame as the origin. This coordinate system linearly normalizes the width and height of the telephoto video frame, converting the image pixel positions into relative coordinates. This ensures that the absolute values ​​of the horizontal and vertical coordinates of the four vertices of the telephoto video frame are all 1. This means that the entire telephoto video frame is mapped to the standardized coordinate interval [-1, 1]. This normalization eliminates scale differences between images of different resolutions, ensuring a unified representation of image data collected by different cameras. The normalized processed telephoto lens video frames are fed into the target detection model, which detects and identifies the tracked objects in the processed telephoto lens video frames, outputting a second detection result in a second image coordinate system. Through the aforementioned acquisition-coordinate system establishment-normalization-model detection process, the original telephoto lens images are converted into standardized target detection results, i.e., the second detection results. This eliminates the impact of image size differences on positioning and provides unified benchmark data for subsequent multi-source fusion.

[0047] In some embodiments, applied to a mobile terminal, the step of obtaining a third detection result of the tracked object includes: Acquiring a mobile terminal video frame captured by a mobile terminal lens of the mobile terminal; Establishing a third image coordinate system with the center point of the mobile terminal video frame as the origin, performing linear normalization processing on the width and height of the mobile terminal video frame to obtain a processed mobile terminal video frame, so that the absolute value of the horizontal axis coordinate or the absolute value of the vertical axis coordinate of each vertex of the processed mobile terminal video frame is a value of one; The processed mobile terminal video frame is input into the target detection model, and the tracked object is detected and identified to obtain a third detection result of the tracked object in the third image coordinate system of the processed mobile terminal video frame, wherein the third detection result includes the third center coordinates of the tracked object in the third image coordinate system, the third scale data of the tracked object and the confidence of the third detection result.

[0048] The third detection result is sent to the multi-axis gimbal.

[0049] Specifically, the third image coordinate system is a normalized coordinate system established with the center of the mobile terminal video frame as the origin (0,0) and the boundary coordinates of the mobile terminal video frame as ±1. The third detection result is the target detection result of the tracked object output by the mobile terminal in the third image coordinate system, including the third center coordinates of the target, the third scale data of the tracked object and the confidence of the third detection result. The third center coordinates are the position of the tracked object in the third image coordinate system, such as (x=0.3, y=0.2), and the third scale data are the normalized values ​​of the width and height of the tracked object relative to the size of the mobile terminal video frame, such as w=0.45, h=0.5. In this application, the normalized value range of the width is w∈[0,2], the normalized value range of the height is h∈[0,2], and the normalized value range of the center coordinates of the tracked object is {(x,y)|x∈[-1,1], y∈[-1,1]}. The third detection result confidence level is used to indicate the reliability of the third detection result. A higher confidence level of the third detection result indicates a more reliable result.

[0050] As an example, a mobile terminal video frame is captured through the mobile terminal lens of a mobile terminal. Subsequently, a third image coordinate system is established with the image center point of the mobile terminal video frame as the origin. The coordinate system takes the image center as the origin, and the width and height of the mobile terminal video frame are linearly normalized. The image pixel position is converted into relative coordinates so that the absolute value of the horizontal or vertical axis coordinates of the four vertices of the mobile terminal video frame is 1, that is, the entire mobile terminal video frame is mapped to the standardized coordinate interval of [-1, 1]. Through normalization, the scale differences between images of different resolutions are eliminated, so that the image data collected by different cameras have a unified expression basis. The processed mobile terminal video frame after normalization is input into the target detection model, and the tracked object in the processed mobile terminal video frame is detected and identified. The third detection result in the third image coordinate system is output and sent to the multi-axis gimbal. Through the above-mentioned process of acquisition - coordinate system establishment - normalization - model detection, the original image of the mobile terminal lens is converted into a standardized target detection result, that is, the third detection result, which eliminates the impact of image size differences on positioning and provides unified benchmark data for subsequent multi-source fusion.

[0051] Step 102 : Read the tracking object position output by the target tracking process corresponding to the target tracking source, wherein the tracking object position is the position of the tracking object in the video image corresponding to the target tracking source.

[0052] Specifically, the target tracking source is the best video input source selected by the system based on priority and confidence, and is used to provide target detection results in the current frame. The target tracking process is a target detection and tracking algorithm process running inside the system, which is responsible for identifying the tracked object from the video source and outputting the position (center coordinates), scale data and confidence of the tracked object in the video screen corresponding to the tracking source. In the present application, the mobile terminal video source, the telephoto lens video source and the wide-angle lens video source each have their own corresponding target tracking processes. The target tracking processes corresponding to the mobile terminal video source, the telephoto lens video source and the wide-angle lens video source are independent of each other and can be executed separately or together. The tracking object position is the coordinate of the tracking object in the video screen corresponding to the tracking source, expressed in normalized coordinates. Optionally, the tracking object position can reflect the position of the tracking object relative to the center of the video screen corresponding to the tracking source. Optionally, the tracking object position can reflect the position of the tracking object relative to the vertex of the video screen corresponding to the tracking source.

[0053] Step 103 : determining the posture adjustment parameters of the multi-axis gimbal according to the position of the tracked object, and controlling the rotation of the multi-axis gimbal based on the posture adjustment parameters.

[0054] Specifically, the pose adjustment parameters are the angles required for the multi-axis gimbal to be adjusted based on the position of the tracked object, including the yaw adjustment vector and the pitch adjustment vector. These are used to re-center the tracked object within the target tracking source's image. The tracked object's position is determined, and the multi-axis gimbal's required angle adjustment parameters are calculated based on the tracked object's position. This is done by converting the offset of the target's center coordinates into an actual spatial angle offset, taking into account the current multi-axis gimbal's pose parameters and field of view parameters, to determine the yaw and pitch adjustment vectors. For example, if the tracked object is to the left of the tracking source's image center, the multi-axis gimbal's yaw angle is increased, rotating it to the left, until the tracked object is re-centered within the tracking source's image center. The calculated gimbal adjustment parameters are then sent to the gimbal controller, which drives the gimbal's yaw and pitch motors. By converting the tracked object's position information into gimbal movements, the tracked object is always centered within the tracking source's image center. This resolves tracking instability caused by dynamic object movement or initial position deviation, improving tracking accuracy and robustness.

[0055] The target tracking shooting method proposed in this application is applied to a pan-tilt control system, which includes a multi-axis pan-tilt, a mobile terminal and a data processor. The mobile terminal can be connected to the pan-tilt through a dedicated fixed component to form a flexible and scalable shooting platform. If the target tracking process is triggered, the most suitable video source is selected from the mobile terminal video source, the telephoto lens video source or the wide-angle lens video source as the current target tracking source according to the priority and the confidence of the target detection result corresponding to each tracking source. Combining the advantages of different cameras, the video source that best suits the current scene is selected according to actual needs, which can avoid tracking interruption caused by failure of a single device. The tracking object position output by the target tracking process corresponding to the target tracking source is read, wherein the tracking object position is the position of the tracking object in the video picture corresponding to the target tracking source, which can provide a basis for subsequent adjustment of the pan-tilt posture. The posture adjustment parameters of the multi-axis pan-tilt are determined according to the tracking object position, and the rotation of the multi-axis pan-tilt is controlled based on the posture adjustment parameters to ensure that the tracking object is always located in the picture center of the target tracking source, solving the technical problem in the prior art that the alternative source cannot be effectively switched when the detection of a single tracking source is unreliable, resulting in tracking interruption, and improving the tracking accuracy and tracking stability. The target tracking shooting method disclosed in this application integrates three video sources: mobile terminal, telephoto, and wide-angle. It can cover the needs of different scenarios. After tracking is triggered, the optimal tracking source is selected based on the preset priority and the real-time target detection confidence of each tracking source. Based on the tracking object position output by the target tracking process corresponding to the target tracking source, the posture adjustment parameters of the multi-axis gimbal are calculated and the multi-axis gimbal is driven to rotate, ensuring that the target is always centered in the image. If a single tracking source fails, it can automatically switch to other tracking sources to avoid tracking interruptions, significantly improving tracking stability and continuity in complex scenarios.

[0056] In some embodiments, after the steps of determining the posture adjustment parameters of the multi-axis gimbal according to the position of the tracked object and controlling the rotation of the multi-axis gimbal based on the posture adjustment parameters, the method further includes: Based on the target tracking source, the tracking algorithm is used to control the multi-axis pan-tilt head to track and shoot the tracked object.

[0057] Specifically, the tracking algorithm is an algorithm used to analyze the position changes of the target in the video stream in real time and predict the future position of the target based on this. The tracking algorithm in this application can be optical flow method, Kalman filtering, kernelized correlation filter (KCF), deep learning-based sorting algorithm (DeepSORT), etc.

[0058] In some embodiments, based on a specific tracking source (such as a telephoto lens), a tracking algorithm, such as KCF, is fused with a Siamese Region Proposal Network Plus Plus (SiamRPN++)-based target tracking algorithm to continuously locate the target. KCF rapidly generates a target response map through frequency-domain correlation filtering, while SiamRPN++ leverages the Siamese network to extract deep features and regress bounding boxes. The combined results are weighted by the tracking source type; for example, in a telephoto scenario, SiamRPN++ uses a weight ≥ 0.8. Simultaneously, a Kalman filter predicts the target's position in the next frame, generating a feedforward control variable to compensate for latency. Closed-loop proportional-integral-derivative (PID) control converts target position deviations into gimbal motor commands, dynamically adjusting the scaling factor to accommodate field-of-view changes and ensure consistent control sensitivity. If the tracked object is lost, the tracking source is switched, and Bezier curve interpolation can be used to achieve smooth gimbal motion transitions during the switching of tracking sources.

[0059] In some embodiments, the step of selecting a target tracking source based on the priority and the confidence level of the target detection result corresponding to each tracking source includes: Obtain the confidence level of target detection results corresponding to each tracking source; Determine the priority of each tracking source; Determine whether the confidence level of the target detection result corresponding to each tracking source is greater than or equal to a first preset threshold in descending order of priority; When the first judgment result is yes, the tracking source corresponding to the judgment result is determined as the target tracking source; Whether to enter the step of determining whether the confidence level of the target detection result corresponding to the non-highest-level tracking source is greater than or equal to the first preset threshold is determined by the determination result of the previous priority tracking source corresponding to the non-highest-level tracking source.

[0060] Specifically, the tracking source is a video input source used for target detection, such as a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source. The first preset threshold is a preset confidence threshold (e.g., 0.7), which is used to determine whether the target detection result is reliable. A non-highest priority tracking source refers to a tracking source that does not have the highest priority. For example, in the priority order of telephoto lens video source > mobile terminal video source > wide-angle lens video source, the mobile terminal video source and the wide-angle lens video source are non-highest priority tracking sources. For example, in the priority order of mobile terminal video source > telephoto lens video source > wide-angle lens video source, the telephoto lens video source and the wide-angle lens video source are non-highest priority tracking sources.

[0061] Optionally, the target detection confidence level corresponding to each tracking source (e.g., mobile terminal video source, telephoto lens video source, and wide-angle lens video source) is obtained. The confidence level reflects the reliability of target detection in the video frame. Subsequently, based on the priority of each tracking source (e.g., telephoto lens video source > mobile terminal video source > wide-angle lens video source), the target detection confidence level of each tracking source is determined, in descending order of priority, to be greater than or equal to a first preset threshold (e.g., 0.7). Once the target detection confidence level corresponding to a tracking source is greater than or equal to the first preset threshold, it is determined as the target tracking source, and subsequent determinations cease. If the currently determined tracking source does not meet the threshold, the target detection confidence level corresponding to the next-highest priority tracking source is determined to be greater than or equal to the first preset threshold. Whether to proceed to determine the target detection confidence level for a non-highest priority tracking source depends on the determination result of the previous-priority tracking source. Specifically, only if the currently determined tracking source does not meet the confidence level requirement can the target detection confidence level corresponding to the next-highest priority tracking source be determined to be greater than or equal to the first preset threshold.

[0062] In some embodiments, the step of determining the priority of each tracking source includes: In response to a scene selection operation by a user, determining a current scene category; The priority of each tracking source under the current scene category is determined based on the current scene category and the preset priorities of each tracking source under each scene category.

[0063] Specifically, scene selection involves the user actively selecting the current shooting environment type through an interface or command. For example, the scene category could be sports events, stage performances, or outdoor hiking. The current scene category is the specific application scenario identified based on the user's operation, which is used to match the preset priority strategy. The preset priority is the pre-configured order of priority for each tracking source in different scenarios. For example, in a sports event, the telephoto lens video source has the highest priority; in a large-scale stage performance, the wide-angle lens video source has the highest priority; and in a meeting recording scenario, the mobile terminal video source has the highest priority.

[0064] As an example, when a user selects a current shooting scene (e.g., "Sports Event," "Stage Performance," "Outdoor Hiking," etc.) through an operating interface (such as a mobile terminal application or gimbal control panel), the system responds to the scene selection by identifying and determining the current scene category. The system has a pre-set scene-priority mapping table, which stores the preset priority strategies for each tracking source in different scenarios. For example, in a sports event scene, the telephoto lens video source has the highest priority; in a stage performance scene, the wide-angle lens video source has the highest priority. Based on the current scene category, the system searches for the corresponding priority configuration and, accordingly, determines the specific priority order of each tracking source within the current scene category. This allows the system to prioritize the video source most suitable for the current scene for target identification and tracking, achieving scene-adaptive configuration of tracking source priorities. This helps improve the intelligence, adaptability, and stability of the target tracking system, making it particularly suitable for complex application scenarios such as diverse live broadcasts and sports photography.

[0065] Optionally, refer to Figure 7 , Figure 7 A flow chart of a target tracking shooting method is provided, which is used to determine the position of the tracked object and the tracking source.

[0066] In the priority ranking of mobile terminal video source > telephoto lens video source > wide-angle lens video source, when the first detection result is a detection output of the wide-angle lens on the multi-axis gimbal for the tracked object, the second detection result is a detection output of the telephoto lens of the multi-axis gimbal for the tracked object, and the third detection result is a detection output of the mobile terminal lens of the mobile terminal, determining whether a confidence level of a target detection result corresponding to each tracking source is greater than or equal to a first preset threshold in descending order of priority; if a positive result is obtained for the first determination, determining the tracking source corresponding to the determination result as the target tracking source includes: determining a confidence level of the third detection result in a third image coordinate system based on the third detection result; Comparing the confidence level of the third detection result with the first preset threshold; If the confidence level of the third detection result is greater than or equal to the first preset threshold, determining the mobile terminal video source corresponding to the third detection result as the target tracking source; If the confidence level of the third detection result is less than the first preset threshold, comparing the confidence level of the second detection result with the first preset threshold; If the confidence level of the second detection result is greater than or equal to a first preset threshold, determining the telephoto lens video source corresponding to the second detection result as the target tracking source; If the confidence level of the second detection result is less than the first preset threshold, the wide-angle lens video source corresponding to the third detection result is determined as the target tracking source.

[0067] Specifically, the first preset threshold is a preset confidence critical value (such as 0.7), which is used to determine whether the detection result is reliable. The mobile terminal main control chip has high computing power and can process the detection results captured by the mobile terminal lens in real time at a high frame rate. The multi-axis gimbal main control chip has relatively limited computing power, but it can give full play to the hardware advantages of its telephoto lens and wide-angle lens. When the mobile terminal lens fails to detect the tracking object, the gimbal dual camera is used to assist in target detection and tracking, thereby achieving full-scene target detection and coverage. On this basis, when the confidence level of the mobile terminal's target detection result is higher than the preset threshold and the task is reliable, the mobile terminal video source is preferentially selected as the target tracking source.

[0068] As an example, a third detection result confidence level (indicating the reliability of the detection result, typically a value between 0 and 1) is determined based on the third detection result. Subsequently, the third detection result confidence level is compared with a first preset threshold (e.g., 0.7). If the third detection result confidence level is greater than or equal to the first preset threshold, it indicates that the mobile terminal's lens's current recognition result of the target has a high degree of confidence, and the mobile terminal's video source is determined as the target tracking source. This means that subsequent pan / tilt adjustments will be based on the video source provided by the mobile terminal.

[0069] If the confidence level of the third detection result is less than the first preset threshold, it indicates that the current detection result of the mobile terminal lens is not reliable enough. At this time, the next level of screening logic will be entered, that is, the confidence level of the second detection result is compared with the first preset threshold. If the confidence level of the second detection result is greater than or equal to the first preset threshold, the telephoto lens video source corresponding to the second detection result is determined as the target tracking source. That is, subsequent gimbal adjustments will be based on the video source provided by the telephoto lens.

[0070] If the confidence level of the second detection result is less than the first preset threshold, indicating that the telephoto lens is also unable to provide reliable target information, the wide-angle lens video source corresponding to the third detection result will be determined as the target tracking source by default. Although wide-angle lenses are not as accurate as mobile phones or telephoto lenses in target recognition, they have the advantages of a large field of view and wide coverage. In the event of target loss or unstable detection, they can provide basic tracking capabilities to prevent the target from completely falling out of the frame. Through the above-mentioned quantitative confidence evaluation and step-by-step screening mechanism, it is ensured that the multi-axis gimbal is always adjusted based on the most reliable detection data, achieving stable and accurate tracking of targets in a wide range of motion scenes. It is suitable for scenarios such as live sports broadcasts and security monitoring.

[0071] In some embodiments, the step of determining the posture adjustment parameters of the multi-axis gimbal according to the position of the tracked object includes: Calculate the offset between the position of the tracked object and the center of the image of the target tracking source; Get the current field of view angle parameters of the multi-axis gimbal, and determine the corresponding posture adjustment parameters based on the current field of view angle parameters and offset.

[0072] Specifically, the current field of view parameters include the current horizontal field of view angle and the current vertical field of view angle. The current field of view angle parameters determine the viewing angle range that the multi-axis gimbal can cover. The offset is the difference between the center coordinates of the tracked object and the center of the image.

[0073] By calculating the adjustment parameters of the multi-axis gimbal based on the position of the tracked object and controlling the rotation of the multi-axis gimbal accordingly, accurate and stable tracking of the tracked object can be achieved in a dynamic environment, improving the accuracy and robustness of tracking, as well as enhancing the flexibility and adaptability of the multi-axis gimbal. It is suitable for target tracking and shooting tasks in complex scenes.

[0074] In some embodiments, the step of determining corresponding posture adjustment parameters based on the current field of view angle parameter and the offset includes: Determine a current horizontal field of view angle and a current vertical field of view angle based on the current field of view angle parameter, and determine a horizontal offset vector and a vertical offset vector based on the offset; Calculating a corresponding yaw angle adjustment vector based on the current horizontal field of view angle and the horizontal offset vector, and calculating a corresponding pitch angle adjustment vector based on the current vertical field of view angle and the vertical offset vector; The yaw angle adjustment vector and the pitch angle adjustment vector are determined as posture adjustment parameters.

[0075] Specifically, the horizontal offset vector is the horizontal offset, and the vertical offset vector is the vertical offset. The yaw angle adjustment vector is the horizontal angle change that the multi-axis gimbal needs to adjust to bring the tracked object back to the center of the video image corresponding to the target tracking source. The pitch angle adjustment vector is the vertical angle change that the multi-axis gimbal needs to adjust to bring the tracked object back to the center of the video image corresponding to the target tracking source.

[0076] As an example, the data processor reads the current field of view parameters stored internally in the gimbal, or measures them in real time using sensors. The current horizontal field of view reflects the camera's horizontal coverage range, while the current vertical field of view corresponds to the vertical coverage range. Together, these two parameters determine the captured image range. The tracked object position, representing the current center coordinates of the target within the video frame corresponding to the target tracking source, indicates the tracked object's position relative to the center. By calculating the difference between the target center coordinates and the center of the video frame corresponding to the target tracking source, horizontal and vertical offset vectors are derived. A larger absolute value of the offset vector indicates a greater degree of deviation of the tracked object from the center of the frame. The horizontal offset vector reflects the horizontal deviation ratio of the tracked object from the center, while the vertical offset vector reflects the vertical deviation ratio. Furthermore, a yaw angle adjustment vector is calculated based on the horizontal field of view and the horizontal offset vector. The yaw angle adjustment vector indicates the horizontal rotation angle required to move the multi-axis gimbal to the center of the frame. Similarly, a pitch angle adjustment vector is calculated based on the vertical field of view and the vertical offset vector. The pitch angle adjustment vector indicates the vertical rotation angle required to move the multi-axis gimbal. The yaw and pitch adjustment vectors are used as adjustment parameters for the multi-axis gimbal, driving the motors to return the tracked object to the center of the video frame corresponding to the target tracking source. Dynamic conversion of the field of view angle and offset allows for precise mapping of the target position to the gimbal's movements, improving tracking accuracy. The system also dynamically adapts to different shooting scenarios, ensuring the tracked object is centered in the video frame corresponding to the target tracking source, providing reliable input for subsequent closed-loop control.

[0077] refer to Figure 8 , Figure 8A flow chart corresponding to another target tracking shooting method is provided for determining posture adjustment parameters. A mobile terminal acquires a mobile terminal video source for target detection, performs target detection on the tracked object based on the mobile terminal video source, and obtains a target detection result confidence corresponding to the mobile terminal video source and a tracked object position corresponding to the mobile terminal video source; a multi-axis gimbal acquires a telephoto lens video source, performs target detection on the tracked object based on the telephoto lens video source, and obtains a target detection result confidence corresponding to the telephoto lens video source and a tracked object position corresponding to the telephoto lens video source; and a multi-axis gimbal acquires a wide-angle lens video source, performs target detection on the tracked object based on the wide-angle lens video source, and obtains a target detection result confidence corresponding to the wide-angle lens video source and a tracked object position corresponding to the wide-angle lens video source. The data processor reads the target detection result confidence level corresponding to the mobile terminal video source, the target detection result confidence level corresponding to the telephoto lens video source, the target detection result confidence level corresponding to the wide-angle lens video source, and the priority level of each tracking source. Based on the target detection result confidence level corresponding to the mobile terminal video source, the target detection result confidence level corresponding to the telephoto lens video source, the target detection result confidence level corresponding to the wide-angle lens video source, and the priority level of each tracking source, the data processor selects a target tracking source from the target tracking sources of the mobile terminal video source, the telephoto lens video source, and the wide-angle lens video source. The data processor then reads the tracking object position output by the target tracking process corresponding to the target tracking source, thereby obtaining one of the tracking object position corresponding to the mobile terminal video source, the tracking object position corresponding to the telephoto lens video source, or the target detection result confidence level corresponding to the wide-angle lens video source. The data processor determines the posture adjustment parameters of the multi-axis gimbal based on the tracking object position.

[0078] In some embodiments, when the target tracking source is a mobile terminal video source or a telephoto lens video source, the steps of controlling the multi-axis gimbal based on the target tracking source using a tracking algorithm to track and shoot the tracked object include: Read the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source; Based on the tracking object data corresponding to the current frame and the tracking object data corresponding to multiple historical frames, the motion trajectory of the tracked object is predicted to obtain the predicted motion trajectory of the tracked object in the next frame; Based on the tracking object data and motion trajectory prediction corresponding to the current frame, the posture adjustment parameters of the multi-axis gimbal are generated, and the rotation of the multi-axis gimbal is controlled according to the posture adjustment parameters; The continuous video frames include a current frame and multiple historical frames, and the tracking object data includes a tracking object position and a tracking object data confidence level.

[0079] Specifically, the tracking object data is the tracking object position, scale data and confidence information in each video frame.

[0080] As an example, when the tracking source is a mobile terminal camera or a telephoto lens of a multi-axis gimbal, the mobile terminal camera or telephoto lens captures continuous video frames of the tracked object in a moving scene at a fixed frame rate (e.g., 30 fps). For example, it captures the current frame and the 30 historical frames from the most recent second. A tracking algorithm is then used to process each frame, outputting tracking object data corresponding to the continuous video frames, including tracking object data for the current frame (frame N) and tracking object data for multiple historical frames (frames N-1 to N-30). The tracking algorithm combines the tracking object data from the current frame with the historical frames to predict the motion trajectory of the next frame (frame N+1), obtaining a predicted motion trajectory for the next frame. After obtaining the tracking object data corresponding to the current frame and the predicted motion trajectory for the tracked object in the next frame, the predicted motion trajectory is combined with the tracking object data corresponding to the current frame to generate a feedforward control variable, namely, a pose adjustment parameter, to pre-rotate the gimbal so that the multi-axis gimbal is already in a pre-adjusted state when the tracked object enters the center of the video frame corresponding to the target tracking source. Furthermore, the posture adjustment parameters are input into the PID controller to drive the yaw and pitch axis motors to rotate until the position of the tracked object coincides with the center of the video image corresponding to the target tracking source. During the target tracking process, the adjustment parameters are updated and the gimbal position is corrected every time a frame of image is processed, forming a real-time control loop.

[0081] In addition, the control parameters need to be automatically scaled according to the field of view of the tracking source. For example, the adjustment angle corresponding to the same offset under a telephoto lens is larger, while the adjustment angle under a mobile terminal lens is smaller, so as to achieve consistent control response speed at different focal lengths.

[0082] In some embodiments, before the step of predicting the motion trajectory of the tracked object based on the tracked object data corresponding to the current frame and the tracked object data corresponding to multiple historical frames, the step further includes: If the confidence level of the tracking object data corresponding to the current frame is less than a second preset threshold or the tracking object data corresponding to the current frame is empty, reading the first candidate tracking object data corresponding to the current frame and the second candidate tracking object data corresponding to the current frame output by the target tracking process corresponding to the non-target tracking source; Switching the target tracking source based on the first candidate tracking object data and the second candidate tracking object data, and returning to the step of reading the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source; The first candidate tracking object data is tracking object data corresponding to a tracking source with a higher priority among other tracking sources except the target tracking source, and the second candidate tracking object data is tracking object data corresponding to a tracking source with a lower priority among other tracking sources except the target tracking source.

[0083] Specifically, in this application, the priority ranking for each tracking source can be: mobile terminal video source > telephoto lens video source > wide-angle lens video source. The second preset threshold is a preset confidence threshold (e.g., 0.6), which is used to determine the reliability of the tracking object data corresponding to the current frame. Empty tracking object data corresponding to the current frame means that no tracking object is detected in the current frame, and the tracking algorithm outputs no valid data, such as empty fields such as the tracking object position and scale data. The first candidate tracking object data is the tracking object data corresponding to a higher priority tracking source among the tracking sources other than the target tracking source, and the second candidate tracking object data is the tracking object data corresponding to a lower priority tracking source among the tracking sources other than the target tracking source. Optionally, in the case of a tracking source priority ranking of mobile terminal video source > telephoto lens video source > wide-angle lens video source, if the target tracking source is the mobile terminal video source, the first candidate tracking object data is the tracking object data corresponding to the telephoto lens video source, and the second candidate tracking object data is the tracking object data corresponding to the wide-angle lens video source. Optionally, when the target tracking source is a telephoto lens video source, the first candidate tracking object data is the tracking object data corresponding to the mobile terminal video source, and the second candidate tracking object data is the tracking object data corresponding to the wide-angle lens video source.

[0084] As an example, before predicting the target motion trajectory, the reliability of the tracking object data of the current frame is determined. Specifically, if the confidence level of the tracking object data corresponding to the current frame is lower than a second preset threshold (e.g., 0.6), or if no tracking object is detected in the current frame (i.e., the tracking object data is empty), this indicates that the video source information provided by the currently used camera is unreliable or missing and cannot be used for subsequent trajectory prediction and pan / tilt control. To avoid target loss or pan / tilt malfunction, a multi-camera collaborative mechanism is adopted to obtain first and second candidate tracking object data. If the confidence level of the first candidate data is high and the target information is complete, the tracking source corresponding to the first candidate tracking object data is set as the new target tracking source. If the confidence level of the first candidate detection result is lower than the second preset threshold or the second candidate tracking object data is empty, but the second candidate data is available (i.e., the confidence level of the second candidate detection result is greater than or equal to the second preset threshold), the tracking source corresponding to the second candidate tracking object data is set as the new target tracking source. After the target tracking source is switched, the target detection process based on the tracking algorithm is executed again, and the continuous video frames collected by the new tracking source are re-analyzed to obtain the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source, and then continue with subsequent trajectory prediction and pan / tilt control operations. By introducing the above-mentioned multi-camera collaboration mechanism and intelligent switching strategy, if the current target tracking source fails or becomes unstable, it automatically switches to another tracking source and uses the most reliable tracking object data for tracking decisions, avoiding tracking interruptions caused by single camera detection failures. This provides technical support for achieving continuous, stable, and high-quality automatic tracking shooting.

[0085] In some embodiments, the step of switching the target tracking source based on the first candidate tracking object data and the second candidate tracking object data includes: determining a first candidate detection result confidence level based on the first candidate tracking object data; Comparing the confidence level of the first candidate detection result with a second preset threshold; If the confidence level of the first candidate detection result is greater than or equal to a second preset threshold, switching the tracking source corresponding to the first candidate tracking object data to the target tracking source; If the confidence level of the first candidate detection result is less than a second preset threshold or the second candidate tracking object data is empty, determining the confidence level of the second candidate detection result based on the second candidate tracking object data; Comparing the confidence level of the second candidate detection result with a second preset threshold; If the confidence level of the second candidate detection result is greater than or equal to a second preset threshold, the tracking source corresponding to the second candidate tracking object data is switched to the target tracking source.

[0086] Specifically, when it is detected that the confidence level of the tracking object data corresponding to the current frame is lower than a second preset threshold, or when no tracking object is detected in the current frame, first candidate tracking object data is obtained, and a first candidate detection result confidence level of the data is extracted from the first candidate tracking object data. The first candidate detection result confidence level is compared with the second preset threshold. If the first candidate detection result confidence level is greater than or equal to the second preset threshold, indicating that the target detection result provided by the tracking source corresponding to the first candidate tracking object data is sufficiently reliable, the tracking source corresponding to the first candidate tracking object data is switched to the target tracking source, and the target detection and tracking process based on the tracking algorithm is restarted. If the first candidate detection result confidence level is still lower than the second preset threshold, or the first candidate tracking object data is empty, a second candidate detection result confidence level corresponding to the second candidate tracking object data is extracted, and the second candidate detection result confidence level is compared with the second preset threshold. If the second candidate detection result confidence level is greater than or equal to the second preset threshold, the tracking source corresponding to the second candidate tracking object data is switched to the target tracking source. If the requirements are still not met, a global search mechanism is initiated or a user intervention is prompted.

[0087] In some embodiments, when the target tracking source is a wide-angle lens video source, the steps of controlling a multi-axis gimbal based on the target tracking source using a tracking algorithm to track and shoot the tracked object include: Reading tracking object data corresponding to continuous video frames output by a target tracking process corresponding to a target tracking source, and reading first candidate tracking object data and second candidate tracking object data corresponding to continuous video frames output by a target tracking process corresponding to a non-target tracking source; When it is detected that any detection and identification result of the first candidate tracking object data or the second candidate tracking object data is not empty, the target tracking source is switched, and based on the switched target tracking source, the multi-axis gimbal is controlled by using a tracking algorithm to track and shoot the tracking object.

[0088] Specifically, when the target tracking source is a wide-angle lens video source, which is typically used for large-scale target searches, wide-angle lens video source frames have wide coverage but weak detail recognition capabilities. Using the wide-angle lens video source as the current target tracking source, the tracking algorithm processes the video frames captured by the wide-angle lens to generate tracking object data for the current frame (including center coordinates, scale, and confidence). Simultaneously, the system calls on the remaining cameras (such as the telephoto lens and the mobile terminal lens) to perform parallel detection on the same tracked object, generating first candidate tracking object data (detection results from the mobile terminal) and second candidate tracking object data (detection results corresponding to the telephoto lens video source). If either the first candidate tracking object data or the second candidate tracking object data is not empty, the target is detected, the target tracking source is switched, and the tracking algorithm process is reset.

[0089] When either the first candidate tracking object data or the second candidate tracking object data is not empty, optionally, if the first candidate tracking object data is not empty and the second candidate tracking object data is empty, the tracking source corresponding to the first candidate tracking object data is switched to the target tracking source. Optionally, if the first candidate tracking object data is empty and the second candidate tracking object data is not empty, the tracking source corresponding to the second candidate tracking object data is switched to the target tracking source.

[0090] Optionally, when both the first alternative tracking object data and the second alternative tracking object data are not empty, the first alternative detection result confidence is determined based on the first alternative tracking object data; the first alternative detection result confidence is compared with the second preset threshold; if the first alternative detection result confidence is greater than or equal to the second preset threshold, the tracking source corresponding to the first alternative tracking object data is switched to the target tracking source; if the first alternative detection result confidence is less than the second preset threshold, the second alternative detection result confidence is determined based on the second alternative tracking object data; the second alternative detection result confidence is compared with the second preset threshold; if the second alternative detection result confidence is greater than or equal to the second preset threshold, the tracking source corresponding to the second alternative tracking object data is switched to the target tracking source.

[0091] After the switch is complete, the tracking process restarts based on the switched tracking source. This means the tracking algorithm continues to analyze the continuous video frames captured by the switched camera. Based on this analysis, the gimbal control instructions are generated to drive the multi-axis gimbal to adjust its posture, re-centering the tracked object and maintaining tracking. By incorporating high-precision detection results from other cameras, the limited target recognition accuracy of wide-angle lenses is compensated, enabling intelligent switching of tracking sources and dynamic optimization of tracking strategies.

[0092] In addition, during the process of tracking and shooting the tracked object, the mobile terminal can obtain its camera image through the application development interface to achieve real-time tracking, shooting, storage and live broadcast functions. The multi-axis gimbal can also achieve the same functions.

[0093] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the target tracking and shooting method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0094] This application also provides a target tracking and shooting device, please refer to Figure 9 , the target tracking and shooting device includes: Tracking source determination module 901 is used to select a target tracking source based on the priority and the confidence level of the target detection results corresponding to each tracking source when the target tracking process is triggered. The target tracking source can be a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source. A position determination module 902 is configured to read a tracking object position output by a target tracking process corresponding to a target tracking source, wherein the tracking object position is a position of the tracking object in a video image corresponding to the target tracking source; The control module 903 is used to determine the posture adjustment parameters of the multi-axis gimbal according to the position of the tracked object, and control the rotation of the multi-axis gimbal based on the posture adjustment parameters.

[0095] The target tracking and shooting device provided in this application, which utilizes the target tracking and shooting method in the above-described embodiments, can resolve the technical problem in the prior art of being unable to effectively switch to an alternative source when detection of a single tracking source is unreliable, leading to tracking interruption. Compared to the prior art, the beneficial effects of the target tracking and shooting device provided in this application are the same as those of the target tracking and shooting method provided in the above-described embodiments, and the other technical features of the target tracking and shooting device are the same as those disclosed in the above-described embodiments and are not further described here.

[0096] The present application provides a target tracking and shooting device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the target tracking and shooting method in the above-mentioned embodiment one.

[0097] Reference below Figure 10 , which shows a schematic structural diagram of a target tracking and shooting device suitable for implementing an embodiment of the present application. Figure 10 The target tracking and shooting device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0098] like Figure 10As shown, the target tracking and photographing device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the target tracking and photographing device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007, including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008, including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003, including, for example, a magnetic tape, hard disk, etc.; and communication devices 1009. Communication devices 1009 can allow the target tracking and shooting device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a target tracking and shooting device with various systems, it should be understood that implementation or presence of all the illustrated systems is not required. More or fewer systems may alternatively be implemented or present.

[0099] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are performed.

[0100] The target tracking and shooting device provided in this application, which utilizes the target tracking and shooting method in the above-described embodiment, can resolve the technical problem in the prior art of being unable to effectively switch to an alternative source when detection of a single tracking source is unreliable, resulting in tracking interruption. Compared to the prior art, the beneficial effects of the target tracking and shooting device provided in this application are the same as those of the target tracking and shooting method provided in the above-described embodiment. Other technical features of this target tracking and shooting device are the same as those disclosed in the method in the above-described embodiment and are not further described here.

[0101] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0102] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0103] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer program) stored thereon, wherein the computer-readable program instructions are used to execute the target tracking and shooting method in the above embodiment.

[0104] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0105] The computer-readable storage medium may be included in the target tracking and photographing device; or may exist independently without being assembled into the target tracking and photographing device.

[0106] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a target tracking and shooting device, the target tracking and shooting device: if a target tracking process is triggered, selects a target tracking source based on the priority and the confidence level of the target detection results corresponding to each tracking source, wherein the target tracking source is a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source; reads the tracking object position output by the target tracking process corresponding to the target tracking source, wherein the tracking object position is the position of the tracking object in the video screen corresponding to the target tracking source; determines the posture adjustment parameters of the multi-axis gimbal according to the tracking object position, and controls the rotation of the multi-axis gimbal based on the posture adjustment parameters.

[0107] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0109] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0110] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned target tracking and photographing method. This computer-readable storage medium can address the prior art technical issue of inability to effectively switch to an alternative source when detection of a single tracking source is unreliable, leading to tracking interruption. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the target tracking and photographing method provided in the aforementioned embodiments and are not further elaborated here.

[0111] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned target tracking and shooting method when executed by a processor.

[0112] The computer program product provided in this application can resolve the technical problem in the prior art of tracking interruption caused by the inability to effectively switch to an alternative source when detection of a single tracking source becomes unreliable. Compared to the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the target tracking and shooting method provided in the aforementioned embodiments, and are not further elaborated here.

[0113] The above are only some embodiments of the present application and are not intended to limit the patent scope of the present application. All equivalent structural transformations made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A target tracking and shooting method, characterized in that: Applied to a pan / tilt control system, the pan / tilt control system includes a multi-axis pan / tilt, a mobile terminal, and a data processor, the multi-axis pan / tilt is provided with a telephoto lens, a wide-angle lens, and a mobile terminal fixing component, the mobile terminal is detachably connected to the multi-axis pan / tilt via the mobile terminal fixing component, and the target tracking and shooting method includes: If the target tracking process is triggered, a target tracking source is selected based on the priority and the confidence level of the target detection results corresponding to each tracking source, wherein the target tracking source is a mobile terminal video source, a telephoto lens video source, or a wide-angle lens video source; Reading a tracking object position output by a target tracking process corresponding to the target tracking source, wherein the tracking object position is a position of the tracking object in a video image corresponding to the target tracking source; The multi-axis gimbal is controlled to rotate based on the posture adjustment parameters.

2. The target tracking and shooting method according to claim 1, wherein: The step of selecting a target tracking source according to the priority and the confidence level of the target detection result corresponding to each tracking source includes: Obtaining the target detection result confidence level corresponding to each tracking source; In response to a scene selection operation by a user, determining a current scene category; Determining the priority of each tracking source under the current scene category based on the current scene category and the preset priorities of each tracking source under each preset scene category; Determining, in descending order of priority, whether the confidence level of the target detection result corresponding to each tracking source is greater than or equal to a first preset threshold; When the first judgment result is yes, determining the tracking source corresponding to the judgment result as the target tracking source; Whether to proceed to the step of determining whether the confidence level of the target detection result corresponding to the non-highest-level tracking source is greater than or equal to the first preset threshold is determined by the determination result of the previous priority tracking source corresponding to the non-highest-level tracking source.

3. The target tracking and shooting method according to claim 1, wherein: The step of determining the posture adjustment parameters of the multi-axis gimbal according to the position of the tracked object includes: Calculating the offset between the position of the tracked object and the center of the image of the target tracking source; Acquiring a current field of view angle parameter of the multi-axis gimbal, determining a current horizontal field of view angle and a current vertical field of view angle based on the current field of view angle parameter, and determining a horizontal offset vector and a vertical offset vector based on the offset; Calculating a corresponding yaw angle adjustment vector based on the current horizontal field of view angle and the horizontal offset vector, and calculating a corresponding pitch angle adjustment vector based on the current vertical field of view angle and the vertical offset vector; The yaw angle adjustment vector and the pitch angle adjustment vector are determined as the posture adjustment parameters.

4. The target tracking and shooting method according to claim 1, wherein: After the steps of determining the posture adjustment parameters of the multi-axis gimbal according to the position of the tracked object and controlling the rotation of the multi-axis gimbal based on the posture adjustment parameters, the method further includes: Based on the target tracking source, a tracking algorithm is used to control the multi-axis pan-tilt head to track and shoot the tracked object.

5. The target tracking and shooting method according to claim 1, wherein: In the case where the target tracking source is a mobile terminal video source or the telephoto lens video source, the step of controlling the multi-axis gimbal using a tracking algorithm based on the target tracking source to track and shoot the tracked object includes: Reading tracking object data corresponding to continuous video frames output by the target tracking process corresponding to the target tracking source; Predicting a motion trajectory of the tracked object based on the tracked object data corresponding to the current frame and the tracked object data corresponding to multiple historical frames to obtain a predicted motion trajectory of the tracked object in the next frame; generating a posture adjustment parameter of the multi-axis gimbal based on the tracking object data corresponding to the current frame and the motion trajectory prediction, and controlling the rotation of the multi-axis gimbal according to the posture adjustment parameter; The continuous video frames include a current frame and multiple historical frames, and the tracked object data includes a tracked object position and a tracked object data confidence level.

6. The target tracking and shooting method according to claim 5, wherein: Before the step of predicting the motion trajectory of the tracked object based on the tracked object data corresponding to the current frame and the tracked object data corresponding to the plurality of historical frames, the method further includes: If the confidence level of the tracking object data corresponding to the current frame is less than a second preset threshold or the tracking object data corresponding to the current frame is empty, reading first candidate tracking object data corresponding to the current frame and second candidate tracking object data corresponding to the current frame output by the target tracking process that does not correspond to the target tracking source; Switching the target tracking source based on the first candidate tracking object data and the second candidate tracking object data, and returning to the step of reading the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source; The first candidate tracking object data is tracking object data corresponding to a tracking source with a higher priority among other tracking sources except the target tracking source, and the second candidate tracking object data is tracking object data corresponding to a tracking source with a lower priority among other tracking sources except the target tracking source.

7. The target tracking and shooting method according to claim 6, wherein: The step of switching the target tracking source based on the first candidate tracking object data and the second candidate tracking object data includes: determining a first candidate detection result confidence level based on the first candidate tracking object data; Comparing the confidence level of the first candidate detection result with the second preset threshold; If the confidence level of the first candidate detection result is greater than or equal to the second preset threshold, switching the tracking source corresponding to the first candidate tracking object data to the target tracking source; If the confidence level of the first candidate detection result is less than the second preset threshold or the second candidate tracking object data is empty, determining the confidence level of the second candidate detection result based on the second candidate tracking object data; Comparing the confidence level of the second candidate detection result with the second preset threshold; If the confidence level of the second candidate detection result is greater than or equal to the second preset threshold, the tracking source corresponding to the second candidate tracking object data is switched to the target tracking source.

8. The target tracking and shooting method according to claim 5, wherein: In the case where the target tracking source is a wide-angle lens video source, the step of controlling the multi-axis pan-tilt head using a tracking algorithm based on the target tracking source to track and shoot the tracked object includes: Reading the tracking object data corresponding to the continuous video frames output by the target tracking process corresponding to the target tracking source, and reading the first candidate tracking object data and the second candidate tracking object data corresponding to the continuous video frames output by the target tracking process not corresponding to the target tracking source; When it is detected that the detection and identification result of either the first candidate tracking object data or the second candidate tracking object data is not empty, the target tracking source is switched, and based on the switched target tracking source, a tracking algorithm is used to control the multi-axis gimbal to track and shoot the tracking object.

9. A target tracking and shooting device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the target tracking and shooting method according to any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the target tracking and shooting method according to any one of claims 1 to 8 are implemented.

Images