A target intelligent tracking system and method for a mobile phone holder

Abstract

This invention relates to the technical field of handheld gimbal stabilization, and discloses a target intelligent tracking system and method for a mobile phone gimbal. The tracking method is applied to the gimbal device and specifically includes the following steps: S11: After the device is started, the gimbal control module drives the three-axis brushless motor to perform an attitude self-calibration program to reset to the preset horizontal initial position. The light sensor built into the image acquisition module detects the light intensity data of the current shooting environment in real time and transmits the data to the target recognition module in real time. This invention effectively avoids the tedious operation of manual selection and switching and the problem of accidental tracking loss. When the target is lost due to occlusion, the occlusion recovery unit based on Kalman filter trajectory prediction can automatically start a low-speed scanning search, and combined with the feature cache matching mechanism, achieves fast relocking, solving the long-standing pain point of the inability to automatically recover after the target is lost in the prior art.

Description

Technical Field

[0001] This invention relates to the technical field of handheld gimbal stabilization, and more specifically, to a target intelligent tracking system and method for a mobile phone gimbal. Background Technology

[0002] With the increasing power of smartphone photography functions and the explosive growth of new media formats such as short videos and live streaming, smartphone gimbals (commonly known as handheld stabilizers) have become an indispensable shooting aid for content creators and ordinary consumers. In order to further improve the shooting experience and the success rate of shooting, intelligent target tracking function has emerged and has quickly become a core selling point of mid-to-high-end smartphone gimbals.

[0003] Currently, most mainstream mobile gimbal target tracking technologies rely on the phone's own visual computing capabilities. They use a dedicated app developed by the gimbal manufacturer to access the phone's camera and employ computer vision algorithms to identify and locate selected faces, bodies, or specific objects in the shot. The gimbal then receives the target's position offset calculated by the phone via Bluetooth Low Energy or a wired connection, driving a motor to rotate and keep the target centered in the frame or in a preset composition position. This approach, to some extent, frees the photographer's hands, achieving a "follow-the-camera" effect and reducing shooting difficulty. However, target tracking solutions relying on mobile phone computing power and dedicated apps still have the following drawbacks in practical applications: (1) Since all image processing and target recognition algorithms run on the mobile phone, the real-time performance and stability of tracking are severely limited by the mobile phone model, processor performance and system resource usage, resulting in huge differences in tracking effects between different mobile phones. Moreover, when recording 4K high-definition, the mobile phone's computing power is difficult to simultaneously handle video encoding and target tracking, often resulting in screen stuttering, severe frame skipping in the tracking frame, or even tracking loss. The total delay is generally as high as 200 milliseconds or more, making the gimbal movement always half a beat slow and unable to achieve smooth tracking. (2) Existing visual tracking solutions mostly use a single RGB camera, which is extremely sensitive to the lighting environment. In low-light scenarios such as indoor lighting, dusk or night scene, the accuracy of the target detection algorithm drops sharply, the frame loss rate and false recognition rate surge, seriously affecting the shooting experience.

[0004] In summary, existing mobile phone gimbal target tracking technologies have technical bottlenecks in terms of perception accuracy, processing latency, and scene adaptability. There is an urgent need for a new system and method to overcome these limitations in order to meet users' growing demand for high-quality and intelligent shooting. Summary of the Invention

[0005] The purpose of this invention is to provide a target intelligent tracking system and method for mobile phone gimbals. When a target is lost due to occlusion, the occlusion recovery unit based on Kalman filter trajectory prediction can automatically start a low-speed scanning search and achieve rapid relocking by combining a feature cache matching mechanism, thus solving the long-standing problem of the inability to automatically recover after the target is lost in the prior art.

[0006] This invention is implemented as follows: a target intelligent tracking method for a mobile phone gimbal, applied to a gimbal device, specifically includes the following steps: S11: After the device is started, the gimbal control module drives the three-axis brushless motor to perform the attitude self-calibration program and reset to the preset horizontal initial position. The light sensor built into the image acquisition module detects the light intensity data of the current shooting environment in real time and transmits the data to the target recognition module in real time. At the same time, the target recognition module loads the preset fusion recognition algorithm model, initializes the multi-target priority judgment rule according to the user's historical usage records, and completes the resource preparation before target tracking. S12: The image acquisition module synchronously acquires the original image data within the current field of view. The acquired original image data is transmitted in real time to the ISP acceleration unit integrated in the target recognition module via the high-speed data transmission interface. The ISP acceleration unit performs two-dimensional noise reduction filtering, image contrast enhancement, edge sharpening and color restoration correction on the original image data to generate a sequence of image frames to be recognized that meets the requirements. S13: The target recognition module receives the image frame sequence to be recognized after preprocessing by the ISP acceleration unit, performs real-time dynamic sorting of the image frame sequence through a weighted summation algorithm, accurately selects the target with the highest comprehensive score from multiple potential targets in a complex scene as the current tracking target, and continuously outputs the precise pixel coordinates of the current tracking target in the image frame and the predicted motion trajectory data to the gimbal control module. S14: The gimbal control module receives the precise position coordinates and predicted motion trajectory data of the currently tracked target in real time, and performs high-frequency fusion of the visual data with the current roll angle, pitch angle and yaw angle attitude data of the gimbal device collected and calculated in real time by the attitude sensor. It calculates the real-time rotation angle and angular velocity of the horizontal motor and pitch motor of the gimbal device required to relock the target to the preset composition point, and generates the corresponding PWM drive command. The three-axis brushless motor rotates according to the drive command, driving the gimbal device to adjust its attitude in real time to achieve target following. S15: During continuous tracking, if the target recognition module fails to detect the feature information of the currently tracked target in more than a preset number of image frames, it is determined that the target has entered an occlusion state or temporarily moved out of the shooting frame. The occlusion recovery unit is immediately triggered to start working. When the gimbal device receives a stop command or the target loss timeout occurs, it stops tracking and resets. At the same time, it automatically saves the personalized configuration for the next call.

[0007] Furthermore, in S11, the light sensor built into the image acquisition module detects the light intensity data of the current shooting environment in real time, including: The target recognition module dynamically decides and performs adaptive adjustments to the image acquisition strategy based on the received light intensity data. When the detected light intensity is higher than the preset light threshold, it controls the RGB camera to start working as the main acquisition unit, while turning off the infrared camera and putting the adjustable fill light into standby mode. When the detected light intensity is lower than or equal to the preset light threshold, the RGB camera and infrared camera are activated simultaneously, and the adjustable fill light is also started. The target recognition module calculates and dynamically adjusts the brightness level and color temperature parameters of the fill light in real time according to the real-time light intensity value through the built-in fill light control algorithm, so as to realize the synchronous enhanced acquisition of target detail information and infrared contour information in low-light environment.

[0008] Furthermore, in S12, the image acquisition module synchronously acquires raw image data within the current field of view, including: When the ambient light is sufficient, a high-definition first image stream containing target color and texture detail information is captured by an RGB camera alone; When ambient light is insufficient, the RGB camera and the infrared camera work together to acquire a first image stream containing the target's color and texture details and a second image stream containing the target's infrared radiation contour features, respectively, forming dual-mode image data with complementary information characteristics.

[0009] Further, in S13, the target recognition module receives the image frame sequence to be recognized after preprocessing by the ISP acceleration unit, and performs real-time dynamic sorting of the image frame sequence using a weighted summation algorithm, including: The sequence of image frames to be identified is identified and located in real time using the hardware-accelerated YOLOv8 algorithm. The biometrics and contour features of each candidate target are extracted. Based on user-preset rules, target motion state and multi-dimensional indicators of image clarity, the candidate targets are dynamically prioritized to determine the highest priority tracking object. Simultaneously, the Kalman filter algorithm running on the target recognition module is used to predict the motion trajectory, instantaneous velocity and acceleration of each candidate target in subsequent frames in real time based on the historical position information and motion velocity of each candidate target in consecutive multi-frame images. The predicted data is then fused with the current recognition results to provide the gimbal control module with an advanced position correction amount.

[0010] Furthermore, the sequence of image frames to be identified is used in real time to identify and locate all faces, human bodies, and objects in the image using a hardware-accelerated YOLOv8 algorithm, extracting the biometric and contour features of each candidate target, including: The hardware-accelerated YOLOv8 algorithm identifies and accurately locates the bounding boxes of all faces, human bodies, pets, and specific objects in image frames in real time at a processing speed of no less than 30 frames per second. It simultaneously extracts the biometric vectors of each candidate target and stores them in the feature cache unit. The synchronous extraction algorithm is as follows: ; The parameters include: I t The single frame image input at time t; Θ: The set of weight parameters that are fixed after the YOLOv8 network is trained; H: The function for the entire forward inference process, which takes the image and network parameters as input and outputs the set of detected targets. N t The total number of targets detected in the image at time t; bi (t) : The bounding box parameters of the i-th target, representing the center coordinates, width, and height (x, y, y). i ,y i ,w i ,h i ); FΘ(I t ): A feature map output by an intermediate layer during the forward propagation of a network; Ψ: Synchronous feature extraction operator, which uses RoIAlign to crop the region corresponding to the bounding box from the feature map and maps it to a d-dimensional feature vector through a lightweight fully connected network. f i (t) The biometric vector extracted from the i-th target is used for subsequent target matching and re-identification. The multi-target priority determination unit triggers the construction of a multi-dimensional evaluation index model, which comprehensively considers the tracking preference rules preset by the user through the interaction module, the motion state attributes of each candidate target, and the target feature clarity calculated based on the image clarity evaluation function, and performs real-time dynamic sorting of candidate targets.

[0011] Furthermore, in S14, the visual data is fused with the current roll, pitch, and yaw attitude data of the gimbal device collected and calculated in real time by the attitude sensor at a high frequency. This process calculates the real-time rotation angles and angular velocities of the gimbal device's horizontal and pitch motors required to relock the target to the preset mapping point, including: The gimbal control module performs high-frequency fusion of visual data and attitude data of the gimbal device, which are collected and calculated in real time by the attitude sensor, including the current roll angle, pitch angle, and yaw angle. It calculates the positional deviation of the target from the preset composition point and, based on this, calculates the real-time rotation angle and angular velocity of the horizontal and pitch motors required to relock the target to the composition point. By using a PID closed-loop control algorithm running in the gimbal control module, the motor response is driven based on the target's position deviation from the center of the screen and the target's predicted motion speed. If a sudden change in the target's motion state is detected during continuous tracking, the proportional, integral, and derivative coefficients of the PID control algorithm are dynamically and adaptively tuned according to the target's velocity change rate and acceleration change rate. The motor's response speed, maximum torque output, and tracking sensitivity threshold are adjusted synchronously to ensure that the target remains stably locked in the center of the screen even when moving rapidly or changing direction, thus achieving linkage control between the visual recognition system and the three-axis brushless motor.

[0012] Furthermore, in S15, the occlusion recovery unit is immediately triggered to start operation, including: When the target recognition module determines that the currently tracked target has been lost due to occlusion or moving out of the frame, it immediately triggers the occlusion recovery unit to start working. The occlusion recovery unit predicts and caches the last valid motion trajectory data before the target is lost based on the Kalman filter algorithm, extracts the instantaneous velocity vector, motion direction angle and acceleration value at the moment the target is lost, and then extrapolates the spatial location area where the target is most likely to appear at the current moment by combining the kinematic equation with the loss time, and generates a predicted scanning path covering high probability points based on the probability distribution characteristics of the area. The occlusion recovery unit controls the PTZ device to enter a low-speed search and scan mode, and performs a traversal scan within the predicted area at a constant angular velocity of 5° / s according to a preset gridded or spiral path. Meanwhile, the feature caching unit continuously outputs the biometric vector of the lost target and performs high-frequency fast feature comparison with the image frames acquired in real time during the scanning process. Once a candidate target matching the cached feature data is detected in the scanning area, the occlusion recovery unit immediately terminates the scanning action and, based on the target's current position and the attitude deviation of the gimbal device, quickly adjusts the attitude of the gimbal device with an optimized speed curve to accurately lock onto the target again, automatically and seamlessly returning to the normal tracking process.

[0013] Furthermore, the PTZ device stops tracking and resets upon receiving a stop command or when the target is lost and a timeout occurs, while automatically saving personalized configurations for future use, including: When a user issues a stop tracking command through the interaction module, or when the occlusion recovery unit continues to search for a target that has been lost for a longer period than the preset maximum loss time threshold, the PTZ device immediately terminates all tracking actions, the visual recognition algorithm pauses target locking, the motor drive signal is interrupted, and the device switches from dynamic following state to standby processing stage. The gimbal control module drives the three-axis brushless motor to automatically reset the gimbal device to its initial standby posture. At the same time, the target recognition module saves the personalized parameter configurations generated during this tracking process or manually adjusted by the user, including priority weights, tracking sensitivity, and supplementary lighting parameters, to the local memory for quick recall upon the next startup, realizing the user's habit memory for quick subsequent recall.

[0014] Compared with the prior art, the target intelligent tracking system and method for mobile phone gimbals provided by the present invention have the following beneficial effects: 1. By adopting a dual-camera fusion design of RGB and infrared, combined with a light sensor and adjustable fill light, scene-adaptive acquisition is achieved. Even in low-light environments, it can accurately capture target details and contours. At the same time, the gimbal device has built-in hardware-accelerated YOLOv8 algorithm and Kalman filter algorithm, which can independently complete target recognition and trajectory prediction without relying on the computing power of mobile phones, with latency controlled within 90 milliseconds. By constructing a multi-target priority judgment model, and comprehensively considering user-preset rules, target motion state and clarity and other multi-dimensional indicators, intelligent sorting and seamless switching of multiple targets in complex scenes are achieved. This effectively avoids the tedious operation of manual selection and switching and the problem of mistracking. When a target is lost due to occlusion, the occlusion recovery unit based on Kalman filter trajectory prediction can automatically start low-speed scanning search, and combined with feature cache matching mechanism to achieve fast relocking, solving the long-standing pain point of the inability to automatically recover targets after loss in existing technologies. 2. The image acquisition module adopts a magnetic quick-release and magnetic charging integrated design. The multi-target priority judgment in the target recognition module constructs a dynamic comprehensive scoring model. Through real-time weighted calculation of user preset preferences, motion state characteristics, and image clarity, it achieves human-like intelligent decision-making. This multi-dimensional dynamic logic enables deep understanding and optimization of complex shooting scenarios. The gimbal control module, through adaptive tuning of PID parameters, can adjust the motor response characteristics in real time according to changes in the target's motion speed and acceleration, achieving deep collaboration between visual recognition and the motor, ensuring stable locking under rapid directional changes. The interaction module uses grayscale difference and contour recognition dual verification to break the ecological limitations of dedicated APPs. The power module can automatically switch to deep sleep mode according to the system's working status, significantly extending battery life. After tracking ends, the system can automatically save the user's personalized parameter configuration, realizing the memory and rapid recovery of usage habits. At the same time, the various features work together to form a high-precision, low-latency, and highly adaptable intelligent tracking system.

[0015] A target intelligent tracking system for a mobile phone gimbal, used to execute the above-described target intelligent tracking method, the system comprising: The image acquisition module is used to detect ambient light intensity and adaptively acquire RGB and infrared multimodal image data; The target recognition module is used for local real-time identification and tracking of targets, prediction of motion trajectories, determination of the priority of multiple targets, and occlusion recovery. The gimbal control module is used to drive a three-axis brushless motor to achieve high-precision following based on target data, and to achieve closed-loop control by integrating attitude sensor data; The interaction module is used to receive user gestures and remote control commands and provide feedback on the system status; The power supply module is used to supply power to each module and perform intelligent power consumption scheduling.

[0016] Specifically, the target recognition module includes: The multi-target priority determination unit is used to dynamically score and rank candidate targets by comprehensively considering user-preset rules, target motion status, and clarity. The feature caching unit is used to store the target biometric vector for matching and re-identification; An occlusion recovery unit is used to initiate a low-speed scan and quickly re-lock on the target after it is lost, based on a Kalman filter-predicted trajectory. The ISP acceleration unit is used to perform noise reduction, enhancement, and color correction preprocessing on the original image. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a target intelligent tracking method for a mobile phone gimbal proposed in this invention. Figure 2 This is a schematic diagram of the structure of a target intelligent tracking system for a mobile phone gimbal proposed in this invention; Figure 3 This is a schematic diagram of the target recognition module in a target intelligent tracking system for mobile phone gimbals proposed in this invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] The implementation of the present invention will be described in detail below with reference to specific embodiments.

[0020] In the accompanying drawings of this embodiment, the same or similar reference numerals correspond to the same or similar components. In the description of this invention, it should be understood that if terms such as "upper," "lower," "left," and "right" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting this invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0021] Reference Figure 1 As shown, a target intelligent tracking method for a mobile phone gimbal, applied to a gimbal device, specifically includes the following steps: S11: After the device is started, the gimbal control module drives the three-axis brushless motor to perform an attitude self-calibration program to reset to the preset horizontal initial position. At the same time, the six-axis attitude sensor completes initialization and prepares to collect gimbal attitude data in real time with an output frequency of not less than 1kHz. The light sensor built into the image acquisition module detects the light intensity data of the current shooting environment in real time, and judges the ambient light level according to the preset light threshold, and transmits the data to the target recognition module in real time. The image acquisition module adopts a magnetic quick-release structure, and integrates a micro battery and magnetic charging contacts. It supports automatic adsorption and charging when switching between front and rear lenses, ensuring that the module can maintain sufficient power in any use state. At the same time, the target recognition module loads a preset fusion recognition algorithm model. This model is based on the YOLOv8 target detection algorithm and Kalman filter prediction algorithm. The chip has a built-in neural network acceleration unit and ISP acceleration unit, which can independently complete the real-time processing of images. It initializes multi-target priority judgment rules according to the user's historical usage records, including preset preferences such as face priority and pet priority, and reads the personalized parameter configuration stored in the local memory to complete the resource preparation before target tracking. S12: The image acquisition module synchronously acquires the original image data within the current field of view. When the ambient light is sufficient, only the RGB camera acquires the high-definition first image stream. When the ambient light is insufficient, the RGB camera and the infrared camera work together to acquire the first image stream of target color and texture details and the second image stream of infrared radiation contour features, respectively, forming dual-mode image data with complementary information characteristics. The acquired original image data is transmitted in real time to the ISP acceleration unit integrated in the target recognition module via the high-speed data transmission interface. The ISP acceleration unit performs two-dimensional noise reduction filtering, image contrast enhancement, edge sharpening and color restoration correction on the original image data. At the same time, it performs high dynamic range synthesis and bad pixel correction to generate a high-quality, low-noise image frame sequence to be recognized and stores it in a circular buffer for subsequent processing. S13: The target recognition module receives the image frame sequence to be recognized after preprocessing by the ISP acceleration unit. Using the hardware-accelerated YOLOv8 algorithm, it identifies and accurately locates the bounding boxes of all faces, humans, pets, and specific objects appearing in the image frames in real time at a processing speed of no less than 30 frames per second. Simultaneously, it extracts the biometric vectors of each candidate target and stores them in the feature cache unit. A weighted summation algorithm is used to dynamically sort the image frame sequence in real time. The multi-target priority determination unit constructs a decision model that integrates multi-dimensional evaluation indicators, comprehensively considering the tracking preference rules preset by the user through the interaction module, the motion state attributes of each candidate target, and graph-based... The system calculates the sharpness of target features using a sharpness evaluation function. It then uses a weighted summation algorithm to dynamically score all candidate targets in real time. From multiple potential targets in a complex scene, it accurately selects the target with the highest comprehensive score as the current tracking target. Simultaneously, it utilizes the Kalman filter algorithm running on the target recognition module to predict the motion trajectory, instantaneous velocity, and acceleration of each candidate target in subsequent frames based on the historical position information and motion velocity of each candidate target in multiple consecutive frames. The predicted data is then fused with the current recognition result, and the system continuously outputs the precise pixel coordinates of the currently tracked target in the image frame and the predicted motion trajectory data to the gimbal control module. S14: The gimbal control module receives the precise position coordinates and predicted motion trajectory data of the currently tracked target in real time. It then performs high-frequency fusion of the visual data with the gimbal's current roll, pitch, and yaw attitude data collected and calculated in real time by the attitude sensor. This calculates the real-time rotation angle and angular velocity of the gimbal's horizontal and pitch motors required to relock the target to the preset composition point. Through the PID closed-loop control algorithm running in the gimbal control module, the motor response is driven based on the target's position deviation from the center of the image and the target's predicted motion speed. If a sudden change in the target's motion state is detected during continuous tracking, the proportional, integral, and derivative coefficients of the PID control algorithm are dynamically and adaptively tuned according to the target's velocity and acceleration change rates. This synchronously adjusts the motor's response speed, maximum torque output, and tracking sensitivity threshold to ensure that the target remains stably locked in the center of the image even when moving rapidly or changing direction. This achieves deep collaborative control between the visual recognition system and the three-axis brushless motor, stabilizing the angle control error within ±0.05°. The module generates corresponding PWM drive commands, and the three-axis brushless motor rotates according to the drive commands, causing the gimbal to adjust its attitude in real time to achieve target following. S15: During continuous tracking, if the target recognition module fails to detect the feature information of the currently tracked target in more than a preset number of image frames, it determines that the target has entered an occlusion state or temporarily moved out of the shooting frame, and immediately triggers the occlusion recovery unit to start working: This unit predicts and caches the last valid motion trajectory data before the target is lost based on the Kalman filter algorithm, extracts the instantaneous velocity vector, motion direction angle, and acceleration value at the moment the target is lost, and then extrapolates the spatial location area where the target is most likely to appear at the current moment by combining the kinematic equation with the loss time, and generates a predicted scanning path covering high probability points based on the probability distribution characteristics of the area; the occlusion recovery unit controls the pan-tilt device to enter a low-speed search scanning mode, and performs traversal scanning in the predicted area according to a preset gridded or spiral path at a constant angular velocity of 5° / s; at the same time, the feature caching unit continuously outputs the biometric vector of the lost target and performs high-frequency fast feature comparison with the image frames acquired in real time during the scanning process. Once the target is detected in the scan, the occlusion recovery unit will immediately detect the target's biometric vector. When a candidate target matching the cached feature data is detected within the area, the occlusion recovery unit immediately terminates the scanning action and, based on the target's current position and the gimbal's attitude deviation, quickly adjusts the gimbal's attitude with an optimized speed curve to accurately re-lock the target, automatically and seamlessly resuming the normal tracking process. The gimbal stops tracking and resets upon receiving a stop command or when the target loss timeout occurs. Specifically, when a stop tracking command issued by the user through the interaction module is detected, or when the occlusion recovery unit continues searching for a target after loss for a period exceeding the preset maximum loss time threshold, the gimbal immediately terminates all tracking actions, the visual recognition algorithm pauses target locking, the motor drive signal is interrupted, and the device switches from dynamic following state to standby processing stage. The gimbal control module drives the three-axis brushless motor to automatically reset the gimbal to its initial standby posture. At the same time, the target recognition module saves the personalized parameter configurations generated during this tracking process or manually adjusted by the user, including priority weights, tracking sensitivity, and supplementary lighting parameters, to local memory for quick retrieval upon the next startup.

[0022] In S11 of this embodiment, the light sensor built into the image acquisition module detects the light intensity data of the current shooting environment in real time, including: The target recognition module dynamically decides and performs adaptive adjustments to the image acquisition strategy based on the received light intensity data. When the detected light intensity is higher than the preset light threshold, it controls the RGB camera to start working as the main acquisition unit, while turning off the infrared camera and putting the adjustable fill light into standby mode. When the detected light intensity is lower than or equal to the preset light threshold, the RGB camera and infrared camera are activated simultaneously, and the adjustable fill light is also turned on. The target recognition module calculates and dynamically adjusts the brightness level and color temperature parameters of the fill light in real time based on the real-time light intensity value through the built-in fill light control algorithm, so as to realize the synchronous enhanced acquisition of target detail information and infrared contour information in low-light environment.

[0023] In S12 of this embodiment, the image acquisition module synchronously acquires raw image data within the current field of view, including: When the ambient light is sufficient, a high-definition first image stream containing target color and texture detail information is captured by an RGB camera alone; When ambient light is insufficient, the RGB camera and the infrared camera work together to acquire a first image stream containing the target's color and texture details and a second image stream containing the target's infrared radiation contour features, respectively, forming dual-mode image data with complementary information characteristics.

[0024] In S13 of this embodiment, the target recognition module receives the image frame sequence to be recognized after preprocessing by the ISP acceleration unit, and performs real-time dynamic sorting of the image frame sequence using a weighted summation algorithm, including: The sequence of image frames to be identified uses a hardware-accelerated YOLOv8 algorithm to identify and locate all faces, bodies, and objects in the image in real time. It extracts the biometrics and contour features of each candidate target and dynamically prioritizes them based on user-preset rules, target motion state, and multi-dimensional indicators of image clarity. The algorithm then determines the highest priority tracking object, including: The hardware-accelerated YOLOv8 algorithm identifies and accurately locates the bounding boxes of all faces, human bodies, pets, and specific objects in image frames in real time at a processing speed of no less than 30 frames per second. It simultaneously extracts the biometric vectors of each candidate target and stores them in the feature cache unit. The synchronous extraction algorithm is as follows: ; The parameters include: I t The single frame image input at time t; Θ: The set of weight parameters that are fixed after the YOLOv8 network is trained; H: The function for the entire forward inference process, which takes the image and network parameters as input and outputs the set of detected targets. N t The total number of targets detected in the image at time t; bi (t) : The bounding box parameters of the i-th target, representing the center coordinates, width, and height (x, y, y). i ,y i ,wi ,h i ); FΘ(I t ): A feature map output by an intermediate layer during the forward propagation of a network; Ψ: Synchronous feature extraction operator, which uses RoIAlign to crop the region corresponding to the bounding box from the feature map and maps it to a d-dimensional feature vector through a lightweight fully connected network. f i (t) The biometric vector extracted from the i-th target is used for subsequent target matching and re-identification. The multi-target priority determination unit triggers the construction of a multi-dimensional evaluation index model, which comprehensively considers the tracking preference rules preset by the user through the interaction module, the motion state attributes of each candidate target, and the target feature clarity calculated based on the image clarity evaluation function, and performs real-time dynamic sorting of candidate targets. Simultaneously, the Kalman filter algorithm running on the target recognition module is used to predict the motion trajectory, instantaneous velocity and acceleration of each candidate target in subsequent frames in real time based on the historical position information and motion velocity of each candidate target in consecutive multi-frame images. The predicted data is then fused with the current recognition results to provide the gimbal control module with an advanced position correction amount.

[0025] In S14 of this embodiment, the visual data is fused with the current roll angle, pitch angle, and yaw angle attitude data of the gimbal device collected and calculated in real time by the attitude sensor at a high frequency. The real-time rotation angle and angular velocity of the horizontal and pitch motors of the gimbal device required to relock the target to the preset composition point are calculated, including: The gimbal control module performs high-frequency fusion of visual data and attitude data of the gimbal device, which are collected and calculated in real time by the attitude sensor, including the current roll angle, pitch angle, and yaw angle. It calculates the positional deviation of the target from the preset composition point and, based on this, calculates the real-time rotation angle and angular velocity of the horizontal and pitch motors required to relock the target to the composition point. By using a PID closed-loop control algorithm running in the gimbal control module, the motor response is driven based on the target's position deviation from the center of the screen and the target's predicted motion speed. If a sudden change in the target's motion state is detected during continuous tracking, the proportional, integral, and derivative coefficients of the PID control algorithm are dynamically and adaptively tuned according to the target's velocity change rate and acceleration change rate. The motor's response speed, maximum torque output, and tracking sensitivity threshold are adjusted synchronously to ensure that the target remains stably locked in the center of the screen even when moving rapidly or changing direction, thus achieving linkage control between the visual recognition system and the three-axis brushless motor.

[0026] In S15 of this embodiment, the occlusion recovery unit is immediately triggered to start working, including: When the target recognition module determines that the currently tracked target has been lost due to occlusion or moving out of the frame, it immediately triggers the occlusion recovery unit to start working. The occlusion recovery unit predicts and caches the last valid motion trajectory data before the target is lost based on the Kalman filter algorithm, extracts the instantaneous velocity vector, motion direction angle and acceleration value at the moment the target is lost, and then extrapolates the spatial location area where the target is most likely to appear at the current moment by combining the kinematic equation with the loss time, and generates a predicted scanning path covering high probability points based on the probability distribution characteristics of the area. The algorithm for predicting the scan path generation is as follows: ; in: r(s): The position coordinates of a point on the scan path, where s is the arc length parameter; : The predicted center location of the target; R(θ): Rotation matrix that aligns the scan direction to the principal axis of the covariance matrix, where θ is determined by Σ. k∣k-1 The eigenvectors are determined; λ1,λ2: Covariance matrix Σ k∣k-1 The eigenvalues ​​represent the lengths of the major and minor semi-axes of the uncertain ellipse; α, β: scaling factors used to control the scan coverage area; s: Normalized scan parameter, incrementing from 0 to s max , where s max Determined by the preset maximum search radius (s) max =max(3λ1,3λ2); 2πs: Phase angle of the spiral scan, ensuring that the path covers the prediction area in a spiral pattern; The scanning path is essentially an Archimedean spiral stretched along the principal axis of an uncertain ellipse. Its pitch is adaptively adjusted by λ1 and λ2, enabling priority dense scanning in high-probability areas where the target is most likely to appear, and sparse scanning in low-probability areas, thus balancing search efficiency and recapture success rate. The occlusion recovery unit controls the PTZ device to enter a low-speed search and scan mode, and performs a traversal scan within the predicted area at a constant angular velocity of 5° / s according to a preset gridded or spiral path. Meanwhile, the feature caching unit continuously outputs the biometric vector of the lost target and performs high-frequency fast feature comparison with the image frames acquired in real time during the scanning process. Once a candidate target matching the cached feature data is detected in the scanning area, the occlusion recovery unit immediately terminates the scanning action and, based on the target's current position and the attitude deviation of the gimbal device, quickly adjusts the attitude of the gimbal device with an optimized speed curve to accurately lock onto the target again, automatically and seamlessly returning to the normal tracking process.

[0027] In this embodiment, the gimbal device stops tracking and resets upon receiving a stop command or when the target is lost and a timeout occurs. Simultaneously, it automatically saves the personalized configuration for future use, including: When a user issues a stop tracking command through the interaction module, or when the occlusion recovery unit continues to search for a target that has been lost for a longer period than the preset maximum loss time threshold, the PTZ device immediately terminates all tracking actions, the visual recognition algorithm pauses target locking, the motor drive signal is interrupted, and the device switches from dynamic following state to standby processing stage. The gimbal control module drives the three-axis brushless motor to automatically reset the gimbal device to its initial standby posture. At the same time, the target recognition module saves the personalized parameter configurations generated during this tracking process or manually adjusted by the user, including priority weights, tracking sensitivity, and supplementary lighting parameters, to the local memory for quick recall upon the next startup, realizing the user's habit memory for quick subsequent recall.

[0028] This technical solution employs a dual-camera fusion design combining RGB and infrared sensors, along with a light sensor and adjustable fill light, to achieve scene-adaptive acquisition. Even in low-light environments, it can accurately capture target details and outlines. Simultaneously, the gimbal device incorporates hardware-accelerated YOLOv8 and Kalman filter algorithms, enabling independent target recognition and trajectory prediction without relying on mobile phone computing power, with latency controlled within 90 milliseconds. By constructing a multi-target priority judgment model, integrating user-preset rules, target motion state, and clarity, it achieves intelligent sorting and seamless switching of multiple targets in complex scenes, effectively avoiding the tedious operation of manual selection and switching and the problem of accidental loss. When a target is lost due to occlusion, the occlusion recovery unit based on Kalman filter-predicted trajectory can automatically initiate a low-speed scanning search, combined with a feature cache matching mechanism to achieve rapid re-locking, solving the long-standing pain point of existing technologies where targets cannot be automatically recovered after loss. Reference Figure 2-3As shown, a target intelligent tracking system for a mobile phone gimbal is used to execute the aforementioned target intelligent tracking method. The system includes: an image acquisition module for detecting ambient light intensity and adaptively acquiring RGB and infrared multimodal image data; a target recognition module for local real-time recognition of tracking targets, prediction of motion trajectories, determination of multi-target priorities, and occlusion recovery; a gimbal control module for driving a three-axis brushless motor to achieve high-precision following based on target data and fusing attitude sensor data to achieve closed-loop control; an interaction module for receiving user gestures and remote control commands and providing feedback on system status; and a power supply module for supplying power to each module and performing intelligent power consumption scheduling. By constructing a multi-target priority determination model and comprehensively considering user-preset rules, target motion status, and clarity, the system achieves intelligent sorting and seamless switching of multiple targets in complex scenarios, effectively avoiding the tedious operation of manual selection and switching and the problem of accidental tracking loss. When a target is lost due to occlusion, the occlusion recovery unit based on Kalman filter-predicted trajectory can automatically start a low-speed scanning search and achieve rapid relocking by combining a feature cache matching mechanism, solving the long-standing pain point of the inability to automatically recover targets after loss in existing technologies.

[0029] Specifically, the target recognition module includes: a multi-target priority determination unit, used to dynamically score and rank candidate targets based on user-preset rules, target motion state, and sharpness; a feature caching unit, used to store target biometric vectors for matching and re-identification; an occlusion recovery unit, used to initiate low-speed scanning and quickly re-lock after target loss based on Kalman filter trajectory prediction; and an ISP acceleration unit, used to perform noise reduction, enhancement, and color correction preprocessing on the original image. The image acquisition module adopts an integrated design of magnetic quick-release and magnetic charging. The multi-target priority determination in the target recognition module constructs a dynamic comprehensive scoring model, which realizes human-like intelligent decision-making through real-time weighted calculation of user-preset preferences, motion state features, and image sharpness. This multi-dimensional dynamic logic enables deep understanding and optimization of complex shooting scenarios.

[0030] The gimbal control module of this technical solution can adjust the motor response characteristics in real time according to the changes in the target's speed and acceleration through adaptive tuning of PID parameters, realizing deep collaboration between visual recognition and motor, and ensuring stable locking under rapid directional changes. The interaction module adopts dual verification of grayscale difference and contour recognition to break the ecological limitations of dedicated APP. The power module can automatically switch to deep sleep mode according to the system's working status, greatly extending the battery life. After tracking ends, the system can automatically save the user's personalized parameter configuration, realizing the memory and rapid recovery of usage habits. At the same time, the various features work together to form a high-precision, low-latency, and highly adaptable intelligent tracking system.

[0031] In this embodiment, the entire operation process can be automated by computer control. In each operation stage, sensors can be set up to provide signal feedback and ensure that the steps are performed sequentially. These are all conventional knowledge of current automation control, and will not be elaborated on in this embodiment.

[0032] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A target intelligent tracking method for a mobile phone holder, characterized in that, When applied to gimbal devices, the specific steps include: S11: After the device is started, the gimbal control module drives the three-axis brushless motor to perform the attitude self-calibration program and reset to the preset horizontal initial position. The light sensor built into the image acquisition module detects the light intensity data of the current shooting environment in real time and transmits the data to the target recognition module in real time. At the same time, the target recognition module loads the preset fusion recognition algorithm model, initializes the multi-target priority judgment rule according to the user's historical usage records, and completes the resource preparation before target tracking. S12: The image acquisition module synchronously acquires the original image data within the current field of view. The acquired original image data is transmitted in real time to the ISP acceleration unit integrated in the target recognition module via the high-speed data transmission interface. The ISP acceleration unit performs two-dimensional noise reduction filtering, image contrast enhancement, edge sharpening and color restoration correction on the original image data to generate a sequence of image frames to be recognized that meets the requirements. S13: The target recognition module receives the image frame sequence to be recognized after preprocessing by the ISP acceleration unit, performs real-time dynamic sorting of the image frame sequence through a weighted summation algorithm, accurately selects the target with the highest comprehensive score from multiple potential targets in a complex scene as the current tracking target, and continuously outputs the precise pixel coordinates of the current tracking target in the image frame and the predicted motion trajectory data to the gimbal control module. S14: The gimbal control module receives the precise position coordinates and predicted motion trajectory data of the currently tracked target in real time, and performs high-frequency fusion of the visual data with the current roll angle, pitch angle and yaw angle attitude data of the gimbal device collected and calculated in real time by the attitude sensor. It calculates the real-time rotation angle and angular velocity of the horizontal motor and pitch motor of the gimbal device required to relock the target to the preset composition point, and generates the corresponding PWM drive command. The three-axis brushless motor rotates according to the drive command, driving the gimbal device to adjust its attitude in real time to achieve target following. S15: During continuous tracking, if the target recognition module fails to detect the feature information of the currently tracked target in more than a preset number of image frames, it is determined that the target has entered an occlusion state or temporarily moved out of the shooting frame. The occlusion recovery unit is immediately triggered to start working. When the gimbal device receives a stop command or the target loss timeout occurs, it stops tracking and resets. At the same time, it automatically saves the personalized configuration for the next call.

2. The target intelligent tracking method for a mobile phone holder according to claim 1, wherein, In S11, the built-in light sensor of the image acquisition module detects the light intensity data of the current shooting environment in real time, including: The target recognition module dynamically decides and performs adaptive adjustments to the image acquisition strategy based on the received light intensity data. When the detected light intensity is higher than the preset light threshold, it controls the RGB camera to start working as the main acquisition unit, while turning off the infrared camera and putting the adjustable fill light into standby mode. When the detected light intensity is lower than or equal to the preset light threshold, the RGB camera and infrared camera are activated simultaneously, and the adjustable fill light is also started. The target recognition module calculates and dynamically adjusts the brightness level and color temperature parameters of the fill light in real time according to the real-time light intensity value through the built-in fill light control algorithm, so as to realize the synchronous enhanced acquisition of target detail information and infrared contour information in low-light environment.

3. The target intelligent tracking method for a mobile phone gimbal as described in claim 2, characterized in that, In S12, the image acquisition module synchronously acquires raw image data within the current field of view, including: When the ambient light is sufficient, a high-definition first image stream containing target color and texture detail information is captured by an RGB camera alone; When ambient light is insufficient, the RGB camera and the infrared camera work together to acquire a first image stream containing the target's color and texture details and a second image stream containing the target's infrared radiation contour features, respectively, forming dual-mode image data with complementary information characteristics.

4. The target intelligent tracking method for a mobile phone gimbal as described in claim 3, characterized in that, In S13, the target recognition module receives the image frame sequence to be recognized after preprocessing by the ISP acceleration unit, and performs real-time dynamic sorting of the image frame sequence using a weighted summation algorithm, including: The sequence of image frames to be identified is identified and located in real time using the hardware-accelerated YOLOv8 algorithm. The biometrics and contour features of each candidate target are extracted. Based on user-preset rules, target motion state and multi-dimensional indicators of image clarity, the candidate targets are dynamically prioritized to determine the highest priority tracking object. Simultaneously, the Kalman filter algorithm running on the target recognition module is used to predict the motion trajectory, instantaneous velocity and acceleration of each candidate target in subsequent frames in real time based on the historical position information and motion velocity of each candidate target in consecutive multi-frame images. The predicted data is then fused with the current recognition results to provide the gimbal control module with an advanced position correction amount.

5. The target intelligent tracking method for a mobile phone holder according to claim 4, wherein, The sequence of image frames to be identified is used in real time to identify and locate all faces, human bodies, and objects in the image using a hardware-accelerated YOLOv8 algorithm, extracting the biometric and contour features of each candidate target, including: The hardware-accelerated YOLOv8 algorithm identifies and accurately locates the bounding boxes of all faces, human bodies, pets, and specific objects in image frames in real time at a processing speed of no less than 30 frames per second. It simultaneously extracts the biometric vectors of each candidate target and stores them in the feature cache unit. The synchronous extraction algorithm is as follows: ； The parameters include: I t : single frame image input at time t; Θ: The set of weight parameters that are fixed after the YOLOv8 network is trained; H: The function for the entire forward inference process, which takes the image and network parameters as input and outputs the set of detected targets. N t : total number of targets detected in the image at time t; bi (t) : bounding box parameters of the i-th target, respectively representing the center coordinates, width and height (x i ,y i ,w i ,h i ) FΘ(I t ): feature map of an intermediate layer output in the network forward propagation process; Ψ: Synchronous feature extraction operator, which uses RoIAlign to crop the region corresponding to the bounding box from the feature map and maps it to a d-dimensional feature vector through a lightweight fully connected network. f i (t) : the i-th target extracted biometric feature vector, used for subsequent target matching and re-identification; The multi-target priority determination unit triggers the construction of a multi-dimensional evaluation index model, which comprehensively considers the tracking preference rules preset by the user through the interaction module, the motion state attributes of each candidate target, and the target feature clarity calculated based on the image clarity evaluation function, and performs real-time dynamic sorting of candidate targets.

6. The target intelligent tracking method for a mobile phone holder according to claim 5, wherein, In S14, visual data is fused with real-time roll, pitch, and yaw attitude data of the gimbal device collected and calculated by the attitude sensor at high frequency. This process calculates the real-time rotation angles and angular velocities of the gimbal device's horizontal and pitch motors required to relock the target to the preset positioning point, including: The gimbal control module performs high-frequency fusion of visual data and attitude data of the gimbal device, which are collected and calculated in real time by the attitude sensor, including the current roll angle, pitch angle, and yaw angle. It calculates the positional deviation of the target from the preset composition point and, based on this, calculates the real-time rotation angle and angular velocity of the horizontal and pitch motors required to relock the target to the composition point. By using a PID closed-loop control algorithm running in the gimbal control module, the motor response is driven based on the target's position deviation from the center of the screen and the target's predicted motion speed. If a sudden change in the target's motion state is detected during continuous tracking, the proportional, integral, and derivative coefficients of the PID control algorithm are dynamically and adaptively tuned according to the target's velocity change rate and acceleration change rate. The motor's response speed, maximum torque output, and tracking sensitivity threshold are adjusted synchronously to ensure that the target remains stably locked in the center of the screen even when moving rapidly or changing direction, thus achieving linkage control between the visual recognition system and the three-axis brushless motor.

7. The target intelligent tracking method for a mobile phone holder according to claim 6, wherein, In S15, the occlusion recovery unit is immediately triggered to start working, including: When the target recognition module determines that the currently tracked target has been lost due to occlusion or moving out of the frame, it immediately triggers the occlusion recovery unit to start working. The occlusion recovery unit predicts and caches the last valid motion trajectory data before the target is lost based on the Kalman filter algorithm, extracts the instantaneous velocity vector, motion direction angle and acceleration value at the moment the target is lost, and then extrapolates the spatial location area where the target is most likely to appear at the current moment by combining the kinematic equation with the loss time, and generates a predicted scanning path covering high probability points based on the probability distribution characteristics of the area. The occlusion recovery unit controls the PTZ device to enter a low-speed search and scan mode, and performs a traversal scan within the predicted area at a constant angular velocity of 5° / s according to a preset gridded or spiral path. Meanwhile, the feature caching unit continuously outputs the biometric vector of the lost target and performs high-frequency fast feature comparison with the image frames acquired in real time during the scanning process. Once a candidate target matching the cached feature data is detected in the scanning area, the occlusion recovery unit immediately terminates the scanning action and, based on the target's current position and the attitude deviation of the gimbal device, quickly adjusts the attitude of the gimbal device with an optimized speed curve to accurately lock onto the target again, automatically and seamlessly returning to the normal tracking process.

8. The target intelligent tracking method for a mobile phone holder according to claim 7, wherein, The PTZ device stops tracking and resets upon receiving a stop command or when the target is lost and a timeout occurs. It also automatically saves its personalized configuration for future use, including: When a user issues a stop tracking command through the interaction module, or when the occlusion recovery unit continues to search for a target that has been lost for a longer period than the preset maximum loss time threshold, the PTZ device immediately terminates all tracking actions, the visual recognition algorithm pauses target locking, the motor drive signal is interrupted, and the device switches from dynamic following state to standby processing stage. The gimbal control module drives the three-axis brushless motor to automatically reset the gimbal device to its initial standby posture. At the same time, the target recognition module saves the personalized parameter configurations generated during this tracking process or manually adjusted by the user, including priority weights, tracking sensitivity, and supplementary lighting parameters, to the local memory for quick recall upon the next startup, realizing the user's habit memory for quick subsequent recall.

9. A target intelligent tracking system for a mobile phone holder, characterized in that, The system is used to perform the intelligent target tracking method according to any one of claims 1-8, the system comprising: The image acquisition module is used to detect ambient light intensity and adaptively acquire RGB and infrared multimodal image data; The target recognition module is used for local real-time identification and tracking of targets, prediction of motion trajectories, determination of the priority of multiple targets, and occlusion recovery. The gimbal control module is used to drive a three-axis brushless motor to achieve high-precision following based on target data, and to achieve closed-loop control by integrating attitude sensor data; The interaction module is used to receive user gestures and remote control commands and provide feedback on the system status; The power supply module is used to supply power to each module and perform intelligent power consumption scheduling.

10. The target intelligent tracking system for a mobile phone holder of claim 9, wherein, The target recognition module includes: The multi-target priority determination unit is used to dynamically score and rank candidate targets by comprehensively considering user-preset rules, target motion status, and clarity. The feature caching unit is used to store the target biometric vector for matching and re-identification; An occlusion recovery unit is used to initiate a low-speed scan and quickly re-lock on the target after it is lost, based on a Kalman filter-predicted trajectory. The ISP acceleration unit is used to perform noise reduction, enhancement, and color correction preprocessing on the original image.

Images