Gimbal tracking control method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of a gimbal tracking technology, and discloses a gimbal tracking control method and device, electronic equipment and a storage medium, the method comprising the following steps: acquiring a camera preview frame in real time, performing multi-target detection on the camera preview frame, and tracking each target detected; screening a main target from the targets according to a business scenario, obtaining a center parameter and a size parameter of a main target frame; determining a visual center distance based on the center parameter and a center coordinate of a region of interest; determining a target zoom ratio based on the size parameter, a zoom interval and a current zoom ratio, and adjusting to the target zoom ratio; and based on the visual center distance and the target zoom ratio, controlling gimbal movement by using a pre-constructed motor control function, so that the center parameter of the main target frame is adjusted to the center coordinate of the region of interest. The application maps a visual offset to a motor control function, avoids complex angle calibration work, and improves the flexibility and stability of gimbal tracking.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of gimbal tracking, in particular to a gimbal tracking control method and device, electronic equipment and storage medium. BACKGROUND

[0002] In the fields of video monitoring, photography tracking, unmanned aerial vehicle tracking, etc., accurate tracking of dynamic targets is a key technology. As a device that can flexibly adjust the viewing angle, the gimbal is widely used in the tracking of dynamic targets.

[0003] With the explosive growth of intelligent security, unmanned aerial vehicle inspection, live tracking and shooting, and industrial inspection scenarios, the gimbal visual tracking system based on visible light or infrared cameras has become the core front-end device in the "end-edge-cloud" architecture. The existing technology usually adopts a "detection-filter-drive" three-stage closed loop to realize tracking:

[0004] 1) using deep neural networks (YOLO, etc.) or traditional KCF (kernelized Correlation Filters, a kind of kernel correlation filter algorithm), CSRT (Channel and Spatial Reliability Tracker, a target tracking algorithm based on correlation filter) to detect the specific target in the picture frame by frame;

[0005] 2) using SORT (Simple Online and Realtime Tracking, a multi-target tracking algorithm) or DeepSORT (Deep Simple Online and Realtime Tracking, a multi-target tracking algorithm) framework to track the target based on the obtained pixel coordinates, and generating the expected angle of the gimbal;

[0006] 3) converting the angle offset into motor speed through PID (Proportional-Integral-Derivative engineering control technology) or fuzzy PID, and driving the gimbal to rotate to keep the target in the center area of the picture.

[0007] The above scheme has the following problems when engineering is landed:

[0008] 1. The existing technology lacks a main target screening mechanism based on business needs in the scene where multiple targets appear at the same time. When there are multiple targets in the picture, the gimbal is easy to switch between different targets irregularly, resulting in tracking confusion.

[0009] 2. The tracking target positioning logic of the existing technology is fixed, and only the geometric center of the picture is used as the tracking target resting area. In some scenes where there are specific requirements for the tracking target positioning area, the existing technology cannot adjust the tracking target resting area.

[0010] 3. Prior art first converts pixel offset into intermediate angle parameters such as pan angle, tilt angle, etc. of the gimbal, and then calculates motor control instructions based on the angle parameters. This process not only increases the complexity of the calculation link, but also requires professionals to perform tedious angle calibration. If the calibration error is large, it may lead to a decrease in motor control accuracy.

[0011] 4. The zoom adjustment of the prior art is mostly fixed mode or manual control, and cannot dynamically adapt the zoom ratio according to the actual size of the main target. When the size of the main target is small, if the zoom ratio is not increased in time, the main target will occupy too small a proportion in the picture; when the size of the main target is large, if the zoom ratio is not reduced in time, the main target will be out of the picture range.

[0012] 5. The motor control of the prior art mostly only relies on a single pixel offset, without considering the zoom ratio, business scenario, etc., causing problems such as shaking during gimbal tracking and inability to adapt to different business scenarios.

[0013] 6. The prior art either sacrifices hardware cost or tracking robustness, and cannot meet the engineering requirements of "no loss, no shaking, no collision, and no error" under the four constraints of "low cost, high zoom, strong occlusion, and multiple targets".

[0014] Therefore, this paper redesigns the gimbal tracking control method at the algorithm-control-hardware coordination level, especially optimizing the above-mentioned pain points at the control level. SUMMARY

[0015] To solve the above problems, the embodiments of the present application provide a gimbal tracking control method and device, electronic equipment and storage medium, which directly maps the visual offset to the motor control function, omits the intermediate calculation process of the pan angle and tilt angle, avoids the complex angle calibration process, is simple and practical, easy to implement, and improves the flexibility and stability of the gimbal tracking, to overcome or at least partially overcome the deficiencies of the prior art.

[0016] The embodiments of the present application adopt the following technical solutions:

[0017] In a first aspect, the present application provides a gimbal tracking control method, which comprises:

[0018] real-time acquisition of a camera preview frame, multi-target detection of the camera preview frame, and tracking of each detected target;

[0019] selecting a main target from the targets according to a business scenario, and obtaining center parameters and size parameters of a main target frame;

[0020] determining a visual center distance based on the center parameters and the center coordinates of the region of interest;

[0021] determine a target zoom ratio based on the size parameter, the zoom interval and the current zoom ratio, and adjust to the target zoom ratio;

[0022] based on the visual center distance and the target zoom ratio, control the gimbal movement by using a pre-constructed motor control function, so that the center parameter of the main target frame is adjusted to the center coordinate of the region of interest.

[0023] In a second aspect, the present application provides a gimbal tracking control device, the device comprising:

[0024] a target detection unit configured to acquire camera preview frames in real time, perform multi-target detection on the camera preview frames, and track each target detected;

[0025] a main target determination unit configured to select a main target from the targets according to a business scenario, and obtain a center parameter and a size parameter of a main target frame;

[0026] a visual measurement unit configured to determine a visual center distance based on the center parameter and a center coordinate of a region of interest;

[0027] an optical zoom unit configured to determine a target zoom ratio based on the size parameter, the zoom interval and the current zoom ratio, and adjust to the target zoom ratio;

[0028] a motor control unit configured to control the gimbal movement by using a pre-constructed motor control function based on the visual center distance and the target zoom ratio, so that the center parameter of the main target frame is adjusted to the center coordinate of the region of interest.

[0029] In a third aspect, the present application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned gimbal tracking control method when executing the computer program.

[0030] In a fourth aspect, the present application further provides a computer readable storage medium storing a computer program, wherein the computer program implements the steps of the above-mentioned gimbal tracking control method when being instructed by a processor.

[0031] The above-mentioned at least one technical solution adopted by the embodiments of the present application can achieve the following beneficial effects:

[0032] This application acquires camera preview frames in real time, performs multi-target detection on the camera preview frames, and tracks each detected target; it selects the main target from the targets according to the business scenario, and obtains the center parameters and size parameters of the main target bounding box; it determines the visual center distance based on the center parameters and the center coordinates of the region of interest; it determines the target zoom ratio based on the size parameters, zoom range, and current zoom ratio, and adjusts it to the target zoom ratio; based on the visual center distance and the target zoom ratio, it uses a pre-built motor control function to control the gimbal movement so that the center parameters of the main target bounding box are adjusted to the center coordinates of the region of interest.

[0033] The visual tracking and gimbal (including a stepper motor) proposed in this embodiment form a closed-loop control system. Throughout the tracking process, feedback values ​​from the visual algorithm are continuously received, and the gimbal is dynamically adjusted. The gimbal's movement speed is flexible and variable, enabling flexible and stable tracking (i.e., sensitivity + stability) for different business scenarios.

[0034] The method proposed in this embodiment directly maps the visual offset to the motor control function, omitting the intermediate calculation process of yaw angle and pitch angle. Even non-professionals can use the experience-based parameter tuning method, avoiding the complicated angle calibration process. It is simple, practical and easy to implement. Attached Figure Description

[0035] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0036] Figure 1 A flowchart illustrating a gimbal tracking control method according to an embodiment of this application is shown;

[0037] Figure 2 shows a schematic diagram of the region of interest according to an embodiment of the present application, wherein Figure 2(a) shows the region of interest located at the center of the screen, and Figure 2(b) shows the region of interest located at the right 2 / 3 of the screen;

[0038] Figure 3 A schematic flowchart illustrating the process of determining a target zoom ratio according to an embodiment of this application is shown;

[0039] Figure 4 Different embodiments according to this application are shown. Down and Relationship curve diagram;

[0040] Figure 5 This diagram illustrates the structure of a gimbal tracking control device according to an embodiment of the present application.

[0041] Figure 6A schematic diagram of the structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0043] To enable those skilled in the art to more clearly understand the technical solutions provided in the various embodiments of this application, the technical concept of this application will first be described.

[0044] Figure 1 A schematic flowchart of a gimbal tracking control method according to an embodiment of this application is shown. (Refer to...) Figure 1 As shown, the method provided in this embodiment includes:

[0045] Step S110: Acquire camera preview frames in real time, perform multi-target detection on the camera preview frames, and track each detected target;

[0046] Step S120: Select the main target from each target according to the business scenario, and obtain the center parameters and size parameters of the main target bounding box;

[0047] Step S130: Determine the visual center distance based on the center parameters and the center coordinates of the region of interest;

[0048] Step S140: Based on the size parameters, zoom range and current zoom ratio, determine the target zoom ratio and adjust to the target zoom ratio;

[0049] Step S150: Based on the visual center distance and target zoom ratio, the gimbal movement is controlled using a pre-built motor control function so that the center parameters of the main target box are adjusted to the center coordinates of the region of interest.

[0050] In practical applications, the gimbal needs to continuously track the camera video stream, therefore steps S110 to S150 above are executed cyclically. That is, based on each camera preview frame of the camera video stream, the closed-loop process of steps S110 to S150 above is continuously executed until the visual center distance approaches 0 and the main target bounding box is stably within the region of interest, maintaining the gimbal state; if the main target deviates again, the above closed-loop process is retried.

[0051] The following describes the gimbal tracking control method proposed in this embodiment by executing the above closed-loop process once.

[0052] In some optional implementations, step S110, acquiring camera preview frames in real time, performing multi-target detection on the camera preview frames, and tracking each detected target, includes: acquiring camera preview frames in real time; performing multi-target detection on the camera preview frames using a pre-trained target detector to obtain a target detection result set; assigning a unique identifier to each target in the target detection result set using a multi-target tracking framework, and recording the corresponding tracking time.

[0053] This embodiment can be applied to classroom observation scenarios, where camera video streams can be read from classroom monitoring equipment. Recommended, but not limited to, camera video stream resolution is 1280×720. This embodiment can also be applied to live streaming and follow-up shooting scenarios, where camera video streams can be read from recording equipment. The above application scenarios are merely illustrative and are not intended to limit the scope of the application.

[0054] Based on the camera video stream, read real-time camera preview frames. A camera preview frame represents a preview image at the current timestamp in the camera video stream.

[0055] Pre-train an object detector, which can be implemented based on any existing network architecture, as long as it can achieve multi-object detection. In a classroom observation scenario, the object detector can be a human face detector. The detection categories for the human face detector can be head, head and shoulders, or human body, etc. It is recommended that the detection category be head.

[0056] Head targets are detected in the camera preview frame using an object detector. Each head target (selected as a bounding box in practical applications) can be represented by a four-dimensional vector. express:

[0057] Equation (1);

[0058] in, This represents the x-coordinate of the top-left vertex of the detection box. This represents the y-coordinate of the top-left vertex of the detection box. This indicates the width of the detection frame. Indicates the height of the detection frame.

[0059] Object detection result set Represented as:

[0060] Equation (2);

[0061] Among them, the headcount target includes ~ .

[0062] Each detected target is assigned a unique trackID and its corresponding tracking time is recorded, serving as the tracking object. This embodiment can employ existing multi-target tracking frameworks, such as SORT or DeepSORT, to achieve multi-target tracking.

[0063] In multi-target tracking, each head target detected in the camera preview frame image can be associated with several existing tracking objects. If a head target is associated with an existing tracking object, the trackID of the existing tracking object is assigned to the head target. If a head target is not associated with an existing tracking object, a new unique trackID is assigned to the head target.

[0064] For each head target in the target detection result set, assign a unique trackID to each head target (this trackID does not change over time) and record the tracking time to obtain the following vector. express:

[0065] Equation (3);

[0066] in, An identifier representing the target person. This indicates the corresponding tracking time.

[0067] After performing multi-target detection and tracking on the camera preview frames, the primary target for gimbal tracking can be selected based on the business scenario. In practical applications, business scenarios can include, but are not limited to, intelligent security, drone inspection, live streaming, industrial inspection, and classroom observation. Depending on the business scenario, there are typically two methods for extracting the primary target: one is to use the largest target (which can be considered the closest target) as the primary target, and the other is to use the entire target as the primary target. For example, in a live streaming scenario, the largest target is used as the primary target; in an industrial inspection scenario, the entire target is used as the primary target.

[0068] In some optional implementations, step S120, selecting the main target from each target according to the business scenario and obtaining the center parameter and size parameter of the main target bounding box, includes: selecting the target with the largest detection box area among each target as the main target and obtaining the center parameter and size parameter of the main target bounding box.

[0069] Specifically, the detection box area for each target is calculated separately. :

[0070] Equation (4);

[0071] Sort the detection bounding boxes by area from largest to smallest, and select the target corresponding to the largest detection bounding box area in the ranking as the primary target. :

[0072] Equation (5).

[0073] The center and size parameters of the main target bounding box are calculated and expressed as follows:

[0074] , Equation (6);

[0075] in, , , , .

[0076] In some optional implementations, step S120, selecting the main target from each target according to the business scenario and obtaining the center parameters and size parameters of the main target bounding box, includes: using the bounding box of each target's detection box as the main target and obtaining the center parameters and size parameters of the main target bounding box.

[0077] Specifically, based on the four-dimensional vector representation of each head target, it can be converted into the coordinates of the four vertices of the detection box for each target:

[0078] Equation (7);

[0079] in, , .

[0080] The minimum x-coordinate of the top-left corner vertex, the minimum y-coordinate of the top-left corner vertex, the maximum x-coordinate of the bottom-right corner vertex, and the maximum y-coordinate of the bottom-right corner vertex of all detection boxes are used to construct the bounding box of the detection boxes, which serves as the primary target. :

[0081] Equation (8);

[0082] in, , , , .

[0083] The center and size parameters of the main target bounding box are calculated and expressed as follows:

[0084] Equation (9);

[0085] in, , , , .

[0086] After determining the main target, the visual center distance and target zoom ratio are calculated based on the center parameters and size parameters of the main target.

[0087] In some optional implementations, step S130, determining the visual center distance based on the center parameters and the center coordinates of the region of interest, includes: obtaining the center coordinates; wherein the center coordinates are represented by a normalized vector representing the center x-coordinate and center y-coordinate; normalizing the center parameters to obtain the center point x-coordinate and center point y-coordinate; and determining the visual center distance based on the difference between the center point x-coordinate and the center x-coordinate, and the difference between the center point y-coordinate and the center y-coordinate.

[0088] The region of interest is also known as the ROI. In existing technologies, the ROI is usually the exact center of the image (i.e., the visual center). For example, in a live streaming scenario, the main target (i.e., the streamer) is positioned in the exact center of the image. However, in other gimbal tracking scenarios, such as classroom observation, placing the ROI in the exact center of the image may not conform to photographic aesthetics or may affect the viewing angle of the blackboard. Therefore, in this embodiment, the center coordinates of the ROI can be obtained so that the main target is shifted towards the desired direction of the ROI during subsequent gimbal tracking.

[0089] Figure 2 shows a schematic diagram of the region of interest (ROI). Figure 2(a) shows the ROI located at the center of the image, and Figure 2(b) shows the ROI located in the right two-thirds of the image. Referring to Figure 2(a), the ROI is defined as... :

[0090] Equation (10);

[0091] in, Represents the normalized x-coordinate of the ROI. The y-coordinate of the normalized center of the ROI is represented by... This represents the normalized width of the ROI. This represents the normalized height of the ROI.

[0092] When the ROI in Figure 2(a) is located in the exact center of the image, initialization... The default value can be In the case shown in Figure 2(b), No longer in the visual center, but in the right two-thirds of the screen. .

[0093] ROI acquisition can be done manually or automatically.

[0094] Manual acquisition can introduce human-computer interaction, such as providing buttons or UI interfaces, to allow users to set the center coordinates of the ROI themselves.

[0095] Automatic acquisition can be tailored to business needs. Taking classroom observation as an example, if the main target is the teacher, but the live broadcast also needs to include close-up shots of the blackboard writing, then the center coordinates of the ROI can be automatically set based on the relative position of the teacher and the blackboard.

[0096] The central parameters are normalized to obtain the normalized central parameters. :

[0097] , Equation (11);

[0098] in, The normalized x-coordinate of the center point representing the center parameter. The ordinate of the normalized center point represents the central parameter.

[0099] So, based on the main objective By using the normalized center point location and the obtained center coordinates of the ROI, the visual center distance can be determined. :

[0100] Equation (12);

[0101] in, , .

[0102] In some optional implementations, step S140, determining and adjusting to the target zoom ratio based on size parameters, zoom range, and current zoom ratio, includes: determining the area of ​​the main target frame based on size parameters; obtaining the zoom range and current zoom ratio; wherein the zoom range includes a minimum area threshold and a maximum area threshold; when the area of ​​the main target frame is not less than the minimum area threshold and not greater than the maximum area threshold, determining the current zoom ratio as the target zoom ratio; when the area of ​​the main target frame is less than the minimum area threshold or greater than the maximum area threshold, determining the target zoom ratio based on the current zoom ratio, the area of ​​the main target frame, the minimum area threshold, and the maximum area threshold; and zooming according to the target zoom ratio.

[0103] Figure 3 A flowchart illustrating the process of determining the target zoom ratio is shown. (Refer to...) Figure 3 As shown, after determining the size parameters of the main target frame, the target zoom ratio can be determined based on the current zoom ratio, so as to zoom from the current zoom ratio to the target zoom ratio.

[0104] Calculate the area of ​​the main target bounding box based on its dimensions. :

[0105] Equation (13).

[0106] Obtain zoom range ,in, This represents the minimum area threshold. This represents the maximum area threshold.

[0107] Get current zoom level .

[0108] if Then let ,in, Indicates the target zoom level.

[0109] if or Then let .

[0110] According to the target zoom ratio Zoom in.

[0111] Under non-overload conditions, the operating characteristics of a motor exhibit clear and controllable patterns. Generally, its stopping position is determined solely by the number of input pulses (pulse emission count), while its speed is determined solely by the input duty cycle and is unaffected by load changes.

[0112] When a stepper driver receives a pulse signal, it drives the motor to rotate a fixed angle in a preset direction. This angle is defined as the "step angle." This characteristic makes the motor's rotation operate in a "step-by-step" manner—each pulse corresponds to one fixed angle rotation. Therefore, by precisely controlling the number of input pulses, the total angular displacement of the motor can be directly adjusted, thereby achieving accurate positioning of the gimbal (for example, when the gimbal needs to rotate a specific angle, only the number of pulses corresponding to a multiple of the step angle needs to be output).

[0113] By employing PWM (Pulse Width Modulation) technology, the average voltage across the motor can be adjusted by changing the duty cycle of the pulse signal (i.e., the ratio of the high-level duration within one pulse cycle to the total duration of the cycle). Since motor speed is positively correlated with the input average voltage (the higher the average voltage, the faster the motor speed), flexible control of the motor speed can be achieved by adjusting the duty cycle.

[0114] In gimbal tracking control, based on the above principles, precise positioning and speed regulation of the motor can be achieved by independently adjusting the number of pulses and the duty cycle.

[0115] In some optional implementations, step S150, based on the visual center distance and target zoom ratio, uses a pre-built motor control function to control the gimbal movement so that the center parameter of the main target box is adjusted to the center coordinate of the region of interest, includes: pre-building a motor control function; wherein the motor control function includes: an electrical pulse control function for adjusting the positioning accuracy of the gimbal and a duty cycle control function for adjusting the rotation speed of the gimbal; the visual center distance and target zoom ratio are respectively input into the electrical pulse control function and the duty cycle control function to perform motion control of the gimbal.

[0116] The motion control of the gimbal is divided into controlling the positioning accuracy of the gimbal and controlling the rotation speed of the gimbal. Specifically, the electrical pulse control function is used to determine how many steps the motor rotates, and the duty cycle control function is used to determine how fast the motor rotates.

[0117] In some optional implementations, the electrical pulse control functions include: a yaw angle electrical pulse control function and a pitch angle electrical pulse control function; both the yaw angle electrical pulse control function and the pitch angle electrical pulse control function are linear functions of the visual center distance and the target zoom ratio.

[0118] Specifically, the number of pulses emitted is calculated using the following formulas (14) to (15), with each degree of freedom (yaw and pitch) calculated separately. For a 2DoF gimbal:

[0119] Define the yaw angle electrical pulse control function of the gimbal as follows:

[0120] , Equation (14);

[0121] Define the pitch angle electrical pulse control function of the gimbal as follows:

[0122] Equation (15);

[0123] in, , , , , , These represent the calibrated electrical pulse control parameters.

[0124] The six electrical pulse control parameters can be calibrated using the following method.

[0125] One method is manual parameter tuning. Based on observation of a reference object, the parameters are gradually adjusted until a satisfactory control effect is achieved.

[0126] Taking yaw angle as an example:

[0127] Will and Set it to 0, set the optical zoom ratio to 1, and set a small duty cycle. Set to an arbitrarily small initial value (the range may vary between different suppliers, e.g., 128), and gradually increase it in specific increments. This continues until the motor braking is triggered (i.e., the minimum number of pulses required to brake the motor is reached), at which point the final value is fixed. ;

[0128] Set up any first reference object (such as a model of a human head), the first reference object , (in, This represents the horizontal component of the calibrated visual center distance. (representing the vertical component of the calibrated visual center distance), keep Set to 0, maintain optical zoom ratio of 1, and maintain the final... Gradually increase Control the motor to rotate in a specific direction, increasing the speed each time. The motor will return to its original position until... And without oscillation, the final result is fixed. ;

[0129] Keep the final And the final Gradually reduce each time decrease The optical zoom ratio is changed at each time until a certain optical zoom ratio is reached. And without oscillation, the final result is fixed. .

[0130] Thus, for the yaw angle electrical pulse control function , , Perform calibration.

[0131] For pitch angle electrical pulse control function , , The calibration method can be the same. That is:

[0132] Will and Set it to 0, set the optical zoom ratio to 1, and set a small duty cycle. Set to an arbitrarily small initial value (the range may vary between different suppliers, e.g., 128), and gradually increase it in specific increments. This continues until the motor braking is triggered (i.e., the minimum number of pulses required to brake the motor is reached), at which point the final value is fixed. ;

[0133] Set up any second reference object (such as a model of a human head), the second reference object , ,Keep Set to 0, maintain optical zoom ratio of 1, and maintain the final... Gradually increase Control the motor to rotate in a specific direction, increasing the speed each time. The motor will return to its original position until... And without oscillation, the final result is fixed. ;

[0134] Keep the final And the final Gradually reduce each time decrease The optical zoom ratio is changed at each time until a certain optical zoom ratio is reached. And without oscillation, the final result is fixed. .

[0135] Another method is automatic parameter tuning. The electrical pulse control parameters are determined by fitting the geometric planes of the yaw angle electrical pulse control function and the pitch angle electrical pulse control function, respectively.

[0136] Taking yaw angle as an example:

[0137] Set a first reference point. , ;

[0138] Multiple preset first optical zoom levels;

[0139] For each first optical zoom level, control the gimbal movement to make And there is no oscillation; record the corresponding first pulse emission count data.

[0140] By summarizing all first optical zoom levels and their corresponding first pulse emission counts, a system is constructed based on... The first sample dataset, with the first optical zoom level as the input variable and the first pulse emission count as the output variable;

[0141] Based on the first sample dataset, the least squares method was used for fitting to obtain... , , .

[0142] For pitch angle electrical pulse control function , , The calibration method can be the same. That is:

[0143] Set a second reference point. , ;

[0144] Multiple preset second optical zoom levels;

[0145] For each second optical zoom level, control the gimbal movement to make And there was no oscillation; the corresponding second pulse emission count data was recorded.

[0146] By summarizing all second optical zoom levels and their corresponding second pulse emission counts, a system is constructed based on... The second sample dataset uses the second optical zoom level as the input variable and the second pulse emission number data as the output variable;

[0147] Based on the first sample dataset, the least squares method was used for fitting to obtain... , , .

[0148] In some optional implementations, the duty cycle control function includes: a yaw angle duty cycle control function and a pitch angle duty cycle control function; both the yaw angle duty cycle control function and the pitch angle duty cycle control function are functions that include a dynamic adjustment term, an exponential adjustment term, and a fixed term; wherein, the dynamic adjustment term is related to the target zoom magnification, and the exponential adjustment term is related to the visual center distance.

[0149] Specifically, the duty cycle is calculated using the following formulas (16) to (17), with each degree of freedom (yaw and pitch) calculated separately. For a 2DoF gimbal:

[0150] Define the yaw angle duty cycle control function of the gimbal as follows:

[0151] , Equation (16);

[0152] Define the gimbal's pitch angle duty cycle control function as follows:

[0153] , Equation (17);

[0154] in, , , , , , These represent the calibrated duty cycle control parameters, respectively. In formulas (16) and (17) above: and This is a dynamic adjustment term, which is related to the target zoom level; and This is an exponential adjustment term, which is related to the visual center distance; and This is a fixed item.

[0155] The function of the index adjustment term is as follows:

[0156] (1) The exponential adjustment term can normalize (or compress) the numerical range to between [0,1], and then through... , The adaptive zoom is resized to the specified range, effectively limiting the output amplitude.

[0157] (2) and When the exponential adjustment term is monotonic and has no inflection point, it has less noise impact during numerical fitting compared to other forms such as the polynomial adjustment term.

[0158] (3) The exponential adjustment term is more useful in practical applications; some parameters that are very important in engineering, such as the brightness error-gain convergence curve of automatic exposure AE and the nonlinear relationship of excitation / torque response of FOC current loop dq axis, are all expressed by exponential expression.

[0159] The basic idea of ​​variable speed control is: the farther the center parameter of the main target box is from the center coordinate of the ROI, the greater the gimbal tracking speed; the closer the center parameter of the main target box is to the center coordinate of the ROI, the smaller the gimbal tracking speed.

[0160] parameter and It is used to limit the range of values ​​for the dynamic duty cycle of the gimbal. Its core function is to limit the dynamic adjustment of the duty cycle within a reasonable range.

[0161] The steepness of the speed change in variable speed motion is determined by... and Confirmed. The smaller the value, the more drastic the change in speed with the deviation of the main target box; the closer the value is to 0, the smoother the change in speed with the deviation of the main target box, which can reduce tracking jitter caused by sudden changes in rotation speed.

[0162] and These are the weighting coefficients for different optical zoom ratios; and This is a fixed term, representing the minimum duty cycle (corresponding to the minimum rotational speed). That is, when... or At that time, the gimbal operates at a constant minimum rotation speed. or sports.

[0163] The duty cycle control parameters in the yaw angle duty cycle control function and pitch angle duty cycle control function mentioned above can be obtained by calibration using the following methods.

[0164] Taking yaw angle as an example:

[0165] make To maximize the optical zoom ratio, Set a small initial value. At this point, regardless of the optical zoom level, the gimbal will... Move at the corresponding speed. Adjust the optical zoom to maximum and gradually increase it. Until the maximum optical zoom magnification, the gimbal tracking is oscillating, and the final position is fixed. ;

[0166] Maintain the maximum optical zoom ratio, and maintain the final zoom level. Gradually increase the amount of data while maintaining a stable, oscillating performance. Until the gimbal tracking speed reaches the desired standard, the final position is fixed. ;

[0167] Keep the final And the final The optical zoom ratio is changed to determine the smoothness of gimbal tracking and the business scenario based on different optical zoom ratios. For example, some business scenarios require faster gimbal tracking speeds. It can be smaller; some business scenarios require more stable gimbal tracking, then... It can get closer to 0. For example, at low optical zoom levels where the shifting speed is slow, it can be reduced. .

[0168] Thus, in the yaw angle duty cycle control function , , Perform calibration.

[0169] For the pitch angle duty cycle control function , , The calibration method can be the same. That is:

[0170] make To maximize the optical zoom ratio, Set a small initial value. At this point, regardless of the optical zoom level, the gimbal will... Move at the corresponding speed. Adjust the optical zoom to maximum and gradually increase it. Until the maximum optical zoom magnification, the gimbal tracking is oscillating, and the final position is fixed. ;

[0171] Maintain the maximum optical zoom ratio, and maintain the final zoom level. Gradually increase the amount of data while maintaining a stable, oscillating performance. Until the gimbal tracking speed reaches the desired standard, the final position is fixed. ;

[0172] Keep the final And the final The optical zoom ratio is changed to determine the smoothness of gimbal tracking and the business scenario based on different optical zoom ratios. For example, some business scenarios require faster gimbal tracking speeds. It can be smaller; some business scenarios require more stable gimbal tracking, then... It can get closer to 0. For example, at low optical zoom levels where the shifting speed is slow, it can be reduced. .

[0173] Figure 4 Show different Down and The relationship curve diagram. In Figure 4 In the middle, Written as , =1、 , .

[0174] The gimbal tracking control method proposed in this embodiment involves: acquiring camera preview frames in real time; using a target detector to detect the camera preview frame image and obtain a target detection box (containing the target to be tracked); using a multi-target tracking framework to track the detection results; determining the main target; acquiring the ROI, calculating the visual center distance and target zoom ratio, zooming according to the target zoom ratio, and inputting the visual center distance and target zoom ratio into the motor control function; using the calculated pulse emission number and duty cycle to control the stepper motor of the gimbal to rotate; continuing to acquire the next camera preview frame and repeating the aforementioned steps until the center parameter of the main target box remains at the center coordinates of the ROI.

[0175] In existing technologies, stepper motors are generally controlled in an open-loop manner. When the target zoom ratio is very high (e.g., 10x), existing technologies commonly use a single-loop PID controller that tracks "pixel deviation → angle deviation → speed," without planning for motor acceleration. When the main target suddenly appears at the edge, the large error causes the PID controller to saturate instantaneously, and the gimbal rushes towards the main target with maximum acceleration, resulting in severe overshoot due to the lack of a predicted deceleration phase. Furthermore, current mainstream solutions lock the camera output at 30 fps or higher. However, the detection frame rate of the deep learning visual model is not synchronized with the complete cycle of the motor control command from execution to control response. At high detection frame rates, the pixel displacement of the main target between adjacent frames is extremely small, but it is repeatedly amplified by the PID controller, causing jitter.

[0176] The visual tracking and gimbal (including a stepper motor) proposed in this embodiment form a closed-loop control system. Throughout the tracking process, feedback values ​​from the visual algorithm are continuously received, and the gimbal is dynamically adjusted. The gimbal's movement speed is flexible and variable, enabling flexible and stable tracking (i.e., sensitivity + stability), and it is suitable for different business scenarios.

[0177] The method proposed in this embodiment directly maps the visual offset to the motor control function, omitting the intermediate calculation process of yaw angle and pitch angle. Even non-professionals can use the experience-based parameter tuning method, avoiding the complicated angle calibration process. It is simple, practical and easy to implement.

[0178] Figure 5 A schematic diagram of a gimbal tracking control device according to an embodiment of this application is shown, with reference to... Figure 5 As shown, the device 500 includes:

[0179] The target detection unit 510 is used to acquire camera preview frames in real time, perform multi-target detection on the camera preview frames, and track each detected target.

[0180] The main target determination unit 520 is used to filter out the main target from each target according to the business scenario and obtain the center parameters and size parameters of the main target box;

[0181] The visual measurement unit 530 is used to determine the visual center distance based on the center parameters and the center coordinates of the region of interest;

[0182] The optical zoom unit 540 is used to determine the target zoom ratio and adjust to the target zoom ratio based on the size parameters, zoom range and current zoom ratio;

[0183] The motor control unit 550 is used to control the gimbal movement based on the visual center distance and target zoom ratio using a pre-built motor control function, so that the center parameters of the main target box are adjusted to the center coordinates of the region of interest.

[0184] In some optional implementations, the target detection unit 510 is used to: acquire camera preview frames in real time; perform multi-target detection on the camera preview frames using a pre-trained target detector to obtain a target detection result set; assign a unique identifier to each target in the target detection result set using a multi-target tracking framework, and record the corresponding tracking time.

[0185] In some optional implementations, the main target determination unit 520 is used to: select the target with the largest detection box area among all targets as the main target, and obtain the center parameters and size parameters of the main target box.

[0186] In some optional implementations, the main target determination unit 520 is specifically used to: calculate the detection box area of ​​each target respectively, sort the detection box areas from largest to smallest; select the target corresponding to the detection box area with the largest sorted area as the main target; and determine the center parameters and size parameters of the main target's main target box.

[0187] In some optional implementations, the main target determination unit 520 is used to: use the bounding boxes of the detection frames of each target as the main target, and obtain the center parameters and size parameters of the main target frame.

[0188] In some optional implementations, the main target determination unit 520 is specifically used to: calculate the x-coordinate of the top left corner vertex, the y-coordinate of the top left corner vertex, the x-coordinate of the bottom right corner vertex, and the y-coordinate of the bottom right corner vertex of each target's detection box; take the minimum value of the x-coordinate of each top left corner vertex, the minimum value of the y-coordinate of each top left corner vertex, the maximum value of the x-coordinate of each bottom right corner vertex, and the maximum value of the y-coordinate of each bottom right corner vertex, and synthesize the detection box bounding box as the main target; and determine the center parameters and size parameters of the main target's main target box.

[0189] In some optional implementations, the visual measurement unit 530 is used to: acquire center coordinates; wherein the center abscissa and center ordinate are represented by a normalized vector; normalize the center parameters to obtain the center point abscissa and center point ordinate; and determine the visual center distance based on the difference between the center point abscissa and the center abscissa, and the difference between the center point ordinate and the center ordinate.

[0190] In some optional implementations, the optical zoom unit 540 is used to: determine the area of ​​the main target frame based on size parameters; obtain the zoom range and the current zoom ratio; wherein the zoom range includes a minimum area threshold and a maximum area threshold; when the area of ​​the main target frame is not less than the minimum area threshold and not greater than the maximum area threshold, determine the current zoom ratio as the target zoom ratio; when the area of ​​the main target frame is less than the minimum area threshold or greater than the maximum area threshold, determine the target zoom ratio based on the current zoom ratio, the area of ​​the main target frame, the minimum area threshold, and the maximum area threshold; and zoom according to the target zoom ratio.

[0191] In some optional implementations, the motor control unit 550 is used to: pre-build motor control functions; wherein the motor control functions include: an electrical pulse control function for adjusting the positioning accuracy of the gimbal and a duty cycle control function for adjusting the rotation speed of the gimbal; and input the visual center distance and the target zoom ratio into the electrical pulse control function and the duty cycle control function, respectively, to perform motion control on the gimbal.

[0192] In some optional implementations, the electrical pulse control functions include: a yaw angle electrical pulse control function and a pitch angle electrical pulse control function; both the yaw angle electrical pulse control function and the pitch angle electrical pulse control function are linear functions of the visual center distance and the target zoom ratio.

[0193] In some optional implementations, the yaw angle electrical pulse control function and the pitch angle electrical pulse control function are respectively expressed as:

[0194] ;

[0195] ;

[0196] in, This represents the yaw angle electrical pulse control function. This represents the pitch angle electrical pulse control function. The horizontal component representing the visual center distance, The vertical component representing the visual center distance, Indicates the target zoom ratio. , , , , , These represent the calibrated electrical pulse control parameters.

[0197] In some optional implementations, the electrical pulse control parameters of the yaw angle electrical pulse control function are calibrated by the following method:

[0198] make Let the optical zoom ratio be 1. Set as the initial value, and only gradually increase it. Until the motor brake is triggered, the final position is fixed. ;

[0199] Set a first reference object. The horizontal component of the calibrated visual center distance of the first reference object is not less than 0.1, and the vertical component of the calibrated visual center distance of the first reference object is 0. Maintain the optical zoom ratio of 1, and maintain the final... Only gradually increase Each time it increases The motor is returned to its original position until the horizontal component of the calibrated visual center distance approaches 0 and there is no oscillation, at which point the final position is fixed. ;

[0200] Keep the final And the final Only gradually decrease each time decrease The optical zoom magnification is changed periodically until the horizontal component of the calibrated visual center distance at any optical zoom magnification approaches 0 and shows no oscillation, at which point the final magnification is fixed. .

[0201] In some optional implementations, the electrical pulse control parameters of the pitch angle electrical pulse control function are calibrated by the following method:

[0202] make Let the optical zoom ratio be 1. Set as the initial value, and only gradually increase it. Until the motor brake is triggered, the final position is fixed. ;

[0203] A second reference object is set up, with the vertical component of the calibration visual center distance of the second reference object being no less than 0.1 and the horizontal component of the calibration visual center distance of the second reference object being 0. Maintain the optical zoom ratio of 1, and maintain the final... Only gradually increase Each time it increases The motor is returned to its original position until the vertical component of the calibrated visual center distance approaches zero and there is no oscillation, at which point the final position is fixed. ;

[0204] Keep the final And the final Only gradually decrease each time decrease The optical zoom ratio is changed periodically until the vertical component of the calibrated visual center distance at any optical zoom ratio approaches 0 and shows no oscillation; the final value is then fixed. .

[0205] In some optional implementations, the electrical pulse control parameters of the yaw angle electrical pulse control function are calibrated by the following method:

[0206] Set a first reference object, the horizontal component of the calibration visual center distance of the first reference object is not less than 0.1, and the vertical component of the calibration visual center distance of the first reference object is 0;

[0207] Multiple preset first optical zoom levels;

[0208] For each first optical zoom level, control the gimbal movement to make the horizontal component of the calibrated visual center distance approach 0 and without oscillation, and record the corresponding first pulse emission data;

[0209] Summarize all first optical zoom magnification levels and corresponding first pulse emission counts to construct a first sample dataset with the horizontal component of the calibrated visual center distance and the first optical zoom magnification level as input variables and the first pulse emission counts as output variables;

[0210] Based on the first sample dataset, the least squares method is used for fitting to obtain the electrical pulse control parameters of the yaw angle electrical pulse control function.

[0211] In some optional implementations, the electrical pulse control parameters of the pitch angle electrical pulse control function are calibrated by the following method:

[0212] A second reference object is set up, the vertical component of the calibration visual center distance of the second reference object is not less than 0.1, and the horizontal component of the calibration visual center distance of the second reference object is 0.

[0213] Multiple preset second optical zoom levels;

[0214] For each second optical zoom level, control the gimbal movement to make the vertical component of the calibrated visual center distance approach 0 and without oscillation, and record the corresponding second pulse emission data;

[0215] By summarizing all the second optical zoom magnification levels and the corresponding second pulse emission count data, a second sample dataset is constructed, with the vertical component of the calibrated visual center distance and the second optical zoom magnification level as input variables and the second pulse emission count data as output variables.

[0216] Based on the second sample dataset, the least squares method is used for fitting to obtain the electrical pulse control parameters of the pitch angle electrical pulse control function.

[0217] In some optional implementations, the duty cycle control function includes: a yaw angle duty cycle control function and a pitch angle duty cycle control function; both the yaw angle duty cycle control function and the pitch angle duty cycle control function are functions that include a dynamic adjustment term, an exponential adjustment term, and a fixed term; wherein, the dynamic adjustment term is related to the target zoom magnification, and the exponential adjustment term is related to the visual center distance.

[0218] In some optional implementations, the yaw angle duty cycle control function and the pitch angle duty cycle control function are expressed as follows:

[0219] ;

[0220] ;

[0221] in, This represents the yaw angle duty cycle control function. This represents the pitch angle duty cycle control function. The horizontal component representing the visual center distance, The vertical component representing the visual center distance, Indicates the target zoom ratio. , , , , , These represent the calibrated duty cycle control parameters.

[0222] In some optional implementations, the duty cycle control parameters of the yaw angle duty cycle control function are calibrated using the following methods:

[0223] make To maximize the optical zoom ratio, Set as the initial value, and only gradually increase it. Until the maximum optical zoom magnification, the gimbal tracking is oscillating, and the final position is fixed. ;

[0224] Maintain the maximum optical zoom ratio, and maintain the final zoom level. Under the premise of no oscillation, only gradually increase Until the gimbal tracking speed reaches the desired standard, the final position is fixed. ;

[0225] Keep the final And the final The optical zoom ratio is changed to determine the smoothness of gimbal tracking and the business scenario based on different optical zoom ratios. .

[0226] In some optional implementations, the duty cycle control parameters of the pitch angle duty cycle control function are calibrated using the following methods:

[0227] make To maximize the optical zoom ratio, Set as the initial value, and only gradually increase it. Until the maximum optical zoom magnification, the gimbal tracking is oscillating, and the final position is fixed. ;

[0228] Maintain the maximum optical zoom ratio, and maintain the final zoom level. Under the premise of no oscillation, only gradually increase Until the gimbal tracking speed reaches the desired standard, the final position is fixed. ;

[0229] Keep the final And the final The optical zoom ratio is changed to determine the smoothness of gimbal tracking and the business scenario based on different optical zoom ratios. .

[0230] It should be noted that the aforementioned gimbal tracking control device 500 can implement the aforementioned gimbal tracking control method, which will not be elaborated further.

[0231] Figure 6This invention illustrates a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 6 As shown, the electronic device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used for communication with external devices via a network connection. When the computer program is executed by the processor, it implements the functions or steps of the gimbal tracking control method.

[0232] In one embodiment, the electronic device provided in this application includes a memory and a processor. The memory stores a database and a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the aforementioned gimbal tracking control method.

[0233] The above is as stated in this application. Figure 5 The method executed by the gimbal tracking control device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0234] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the aforementioned gimbal tracking control method.

[0235] It should be noted that the functions or steps that the above-mentioned electronic devices or computer-readable storage media can achieve can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0236] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0237] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0238] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A gimbal tracking control method, characterized in that, include: The camera preview frame is acquired in real time, multi-target detection is performed on the camera preview frame, and each detected target is tracked. Based on the business scenario, the main target is selected from the various targets to obtain the center parameters and size parameters of the main target bounding box; The visual center distance is determined based on the center parameters and the center coordinates of the region of interest. Based on the size parameters, zoom range, and current zoom ratio, determine the target zoom ratio and adjust to the target zoom ratio; Based on the visual center distance and the target zoom ratio, a pre-built motor control function is used to control the gimbal movement so that the center parameter of the main target box is adjusted to the center coordinate of the region of interest.

2. The method according to claim 1, characterized in that, The real-time acquisition of camera preview frames, multi-target detection of the camera preview frames, and tracking of each detected target include: Real-time acquisition of camera preview frames; The camera preview frame is used to perform multi-object detection using a pre-trained object detector to obtain a set of object detection results; A multi-target tracking framework is used to assign a unique identifier to each target in the target detection result set and record the corresponding tracking time.

3. The method according to claim 1, characterized in that, The step of selecting the main target from the various targets based on the business scenario, and obtaining the center parameters and size parameters of the main target bounding box, includes: The target with the largest detection box area among all the targets is selected as the main target, and the center parameter and the size parameter of the main target box are obtained.

4. The method according to claim 3, characterized in that, The step of selecting the target with the largest detection box area among all the targets as the main target, and obtaining the center parameter and the size parameter of the main target bounding box, includes: Calculate the area of ​​the detection box for each target and sort the areas of the detection boxes from largest to smallest; The target whose area corresponds to the first selected detection box is taken as the main target. Determine the center parameter and the size parameter of the main target's main target bounding box.

5. The method according to claim 1, characterized in that, The step of selecting the main target from the various targets based on the business scenario, and obtaining the center parameters and size parameters of the main target bounding box, includes: Using the bounding box of each target as the main target, the center parameter and the size parameter of the main target bounding box are obtained.

6. The method according to claim 5, characterized in that, The step of using the bounding boxes of the detection frames of each of the aforementioned targets as the primary target, and obtaining the center parameters and size parameters of the primary target bounding box, includes: Calculate the x-coordinate of the top-left corner vertex, the y-coordinate of the top-left corner vertex, the x-coordinate of the bottom-right corner vertex, and the y-coordinate of the bottom-right corner vertex of the detection box for each target. The minimum value of the x-coordinate of each of the top-left corner vertices, the minimum value of the y-coordinate of each of the top-left corner vertices, the maximum value of the x-coordinate of each of the bottom-right corner vertices, and the maximum value of the y-coordinate of each of the bottom-right corner vertices are taken to synthesize the detection bounding box as the main target; Determine the center parameter and the size parameter of the main target's main target bounding box.

7. The method according to claim 1, characterized in that, Determining the visual center distance based on the center parameters and the center coordinates of the region of interest includes: Obtain the center coordinates; wherein the center x-coordinate and center y-coordinate are represented by a normalized vector. The center parameter is normalized to obtain the x-coordinate and y-coordinate of the center point of the center parameter. The visual center distance is determined based on the difference between the x-coordinate of the center point and the x-coordinate of the center point, and the difference between the y-coordinate of the center point and the y-coordinate of the center point.

8. The method according to claim 1, characterized in that, The step of determining the target zoom ratio and adjusting it to the target zoom ratio based on the size parameters, zoom range, and current zoom ratio includes: Determine the area of ​​the main target frame based on the aforementioned size parameters; Obtain the zoom range and the current zoom ratio; wherein, the zoom range includes a minimum area threshold and a maximum area threshold; When the area of ​​the main target frame is not less than the minimum area threshold and not greater than the maximum area threshold, the current zoom ratio is determined as the target zoom ratio; When the area of ​​the main target frame is less than the minimum area threshold or greater than the maximum area threshold, the target zoom ratio is determined based on the current zoom ratio, the area of ​​the main target frame, the minimum area threshold, and the maximum area threshold; Zoom according to the target zoom ratio.

9. The method according to claim 1, characterized in that, The step of controlling the gimbal movement using a pre-built motor control function based on the visual center distance and the target zoom ratio, so as to adjust the center parameters of the main target bounding box to the center coordinates of the region of interest, includes: The motor control function is pre-constructed; wherein, the motor control function includes: an electrical pulse control function for adjusting the positioning accuracy of the gimbal and a duty cycle control function for adjusting the rotation speed of the gimbal; The visual center distance and the target zoom ratio are respectively input into the electrical pulse control function and the duty cycle control function to perform motion control on the gimbal.

10. The method according to claim 9, characterized in that, The electrical pulse control functions include: yaw angle electrical pulse control function and pitch angle electrical pulse control function; Both the yaw angle electrical pulse control function and the pitch angle electrical pulse control function are linear functions of the visual center distance and the target zoom ratio.

11. The method according to claim 10, characterized in that, The yaw angle electrical pulse control function and the pitch angle electrical pulse control function are respectively expressed as follows: ； ； in, This represents the yaw angle electrical pulse control function. This represents the pitch angle electrical pulse control function. The horizontal component of the visual center distance is represented. This represents the vertical component of the visual center distance. This indicates the target zoom ratio. , , , , , These represent the calibrated electrical pulse control parameters.

12. The method according to claim 11, characterized in that, The electrical pulse control parameters of the yaw angle electrical pulse control function are calibrated by the following method: make Let the optical zoom ratio be 1. Set as the initial value, and only gradually increase it. Until the motor brake is triggered, the final position is fixed. ; A first reference object is set, wherein the horizontal component of the calibrated visual center distance of the first reference object is not less than 0.1, and the vertical component of the calibrated visual center distance of the first reference object is 0, and this is maintained. Maintain the optical zoom ratio of 1, and maintain the final Only gradually increase Each time it increases The motor is returned to its original position until the horizontal component of the calibrated visual center distance approaches 0 and there is no oscillation, then the final position is fixed. ; Keep the final And the final Only gradually decrease each time decrease The optical zoom ratio is changed periodically until the horizontal component of the calibrated visual center distance at any given optical zoom ratio approaches 0 and shows no oscillation, at which point the final value is fixed. .

13. The method according to claim 11, characterized in that, The electrical pulse control parameters of the pitch angle electrical pulse control function are calibrated by the following method: make Let the optical zoom ratio be 1. Set as the initial value, and only gradually increase it. Until the motor brake is triggered, the final position is fixed. ; A second reference object is set, wherein the vertical component of the calibrated visual center distance of the second reference object is not less than 0.1, and the horizontal component of the calibrated visual center distance of the second reference object is 0, maintaining... Maintain the optical zoom ratio of 1, and maintain the final Only gradually increase Each time it increases The motor is returned to its original position until the vertical component of the calibrated visual center distance approaches 0 and there is no oscillation, then the final position is fixed. ; Keep the final And the final Only gradually decrease each time decrease The optical zoom magnification is changed periodically until the vertical component of the calibrated visual center distance at any optical zoom magnification approaches 0 and shows no oscillation, at which point the final value is fixed. .

14. The method according to claim 11, characterized in that, The electrical pulse control parameters of the yaw angle electrical pulse control function are calibrated by the following method: A first reference object is set, wherein the horizontal component of the calibrated visual center distance of the first reference object is not less than 0.1, and the vertical component of the calibrated visual center distance of the first reference object is 0. Multiple preset first optical zoom levels; For each of the first optical zoom levels, control the gimbal movement to make the horizontal component of the calibrated visual center distance approach 0 and without oscillation, and record the corresponding first pulse emission data; Summarize all the first optical zoom magnification levels and the corresponding first pulse emission count data to construct a first sample dataset with the horizontal component of the calibrated visual center distance and the first optical zoom magnification level as input variables and the first pulse emission count data as output variables; Based on the first sample dataset, the least squares method is used for fitting to obtain the electrical pulse control parameters of the yaw angle electrical pulse control function.

15. The method according to claim 11, characterized in that, The electrical pulse control parameters of the pitch angle electrical pulse control function are calibrated by the following method: A second reference object is set, wherein the vertical component of the calibrated visual center distance of the second reference object is not less than 0.1, and the horizontal component of the calibrated visual center distance of the second reference object is 0; Multiple preset second optical zoom levels; For each of the second optical zoom levels, control the gimbal movement to make the vertical component of the calibrated visual center distance approach 0 and without oscillation, and record the corresponding second pulse emission data; Summarize all the second optical zoom magnification levels and the corresponding second pulse emission count data to construct a second sample dataset with the vertical component of the calibrated visual center distance and the second optical zoom magnification level as input variables and the second pulse emission count data as output variables; Based on the second sample dataset, the least squares method is used for fitting to obtain the electrical pulse control parameters of the pitch angle electrical pulse control function.

16. The method according to claim 9, characterized in that, The duty cycle control function includes: yaw angle duty cycle control function and pitch angle duty cycle control function; Both the yaw angle duty cycle control function and the pitch angle duty cycle control function are functions that include a dynamic adjustment term, an exponential adjustment term, and a fixed term; wherein, the dynamic adjustment term is related to the target zoom ratio, and the exponential adjustment term is related to the visual center distance.

17. The method according to claim 16, characterized in that, The yaw angle duty cycle control function and the pitch angle duty cycle control function are respectively expressed as follows: ； ； in, This represents the yaw angle duty cycle control function. This represents the pitch angle duty cycle control function. The horizontal component of the visual center distance is represented. This represents the vertical component of the visual center distance. This indicates the target zoom ratio. , , , , , These represent the calibrated duty cycle control parameters.

18. The method according to claim 17, characterized in that, The duty cycle control parameters of the yaw angle duty cycle control function are calibrated by the following method: make To maximize the optical zoom ratio, Set as the initial value, and only gradually increase it. Until the maximum optical zoom magnification, the gimbal tracking is oscillating, and the final position is fixed. ; Maintaining the maximum optical zoom ratio, maintaining the final... Under the premise of no oscillation, only gradually increase Until the gimbal tracking speed reaches the desired standard, the final position is fixed. ; Keep the final And the final The optical zoom ratio is changed, and the gimbal tracking smoothness is determined based on the different optical zoom ratios and the business scenario. .

19. The method according to claim 17, characterized in that, The duty cycle control parameters of the pitch angle duty cycle control function are calibrated using the following method: make To maximize the optical zoom ratio, Set as the initial value, and only gradually increase it. Until the maximum optical zoom magnification, the gimbal tracking is oscillating, and the final position is fixed. ; Maintaining the maximum optical zoom ratio, maintaining the final... Under the premise of no oscillation, only gradually increase Until the gimbal tracking speed reaches the desired standard, the final position is fixed. ; Keep the final And the final The optical zoom ratio is changed, and the gimbal tracking smoothness is determined based on the different optical zoom ratios and the business scenario. .

20. A gimbal tracking control device, characterized in that, The device includes: The target detection unit is used to acquire camera preview frames in real time, perform multi-target detection on the camera preview frames, and track each detected target. The main target determination unit is used to filter out the main target from the various targets according to the business scenario, and obtain the center parameters and size parameters of the main target box; A visual measurement unit is used to determine the visual center distance based on the center parameters and the center coordinates of the region of interest; An optical zoom unit is used to determine a target zoom ratio and adjust it to the target zoom ratio based on the size parameters, zoom range and current zoom ratio; A motor control unit is used to control the movement of the gimbal using a pre-built motor control function based on the visual center distance and the target zoom ratio, so that the center parameter of the main target box is adjusted to the center coordinate of the region of interest.

21. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the gimbal tracking control method as described in any one of claims 1 to 19.

22. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is instructed by the processor, it implements the steps of the gimbal tracking control method as described in any one of claims 1 to 19.

Images